2024-05-21T14:52:33.143 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-21T14:52:33.144 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/orig.config.yaml 2024-05-21T14:52:33.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T14:52:33.308 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560 branch: wip-pdonnell-testing-20240517.203521-squid-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} email: pdonnell@redhat.com first_in_suite: false job_id: '7718560' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.203521-squid-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.203521-squid-debug sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9749 sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d sleep_before_teardown: 0 subset: 272/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.203521-squid-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjscK/jMkSTwH9F5FQVxO1HUUSB8PjxL3Ma+OrZ9Y6MGAxiLphqRgyRg4XD9lqpZ5hlkZvMBoS2XSy4g4tB7P8= smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSZCSTiexX4aBLD/Dz/gfRoByYUt9+Hi4f2g/PCf0QMXJplxqTMChjKWMZZotvo/BwWGdIyE4Sn0Zr5bzRrRAw= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8VnR5VdbzAHg1bgN3y1VBHeFrzqdDYxdbwfLHiY8WVsp5SZ+EFwA6wZusDZKHuX8RY1TwJeyx7v1I6uu+KB3I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-21_13:18:14 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-21T14:52:33.308 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa; will attempt to use it 2024-05-21T14:52:33.309 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks 2024-05-21T14:52:33.309 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-21T14:52:33.311 INFO:teuthology.task.internal:Checking packages... 2024-05-21T14:52:33.334 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2b507c724ef6c8295ee0a35d185cdab167fd61d' 2024-05-21T14:52:33.334 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-21T14:52:33.334 INFO:teuthology.packaging:ref: None 2024-05-21T14:52:33.334 INFO:teuthology.packaging:tag: None 2024-05-21T14:52:33.334 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-21T14:52:33.334 INFO:teuthology.packaging:sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T14:52:33.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-21T14:52:33.471 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2744.gd2b507c7 2024-05-21T14:52:33.472 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-21T14:52:33.481 INFO:teuthology.task.internal:no buildpackages task found 2024-05-21T14:52:33.481 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-21T14:52:33.495 INFO:teuthology.task.internal:Saving configuration 2024-05-21T14:52:33.518 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-21T14:52:33.527 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-21T14:52:33.553 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 14:47:24.997245', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjscK/jMkSTwH9F5FQVxO1HUUSB8PjxL3Ma+OrZ9Y6MGAxiLphqRgyRg4XD9lqpZ5hlkZvMBoS2XSy4g4tB7P8='} 2024-05-21T14:52:33.574 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 14:47:24.995768', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSZCSTiexX4aBLD/Dz/gfRoByYUt9+Hi4f2g/PCf0QMXJplxqTMChjKWMZZotvo/BwWGdIyE4Sn0Zr5bzRrRAw='} 2024-05-21T14:52:33.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 14:47:24.994258', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8VnR5VdbzAHg1bgN3y1VBHeFrzqdDYxdbwfLHiY8WVsp5SZ+EFwA6wZusDZKHuX8RY1TwJeyx7v1I6uu+KB3I='} 2024-05-21T14:52:33.597 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-21T14:52:33.606 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-21T14:52:33.606 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-21T14:52:33.606 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-21T14:52:33.606 INFO:teuthology.run_tasks:Running task console_log... 2024-05-21T14:52:33.698 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8fa9f3fac0>, signals=[15]) 2024-05-21T14:52:33.698 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-21T14:52:33.706 INFO:teuthology.task.internal:Opening connections... 2024-05-21T14:52:33.706 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T14:52:33.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T14:52:33.786 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-05-21T14:52:33.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T14:52:33.864 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-05-21T14:52:33.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T14:52:33.938 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-21T14:52:33.947 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-05-21T14:52:33.964 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-05-21T14:52:33.965 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-05-21T14:52:34.021 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-05-21T14:52:34.022 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T14:52:34.022 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-05-21T14:52:34.022 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T14:52:34.022 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T14:52:34.022 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T14:52:34.022 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-05-21T14:52:34.041 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-05-21T14:52:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-05-21T14:52:34.059 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-05-21T14:52:34.115 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-05-21T14:52:34.116 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T14:52:34.117 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T14:52:34.117 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T14:52:34.117 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-05-21T14:52:34.140 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-05-21T14:52:34.158 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-05-21T14:52:34.159 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-05-21T14:52:34.218 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-05-21T14:52:34.218 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-05-21T14:52:34.218 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-05-21T14:52:34.218 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-05-21T14:52:34.218 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T14:52:34.219 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T14:52:34.220 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-05-21T14:52:34.240 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-21T14:52:34.252 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-21T14:52:34.263 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-21T14:52:34.263 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-05-21T14:52:34.265 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-05-21T14:52:34.268 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-05-21T14:52:34.284 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-21T14:52:34.296 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-21T14:52:34.296 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-05-21T14:52:34.322 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-05-21T14:52:34.324 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-05-21T14:52:34.433 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-21T14:52:34.512 INFO:teuthology.run_tasks:Running task kernel... 2024-05-21T14:52:34.529 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-21T14:52:34.530 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-21T14:52:34.530 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-21T14:52:34.530 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T14:52:34.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T14:52:34.546 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-05-21T14:52:34.603 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-437.el9.x86_64 2024-05-21T14:52:34.603 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-437.el9.x86_64 2024-05-21T14:52:34.603 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-05-21T14:52:38.240 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.1 MB 00:01 2024-05-21T14:52:44.040 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 19 MB 00:04 2024-05-21T14:54:58.701 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 48 kB/s | 6.2 MB 02:11 2024-05-21T14:55:00.476 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-21T14:55:05.705 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 21 MB 00:05 2024-05-21T14:55:10.877 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-05-21T14:55:12.708 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T14:55:12.708 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T14:55:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-05-21T14:55:12.824 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-05-21T14:55:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-05-21T14:55:12.824 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-05-21T14:55:12.824 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-05-21T14:55:12.824 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout:Install 4 Packages 2024-05-21T14:55:12.825 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T14:55:12.826 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 78 M 2024-05-21T14:55:12.826 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 123 M 2024-05-21T14:55:12.826 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-05-21T14:55:13.129 INFO:teuthology.orchestra.run.smithi078.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 2.3 MB/s | 576 kB 00:00 2024-05-21T14:55:14.462 INFO:teuthology.orchestra.run.smithi078.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 9.8 MB/s | 16 MB 00:01 2024-05-21T14:55:14.954 INFO:teuthology.orchestra.run.smithi078.stdout:(3/4): kernel-modules-core-5.14.0-447.el9.x86_6 15 MB/s | 28 MB 00:01 2024-05-21T14:55:15.438 INFO:teuthology.orchestra.run.smithi078.stdout:(4/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 13 MB/s | 34 MB 00:02 2024-05-21T14:55:15.438 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-05-21T14:55:15.439 INFO:teuthology.orchestra.run.smithi078.stdout:Total 30 MB/s | 78 MB 00:02 2024-05-21T14:55:16.119 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-05-21T14:55:16.458 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-05-21T14:55:16.459 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-05-21T14:55:17.387 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-05-21T14:55:17.388 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-05-21T14:55:19.774 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-05-21T14:55:20.382 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T14:55:20.472 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T14:55:23.123 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T14:55:23.254 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T14:55:27.949 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T14:55:27.990 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T14:55:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T14:56:02.187 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T14:56:02.225 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T14:56:02.599 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T14:56:02.599 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T14:56:02.599 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T14:56:02.599 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T14:56:03.392 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T14:56:03.393 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T14:56:03.393 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-05-21T14:56:03.393 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T14:56:03.393 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T14:56:03.393 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T14:56:03.393 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T14:56:03.393 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T14:56:03.393 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-05-21T14:56:03.716 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T14:56:03.787 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T14:56:03.787 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T14:56:03.787 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T14:56:03.787 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T14:56:03.787 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T14:56:03.788 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-21T14:56:03.788 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi078.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T14:56:03.788 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T14:56:03.837 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T14:56:03.837 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T14:56:03.838 DEBUG:teuthology.orchestra.run.smithi078:> sudo rpm -qi grub2-tools 2024-05-21T14:56:03.868 INFO:teuthology.orchestra.run.smithi078.stdout:Name : grub2-tools 2024-05-21T14:56:03.868 INFO:teuthology.orchestra.run.smithi078.stdout:Epoch : 1 2024-05-21T14:56:03.868 INFO:teuthology.orchestra.run.smithi078.stdout:Version : 2.06 2024-05-21T14:56:03.868 INFO:teuthology.orchestra.run.smithi078.stdout:Release : 68.el9 2024-05-21T14:56:03.868 INFO:teuthology.orchestra.run.smithi078.stdout:Architecture: x86_64 2024-05-21T14:56:03.868 INFO:teuthology.orchestra.run.smithi078.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T14:56:03.868 INFO:teuthology.orchestra.run.smithi078.stdout:Group : Unspecified 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Size : 8264254 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:License : GPLv3+ 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Packager : builder@centos.org 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Vendor : CentOS 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Summary : Support tools for GRUB. 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:Description : 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T14:56:03.869 INFO:teuthology.orchestra.run.smithi078.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T14:56:03.870 INFO:teuthology.orchestra.run.smithi078.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T14:56:03.870 INFO:teuthology.orchestra.run.smithi078.stdout:hardware devices. 2024-05-21T14:56:03.870 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T14:56:03.870 INFO:teuthology.orchestra.run.smithi078.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T14:56:03.871 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T14:56:03.871 INFO:teuthology.task.kernel:Updating grub on smithi078 to boot 5.14.0-447.el9.x86_64 2024-05-21T14:56:03.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T14:56:04.435 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2024-05-21T14:56:05.782 INFO:teuthology.orchestra.run.smithi078.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T14:56:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:done 2024-05-21T14:56:05.821 DEBUG:teuthology.orchestra.run.smithi078:> mktemp 2024-05-21T14:56:05.835 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/tmp.Y7mDjo5dmQ 2024-05-21T14:56:05.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Y7mDjo5dmQ 2024-05-21T14:56:05.900 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0666 /tmp/tmp.Y7mDjo5dmQ 2024-05-21T14:56:06.042 DEBUG:teuthology.orchestra.remote:smithi078:/tmp/tmp.Y7mDjo5dmQ is 6KB 2024-05-21T14:56:06.061 DEBUG:teuthology.orchestra.run.smithi078:> rm -fr /tmp/tmp.Y7mDjo5dmQ 2024-05-21T14:56:06.075 DEBUG:teuthology.orchestra.run.smithi078:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T14:56:06.146 INFO:teuthology.orchestra.run.smithi078.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T14:56:06.146 INFO:teuthology.orchestra.run.smithi078.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T14:56:06.146 INFO:teuthology.orchestra.run.smithi078.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T14:56:06.146 INFO:teuthology.orchestra.run.smithi078.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T14:56:06.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T14:56:06.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo shutdown -r now 2024-05-21T14:56:36.299 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-21T14:56:36.299 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-05-21T14:56:36.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T14:57:36.302 DEBUG:teuthology.orchestra.remote:timed out 2024-05-21T14:57:45.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-05-21T14:57:45.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T14:57:46.753 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-05-21T14:57:47.275 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-05-21T14:57:47.276 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-447.el9.x86_64"... 2024-05-21T14:57:47.276 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-05-21T14:57:47.291 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-447.el9.x86_64 2024-05-21T14:57:47.291 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T14:57:47.291 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T14:57:47.291 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T14:57:48.292 DEBUG:teuthology.parallel:result is None 2024-05-21T14:57:48.293 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-21T14:57:48.300 INFO:teuthology.task.internal:Creating test directory... 2024-05-21T14:57:48.301 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T14:57:48.303 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T14:57:48.306 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T14:57:48.324 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-21T14:57:48.330 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-21T14:57:48.340 INFO:teuthology.task.internal:Creating archive directory... 2024-05-21T14:57:48.340 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T14:57:48.359 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T14:57:48.363 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T14:57:48.465 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-21T14:57:48.473 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-21T14:57:48.473 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T14:57:48.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T14:57:48.490 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T14:57:48.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T14:57:48.507 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T14:57:48.509 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T14:57:48.532 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T14:57:48.569 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T14:57:48.583 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T14:57:48.583 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T14:57:48.602 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T14:57:48.620 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T14:57:48.620 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T14:57:48.622 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-21T14:57:48.629 INFO:teuthology.task.internal:Configuring sudo... 2024-05-21T14:57:48.630 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T14:57:48.632 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T14:57:48.634 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T14:57:48.688 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-21T14:57:48.698 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-21T14:57:48.698 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T14:57:48.700 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T14:57:48.702 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T14:57:48.744 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T14:57:48.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T14:57:48.841 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T14:57:48.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T14:57:48.964 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T14:57:48.964 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T14:57:49.059 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T14:57:49.103 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T14:57:49.160 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T14:57:49.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T14:57:49.284 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T14:57:49.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T14:57:49.350 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T14:57:49.394 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T14:57:49.445 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T14:57:49.528 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T14:57:49.570 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T14:57:49.570 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T14:57:49.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-05-21T14:57:49.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-05-21T14:57:49.640 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-05-21T14:57:49.674 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T14:57:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T14:57:49.709 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T14:57:50.223 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-21T14:57:50.231 INFO:teuthology.task.internal:Starting timer... 2024-05-21T14:57:50.232 INFO:teuthology.run_tasks:Running task pcp... 2024-05-21T14:57:50.242 INFO:teuthology.run_tasks:Running task selinux... 2024-05-21T14:57:50.250 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-21T14:57:50.250 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-05-21T14:57:50.315 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-05-21T14:57:50.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2024-05-21T14:57:50.389 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2024-05-21T14:57:50.391 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-05-21T14:57:50.468 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-05-21T14:57:50.470 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-21T14:57:50.470 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-05-21T14:57:50.488 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-05-21T14:57:50.489 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2024-05-21T14:57:50.544 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2024-05-21T14:57:50.544 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-05-21T14:57:50.566 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-05-21T14:57:50.566 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi084.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2024-05-21T14:57:50.567 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T14:57:50.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T14:57:50.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T14:57:50.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T14:57:50.622 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T14:57:50.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T14:57:50.648 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-21T14:57:50.648 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-05-21T14:57:50.858 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2024-05-21T14:57:50.885 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-05-21T14:57:50.909 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-21T14:57:50.919 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-21T14:57:50.931 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-21T14:57:50.932 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi084.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-21T15:02:58.458 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi084.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2024-05-21T15:02:58.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-05-21T15:02:58.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:02:58.538 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-05-21T15:02:58.617 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-05-21T15:02:58.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-05-21T15:02:58.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:02:58.694 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-05-21T15:02:58.777 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-05-21T15:02:58.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-21T15:02:58.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T15:02:58.855 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-05-21T15:02:58.929 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-05-21T15:02:58.930 INFO:teuthology.run_tasks:Running task clock... 2024-05-21T15:02:58.941 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-21T15:02:58.941 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T15:02:58.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:02:58.944 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T15:02:58.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:02:58.946 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T15:02:58.946 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:02:58.971 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T15:02:58.971 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T15:02:58.984 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T15:02:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T15:02:58.999 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T15:02:59.005 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2024-05-21T15:02:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-05-21T15:02:59.013 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T15:02:59.018 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2024-05-21T15:02:59.020 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-05-21T15:02:59.028 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T15:02:59.034 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T15:02:59.036 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-05-21T15:02:59.042 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T15:02:59.046 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-05-21T15:02:59.048 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T15:02:59.060 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T15:02:59.073 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T15:02:59.102 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:02:59.102 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:02:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:02:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-05-21T15:02:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.107 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:02:59.107 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-05-21T15:02:59.107 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.107 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.107 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.107 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.125 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:02:59.128 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:02:59.129 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-05-21T15:02:59.129 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.129 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.129 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.129 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:02:59.130 INFO:teuthology.run_tasks:Running task install... 2024-05-21T15:02:59.138 DEBUG:teuthology.task.install:project ceph 2024-05-21T15:02:59.138 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}} 2024-05-21T15:02:59.138 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'} 2024-05-21T15:02:59.138 INFO:teuthology.task.install:Using flavor: default 2024-05-21T15:02:59.143 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-21T15:02:59.144 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-21T15:02:59.144 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-21T15:02:59.144 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:02:59.146 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-21T15:02:59.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:02:59.147 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-21T15:02:59.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:02:59.362 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-21T15:02:59.362 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-21T15:02:59.381 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-21T15:02:59.381 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-21T15:02:59.385 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-21T15:02:59.385 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-21T15:02:59.498 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T15:02:59.498 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:02:59.498 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T15:02:59.506 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T15:02:59.507 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:02:59.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T15:02:59.522 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T15:02:59.523 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T15:02:59.531 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T15:02:59.531 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T15:02:59.534 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T15:02:59.534 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:02:59.534 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T15:02:59.561 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T15:02:59.561 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T15:02:59.586 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T15:02:59.596 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T15:02:59.631 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T15:02:59.662 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T15:02:59.668 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T15:02:59.706 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T15:02:59.725 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2024-05-21T15:02:59.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2024-05-21T15:02:59.731 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-05-21T15:02:59.732 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-05-21T15:02:59.771 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-05-21T15:02:59.773 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-05-21T15:02:59.954 INFO:teuthology.orchestra.run.smithi084.stdout:45 files removed 2024-05-21T15:02:59.957 INFO:teuthology.orchestra.run.smithi172.stdout:45 files removed 2024-05-21T15:02:59.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T15:02:59.985 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T15:03:00.006 INFO:teuthology.orchestra.run.smithi078.stdout:45 files removed 2024-05-21T15:03:00.033 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T15:03:00.632 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-05-21T15:03:00.677 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 190 kB/s | 83 kB 00:00 2024-05-21T15:03:00.722 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-05-21T15:03:01.008 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-05-21T15:03:01.035 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-05-21T15:03:01.089 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-05-21T15:03:01.416 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-21T15:03:01.453 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-21T15:03:01.498 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-05-21T15:03:02.108 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-21T15:03:03.403 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.1 MB 00:01 2024-05-21T15:03:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.1 MB 00:02 2024-05-21T15:03:05.509 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-05-21T15:03:05.659 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-05-21T15:03:07.262 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-05-21T15:03:10.194 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-05-21T15:03:10.525 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-05-21T15:03:11.819 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-21T15:03:12.670 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-05-21T15:03:13.451 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-05-21T15:03:14.619 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-21T15:03:15.229 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 21 MB 00:03 2024-05-21T15:03:15.587 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 9.2 kB/s | 16 kB 00:01 2024-05-21T15:03:17.333 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-21T15:03:19.786 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-21T15:03:20.629 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 5.3 kB/s | 1.7 kB 00:00 2024-05-21T15:03:21.954 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:03:21.955 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:03:21.966 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T15:03:21.966 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T15:03:21.967 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T15:03:21.968 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T15:03:21.969 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T15:03:21.971 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T15:03:21.972 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T15:03:22.035 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-21T15:03:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-21T15:03:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T15:03:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T15:03:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T15:03:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T15:03:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T15:03:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T15:03:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-21T15:03:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T15:03:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T15:03:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T15:03:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T15:03:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T15:03:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T15:03:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T15:03:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T15:03:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-05-21T15:03:22.076 INFO:teuthology.orchestra.run.smithi078.stdout:Install 148 Packages 2024-05-21T15:03:22.077 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 11 Packages 2024-05-21T15:03:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:03:22.077 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 222 M 2024-05-21T15:03:22.077 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-05-21T15:03:22.530 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-21T15:03:22.746 INFO:teuthology.orchestra.run.smithi078.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-21T15:03:22.755 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:03:22.755 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:03:22.766 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T15:03:22.767 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T15:03:22.768 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T15:03:22.769 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T15:03:22.770 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T15:03:22.772 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T15:03:22.773 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T15:03:22.835 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-05-21T15:03:22.849 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-21T15:03:22.850 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T15:03:22.851 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-21T15:03:22.852 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T15:03:22.853 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-21T15:03:22.854 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-21T15:03:22.855 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T15:03:22.856 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T15:03:22.857 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T15:03:22.857 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T15:03:22.857 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T15:03:22.857 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T15:03:22.857 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T15:03:22.857 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T15:03:22.857 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T15:03:22.857 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T15:03:22.858 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T15:03:22.859 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T15:03:22.860 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T15:03:22.861 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T15:03:22.862 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout:Install 148 Packages 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 11 Packages 2024-05-21T15:03:22.863 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:03:22.864 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 222 M 2024-05-21T15:03:22.864 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-05-21T15:03:23.105 INFO:teuthology.orchestra.run.smithi078.stdout:(2/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 9.0 MB/s | 5.4 MB 00:00 2024-05-21T15:03:23.147 INFO:teuthology.orchestra.run.smithi078.stdout:(3/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 2.1 MB/s | 863 kB 00:00 2024-05-21T15:03:23.172 INFO:teuthology.orchestra.run.smithi078.stdout:(4/159): ceph-immutable-object-cache-19.0.0-274 2.1 MB/s | 147 kB 00:00 2024-05-21T15:03:23.306 INFO:teuthology.orchestra.run.smithi078.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 11 MB/s | 1.5 MB 00:00 2024-05-21T15:03:23.563 INFO:teuthology.orchestra.run.smithi172.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-21T15:03:23.747 INFO:teuthology.orchestra.run.smithi078.stdout:(6/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 11 MB/s | 4.9 MB 00:00 2024-05-21T15:03:23.798 INFO:teuthology.orchestra.run.smithi078.stdout:(7/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 3.5 MB/s | 2.3 MB 00:00 2024-05-21T15:03:24.131 INFO:teuthology.orchestra.run.smithi172.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.5 MB/s | 863 kB 00:00 2024-05-21T15:03:24.297 INFO:teuthology.orchestra.run.smithi172.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 883 kB/s | 147 kB 00:00 2024-05-21T15:03:24.664 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:03:24.664 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T15:03:24.675 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T15:03:24.676 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T15:03:24.677 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T15:03:24.677 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T15:03:24.679 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T15:03:24.681 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T15:03:24.682 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T15:03:24.697 INFO:teuthology.orchestra.run.smithi172.stdout:(4/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 3.7 MB/s | 5.4 MB 00:01 2024-05-21T15:03:24.744 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-05-21T15:03:24.761 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-05-21T15:03:24.761 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-05-21T15:03:24.761 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-21T15:03:24.762 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T15:03:24.763 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T15:03:24.764 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-05-21T15:03:24.765 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T15:03:24.766 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-21T15:03:24.767 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T15:03:24.768 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T15:03:24.769 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T15:03:24.770 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T15:03:24.771 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T15:03:24.772 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T15:03:24.773 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout:Install 148 Packages 2024-05-21T15:03:24.774 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 11 Packages 2024-05-21T15:03:24.775 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:03:24.775 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 222 M 2024-05-21T15:03:24.775 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-05-21T15:03:24.973 INFO:teuthology.orchestra.run.smithi172.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-05-21T15:03:24.982 INFO:teuthology.orchestra.run.smithi078.stdout:(8/159): ceph-common-19.0.0-2744.gd2b507c7.el9. 7.8 MB/s | 19 MB 00:02 2024-05-21T15:03:25.065 INFO:teuthology.orchestra.run.smithi078.stdout:(9/159): ceph-selinux-19.0.0-2744.gd2b507c7.el9 302 kB/s | 25 kB 00:00 2024-05-21T15:03:25.448 INFO:teuthology.orchestra.run.smithi172.stdout:(6/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 2.0 MB/s | 2.3 MB 00:01 2024-05-21T15:03:26.065 INFO:teuthology.orchestra.run.smithi172.stdout:(7/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 4.5 MB/s | 4.9 MB 00:01 2024-05-21T15:03:26.082 INFO:teuthology.orchestra.run.smithi078.stdout:(10/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 7.6 MB/s | 18 MB 00:02 2024-05-21T15:03:26.089 INFO:teuthology.orchestra.run.smithi084.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 9.6 kB/s | 6.6 kB 00:00 2024-05-21T15:03:26.199 INFO:teuthology.orchestra.run.smithi078.stdout:(11/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el 4.1 MB/s | 9.8 MB 00:02 2024-05-21T15:03:26.224 INFO:teuthology.orchestra.run.smithi078.stdout:(12/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 1.5 MB/s | 221 kB 00:00 2024-05-21T15:03:26.274 INFO:teuthology.orchestra.run.smithi078.stdout:(13/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 361 kB/s | 27 kB 00:00 2024-05-21T15:03:26.300 INFO:teuthology.orchestra.run.smithi078.stdout:(14/159): libcephfs-devel-19.0.0-2744.gd2b507c7 421 kB/s | 32 kB 00:00 2024-05-21T15:03:26.400 INFO:teuthology.orchestra.run.smithi078.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 1.5 MB/s | 158 kB 00:00 2024-05-21T15:03:26.500 INFO:teuthology.orchestra.run.smithi078.stdout:(16/159): librados-devel-19.0.0-2744.gd2b507c7. 1.2 MB/s | 127 kB 00:00 2024-05-21T15:03:26.559 INFO:teuthology.orchestra.run.smithi078.stdout:(17/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 2.6 MB/s | 756 kB 00:00 2024-05-21T15:03:26.789 INFO:teuthology.orchestra.run.smithi084.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.2 MB/s | 863 kB 00:00 2024-05-21T15:03:26.956 INFO:teuthology.orchestra.run.smithi084.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 883 kB/s | 147 kB 00:00 2024-05-21T15:03:27.134 INFO:teuthology.orchestra.run.smithi078.stdout:(18/159): libradosstriper1-19.0.0-2744.gd2b507c 786 kB/s | 497 kB 00:00 2024-05-21T15:03:27.218 INFO:teuthology.orchestra.run.smithi078.stdout:(19/159): python3-ceph-argparse-19.0.0-2744.gd2 539 kB/s | 45 kB 00:00 2024-05-21T15:03:27.318 INFO:teuthology.orchestra.run.smithi078.stdout:(20/159): python3-ceph-common-19.0.0-2744.gd2b5 1.3 MB/s | 135 kB 00:00 2024-05-21T15:03:27.418 INFO:teuthology.orchestra.run.smithi078.stdout:(21/159): python3-cephfs-19.0.0-2744.gd2b507c7. 1.6 MB/s | 161 kB 00:00 2024-05-21T15:03:27.527 INFO:teuthology.orchestra.run.smithi078.stdout:(22/159): python3-rados-19.0.0-2744.gd2b507c7.e 2.9 MB/s | 320 kB 00:00 2024-05-21T15:03:27.627 INFO:teuthology.orchestra.run.smithi078.stdout:(23/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 2.9 MB/s | 299 kB 00:00 2024-05-21T15:03:27.673 INFO:teuthology.orchestra.run.smithi084.stdout:(4/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 2.4 MB/s | 5.4 MB 00:02 2024-05-21T15:03:27.694 INFO:teuthology.orchestra.run.smithi078.stdout:(24/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 1.5 MB/s | 99 kB 00:00 2024-05-21T15:03:27.786 INFO:teuthology.orchestra.run.smithi078.stdout:(25/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 936 kB/s | 86 kB 00:00 2024-05-21T15:03:28.022 INFO:teuthology.orchestra.run.smithi084.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 4.3 MB/s | 1.5 MB 00:00 2024-05-21T15:03:28.195 INFO:teuthology.orchestra.run.smithi078.stdout:(26/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 3.1 MB/s | 5.0 MB 00:01 2024-05-21T15:03:28.273 INFO:teuthology.orchestra.run.smithi084.stdout:(6/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-05-21T15:03:28.321 INFO:teuthology.orchestra.run.smithi078.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 1.3 MB/s | 169 kB 00:00 2024-05-21T15:03:28.437 INFO:teuthology.orchestra.run.smithi078.stdout:(28/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 4.7 MB/s | 3.0 MB 00:00 2024-05-21T15:03:28.462 INFO:teuthology.orchestra.run.smithi078.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2744.g 224 kB/s | 32 kB 00:00 2024-05-21T15:03:28.529 INFO:teuthology.orchestra.run.smithi078.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.5 MB/s | 142 kB 00:00 2024-05-21T15:03:28.799 INFO:teuthology.orchestra.run.smithi084.stdout:(7/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 6.3 MB/s | 4.9 MB 00:00 2024-05-21T15:03:28.873 INFO:teuthology.orchestra.run.smithi172.stdout:(8/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el9 3.5 MB/s | 9.8 MB 00:02 2024-05-21T15:03:28.965 INFO:teuthology.orchestra.run.smithi172.stdout:(9/159): ceph-selinux-19.0.0-2744.gd2b507c7.el9 275 kB/s | 25 kB 00:00 2024-05-21T15:03:29.988 INFO:teuthology.orchestra.run.smithi078.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 2.3 MB/s | 3.5 MB 00:01 2024-05-21T15:03:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 1.5 MB/s | 249 kB 00:00 2024-05-21T15:03:30.215 INFO:teuthology.orchestra.run.smithi084.stdout:(8/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el9 7.0 MB/s | 9.8 MB 00:01 2024-05-21T15:03:30.239 INFO:teuthology.orchestra.run.smithi078.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 593 kB/s | 49 kB 00:00 2024-05-21T15:03:30.299 INFO:teuthology.orchestra.run.smithi084.stdout:(9/159): ceph-selinux-19.0.0-2744.gd2b507c7.el9 302 kB/s | 25 kB 00:00 2024-05-21T15:03:30.322 INFO:teuthology.orchestra.run.smithi078.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2744.gd 197 kB/s | 16 kB 00:00 2024-05-21T15:03:30.573 INFO:teuthology.orchestra.run.smithi078.stdout:(35/159): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-05-21T15:03:30.598 INFO:teuthology.orchestra.run.smithi078.stdout:(36/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 958 kB/s | 264 kB 00:00 2024-05-21T15:03:30.707 INFO:teuthology.orchestra.run.smithi078.stdout:(37/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 168 kB/s | 18 kB 00:00 2024-05-21T15:03:30.857 INFO:teuthology.orchestra.run.smithi078.stdout:(38/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 2.6 MB/s | 763 kB 00:00 2024-05-21T15:03:31.065 INFO:teuthology.orchestra.run.smithi078.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 224 kB/s | 80 kB 00:00 2024-05-21T15:03:31.258 INFO:teuthology.orchestra.run.smithi078.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-21T15:03:31.441 INFO:teuthology.orchestra.run.smithi078.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-21T15:03:31.600 INFO:teuthology.orchestra.run.smithi078.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-05-21T15:03:31.717 INFO:teuthology.orchestra.run.smithi078.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 914 kB/s | 106 kB 00:00 2024-05-21T15:03:31.792 INFO:teuthology.orchestra.run.smithi078.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-05-21T15:03:31.826 INFO:teuthology.orchestra.run.smithi078.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-21T15:03:32.251 INFO:teuthology.orchestra.run.smithi078.stdout:(46/159): ceph-test-19.0.0-2744.gd2b507c7.el9.x 6.5 MB/s | 46 MB 00:07 2024-05-21T15:03:32.277 INFO:teuthology.orchestra.run.smithi078.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 258 kB/s | 125 kB 00:00 2024-05-21T15:03:32.281 INFO:teuthology.orchestra.run.smithi172.stdout:(10/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 2.6 MB/s | 18 MB 00:06 2024-05-21T15:03:32.302 INFO:teuthology.orchestra.run.smithi078.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 451 kB/s | 215 kB 00:00 2024-05-21T15:03:32.407 INFO:teuthology.orchestra.run.smithi172.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 1.7 MB/s | 221 kB 00:00 2024-05-21T15:03:32.418 INFO:teuthology.orchestra.run.smithi078.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 918 kB/s | 107 kB 00:00 2024-05-21T15:03:32.474 INFO:teuthology.orchestra.run.smithi172.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 407 kB/s | 27 kB 00:00 2024-05-21T15:03:32.519 INFO:teuthology.orchestra.run.smithi078.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 217 kB/s | 22 kB 00:00 2024-05-21T15:03:32.540 INFO:teuthology.orchestra.run.smithi172.stdout:(13/159): libcephfs-devel-19.0.0-2744.gd2b507c7 477 kB/s | 32 kB 00:00 2024-05-21T15:03:32.561 INFO:teuthology.orchestra.run.smithi078.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-05-21T15:03:32.628 INFO:teuthology.orchestra.run.smithi078.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-21T15:03:32.662 INFO:teuthology.orchestra.run.smithi078.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-05-21T15:03:32.770 INFO:teuthology.orchestra.run.smithi078.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-21T15:03:32.791 INFO:teuthology.orchestra.run.smithi172.stdout:(14/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 3.0 MB/s | 756 kB 00:00 2024-05-21T15:03:32.866 INFO:teuthology.orchestra.run.smithi172.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 2.1 MB/s | 158 kB 00:00 2024-05-21T15:03:32.870 INFO:teuthology.orchestra.run.smithi078.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-21T15:03:32.966 INFO:teuthology.orchestra.run.smithi172.stdout:(16/159): librados-devel-19.0.0-2744.gd2b507c7. 1.2 MB/s | 127 kB 00:00 2024-05-21T15:03:32.982 INFO:teuthology.orchestra.run.smithi078.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-21T15:03:33.004 INFO:teuthology.orchestra.run.smithi078.stdout:(57/159): automake-1.16.2-8.el9.noarch.rpm 915 kB/s | 689 kB 00:00 2024-05-21T15:03:33.055 INFO:teuthology.orchestra.run.smithi078.stdout:(58/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.0 MB/s | 3.0 MB 00:00 2024-05-21T15:03:33.080 INFO:teuthology.orchestra.run.smithi078.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 449 kB/s | 45 kB 00:00 2024-05-21T15:03:33.117 INFO:teuthology.orchestra.run.smithi172.stdout:(17/159): libradosstriper1-19.0.0-2744.gd2b507c 3.2 MB/s | 497 kB 00:00 2024-05-21T15:03:33.188 INFO:teuthology.orchestra.run.smithi078.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-05-21T15:03:33.199 INFO:teuthology.orchestra.run.smithi084.stdout:(10/159): ceph-common-19.0.0-2744.gd2b507c7.el9 2.5 MB/s | 19 MB 00:07 2024-05-21T15:03:33.307 INFO:teuthology.orchestra.run.smithi084.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 2.0 MB/s | 221 kB 00:00 2024-05-21T15:03:33.322 INFO:teuthology.orchestra.run.smithi078.stdout:(61/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-05-21T15:03:33.347 INFO:teuthology.orchestra.run.smithi078.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 299 kB/s | 47 kB 00:00 2024-05-21T15:03:33.374 INFO:teuthology.orchestra.run.smithi084.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 407 kB/s | 27 kB 00:00 2024-05-21T15:03:33.439 INFO:teuthology.orchestra.run.smithi078.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-05-21T15:03:33.441 INFO:teuthology.orchestra.run.smithi084.stdout:(13/159): libcephfs-devel-19.0.0-2744.gd2b507c7 477 kB/s | 32 kB 00:00 2024-05-21T15:03:33.465 INFO:teuthology.orchestra.run.smithi078.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-21T15:03:33.548 INFO:teuthology.orchestra.run.smithi078.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-05-21T15:03:33.700 INFO:teuthology.orchestra.run.smithi084.stdout:(14/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 2.9 MB/s | 756 kB 00:00 2024-05-21T15:03:33.825 INFO:teuthology.orchestra.run.smithi084.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 1.2 MB/s | 158 kB 00:00 2024-05-21T15:03:33.909 INFO:teuthology.orchestra.run.smithi084.stdout:(16/159): librados-devel-19.0.0-2744.gd2b507c7. 1.5 MB/s | 127 kB 00:00 2024-05-21T15:03:33.999 INFO:teuthology.orchestra.run.smithi078.stdout:(66/159): libtool-2.4.6-45.el9.x86_64.rpm 617 kB/s | 582 kB 00:00 2024-05-21T15:03:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:(17/159): libradosstriper1-19.0.0-2744.gd2b507c 3.2 MB/s | 497 kB 00:00 2024-05-21T15:03:34.067 INFO:teuthology.orchestra.run.smithi172.stdout:(18/159): ceph-common-19.0.0-2744.gd2b507c7.el9 1.8 MB/s | 19 MB 00:10 2024-05-21T15:03:34.074 INFO:teuthology.orchestra.run.smithi078.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-05-21T15:03:34.142 INFO:teuthology.orchestra.run.smithi172.stdout:(19/159): python3-ceph-argparse-19.0.0-2744.gd2 606 kB/s | 45 kB 00:00 2024-05-21T15:03:34.201 INFO:teuthology.orchestra.run.smithi172.stdout:(20/159): python3-ceph-common-19.0.0-2744.gd2b5 2.3 MB/s | 135 kB 00:00 2024-05-21T15:03:34.268 INFO:teuthology.orchestra.run.smithi172.stdout:(21/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 4.4 MB/s | 5.0 MB 00:01 2024-05-21T15:03:34.310 INFO:teuthology.orchestra.run.smithi172.stdout:(22/159): python3-cephfs-19.0.0-2744.gd2b507c7. 1.5 MB/s | 161 kB 00:00 2024-05-21T15:03:34.360 INFO:teuthology.orchestra.run.smithi172.stdout:(23/159): python3-rados-19.0.0-2744.gd2b507c7.e 3.4 MB/s | 320 kB 00:00 2024-05-21T15:03:34.413 INFO:teuthology.orchestra.run.smithi172.stdout:(24/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 1.9 MB/s | 99 kB 00:00 2024-05-21T15:03:34.445 INFO:teuthology.orchestra.run.smithi172.stdout:(25/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 2.2 MB/s | 299 kB 00:00 2024-05-21T15:03:34.470 INFO:teuthology.orchestra.run.smithi172.stdout:(26/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.4 MB/s | 86 kB 00:00 2024-05-21T15:03:34.520 INFO:teuthology.orchestra.run.smithi172.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 3.3 MB/s | 169 kB 00:00 2024-05-21T15:03:34.559 INFO:teuthology.orchestra.run.smithi084.stdout:(18/159): ceph-test-19.0.0-2744.gd2b507c7.el9.x 11 MB/s | 46 MB 00:04 2024-05-21T15:03:34.570 INFO:teuthology.orchestra.run.smithi172.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2744.g 638 kB/s | 32 kB 00:00 2024-05-21T15:03:34.626 INFO:teuthology.orchestra.run.smithi084.stdout:(19/159): python3-ceph-argparse-19.0.0-2744.gd2 680 kB/s | 45 kB 00:00 2024-05-21T15:03:34.629 INFO:teuthology.orchestra.run.smithi172.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 2.4 MB/s | 142 kB 00:00 2024-05-21T15:03:34.693 INFO:teuthology.orchestra.run.smithi084.stdout:(20/159): python3-ceph-common-19.0.0-2744.gd2b5 2.0 MB/s | 135 kB 00:00 2024-05-21T15:03:34.777 INFO:teuthology.orchestra.run.smithi084.stdout:(21/159): python3-cephfs-19.0.0-2744.gd2b507c7. 1.9 MB/s | 161 kB 00:00 2024-05-21T15:03:34.861 INFO:teuthology.orchestra.run.smithi084.stdout:(22/159): python3-rados-19.0.0-2744.gd2b507c7.e 3.8 MB/s | 320 kB 00:00 2024-05-21T15:03:34.941 INFO:teuthology.orchestra.run.smithi078.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.2 MB/s | 4.7 MB 00:01 2024-05-21T15:03:34.944 INFO:teuthology.orchestra.run.smithi084.stdout:(23/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 3.5 MB/s | 299 kB 00:00 2024-05-21T15:03:35.020 INFO:teuthology.orchestra.run.smithi084.stdout:(24/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 1.3 MB/s | 99 kB 00:00 2024-05-21T15:03:35.103 INFO:teuthology.orchestra.run.smithi084.stdout:(25/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-21T15:03:35.187 INFO:teuthology.orchestra.run.smithi172.stdout:(30/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 4.1 MB/s | 3.0 MB 00:00 2024-05-21T15:03:35.207 INFO:teuthology.orchestra.run.smithi078.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 500 kB/s | 133 kB 00:00 2024-05-21T15:03:35.253 INFO:teuthology.orchestra.run.smithi084.stdout:(26/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 4.2 MB/s | 5.0 MB 00:01 2024-05-21T15:03:35.279 INFO:teuthology.orchestra.run.smithi172.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 5.4 MB/s | 3.5 MB 00:00 2024-05-21T15:03:35.312 INFO:teuthology.orchestra.run.smithi084.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 2.8 MB/s | 169 kB 00:00 2024-05-21T15:03:35.325 INFO:teuthology.orchestra.run.smithi078.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 728 kB/s | 911 kB 00:01 2024-05-21T15:03:35.380 INFO:teuthology.orchestra.run.smithi172.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 2.4 MB/s | 249 kB 00:00 2024-05-21T15:03:35.396 INFO:teuthology.orchestra.run.smithi084.stdout:(28/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 10 MB/s | 3.0 MB 00:00 2024-05-21T15:03:35.413 INFO:teuthology.orchestra.run.smithi084.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2744.g 317 kB/s | 32 kB 00:00 2024-05-21T15:03:35.430 INFO:teuthology.orchestra.run.smithi172.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 992 kB/s | 49 kB 00:00 2024-05-21T15:03:35.472 INFO:teuthology.orchestra.run.smithi084.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.8 MB/s | 142 kB 00:00 2024-05-21T15:03:35.480 INFO:teuthology.orchestra.run.smithi172.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2744.gd 329 kB/s | 16 kB 00:00 2024-05-21T15:03:35.639 INFO:teuthology.orchestra.run.smithi172.stdout:(35/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 1.6 MB/s | 264 kB 00:00 2024-05-21T15:03:35.906 INFO:teuthology.orchestra.run.smithi172.stdout:(36/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 2.8 MB/s | 763 kB 00:00 2024-05-21T15:03:35.990 INFO:teuthology.orchestra.run.smithi172.stdout:(37/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 219 kB/s | 18 kB 00:00 2024-05-21T15:03:36.130 INFO:teuthology.orchestra.run.smithi084.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-05-21T15:03:36.175 INFO:teuthology.orchestra.run.smithi078.stdout:(71/159): postgresql-13.14-1.el9.x86_64.rpm 747 kB/s | 1.6 MB 00:02 2024-05-21T15:03:36.213 INFO:teuthology.orchestra.run.smithi084.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 2.9 MB/s | 249 kB 00:00 2024-05-21T15:03:36.297 INFO:teuthology.orchestra.run.smithi084.stdout:(33/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 4.0 MB/s | 3.5 MB 00:00 2024-05-21T15:03:36.384 INFO:teuthology.orchestra.run.smithi078.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 419 kB/s | 87 kB 00:00 2024-05-21T15:03:36.470 INFO:teuthology.orchestra.run.smithi084.stdout:(34/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 2.2 MB/s | 18 MB 00:08 2024-05-21T15:03:36.490 INFO:teuthology.orchestra.run.smithi084.stdout:(35/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 179 kB/s | 49 kB 00:00 2024-05-21T15:03:36.528 INFO:teuthology.orchestra.run.smithi084.stdout:(36/159): ceph-prometheus-alerts-19.0.0-2744.gd 76 kB/s | 16 kB 00:00 2024-05-21T15:03:36.582 INFO:teuthology.orchestra.run.smithi084.stdout:(37/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 2.2 MB/s | 264 kB 00:00 2024-05-21T15:03:36.608 INFO:teuthology.orchestra.run.smithi084.stdout:(38/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 199 kB/s | 18 kB 00:00 2024-05-21T15:03:36.649 INFO:teuthology.orchestra.run.smithi084.stdout:(39/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 5.6 MB/s | 763 kB 00:00 2024-05-21T15:03:36.682 INFO:teuthology.orchestra.run.smithi172.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 116 kB/s | 80 kB 00:00 2024-05-21T15:03:36.743 INFO:teuthology.orchestra.run.smithi078.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 729 kB/s | 1.0 MB 00:01 2024-05-21T15:03:36.774 INFO:teuthology.orchestra.run.smithi172.stdout:(39/159): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-05-21T15:03:36.774 INFO:teuthology.orchestra.run.smithi084.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 418 kB/s | 80 kB 00:00 2024-05-21T15:03:36.860 INFO:teuthology.orchestra.run.smithi078.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-05-21T15:03:36.882 INFO:teuthology.orchestra.run.smithi084.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 837 kB/s | 195 kB 00:00 2024-05-21T15:03:36.908 INFO:teuthology.orchestra.run.smithi084.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-05-21T15:03:36.941 INFO:teuthology.orchestra.run.smithi084.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-05-21T15:03:36.967 INFO:teuthology.orchestra.run.smithi084.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-21T15:03:37.000 INFO:teuthology.orchestra.run.smithi084.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-21T15:03:37.025 INFO:teuthology.orchestra.run.smithi084.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-21T15:03:37.084 INFO:teuthology.orchestra.run.smithi084.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-21T15:03:37.210 INFO:teuthology.orchestra.run.smithi084.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-05-21T15:03:37.210 INFO:teuthology.orchestra.run.smithi078.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 697 kB/s | 244 kB 00:00 2024-05-21T15:03:37.268 INFO:teuthology.orchestra.run.smithi084.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-21T15:03:37.302 INFO:teuthology.orchestra.run.smithi084.stdout:(50/159): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-05-21T15:03:37.327 INFO:teuthology.orchestra.run.smithi084.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-05-21T15:03:37.361 INFO:teuthology.orchestra.run.smithi084.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-05-21T15:03:37.386 INFO:teuthology.orchestra.run.smithi084.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-05-21T15:03:37.390 INFO:teuthology.orchestra.run.smithi172.stdout:(40/159): ceph-test-19.0.0-2744.gd2b507c7.el9.x 5.5 MB/s | 46 MB 00:08 2024-05-21T15:03:37.436 INFO:teuthology.orchestra.run.smithi084.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-21T15:03:37.487 INFO:teuthology.orchestra.run.smithi084.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-21T15:03:37.552 INFO:teuthology.orchestra.run.smithi078.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 726 kB/s | 248 kB 00:00 2024-05-21T15:03:37.553 INFO:teuthology.orchestra.run.smithi084.stdout:(56/159): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-05-21T15:03:37.579 INFO:teuthology.orchestra.run.smithi084.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-05-21T15:03:37.604 INFO:teuthology.orchestra.run.smithi084.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-05-21T15:03:37.661 INFO:teuthology.orchestra.run.smithi078.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-05-21T15:03:37.754 INFO:teuthology.orchestra.run.smithi084.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-21T15:03:37.769 INFO:teuthology.orchestra.run.smithi078.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-21T15:03:37.805 INFO:teuthology.orchestra.run.smithi084.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-05-21T15:03:37.838 INFO:teuthology.orchestra.run.smithi084.stdout:(61/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-05-21T15:03:37.855 INFO:teuthology.orchestra.run.smithi084.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-21T15:03:37.939 INFO:teuthology.orchestra.run.smithi084.stdout:(63/159): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-21T15:03:37.964 INFO:teuthology.orchestra.run.smithi084.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-05-21T15:03:37.989 INFO:teuthology.orchestra.run.smithi084.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-05-21T15:03:38.007 INFO:teuthology.orchestra.run.smithi172.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 606 kB/s | 803 kB 00:01 2024-05-21T15:03:38.033 INFO:teuthology.orchestra.run.smithi172.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 155 kB/s | 195 kB 00:01 2024-05-21T15:03:38.036 INFO:teuthology.orchestra.run.smithi078.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 645 kB/s | 172 kB 00:00 2024-05-21T15:03:38.048 INFO:teuthology.orchestra.run.smithi084.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.4 MB/s | 3.0 MB 00:00 2024-05-21T15:03:38.136 INFO:teuthology.orchestra.run.smithi078.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-05-21T15:03:38.233 INFO:teuthology.orchestra.run.smithi172.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 311 kB/s | 253 kB 00:00 2024-05-21T15:03:38.407 INFO:teuthology.orchestra.run.smithi084.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-05-21T15:03:38.624 INFO:teuthology.orchestra.run.smithi084.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-05-21T15:03:38.682 INFO:teuthology.orchestra.run.smithi084.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-21T15:03:38.783 INFO:teuthology.orchestra.run.smithi172.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 142 kB/s | 106 kB 00:00 2024-05-21T15:03:38.874 INFO:teuthology.orchestra.run.smithi084.stdout:(70/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-05-21T15:03:39.008 INFO:teuthology.orchestra.run.smithi172.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-21T15:03:39.058 INFO:teuthology.orchestra.run.smithi172.stdout:(46/159): python3-pycparser-2.20-6.el9.noarch.r 175 kB/s | 135 kB 00:00 2024-05-21T15:03:39.091 INFO:teuthology.orchestra.run.smithi084.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-05-21T15:03:39.199 INFO:teuthology.orchestra.run.smithi084.stdout:(72/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.8 MB/s | 4.7 MB 00:01 2024-05-21T15:03:39.233 INFO:teuthology.orchestra.run.smithi084.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 617 kB/s | 87 kB 00:00 2024-05-21T15:03:39.292 INFO:teuthology.orchestra.run.smithi084.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-05-21T15:03:39.375 INFO:teuthology.orchestra.run.smithi084.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-21T15:03:39.459 INFO:teuthology.orchestra.run.smithi084.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-21T15:03:39.518 INFO:teuthology.orchestra.run.smithi084.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-05-21T15:03:39.534 INFO:teuthology.orchestra.run.smithi172.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 166 kB/s | 125 kB 00:00 2024-05-21T15:03:39.576 INFO:teuthology.orchestra.run.smithi084.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-05-21T15:03:39.635 INFO:teuthology.orchestra.run.smithi084.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-21T15:03:39.694 INFO:teuthology.orchestra.run.smithi084.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 598 kB/s | 35 kB 00:00 2024-05-21T15:03:39.826 INFO:teuthology.orchestra.run.smithi172.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 899 kB/s | 689 kB 00:00 2024-05-21T15:03:39.851 INFO:teuthology.orchestra.run.smithi172.stdout:(49/159): python3-urllib3-1.26.5-5.el9.noarch.r 263 kB/s | 215 kB 00:00 2024-05-21T15:03:39.852 INFO:teuthology.orchestra.run.smithi084.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-05-21T15:03:39.935 INFO:teuthology.orchestra.run.smithi172.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 983 kB/s | 107 kB 00:00 2024-05-21T15:03:39.944 INFO:teuthology.orchestra.run.smithi084.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-21T15:03:40.036 INFO:teuthology.orchestra.run.smithi084.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-21T15:03:40.043 INFO:teuthology.orchestra.run.smithi172.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-21T15:03:40.095 INFO:teuthology.orchestra.run.smithi084.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-21T15:03:40.118 INFO:teuthology.orchestra.run.smithi172.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 81 kB/s | 22 kB 00:00 2024-05-21T15:03:40.144 INFO:teuthology.orchestra.run.smithi172.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 325 kB/s | 33 kB 00:00 2024-05-21T15:03:40.154 INFO:teuthology.orchestra.run.smithi084.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-05-21T15:03:40.245 INFO:teuthology.orchestra.run.smithi084.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-05-21T15:03:40.252 INFO:teuthology.orchestra.run.smithi172.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-21T15:03:40.304 INFO:teuthology.orchestra.run.smithi084.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-21T15:03:40.353 INFO:teuthology.orchestra.run.smithi172.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-21T15:03:40.379 INFO:teuthology.orchestra.run.smithi084.stdout:(88/159): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-05-21T15:03:40.430 INFO:teuthology.orchestra.run.smithi084.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-21T15:03:40.445 INFO:teuthology.orchestra.run.smithi172.stdout:(56/159): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-05-21T15:03:40.455 INFO:teuthology.orchestra.run.smithi084.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 679 kB/s | 51 kB 00:00 2024-05-21T15:03:40.470 INFO:teuthology.orchestra.run.smithi172.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-21T15:03:40.480 INFO:teuthology.orchestra.run.smithi084.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-21T15:03:40.547 INFO:teuthology.orchestra.run.smithi084.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.0 MB/s | 5.1 MB 00:00 2024-05-21T15:03:40.562 INFO:teuthology.orchestra.run.smithi172.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-05-21T15:03:40.564 INFO:teuthology.orchestra.run.smithi084.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 498 kB/s | 42 kB 00:00 2024-05-21T15:03:40.589 INFO:teuthology.orchestra.run.smithi084.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-21T15:03:40.623 INFO:teuthology.orchestra.run.smithi084.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-21T15:03:40.640 INFO:teuthology.orchestra.run.smithi084.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-05-21T15:03:40.732 INFO:teuthology.orchestra.run.smithi084.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-05-21T15:03:40.762 INFO:teuthology.orchestra.run.smithi172.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-05-21T15:03:40.832 INFO:teuthology.orchestra.run.smithi084.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-05-21T15:03:40.858 INFO:teuthology.orchestra.run.smithi084.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-05-21T15:03:40.863 INFO:teuthology.orchestra.run.smithi172.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-05-21T15:03:40.888 INFO:teuthology.orchestra.run.smithi172.stdout:(61/159): libtool-2.4.6-45.el9.x86_64.rpm 1.7 MB/s | 582 kB 00:00 2024-05-21T15:03:40.891 INFO:teuthology.orchestra.run.smithi084.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-05-21T15:03:40.916 INFO:teuthology.orchestra.run.smithi084.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-21T15:03:40.942 INFO:teuthology.orchestra.run.smithi084.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-05-21T15:03:40.963 INFO:teuthology.orchestra.run.smithi172.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-21T15:03:41.025 INFO:teuthology.orchestra.run.smithi172.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-21T15:03:41.072 INFO:teuthology.orchestra.run.smithi172.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-21T15:03:41.131 INFO:teuthology.orchestra.run.smithi172.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 333 kB/s | 35 kB 00:00 2024-05-21T15:03:41.181 INFO:teuthology.orchestra.run.smithi172.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-05-21T15:03:41.331 INFO:teuthology.orchestra.run.smithi172.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-21T15:03:41.473 INFO:teuthology.orchestra.run.smithi172.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-21T15:03:41.525 INFO:teuthology.orchestra.run.smithi084.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 37 kB/s | 22 kB 00:00 2024-05-21T15:03:41.590 INFO:teuthology.orchestra.run.smithi172.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-21T15:03:41.699 INFO:teuthology.orchestra.run.smithi172.stdout:(70/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 4.4 MB 00:00 2024-05-21T15:03:41.833 INFO:teuthology.orchestra.run.smithi172.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-05-21T15:03:41.950 INFO:teuthology.orchestra.run.smithi084.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-21T15:03:41.958 INFO:teuthology.orchestra.run.smithi172.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 700 kB/s | 87 kB 00:00 2024-05-21T15:03:42.043 INFO:teuthology.orchestra.run.smithi084.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 784 kB/s | 863 kB 00:01 2024-05-21T15:03:42.068 INFO:teuthology.orchestra.run.smithi084.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-05-21T15:03:42.227 INFO:teuthology.orchestra.run.smithi084.stdout:(107/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.2 MB/s | 16 MB 00:01 2024-05-21T15:03:42.344 INFO:teuthology.orchestra.run.smithi084.stdout:(108/159): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-05-21T15:03:42.402 INFO:teuthology.orchestra.run.smithi084.stdout:(109/159): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-21T15:03:42.427 INFO:teuthology.orchestra.run.smithi084.stdout:(110/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-05-21T15:03:42.461 INFO:teuthology.orchestra.run.smithi084.stdout:(111/159): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-05-21T15:03:42.486 INFO:teuthology.orchestra.run.smithi084.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 803 kB/s | 67 kB 00:00 2024-05-21T15:03:42.570 INFO:teuthology.orchestra.run.smithi084.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-05-21T15:03:42.595 INFO:teuthology.orchestra.run.smithi084.stdout:(114/159): luarocks-3.9.2-5.el9.noarch.rpm 904 kB/s | 151 kB 00:00 2024-05-21T15:03:42.637 INFO:teuthology.orchestra.run.smithi084.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-05-21T15:03:42.663 INFO:teuthology.orchestra.run.smithi084.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-05-21T15:03:42.704 INFO:teuthology.orchestra.run.smithi084.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-21T15:03:42.755 INFO:teuthology.orchestra.run.smithi084.stdout:(118/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 839 kB 00:00 2024-05-21T15:03:42.771 INFO:teuthology.orchestra.run.smithi084.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-21T15:03:42.797 INFO:teuthology.orchestra.run.smithi084.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-05-21T15:03:42.839 INFO:teuthology.orchestra.run.smithi084.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-21T15:03:42.898 INFO:teuthology.orchestra.run.smithi084.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-05-21T15:03:42.923 INFO:teuthology.orchestra.run.smithi084.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-21T15:03:42.973 INFO:teuthology.orchestra.run.smithi084.stdout:(124/159): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-05-21T15:03:42.998 INFO:teuthology.orchestra.run.smithi084.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-05-21T15:03:43.024 INFO:teuthology.orchestra.run.smithi084.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-21T15:03:43.049 INFO:teuthology.orchestra.run.smithi084.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-05-21T15:03:43.074 INFO:teuthology.orchestra.run.smithi084.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-21T15:03:43.099 INFO:teuthology.orchestra.run.smithi084.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-05-21T15:03:43.133 INFO:teuthology.orchestra.run.smithi084.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-05-21T15:03:43.145 INFO:teuthology.orchestra.run.smithi078.stdout:(81/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.0 MB/s | 5.1 MB 00:05 2024-05-21T15:03:43.158 INFO:teuthology.orchestra.run.smithi084.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 551 kB/s | 46 kB 00:00 2024-05-21T15:03:43.183 INFO:teuthology.orchestra.run.smithi084.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-05-21T15:03:43.209 INFO:teuthology.orchestra.run.smithi084.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-05-21T15:03:43.220 INFO:teuthology.orchestra.run.smithi078.stdout:(82/159): postgresql-server-13.14-1.el9.x86_64. 739 kB/s | 5.8 MB 00:08 2024-05-21T15:03:43.234 INFO:teuthology.orchestra.run.smithi084.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-21T15:03:43.251 INFO:teuthology.orchestra.run.smithi084.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 243 kB/s | 16 kB 00:00 2024-05-21T15:03:43.276 INFO:teuthology.orchestra.run.smithi084.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-21T15:03:43.293 INFO:teuthology.orchestra.run.smithi084.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-05-21T15:03:43.319 INFO:teuthology.orchestra.run.smithi084.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 457 kB/s | 31 kB 00:00 2024-05-21T15:03:43.360 INFO:teuthology.orchestra.run.smithi084.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-21T15:03:43.386 INFO:teuthology.orchestra.run.smithi084.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 640 kB/s | 59 kB 00:00 2024-05-21T15:03:43.411 INFO:teuthology.orchestra.run.smithi084.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-05-21T15:03:43.436 INFO:teuthology.orchestra.run.smithi084.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-21T15:03:43.470 INFO:teuthology.orchestra.run.smithi084.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-21T15:03:43.495 INFO:teuthology.orchestra.run.smithi084.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-21T15:03:43.503 INFO:teuthology.orchestra.run.smithi078.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 699 kB/s | 250 kB 00:00 2024-05-21T15:03:43.521 INFO:teuthology.orchestra.run.smithi084.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-21T15:03:43.546 INFO:teuthology.orchestra.run.smithi084.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-05-21T15:03:43.579 INFO:teuthology.orchestra.run.smithi078.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 619 kB/s | 222 kB 00:00 2024-05-21T15:03:43.596 INFO:teuthology.orchestra.run.smithi084.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-21T15:03:43.630 INFO:teuthology.orchestra.run.smithi084.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-21T15:03:43.687 INFO:teuthology.orchestra.run.smithi078.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-05-21T15:03:43.713 INFO:teuthology.orchestra.run.smithi078.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 369 kB/s | 77 kB 00:00 2024-05-21T15:03:43.988 INFO:teuthology.orchestra.run.smithi084.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 6.4 MB/s | 2.3 MB 00:00 2024-05-21T15:03:44.005 INFO:teuthology.orchestra.run.smithi078.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-05-21T15:03:44.038 INFO:teuthology.orchestra.run.smithi084.stdout:(150/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-05-21T15:03:44.089 INFO:teuthology.orchestra.run.smithi084.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 832 kB/s | 41 kB 00:00 2024-05-21T15:03:44.146 INFO:teuthology.orchestra.run.smithi078.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 587 kB/s | 269 kB 00:00 2024-05-21T15:03:44.172 INFO:teuthology.orchestra.run.smithi084.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-05-21T15:03:44.206 INFO:teuthology.orchestra.run.smithi084.stdout:(153/159): NetworkManager-libnm-1.47.90-1.el9.x 8.3 MB/s | 1.8 MB 00:00 2024-05-21T15:03:44.223 INFO:teuthology.orchestra.run.smithi084.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 732 kB/s | 37 kB 00:00 2024-05-21T15:03:44.255 INFO:teuthology.orchestra.run.smithi078.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-05-21T15:03:44.257 INFO:teuthology.orchestra.run.smithi084.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.0 MB/s | 207 kB 00:00 2024-05-21T15:03:44.307 INFO:teuthology.orchestra.run.smithi084.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.0 MB/s | 260 kB 00:00 2024-05-21T15:03:44.380 INFO:teuthology.orchestra.run.smithi078.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 429 kB/s | 54 kB 00:00 2024-05-21T15:03:44.415 INFO:teuthology.orchestra.run.smithi084.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 1.3 MB 00:00 2024-05-21T15:03:44.489 INFO:teuthology.orchestra.run.smithi078.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 576 kB/s | 279 kB 00:00 2024-05-21T15:03:44.534 INFO:teuthology.orchestra.run.smithi172.stdout:(73/159): openblas-openmp-0.3.21-2.el9.x86_64.r 1.4 MB/s | 4.7 MB 00:03 2024-05-21T15:03:44.540 INFO:teuthology.orchestra.run.smithi084.stdout:(158/159): librados2-19.0.0-2744.gd2b507c7.el9. 3.3 MB/s | 3.3 MB 00:00 2024-05-21T15:03:44.598 INFO:teuthology.orchestra.run.smithi078.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-05-21T15:03:44.658 INFO:teuthology.orchestra.run.smithi172.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 491 kB/s | 61 kB 00:00 2024-05-21T15:03:44.673 INFO:teuthology.orchestra.run.smithi078.stdout:(93/159): python3-babel-2.9.1-2.el9.noarch.rpm 737 kB/s | 6.0 MB 00:08 2024-05-21T15:03:44.698 INFO:teuthology.orchestra.run.smithi078.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 440 kB/s | 44 kB 00:00 2024-05-21T15:03:44.708 INFO:teuthology.orchestra.run.smithi084.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 2.4 MB 00:00 2024-05-21T15:03:44.711 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:03:44.711 INFO:teuthology.orchestra.run.smithi084.stdout:Total 11 MB/s | 222 MB 00:19 2024-05-21T15:03:44.767 INFO:teuthology.orchestra.run.smithi172.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-21T15:03:44.807 INFO:teuthology.orchestra.run.smithi078.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-05-21T15:03:44.884 INFO:teuthology.orchestra.run.smithi172.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-21T15:03:44.925 INFO:teuthology.orchestra.run.smithi078.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 404 kB/s | 47 kB 00:00 2024-05-21T15:03:45.009 INFO:teuthology.orchestra.run.smithi172.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-05-21T15:03:45.066 INFO:teuthology.orchestra.run.smithi078.stdout:(97/159): qatlib-24.02.0-1.el9.x86_64.rpm 562 kB/s | 221 kB 00:00 2024-05-21T15:03:45.151 INFO:teuthology.orchestra.run.smithi172.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 135 kB/s | 19 kB 00:00 2024-05-21T15:03:45.200 INFO:teuthology.orchestra.run.smithi078.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 433 kB/s | 58 kB 00:00 2024-05-21T15:03:45.268 INFO:teuthology.orchestra.run.smithi172.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-05-21T15:03:45.368 INFO:teuthology.orchestra.run.smithi172.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-05-21T15:03:45.409 INFO:teuthology.orchestra.run.smithi078.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 450 kB/s | 94 kB 00:00 2024-05-21T15:03:45.459 INFO:teuthology.orchestra.run.smithi078.stdout:(100/159): socat-1.7.4.1-5.el9.x86_64.rpm 571 kB/s | 305 kB 00:00 2024-05-21T15:03:45.593 INFO:teuthology.orchestra.run.smithi078.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 335 kB/s | 45 kB 00:00 2024-05-21T15:03:45.644 INFO:teuthology.orchestra.run.smithi078.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 271 kB/s | 64 kB 00:00 2024-05-21T15:03:45.854 INFO:teuthology.orchestra.run.smithi078.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-05-21T15:03:45.891 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-05-21T15:03:45.952 INFO:teuthology.orchestra.run.smithi172.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 1.3 MB/s | 5.8 MB 00:04 2024-05-21T15:03:46.036 INFO:teuthology.orchestra.run.smithi172.stdout:(82/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.6 MB/s | 5.1 MB 00:00 2024-05-21T15:03:46.048 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-05-21T15:03:46.049 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-05-21T15:03:46.288 INFO:teuthology.orchestra.run.smithi078.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-05-21T15:03:46.311 INFO:teuthology.orchestra.run.smithi172.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 700 kB/s | 250 kB 00:00 2024-05-21T15:03:46.378 INFO:teuthology.orchestra.run.smithi172.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-05-21T15:03:46.389 INFO:teuthology.orchestra.run.smithi078.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-21T15:03:46.478 INFO:teuthology.orchestra.run.smithi172.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-05-21T15:03:46.481 INFO:teuthology.orchestra.run.smithi078.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-05-21T15:03:46.503 INFO:teuthology.orchestra.run.smithi172.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 401 kB/s | 77 kB 00:00 2024-05-21T15:03:46.764 INFO:teuthology.orchestra.run.smithi078.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 649 kB/s | 309 kB 00:00 2024-05-21T15:03:46.770 INFO:teuthology.orchestra.run.smithi172.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-05-21T15:03:46.831 INFO:teuthology.orchestra.run.smithi078.stdout:(108/159): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-21T15:03:46.857 INFO:teuthology.orchestra.run.smithi078.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-05-21T15:03:46.912 INFO:teuthology.orchestra.run.smithi172.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 621 kB/s | 269 kB 00:00 2024-05-21T15:03:46.924 INFO:teuthology.orchestra.run.smithi078.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-05-21T15:03:46.982 INFO:teuthology.orchestra.run.smithi078.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-05-21T15:03:47.013 INFO:teuthology.orchestra.run.smithi172.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-05-21T15:03:47.025 INFO:teuthology.orchestra.run.smithi078.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-21T15:03:47.131 INFO:teuthology.orchestra.run.smithi172.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 460 kB/s | 54 kB 00:00 2024-05-21T15:03:47.150 INFO:teuthology.orchestra.run.smithi078.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-21T15:03:47.175 INFO:teuthology.orchestra.run.smithi078.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-05-21T15:03:47.197 INFO:teuthology.orchestra.run.smithi172.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 654 kB/s | 279 kB 00:00 2024-05-21T15:03:47.242 INFO:teuthology.orchestra.run.smithi078.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-05-21T15:03:47.267 INFO:teuthology.orchestra.run.smithi078.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-05-21T15:03:47.305 INFO:teuthology.orchestra.run.smithi172.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-05-21T15:03:47.338 INFO:teuthology.orchestra.run.smithi078.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-05-21T15:03:47.368 INFO:teuthology.orchestra.run.smithi078.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-21T15:03:47.414 INFO:teuthology.orchestra.run.smithi172.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-05-21T15:03:47.435 INFO:teuthology.orchestra.run.smithi078.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-21T15:03:47.469 INFO:teuthology.orchestra.run.smithi078.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-05-21T15:03:47.536 INFO:teuthology.orchestra.run.smithi078.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-21T15:03:47.636 INFO:teuthology.orchestra.run.smithi078.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-21T15:03:47.678 INFO:teuthology.orchestra.run.smithi078.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-05-21T15:03:47.728 INFO:teuthology.orchestra.run.smithi078.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-21T15:03:47.770 INFO:teuthology.orchestra.run.smithi078.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-21T15:03:47.772 INFO:teuthology.orchestra.run.smithi172.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 616 kB/s | 221 kB 00:00 2024-05-21T15:03:47.820 INFO:teuthology.orchestra.run.smithi078.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-05-21T15:03:47.862 INFO:teuthology.orchestra.run.smithi078.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-21T15:03:47.889 INFO:teuthology.orchestra.run.smithi172.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-05-21T15:03:47.913 INFO:teuthology.orchestra.run.smithi078.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-05-21T15:03:47.979 INFO:teuthology.orchestra.run.smithi078.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-05-21T15:03:47.990 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-05-21T15:03:47.991 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-05-21T15:03:47.998 INFO:teuthology.orchestra.run.smithi172.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-05-21T15:03:48.013 INFO:teuthology.orchestra.run.smithi078.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-05-21T15:03:48.080 INFO:teuthology.orchestra.run.smithi078.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-05-21T15:03:48.097 INFO:teuthology.orchestra.run.smithi078.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-05-21T15:03:48.148 INFO:teuthology.orchestra.run.smithi172.stdout:(97/159): python3-babel-2.9.1-2.el9.noarch.rpm 986 kB/s | 6.0 MB 00:06 2024-05-21T15:03:48.181 INFO:teuthology.orchestra.run.smithi078.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-21T15:03:48.198 INFO:teuthology.orchestra.run.smithi078.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-05-21T15:03:48.265 INFO:teuthology.orchestra.run.smithi172.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 496 kB/s | 58 kB 00:00 2024-05-21T15:03:48.273 INFO:teuthology.orchestra.run.smithi078.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 979 kB/s | 90 kB 00:00 2024-05-21T15:03:48.344 INFO:teuthology.orchestra.run.smithi078.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 193 kB/s | 28 kB 00:00 2024-05-21T15:03:48.382 INFO:teuthology.orchestra.run.smithi078.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-05-21T15:03:48.441 INFO:teuthology.orchestra.run.smithi078.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-05-21T15:03:48.449 INFO:teuthology.orchestra.run.smithi172.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 676 kB/s | 305 kB 00:00 2024-05-21T15:03:48.466 INFO:teuthology.orchestra.run.smithi078.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 706 kB/s | 59 kB 00:00 2024-05-21T15:03:48.474 INFO:teuthology.orchestra.run.smithi172.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 449 kB/s | 94 kB 00:00 2024-05-21T15:03:48.533 INFO:teuthology.orchestra.run.smithi078.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-05-21T15:03:48.550 INFO:teuthology.orchestra.run.smithi078.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 607 kB/s | 51 kB 00:00 2024-05-21T15:03:48.584 INFO:teuthology.orchestra.run.smithi172.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-05-21T15:03:48.645 INFO:teuthology.orchestra.run.smithi078.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-21T15:03:48.645 INFO:teuthology.orchestra.run.smithi172.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 346 kB/s | 64 kB 00:00 2024-05-21T15:03:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 894 kB/s | 90 kB 00:00 2024-05-21T15:03:48.734 INFO:teuthology.orchestra.run.smithi078.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-21T15:03:48.760 INFO:teuthology.orchestra.run.smithi078.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-21T15:03:48.844 INFO:teuthology.orchestra.run.smithi078.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-21T15:03:48.885 INFO:teuthology.orchestra.run.smithi078.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-21T15:03:49.158 INFO:teuthology.orchestra.run.smithi172.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-05-21T15:03:49.319 INFO:teuthology.orchestra.run.smithi078.stdout:(148/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 7.1 MB/s | 3.1 MB 00:00 2024-05-21T15:03:49.544 INFO:teuthology.orchestra.run.smithi078.stdout:(149/159): librados2-19.0.0-2744.gd2b507c7.el9. 4.7 MB/s | 3.3 MB 00:00 2024-05-21T15:03:49.583 INFO:teuthology.orchestra.run.smithi172.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 909 kB/s | 863 kB 00:00 2024-05-21T15:03:49.617 INFO:teuthology.orchestra.run.smithi172.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-05-21T15:03:49.701 INFO:teuthology.orchestra.run.smithi172.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-05-21T15:03:50.010 INFO:teuthology.orchestra.run.smithi172.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-21T15:03:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 685 kB/s | 309 kB 00:00 2024-05-21T15:03:50.094 INFO:teuthology.orchestra.run.smithi172.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-05-21T15:03:50.119 INFO:teuthology.orchestra.run.smithi172.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-05-21T15:03:50.169 INFO:teuthology.orchestra.run.smithi172.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-05-21T15:03:50.194 INFO:teuthology.orchestra.run.smithi078.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 2.6 MB/s | 2.3 MB 00:00 2024-05-21T15:03:50.219 INFO:teuthology.orchestra.run.smithi172.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-21T15:03:50.270 INFO:teuthology.orchestra.run.smithi172.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-05-21T15:03:50.311 INFO:teuthology.orchestra.run.smithi078.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 356 kB/s | 41 kB 00:00 2024-05-21T15:03:50.345 INFO:teuthology.orchestra.run.smithi172.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-05-21T15:03:50.370 INFO:teuthology.orchestra.run.smithi172.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-05-21T15:03:50.408 INFO:teuthology.orchestra.run.smithi078.stdout:(152/159): NetworkManager-libnm-1.47.90-1.el9.x 2.1 MB/s | 1.8 MB 00:00 2024-05-21T15:03:50.429 INFO:teuthology.orchestra.run.smithi078.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 2.1 MB/s | 248 kB 00:00 2024-05-21T15:03:50.429 INFO:teuthology.orchestra.run.smithi172.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-05-21T15:03:50.454 INFO:teuthology.orchestra.run.smithi172.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-05-21T15:03:50.512 INFO:teuthology.orchestra.run.smithi078.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 338 kB/s | 37 kB 00:00 2024-05-21T15:03:50.521 INFO:teuthology.orchestra.run.smithi172.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-21T15:03:50.538 INFO:teuthology.orchestra.run.smithi078.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-05-21T15:03:50.555 INFO:teuthology.orchestra.run.smithi172.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-21T15:03:50.614 INFO:teuthology.orchestra.run.smithi172.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-21T15:03:50.621 INFO:teuthology.orchestra.run.smithi078.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-05-21T15:03:50.647 INFO:teuthology.orchestra.run.smithi172.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-21T15:03:50.671 INFO:teuthology.orchestra.run.smithi078.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-21T15:03:50.739 INFO:teuthology.orchestra.run.smithi172.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-05-21T15:03:50.764 INFO:teuthology.orchestra.run.smithi172.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-21T15:03:50.814 INFO:teuthology.orchestra.run.smithi172.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-21T15:03:50.840 INFO:teuthology.orchestra.run.smithi172.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-21T15:03:51.107 INFO:teuthology.orchestra.run.smithi172.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 79 kB/s | 23 kB 00:00 2024-05-21T15:03:51.132 INFO:teuthology.orchestra.run.smithi172.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 66 kB/s | 19 kB 00:00 2024-05-21T15:03:51.174 INFO:teuthology.orchestra.run.smithi172.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-05-21T15:03:51.249 INFO:teuthology.orchestra.run.smithi172.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-05-21T15:03:51.283 INFO:teuthology.orchestra.run.smithi172.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-05-21T15:03:51.333 INFO:teuthology.orchestra.run.smithi172.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-05-21T15:03:51.358 INFO:teuthology.orchestra.run.smithi172.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-05-21T15:03:51.426 INFO:teuthology.orchestra.run.smithi172.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-21T15:03:51.452 INFO:teuthology.orchestra.run.smithi172.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-05-21T15:03:51.501 INFO:teuthology.orchestra.run.smithi172.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-21T15:03:51.526 INFO:teuthology.orchestra.run.smithi172.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-05-21T15:03:51.586 INFO:teuthology.orchestra.run.smithi172.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-05-21T15:03:51.610 INFO:teuthology.orchestra.run.smithi172.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-21T15:03:51.677 INFO:teuthology.orchestra.run.smithi172.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-05-21T15:03:51.703 INFO:teuthology.orchestra.run.smithi172.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-05-21T15:03:51.761 INFO:teuthology.orchestra.run.smithi172.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 611 kB/s | 51 kB 00:00 2024-05-21T15:03:51.787 INFO:teuthology.orchestra.run.smithi172.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-21T15:03:51.845 INFO:teuthology.orchestra.run.smithi172.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-21T15:03:51.870 INFO:teuthology.orchestra.run.smithi172.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-21T15:03:51.912 INFO:teuthology.orchestra.run.smithi172.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-05-21T15:03:51.947 INFO:teuthology.orchestra.run.smithi078.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 2.4 MB 00:01 2024-05-21T15:03:51.963 INFO:teuthology.orchestra.run.smithi172.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-21T15:03:52.055 INFO:teuthology.orchestra.run.smithi172.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-21T15:03:52.405 INFO:teuthology.orchestra.run.smithi172.stdout:(148/159): librados2-19.0.0-2744.gd2b507c7.el9. 7.5 MB/s | 3.3 MB 00:00 2024-05-21T15:03:52.456 INFO:teuthology.orchestra.run.smithi172.stdout:(149/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-05-21T15:03:53.556 INFO:teuthology.orchestra.run.smithi172.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 2.0 MB/s | 2.3 MB 00:01 2024-05-21T15:03:53.610 INFO:teuthology.orchestra.run.smithi172.stdout:(151/159): NetworkManager-libnm-1.47.90-1.el9.x 1.6 MB/s | 1.8 MB 00:01 2024-05-21T15:03:54.181 INFO:teuthology.orchestra.run.smithi172.stdout:(152/159): NetworkManager-team-1.47.90-1.el9.x8 66 kB/s | 41 kB 00:00 2024-05-21T15:03:54.305 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-05-21T15:03:54.386 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T15:03:54.425 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T15:03:54.465 INFO:teuthology.orchestra.run.smithi172.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 286 kB/s | 248 kB 00:00 2024-05-21T15:03:54.486 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T15:03:54.520 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T15:03:54.546 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T15:03:54.589 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T15:03:54.790 INFO:teuthology.orchestra.run.smithi172.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 60 kB/s | 37 kB 00:00 2024-05-21T15:03:54.923 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T15:03:55.030 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:03:55.155 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:03:55.181 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:03:55.265 INFO:teuthology.orchestra.run.smithi172.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 259 kB/s | 207 kB 00:00 2024-05-21T15:03:55.272 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:03:55.302 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:03:55.335 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T15:03:55.539 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T15:03:55.592 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T15:03:55.612 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:03:55.615 INFO:teuthology.orchestra.run.smithi172.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 315 kB/s | 260 kB 00:00 2024-05-21T15:03:55.717 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:03:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:03:55.808 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:03:55.830 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:03:55.853 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:03:56.048 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:03:56.177 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:03:56.363 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:03:56.423 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:03:56.494 INFO:teuthology.orchestra.run.smithi172.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 1.3 MB 00:01 2024-05-21T15:03:56.496 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:03:56.591 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:03:56.787 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:03:56.891 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T15:03:56.899 INFO:teuthology.orchestra.run.smithi172.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.9 MB/s | 2.4 MB 00:01 2024-05-21T15:03:56.984 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T15:03:57.022 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-21T15:03:57.057 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-21T15:03:57.121 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-21T15:03:57.156 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-21T15:03:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-21T15:03:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-21T15:03:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-21T15:03:57.337 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-21T15:03:57.391 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-21T15:03:57.421 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-21T15:03:57.465 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-21T15:03:57.542 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-21T15:03:57.577 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-21T15:03:57.619 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-21T15:03:57.649 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-21T15:03:57.719 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-21T15:03:57.771 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-21T15:03:57.854 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-21T15:03:57.988 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-21T15:03:58.041 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-21T15:03:58.568 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-21T15:03:58.653 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-21T15:03:58.682 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-21T15:03:58.718 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-21T15:03:58.768 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-21T15:03:58.805 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-21T15:03:58.847 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-21T15:03:58.995 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-21T15:03:59.050 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-21T15:03:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-21T15:03:59.178 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T15:03:59.209 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T15:03:59.245 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T15:03:59.292 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T15:03:59.328 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T15:03:59.371 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T15:03:59.413 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T15:03:59.549 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T15:03:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T15:03:59.724 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T15:03:59.834 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T15:03:59.873 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T15:03:59.913 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T15:03:59.939 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T15:03:59.963 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T15:04:00.039 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T15:04:00.166 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T15:04:00.221 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T15:04:00.249 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T15:04:00.283 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T15:04:00.322 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T15:04:00.354 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T15:04:00.384 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T15:04:00.418 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T15:04:00.452 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T15:04:00.493 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T15:04:00.527 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T15:04:00.614 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T15:04:00.658 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T15:04:00.697 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T15:04:00.745 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T15:04:00.844 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T15:04:00.899 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T15:04:01.630 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T15:04:01.688 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T15:04:01.721 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T15:04:01.823 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T15:04:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T15:04:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T15:04:02.010 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T15:04:02.041 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T15:04:02.218 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T15:04:02.247 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T15:04:02.325 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T15:04:02.970 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T15:04:03.086 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T15:04:04.319 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T15:04:04.409 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T15:04:04.437 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T15:04:04.470 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T15:04:04.791 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T15:04:04.833 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:04:04.906 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:04:04.937 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-21T15:04:04.974 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T15:04:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T15:04:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:04:05.548 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:04:05.571 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-21T15:04:07.012 INFO:teuthology.orchestra.run.smithi078.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 735 kB/s | 16 MB 00:22 2024-05-21T15:04:07.014 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:04:07.015 INFO:teuthology.orchestra.run.smithi078.stdout:Total 4.9 MB/s | 222 MB 00:44 2024-05-21T15:04:07.788 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:07.825 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:08.187 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-05-21T15:04:08.342 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-05-21T15:04:08.342 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-05-21T15:04:08.377 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:08.412 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:04:08.500 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:04:08.500 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T15:04:08.500 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T15:04:08.500 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:08.530 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:04:09.665 INFO:teuthology.orchestra.run.smithi172.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 738 kB/s | 16 MB 00:22 2024-05-21T15:04:09.668 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:04:09.669 INFO:teuthology.orchestra.run.smithi172.stdout:Total 4.7 MB/s | 222 MB 00:46 2024-05-21T15:04:10.448 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-05-21T15:04:10.448 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-05-21T15:04:10.868 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-05-21T15:04:11.028 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-05-21T15:04:11.029 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-05-21T15:04:12.898 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-05-21T15:04:12.899 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-05-21T15:04:16.637 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-05-21T15:04:16.725 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T15:04:16.763 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T15:04:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T15:04:16.868 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T15:04:16.893 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T15:04:16.938 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T15:04:17.277 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T15:04:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:04:17.510 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:04:17.537 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:04:17.629 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:04:17.659 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:04:17.716 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T15:04:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T15:04:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T15:04:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:04:18.097 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:04:18.119 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:04:18.181 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:04:18.203 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:04:18.223 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:04:18.431 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:04:18.587 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:04:18.756 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:04:18.828 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:04:18.931 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:04:19.013 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:04:19.112 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-05-21T15:04:19.209 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:04:19.210 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T15:04:19.249 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T15:04:19.311 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T15:04:19.321 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T15:04:19.345 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T15:04:19.371 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T15:04:19.405 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T15:04:19.414 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T15:04:19.453 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-21T15:04:19.487 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-21T15:04:19.552 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-21T15:04:19.587 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-21T15:04:19.630 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-21T15:04:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-21T15:04:19.691 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-21T15:04:19.737 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T15:04:19.760 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-21T15:04:19.815 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-21T15:04:19.844 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-21T15:04:19.863 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:04:19.879 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-21T15:04:19.955 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-21T15:04:19.987 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:04:19.989 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-21T15:04:19.991 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:04:19.991 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2024-05-21T15:04:19.991 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2024-05-21T15:04:19.992 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2024-05-21T15:04:19.992 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2024-05-21T15:04:19.992 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2024-05-21T15:04:19.992 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2024-05-21T15:04:19.992 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2024-05-21T15:04:19.992 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:20.014 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:04:20.025 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-21T15:04:20.054 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-21T15:04:20.087 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:04:20.118 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:04:20.123 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-21T15:04:20.151 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T15:04:20.160 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:04:20.168 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-21T15:04:20.174 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:04:20.174 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:20.174 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T15:04:20.174 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:20.242 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-21T15:04:20.347 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T15:04:20.374 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-21T15:04:20.399 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T15:04:20.420 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-21T15:04:20.427 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:04:20.553 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:04:20.579 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:04:20.624 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-21T15:04:20.647 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:04:20.669 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T15:04:20.813 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:04:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-21T15:04:20.980 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T15:04:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-21T15:04:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-21T15:04:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-21T15:04:21.119 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:04:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-21T15:04:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-21T15:04:21.162 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T15:04:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-21T15:04:21.257 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:04:21.358 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:04:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-21T15:04:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-21T15:04:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-21T15:04:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T15:04:21.556 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T15:04:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T15:04:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T15:04:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T15:04:21.664 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T15:04:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T15:04:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T15:04:21.744 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:04:21.759 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T15:04:21.770 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:04:21.770 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:21.770 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T15:04:21.771 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:04:21.771 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:04:21.771 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T15:04:21.796 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-21T15:04:21.831 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-21T15:04:21.897 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-21T15:04:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T15:04:21.930 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-21T15:04:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T15:04:21.990 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-21T15:04:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T15:04:21.997 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:04:22.018 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:04:22.018 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:22.018 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T15:04:22.019 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:04:22.019 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:04:22.019 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:22.023 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-21T15:04:22.051 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-21T15:04:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T15:04:22.120 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-21T15:04:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T15:04:22.173 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-21T15:04:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T15:04:22.203 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-21T15:04:22.229 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T15:04:22.239 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-21T15:04:22.253 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T15:04:22.316 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-21T15:04:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T15:04:22.350 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-21T15:04:22.385 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-21T15:04:22.413 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-21T15:04:22.470 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T15:04:22.479 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:04:22.484 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-21T15:04:22.502 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:04:22.502 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:22.502 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T15:04:22.502 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:04:22.502 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:04:22.502 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:22.512 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T15:04:22.528 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-21T15:04:22.541 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T15:04:22.575 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T15:04:22.604 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-21T15:04:22.609 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T15:04:22.614 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T15:04:22.645 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T15:04:22.666 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T15:04:22.675 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T15:04:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T15:04:22.720 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T15:04:22.738 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-21T15:04:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T15:04:22.784 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T15:04:22.789 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-21T15:04:22.819 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T15:04:22.906 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T15:04:22.906 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T15:04:22.949 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T15:04:22.954 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T15:04:22.980 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T15:04:23.027 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T15:04:23.031 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T15:04:23.120 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T15:04:23.125 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T15:04:23.174 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T15:04:23.259 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T15:04:23.304 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T15:04:23.325 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-21T15:04:23.345 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T15:04:23.372 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T15:04:23.419 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-21T15:04:23.447 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-21T15:04:23.483 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-21T15:04:23.532 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-21T15:04:23.570 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-21T15:04:23.612 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-21T15:04:23.724 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:04:23.762 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-21T15:04:23.782 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:04:23.823 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-21T15:04:23.823 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-21T15:04:23.877 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-21T15:04:23.889 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-21T15:04:23.897 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T15:04:23.927 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:04:23.945 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T15:04:23.951 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T15:04:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T15:04:23.982 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T15:04:24.018 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T15:04:24.065 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T15:04:24.077 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T15:04:24.101 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T15:04:24.134 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T15:04:24.167 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T15:04:24.182 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T15:04:24.212 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T15:04:24.266 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T15:04:24.307 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T15:04:24.348 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T15:04:24.388 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T15:04:24.429 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T15:04:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T15:04:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T15:04:24.524 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T15:04:24.555 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T15:04:24.594 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T15:04:24.595 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T15:04:24.620 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T15:04:24.645 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T15:04:24.719 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T15:04:24.847 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T15:04:24.894 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T15:04:24.922 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T15:04:24.956 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T15:04:24.964 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:04:24.995 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T15:04:25.012 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:04:25.027 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T15:04:25.057 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T15:04:25.091 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T15:04:25.116 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T15:04:25.149 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T15:04:25.184 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T15:04:25.242 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T15:04:25.272 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T15:04:25.323 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T15:04:25.351 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T15:04:25.354 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T15:04:25.401 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T15:04:25.500 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T15:04:25.547 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T15:04:25.979 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:04:26.004 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:04:26.105 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:04:26.277 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-21T15:04:26.289 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T15:04:26.314 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:04:26.336 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T15:04:26.337 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:04:26.337 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:26.337 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T15:04:26.337 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:04:26.337 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:04:26.338 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:26.369 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T15:04:26.381 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:04:26.392 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:04:26.472 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T15:04:26.491 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-21T15:04:26.525 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T15:04:26.538 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-21T15:04:26.570 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T15:04:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T15:04:26.660 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T15:04:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T15:04:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T15:04:26.714 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T15:04:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T15:04:26.892 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T15:04:26.928 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T15:04:27.007 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T15:04:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T15:04:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:04:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:04:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-21T15:04:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T15:04:27.461 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T15:04:27.501 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:04:27.523 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:04:27.524 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:27.524 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T15:04:27.524 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:04:27.524 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:04:27.524 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:27.568 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:04:27.589 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:04:27.589 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:27.589 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T15:04:27.589 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:04:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:04:27.658 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:27.658 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T15:04:27.658 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:04:27.658 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:04:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:27.662 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T15:04:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T15:04:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:04:27.806 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T15:04:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:04:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-21T15:04:27.977 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:04:28.000 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:04:28.000 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:28.000 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T15:04:28.000 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:04:28.000 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:04:28.001 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:29.078 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T15:04:29.157 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T15:04:29.177 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T15:04:29.201 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T15:04:29.514 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T15:04:29.564 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:04:29.654 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-21T15:04:29.685 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-21T15:04:29.722 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T15:04:30.033 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:30.066 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:30.202 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T15:04:30.234 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:04:30.305 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-21T15:04:30.327 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-21T15:04:30.634 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:04:30.743 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:04:30.743 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T15:04:30.743 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T15:04:30.743 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:30.770 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:04:32.537 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:32.548 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-21T15:04:32.575 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:32.588 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-21T15:04:32.646 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-21T15:04:32.686 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T15:04:32.722 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T15:04:32.910 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-21T15:04:32.951 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T15:04:32.983 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-21T15:04:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T15:04:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T15:04:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:04:33.135 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-21T15:04:33.178 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:04:33.254 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-21T15:04:33.254 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T15:04:33.254 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T15:04:33.254 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:33.288 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:04:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:34.608 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-21T15:04:34.701 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T15:04:34.742 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T15:04:34.826 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T15:04:34.854 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T15:04:34.879 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:04:34.921 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T15:04:34.948 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T15:04:34.948 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:04:35.032 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:04:35.057 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T15:04:35.057 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:04:35.141 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:04:35.167 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T15:04:35.171 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:04:35.171 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:04:35.351 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:04:35.351 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:04:35.436 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:04:35.460 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T15:04:35.524 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:04:41.030 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:04:41.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-21T15:04:41.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-21T15:04:41.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-21T15:04:41.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T15:04:41.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T15:04:41.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T15:04:41.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T15:04:41.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T15:04:41.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T15:04:41.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T15:04:41.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T15:04:41.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-21T15:04:41.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T15:04:41.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T15:04:41.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T15:04:41.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T15:04:41.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T15:04:41.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T15:04:41.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T15:04:42.658 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:04:42.658 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-05-21T15:04:42.658 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-05-21T15:04:42.658 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-05-21T15:04:42.658 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-05-21T15:04:42.658 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-05-21T15:04:42.659 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-05-21T15:04:42.659 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-05-21T15:04:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:42.825 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:04:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:04:42.841 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:42.841 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T15:04:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:43.720 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T15:04:43.720 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:43.720 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-05-21T15:04:43.720 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T15:04:43.720 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T15:04:43.720 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T15:04:43.720 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T15:04:43.721 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.722 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:04:43.723 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.724 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T15:04:43.725 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T15:04:43.726 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T15:04:43.727 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T15:04:43.728 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T15:04:43.729 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.730 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T15:04:43.731 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T15:04:43.732 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T15:04:43.733 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T15:04:43.733 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T15:04:43.733 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T15:04:43.733 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-05-21T15:04:43.733 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-05-21T15:04:44.172 DEBUG:teuthology.parallel:result is None 2024-05-21T15:04:44.436 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:04:44.462 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:04:44.463 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:44.463 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T15:04:44.463 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:04:44.463 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:04:44.463 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:44.686 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:04:44.709 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:04:44.710 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:44.710 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T15:04:44.710 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:04:44.710 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:04:44.710 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:45.098 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-21T15:04:45.099 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-05-21T15:04:45.099 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-05-21T15:04:45.099 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-05-21T15:04:45.099 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-05-21T15:04:45.099 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-05-21T15:04:45.099 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-05-21T15:04:45.099 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-05-21T15:04:45.099 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:45.179 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:04:45.202 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:04:45.203 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:45.203 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T15:04:45.203 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:04:45.203 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:04:45.204 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:45.258 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:04:45.273 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-21T15:04:45.273 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:45.273 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T15:04:45.273 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:45.301 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T15:04:45.357 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T15:04:45.412 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T15:04:45.597 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T15:04:45.646 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T15:04:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T15:04:45.820 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T15:04:45.967 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T15:04:46.012 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T15:04:46.052 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T15:04:46.080 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T15:04:46.468 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:04:46.496 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:04:46.527 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-21T15:04:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-21T15:04:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:04:46.860 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:04:46.888 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-21T15:04:46.889 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:46.889 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T15:04:46.889 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:04:46.889 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T15:04:46.889 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:47.119 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:04:47.146 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-21T15:04:47.146 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:47.146 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T15:04:47.146 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:04:47.146 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T15:04:47.146 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:47.612 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:04:47.635 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-21T15:04:47.635 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:47.635 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T15:04:47.635 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:04:47.635 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T15:04:47.635 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:47.637 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:04:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:04:47.742 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T15:04:47.799 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T15:04:47.855 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T15:04:48.049 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T15:04:48.096 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T15:04:48.175 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T15:04:48.262 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T15:04:48.402 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T15:04:48.454 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T15:04:48.494 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T15:04:48.530 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T15:04:48.635 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:04:48.673 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:04:48.770 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:04:48.931 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-21T15:04:48.959 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:04:48.982 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:04:48.982 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:48.982 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T15:04:48.982 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:04:48.983 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:04:48.983 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:48.999 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:04:49.026 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:04:49.029 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-21T15:04:49.037 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:04:49.060 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-21T15:04:49.125 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-21T15:04:49.135 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-21T15:04:49.152 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:04:49.205 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-21T15:04:50.118 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T15:04:50.145 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:04:50.169 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:04:50.169 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:50.169 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T15:04:50.169 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:04:50.169 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:04:50.169 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:50.194 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-21T15:04:50.221 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:04:50.237 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:04:50.244 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:04:50.244 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:50.244 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T15:04:50.244 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:50.305 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:04:50.328 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:04:50.328 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:50.328 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T15:04:50.328 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:04:50.328 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:04:50.329 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:50.647 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:04:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:04:50.671 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:50.671 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T15:04:50.671 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:04:50.671 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:04:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:04:51.206 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-21T15:04:51.237 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:04:51.339 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-21T15:04:51.501 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-21T15:04:51.539 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:04:51.562 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-21T15:04:51.563 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:51.563 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T15:04:51.563 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:04:51.563 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T15:04:51.563 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:51.615 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:04:51.626 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-21T15:04:51.734 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-21T15:04:51.779 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-21T15:04:52.692 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T15:04:52.726 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:04:52.748 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-21T15:04:52.748 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:52.748 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T15:04:52.748 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:04:52.748 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T15:04:52.748 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:52.801 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:04:52.823 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-21T15:04:52.823 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:52.823 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T15:04:52.823 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:52.869 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:04:52.891 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-21T15:04:52.891 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:52.891 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T15:04:52.891 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:04:52.891 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T15:04:52.891 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:53.211 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:04:53.233 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-21T15:04:53.233 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T15:04:53.233 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T15:04:53.233 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:04:53.233 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T15:04:53.234 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:04:55.217 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-21T15:04:55.259 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-21T15:04:55.300 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-21T15:04:55.331 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T15:04:55.359 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T15:04:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-21T15:04:55.613 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T15:04:55.663 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-21T15:04:55.703 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T15:04:55.732 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T15:04:55.771 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:04:56.403 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:56.443 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:56.464 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:57.294 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-21T15:04:57.390 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T15:04:57.431 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T15:04:57.522 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T15:04:57.560 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T15:04:57.584 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:04:57.610 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T15:04:57.638 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T15:04:57.638 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:04:57.721 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:04:57.754 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T15:04:57.754 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:04:57.772 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-21T15:04:57.814 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-21T15:04:57.838 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:04:57.855 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-21T15:04:57.887 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T15:04:57.888 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T15:04:57.893 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:04:57.893 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:04:57.914 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T15:04:58.091 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:04:58.091 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:04:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-21T15:04:58.151 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T15:04:58.175 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:04:58.186 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-21T15:04:58.207 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T15:04:58.224 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T15:04:58.254 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T15:04:58.274 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:04:58.292 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:04:58.882 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:58.923 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:58.942 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T15:04:59.788 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-21T15:04:59.877 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T15:04:59.926 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T15:05:00.017 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T15:05:00.055 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T15:05:00.080 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:05:00.106 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T15:05:00.132 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T15:05:00.132 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:05:00.234 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T15:05:00.258 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T15:05:00.258 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:05:00.334 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T15:05:00.359 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T15:05:00.364 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:05:00.364 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:05:00.553 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T15:05:00.553 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:05:00.628 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T15:05:00.653 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T15:05:00.716 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:05:04.026 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:05:04.026 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-21T15:05:04.026 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-21T15:05:04.026 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-21T15:05:04.026 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-21T15:05:04.026 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-21T15:05:04.026 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-21T15:05:04.026 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-21T15:05:04.027 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-21T15:05:04.028 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T15:05:04.029 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T15:05:04.030 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T15:05:04.031 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T15:05:04.032 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T15:05:04.033 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T15:05:04.034 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T15:05:04.035 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T15:05:04.036 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:05:04.037 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T15:05:04.038 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T15:05:04.039 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T15:05:04.039 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T15:05:06.476 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-21T15:05:06.477 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-21T15:05:06.478 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-21T15:05:06.479 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-21T15:05:06.479 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-21T15:05:06.479 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-21T15:05:06.479 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-21T15:05:06.479 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-21T15:05:06.492 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-21T15:05:06.492 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-21T15:05:06.492 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-21T15:05:06.492 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-21T15:05:06.492 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-21T15:05:06.492 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-21T15:05:06.492 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-21T15:05:06.492 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T15:05:06.493 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T15:05:06.494 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T15:05:06.495 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T15:05:06.496 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T15:05:06.496 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T15:05:06.496 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T15:05:06.496 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T15:05:06.496 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T15:05:06.496 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T15:05:06.496 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T15:05:06.496 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T15:05:06.497 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T15:05:06.497 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T15:05:06.497 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T15:05:06.497 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T15:05:06.497 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T15:05:06.497 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T15:05:06.497 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T15:05:06.497 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T15:05:06.498 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T15:05:06.499 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T15:05:06.500 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T15:05:06.501 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T15:05:06.502 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T15:05:06.503 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T15:05:06.504 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T15:05:06.505 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T15:05:06.505 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T15:05:06.505 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T15:05:06.505 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T15:05:06.505 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T15:05:06.505 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T15:05:06.505 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T15:05:06.505 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T15:05:06.547 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T15:05:06.547 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T15:05:06.548 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.549 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T15:05:06.551 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T15:05:06.552 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T15:05:06.553 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T15:05:06.554 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T15:05:06.555 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T15:05:06.556 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T15:05:06.557 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T15:05:06.558 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T15:05:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T15:05:06.560 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T15:05:06.561 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T15:05:06.561 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:05:06.561 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-05-21T15:05:06.957 DEBUG:teuthology.parallel:result is None 2024-05-21T15:05:09.287 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T15:05:09.287 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:05:09.287 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-05-21T15:05:09.287 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T15:05:09.287 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T15:05:09.287 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T15:05:09.287 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T15:05:09.287 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T15:05:09.288 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.289 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-21T15:05:09.290 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.291 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T15:05:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T15:05:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T15:05:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T15:05:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T15:05:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T15:05:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:05:09.300 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-05-21T15:05:09.765 DEBUG:teuthology.parallel:result is None 2024-05-21T15:05:09.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:05:09.905 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T15:05:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-21T15:05:09.931 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-21T15:05:09.931 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-21T15:05:09.933 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:05:10.151 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T15:05:10.177 INFO:teuthology.orchestra.run.smithi084.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-21T15:05:10.177 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-21T15:05:10.177 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-21T15:05:10.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:05:10.391 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T15:05:10.414 INFO:teuthology.orchestra.run.smithi172.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-21T15:05:10.414 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-21T15:05:10.415 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-21T15:05:10.417 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-21T15:05:10.417 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:05:10.417 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T15:05:10.449 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:05:10.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T15:05:10.480 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:05:10.480 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T15:05:10.508 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-21T15:05:10.508 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:05:10.508 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T15:05:10.536 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T15:05:10.602 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:05:10.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T15:05:10.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T15:05:10.699 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:05:10.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T15:05:10.724 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T15:05:10.789 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-21T15:05:10.789 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:05:10.789 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T15:05:10.816 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T15:05:10.881 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:05:10.882 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T15:05:10.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T15:05:10.970 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:05:10.970 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T15:05:10.994 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T15:05:11.058 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-21T15:05:11.059 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:05:11.059 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T15:05:11.084 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T15:05:11.150 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:05:11.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T15:05:11.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T15:05:11.237 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:05:11.237 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T15:05:11.262 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T15:05:11.326 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-21T15:05:11.421 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-21T15:05:11.421 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:05:11.422 INFO:tasks.cephadm:Cluster fsid is 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:05:11.422 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-21T15:05:11.422 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.78', 'mon.b': '172.21.15.84', 'mon.c': '172.21.15.172'} 2024-05-21T15:05:11.422 INFO:tasks.cephadm:First mon is mon.a on smithi078 2024-05-21T15:05:11.422 INFO:tasks.cephadm:First mgr is x 2024-05-21T15:05:11.422 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-21T15:05:11.423 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-05-21T15:05:11.448 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname $(hostname -s) 2024-05-21T15:05:11.475 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname $(hostname -s) 2024-05-21T15:05:11.499 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-21T15:05:11.500 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:05:11.718 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79503/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2744-gd2b507c7', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2744.gd2b507c7'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-17 22:05:47.367530', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-21T15:05:11.942 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240517.203521-squid-debug, sha1 d2b507c724ef6c8295ee0a35d185cdab167fd61d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:05:11.943 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm 2024-05-21T15:05:11.944 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:12.392 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 21 15:05 /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:12.393 DEBUG:teuthology.orchestra.run.smithi084:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:13.028 INFO:teuthology.orchestra.run.smithi084.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 21 15:05 /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:13.028 DEBUG:teuthology.orchestra.run.smithi172:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:13.804 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r--. 1 ubuntu ubuntu 782070 May 21 15:05 /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:13.804 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:13.823 DEBUG:teuthology.orchestra.run.smithi084:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:13.842 DEBUG:teuthology.orchestra.run.smithi172:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T15:05:13.875 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d on all hosts... 2024-05-21T15:05:13.876 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-21T15:05:13.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-21T15:05:13.884 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d pull 2024-05-21T15:05:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:05:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:05:14.838 INFO:teuthology.orchestra.run.smithi172.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:05:42.869 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:05:42.869 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-21T15:05:42.869 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-21T15:05:42.869 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-05-21T15:05:42.869 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-21T15:05:42.869 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:05:42.869 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:05:49.488 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-21T15:05:49.488 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-21T15:05:49.488 INFO:teuthology.orchestra.run.smithi084.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-21T15:05:49.488 INFO:teuthology.orchestra.run.smithi084.stdout: "repo_digests": [ 2024-05-21T15:05:49.488 INFO:teuthology.orchestra.run.smithi084.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-21T15:05:49.489 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-05-21T15:05:49.489 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-21T15:05:58.091 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-05-21T15:05:58.092 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-21T15:05:58.092 INFO:teuthology.orchestra.run.smithi172.stdout: "image_id": "cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3", 2024-05-21T15:05:58.092 INFO:teuthology.orchestra.run.smithi172.stdout: "repo_digests": [ 2024-05-21T15:05:58.092 INFO:teuthology.orchestra.run.smithi172.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708" 2024-05-21T15:05:58.092 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-05-21T15:05:58.092 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-05-21T15:05:58.116 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-05-21T15:05:58.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph 2024-05-21T15:05:58.180 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph 2024-05-21T15:05:58.207 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-05-21T15:05:58.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /etc/ceph 2024-05-21T15:05:58.265 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /etc/ceph 2024-05-21T15:05:58.288 INFO:tasks.cephadm:Writing seed config... 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-21T15:05:58.289 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-21T15:05:58.290 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-21T15:05:58.291 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-21T15:05:58.292 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-21T15:05:58.293 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:05:58.293 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-21T15:05:58.310 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 852c9762-1783-11ef-bc9a-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-21T15:05:58.310 DEBUG:teuthology.orchestra.run.smithi078:mon.a> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.a.service 2024-05-21T15:05:58.352 DEBUG:teuthology.orchestra.run.smithi078:mgr.x> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-21T15:05:58.395 INFO:tasks.cephadm:Bootstrapping... 2024-05-21T15:05:58.395 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d -v bootstrap --fsid 852c9762-1783-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.78 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:05:58.586 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-05-21T15:05:58.586 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d', '-v', 'bootstrap', '--fsid', '852c9762-1783-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.78', '--skip-admin-label'] 2024-05-21T15:05:58.586 INFO:teuthology.orchestra.run.smithi078.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-21T15:05:58.586 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying podman|docker is present... 2024-05-21T15:05:58.611 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 5.0.2 2024-05-21T15:05:58.611 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying lvm2 is present... 2024-05-21T15:05:58.611 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying time synchronization is in place... 2024-05-21T15:05:58.619 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T15:05:58.619 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T15:05:58.625 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T15:05:58.625 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-05-21T15:05:58.632 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-05-21T15:05:58.638 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-05-21T15:05:58.638 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-05-21T15:05:58.638 INFO:teuthology.orchestra.run.smithi078.stdout:Repeating the final host check... 2024-05-21T15:05:58.661 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 5.0.2 2024-05-21T15:05:58.661 INFO:teuthology.orchestra.run.smithi078.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-21T15:05:58.661 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl is present 2024-05-21T15:05:58.662 INFO:teuthology.orchestra.run.smithi078.stdout:lvcreate is present 2024-05-21T15:05:58.668 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T15:05:58.668 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T15:05:58.674 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T15:05:58.674 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-05-21T15:05:58.681 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-05-21T15:05:58.687 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-05-21T15:05:58.687 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-05-21T15:05:58.688 INFO:teuthology.orchestra.run.smithi078.stdout:Host looks OK 2024-05-21T15:05:58.688 INFO:teuthology.orchestra.run.smithi078.stdout:Cluster fsid: 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:05:58.688 INFO:teuthology.orchestra.run.smithi078.stdout:Acquiring lock 140095162033536 on /run/cephadm/852c9762-1783-11ef-bc9a-c7b262605968.lock 2024-05-21T15:05:58.688 INFO:teuthology.orchestra.run.smithi078.stdout:Lock 140095162033536 acquired on /run/cephadm/852c9762-1783-11ef-bc9a-c7b262605968.lock 2024-05-21T15:05:58.688 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 3300 ... 2024-05-21T15:05:58.688 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 6789 ... 2024-05-21T15:05:58.689 INFO:teuthology.orchestra.run.smithi078.stdout:Base mon IP(s) is [172.21.15.78:3300, 172.21.15.78:6789], mon addrv is [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-05-21T15:05:58.692 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.78 metric 100 2024-05-21T15:05:58.692 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.78 metric 100 2024-05-21T15:05:58.694 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-21T15:05:58.694 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-21T15:05:58.694 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1755sec hoplimit 64 pref medium 2024-05-21T15:05:58.697 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-21T15:05:58.697 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-21T15:05:58.697 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T15:05:58.697 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-21T15:05:58.697 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-05-21T15:05:58.697 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T15:05:58.698 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-05-21T15:05:58.698 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-05-21T15:05:58.698 INFO:teuthology.orchestra.run.smithi078.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-21T15:05:58.698 INFO:teuthology.orchestra.run.smithi078.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-21T15:05:58.699 INFO:teuthology.orchestra.run.smithi078.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:06:02.469 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 2024-05-21T15:06:02.469 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d... 2024-05-21T15:06:02.470 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Getting image source signatures 2024-05-21T15:06:02.470 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:506d33f63dccdaac2c87f708e3d3ef98b8f2fc00666d50b72896243431ca9bae 2024-05-21T15:06:02.470 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-21T15:06:02.470 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying config sha256:cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 2024-05-21T15:06:02.470 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-21T15:06:11.944 INFO:teuthology.orchestra.run.smithi078.stdout:ceph: stdout ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-21T15:06:11.945 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph version: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev) 2024-05-21T15:06:11.945 INFO:teuthology.orchestra.run.smithi078.stdout:Extracting ceph user uid/gid from container image... 2024-05-21T15:06:14.127 INFO:teuthology.orchestra.run.smithi078.stdout:stat: stdout 167 167 2024-05-21T15:06:14.127 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial keys... 2024-05-21T15:06:15.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBmuExmGs09HhAAKKVkeTQ+bhUvJx/XHoXgvw== 2024-05-21T15:06:15.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBnuExmyVv0ExAAtpxX+dVYWZl7J4KSMXJ/Zw== 2024-05-21T15:06:16.769 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBouExmYAF3DxAAvGazKcgFx3NO20ed2IeVlw== 2024-05-21T15:06:16.770 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial monmap... 2024-05-21T15:06:18.026 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T15:06:18.061 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-21T15:06:18.061 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:18.062 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T15:06:18.062 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool for a [v2:172.21.15.78:3300,v1:172.21.15.78:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T15:06:18.062 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = quincy 2024-05-21T15:06:18.062 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: set fsid to 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:18.064 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T15:06:18.064 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:06:18.064 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mon... 2024-05-21T15:06:19.571 INFO:teuthology.orchestra.run.smithi078.stdout:create mon.a on 2024-05-21T15:06:19.779 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-21T15:06:19.916 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-21T15:06:20.097 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-852c9762-1783-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-852c9762-1783-11ef-bc9a-c7b262605968.target. 2024-05-21T15:06:20.098 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-852c9762-1783-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-852c9762-1783-11ef-bc9a-c7b262605968.target. 2024-05-21T15:06:20.329 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.a 2024-05-21T15:06:20.329 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-21T15:06:20.485 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-852c9762-1783-11ef-bc9a-c7b262605968.target.wants/ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-852c9762-1783-11ef-bc9a-c7b262605968@.service. 2024-05-21T15:06:21.010 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T15:06:21.011 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-05-21T15:06:21.017 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T15:06:21.017 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-05-21T15:06:21.017 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-05-21T15:06:21.018 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T15:06:21.018 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon to start... 2024-05-21T15:06:21.018 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon... 2024-05-21T15:06:22.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:21 smithi078 ceph-mon[31625]: from='client.? 172.21.15.78:0/1020546104' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-21T15:06:22.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout cluster: 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout id: 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout services: 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.60855s) 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout data: 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-21T15:06:22.171 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pgs: 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.511+0000 7fa006faf700 1 Processor -- start 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.512+0000 7fa006faf700 1 -- start start 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.512+0000 7fa006faf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00007c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.512+0000 7fa006faf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa00007c760 con 0x7fa00007dd30 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.512+0000 7fa004d4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00007c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.512+0000 7fa004d4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00007c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34116/0 (socket says 172.21.15.78:34116) 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.512+0000 7fa004d4b700 1 -- 172.21.15.78:0/2605072381 learned_addr learned my addr 172.21.15.78:0/2605072381 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.512+0000 7fa004d4b700 1 -- 172.21.15.78:0/2605072381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00007c8e0 con 0x7fa00007dd30 2024-05-21T15:06:22.172 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.513+0000 7fa004d4b700 1 --2- 172.21.15.78:0/2605072381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00007c220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40098a0 tx=0x7f9ff4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4200d1777f9953f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.513+0000 7f9fff7fe700 1 -- 172.21.15.78:0/2605072381 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ff4004160 con 0x7fa00007dd30 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.513+0000 7f9fff7fe700 1 -- 172.21.15.78:0/2605072381 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff4004300 con 0x7fa00007dd30 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.513+0000 7f9fff7fe700 1 -- 172.21.15.78:0/2605072381 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ff40036a0 con 0x7fa00007dd30 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 -- 172.21.15.78:0/2605072381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 msgr2=0x7fa00007c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 --2- 172.21.15.78:0/2605072381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00007c220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40098a0 tx=0x7f9ff4009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 -- 172.21.15.78:0/2605072381 shutdown_connections 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 --2- 172.21.15.78:0/2605072381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00007c220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 -- 172.21.15.78:0/2605072381 >> 172.21.15.78:0/2605072381 conn(0x7fa000104410 msgr2=0x7fa0001068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 -- 172.21.15.78:0/2605072381 shutdown_connections 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 -- 172.21.15.78:0/2605072381 wait complete. 2024-05-21T15:06:22.173 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 Processor -- start 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.514+0000 7fa006faf700 1 -- start start 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7fa006faf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7fa006faf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa00007ce10 con 0x7fa00007dd30 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7fa004d4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7fa004d4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34126/0 (socket says 172.21.15.78:34126) 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7fa004d4b700 1 -- 172.21.15.78:0/1020546104 learned_addr learned my addr 172.21.15.78:0/1020546104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7fa004d4b700 1 -- 172.21.15.78:0/1020546104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00019ef80 con 0x7fa00007dd30 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7fa004d4b700 1 --2- 172.21.15.78:0/1020546104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00019ea40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4000c00 tx=0x7f9ff4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7f9ffdffb700 1 -- 172.21.15.78:0/1020546104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ff4003db0 con 0x7fa00007dd30 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7f9ffdffb700 1 -- 172.21.15.78:0/1020546104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff4011630 con 0x7fa00007dd30 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00019f210 con 0x7fa00007dd30 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.515+0000 7f9ffdffb700 1 -- 172.21.15.78:0/1020546104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ff4011950 con 0x7fa00007dd30 2024-05-21T15:06:22.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.516+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0001a1e20 con 0x7fa00007dd30 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.516+0000 7f9ffdffb700 1 -- 172.21.15.78:0/1020546104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9ff4011af0 con 0x7fa00007dd30 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.516+0000 7f9ffdffb700 1 -- 172.21.15.78:0/1020546104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9ff401c450 con 0x7fa00007dd30 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.517+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa000063ca0 con 0x7fa00007dd30 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.519+0000 7f9ffdffb700 1 -- 172.21.15.78:0/1020546104 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9ff4032020 con 0x7fa00007dd30 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.590+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa00002df60 con 0x7fa00007dd30 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.590+0000 7f9ffdffb700 1 -- 172.21.15.78:0/1020546104 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f9ff4004260 con 0x7fa00007dd30 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.591+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 msgr2=0x7fa00019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.591+0000 7fa006faf700 1 --2- 172.21.15.78:0/1020546104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00019ea40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4000c00 tx=0x7f9ff4004750 comp rx=0 tx=0).stop 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.591+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 shutdown_connections 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.591+0000 7fa006faf700 1 --2- 172.21.15.78:0/1020546104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa00007dd30 0x7fa00019ea40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.591+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 >> 172.21.15.78:0/1020546104 conn(0x7fa000104410 msgr2=0x7fa0001065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.591+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 shutdown_connections 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:21.591+0000 7fa006faf700 1 -- 172.21.15.78:0/1020546104 wait complete. 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:mon is available 2024-05-21T15:06:22.175 INFO:teuthology.orchestra.run.smithi078.stdout:Assimilating anything we can from ceph.conf... 2024-05-21T15:06:23.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:22 smithi078 ceph-mon[31625]: from='client.? 172.21.15.78:0/2571294508' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-21T15:06:23.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:22 smithi078 ceph-mon[31625]: from='client.? 172.21.15.78:0/2571294508' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:23.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.579+0000 7fa44d0ec700 1 Processor -- start 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.579+0000 7fa44d0ec700 1 -- start start 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.579+0000 7fa44d0ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa448108020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:23.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.579+0000 7fa44d0ec700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa448108610 con 0x7fa448107c10 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.579+0000 7fa446d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa448108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.579+0000 7fa446d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa448108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34136/0 (socket says 172.21.15.78:34136) 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.579+0000 7fa446d9d700 1 -- 172.21.15.78:0/3532410457 learned_addr learned my addr 172.21.15.78:0/3532410457 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.579+0000 7fa446d9d700 1 -- 172.21.15.78:0/3532410457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa448108ea0 con 0x7fa448107c10 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa446d9d700 1 --2- 172.21.15.78:0/3532410457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa448108020 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa4380098a0 tx=0x7fa438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65ea90107dc1bba server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa445d9b700 1 -- 172.21.15.78:0/3532410457 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa438004160 con 0x7fa448107c10 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa445d9b700 1 -- 172.21.15.78:0/3532410457 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa438004300 con 0x7fa448107c10 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa445d9b700 1 -- 172.21.15.78:0/3532410457 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4380036a0 con 0x7fa448107c10 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa44d0ec700 1 -- 172.21.15.78:0/3532410457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 msgr2=0x7fa448108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa44d0ec700 1 --2- 172.21.15.78:0/3532410457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa448108020 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa4380098a0 tx=0x7fa438009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa44d0ec700 1 -- 172.21.15.78:0/3532410457 shutdown_connections 2024-05-21T15:06:23.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa44d0ec700 1 --2- 172.21.15.78:0/3532410457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa448108020 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa44d0ec700 1 -- 172.21.15.78:0/3532410457 >> 172.21.15.78:0/3532410457 conn(0x7fa4481033d0 msgr2=0x7fa448105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.580+0000 7fa44d0ec700 1 -- 172.21.15.78:0/3532410457 shutdown_connections 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.581+0000 7fa44d0ec700 1 -- 172.21.15.78:0/3532410457 wait complete. 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.581+0000 7fa44d0ec700 1 Processor -- start 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa44d0ec700 1 -- start start 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa44d0ec700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa44807b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa44d0ec700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481093d0 con 0x7fa448107c10 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa446d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa44807b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa446d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa44807b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34150/0 (socket says 172.21.15.78:34150) 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa446d9d700 1 -- 172.21.15.78:0/2571294508 learned_addr learned my addr 172.21.15.78:0/2571294508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa446d9d700 1 -- 172.21.15.78:0/2571294508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44807be90 con 0x7fa448107c10 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa446d9d700 1 --2- 172.21.15.78:0/2571294508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa44807b950 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa438000c00 tx=0x7fa438004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.582+0000 7fa42ffff700 1 -- 172.21.15.78:0/2571294508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa438003db0 con 0x7fa448107c10 2024-05-21T15:06:23.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.583+0000 7fa42ffff700 1 -- 172.21.15.78:0/2571294508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa438011630 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.583+0000 7fa42ffff700 1 -- 172.21.15.78:0/2571294508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa438011950 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.583+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44819f5e0 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.583+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44819fa60 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.583+0000 7fa42ffff700 1 -- 172.21.15.78:0/2571294508 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa438011af0 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.583+0000 7fa42ffff700 1 -- 172.21.15.78:0/2571294508 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa43801c450 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.584+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa448063ca0 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.586+0000 7fa42ffff700 1 -- 172.21.15.78:0/2571294508 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa438032020 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.653+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa44802d2b0 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.657+0000 7fa42ffff700 1 -- 172.21.15.78:0/2571294508 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+454 (secure 0 0 0) 0x7fa438004260 con 0x7fa448107c10 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.658+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 msgr2=0x7fa44807b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.658+0000 7fa44d0ec700 1 --2- 172.21.15.78:0/2571294508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa44807b950 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa438000c00 tx=0x7fa438004750 comp rx=0 tx=0).stop 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.658+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 shutdown_connections 2024-05-21T15:06:23.230 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.658+0000 7fa44d0ec700 1 --2- 172.21.15.78:0/2571294508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa448107c10 0x7fa44807b950 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:23.231 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.658+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 >> 172.21.15.78:0/2571294508 conn(0x7fa4481033d0 msgr2=0x7fa44818f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:23.231 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.658+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 shutdown_connections 2024-05-21T15:06:23.231 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:22.658+0000 7fa44d0ec700 1 -- 172.21.15.78:0/2571294508 wait complete. 2024-05-21T15:06:23.231 INFO:teuthology.orchestra.run.smithi078.stdout:Generating new minimal ceph.conf... 2024-05-21T15:06:24.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:23 smithi078 ceph-mon[31625]: from='client.? 172.21.15.78:0/2920726244' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:06:24.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.670+0000 7f937a8f5700 1 Processor -- start 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.670+0000 7f937a8f5700 1 -- start start 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.670+0000 7f937a8f5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f9374109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.670+0000 7f937a8f5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9374109710 con 0x7f9374108d10 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.670+0000 7f9373fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f9374109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.671+0000 7f9373fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f9374109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34160/0 (socket says 172.21.15.78:34160) 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.671+0000 7f9373fff700 1 -- 172.21.15.78:0/4031304822 learned_addr learned my addr 172.21.15.78:0/4031304822 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.671+0000 7f9373fff700 1 -- 172.21.15.78:0/4031304822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9374109fa0 con 0x7f9374108d10 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.671+0000 7f9373fff700 1 --2- 172.21.15.78:0/4031304822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f9374109120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93640098a0 tx=0x7f9364009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b49014b3ef82d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.671+0000 7f9372ffd700 1 -- 172.21.15.78:0/4031304822 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9364004160 con 0x7f9374108d10 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.671+0000 7f9372ffd700 1 -- 172.21.15.78:0/4031304822 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9364004300 con 0x7f9374108d10 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.671+0000 7f9372ffd700 1 -- 172.21.15.78:0/4031304822 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9364011b50 con 0x7f9374108d10 2024-05-21T15:06:24.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.672+0000 7f937a8f5700 1 -- 172.21.15.78:0/4031304822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 msgr2=0x7f9374109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.672+0000 7f937a8f5700 1 --2- 172.21.15.78:0/4031304822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f9374109120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93640098a0 tx=0x7f9364009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.672+0000 7f937a8f5700 1 -- 172.21.15.78:0/4031304822 shutdown_connections 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.672+0000 7f937a8f5700 1 --2- 172.21.15.78:0/4031304822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f9374109120 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.672+0000 7f937a8f5700 1 -- 172.21.15.78:0/4031304822 >> 172.21.15.78:0/4031304822 conn(0x7f9374104410 msgr2=0x7f93741068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.672+0000 7f937a8f5700 1 -- 172.21.15.78:0/4031304822 shutdown_connections 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.672+0000 7f937a8f5700 1 -- 172.21.15.78:0/4031304822 wait complete. 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f937a8f5700 1 Processor -- start 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f937a8f5700 1 -- start start 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f937a8f5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f937419d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f937a8f5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f937410d350 con 0x7f9374108d10 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f9373fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f937419d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f9373fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f937419d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34174/0 (socket says 172.21.15.78:34174) 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f9373fff700 1 -- 172.21.15.78:0/2920726244 learned_addr learned my addr 172.21.15.78:0/2920726244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:24.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f9373fff700 1 -- 172.21.15.78:0/2920726244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937419d700 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.673+0000 7f9373fff700 1 --2- 172.21.15.78:0/2920726244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f937419d1c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9364009fb0 tx=0x7f9364011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.674+0000 7f93717fa700 1 -- 172.21.15.78:0/2920726244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93640038b0 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.674+0000 7f93717fa700 1 -- 172.21.15.78:0/2920726244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9364011e50 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.674+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937419d990 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.674+0000 7f93717fa700 1 -- 172.21.15.78:0/2920726244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f936401a430 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.674+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93741a05a0 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.674+0000 7f93717fa700 1 -- 172.21.15.78:0/2920726244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f936401aa80 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.674+0000 7f93717fa700 1 -- 172.21.15.78:0/2920726244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9364019700 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.675+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9374063ca0 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.677+0000 7f93717fa700 1 -- 172.21.15.78:0/2920726244 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9364003a50 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.744+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f93741a0890 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.744+0000 7f93717fa700 1 -- 172.21.15.78:0/2920726244 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f936402dcb0 con 0x7f9374108d10 2024-05-21T15:06:24.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.745+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 msgr2=0x7f937419d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:24.317 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.745+0000 7f937a8f5700 1 --2- 172.21.15.78:0/2920726244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f937419d1c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9364009fb0 tx=0x7f9364011a00 comp rx=0 tx=0).stop 2024-05-21T15:06:24.317 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.745+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 shutdown_connections 2024-05-21T15:06:24.317 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.745+0000 7f937a8f5700 1 --2- 172.21.15.78:0/2920726244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9374108d10 0x7f937419d1c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:24.317 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.745+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 >> 172.21.15.78:0/2920726244 conn(0x7f9374104410 msgr2=0x7f9374104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:24.317 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.745+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 shutdown_connections 2024-05-21T15:06:24.317 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:23.745+0000 7f937a8f5700 1 -- 172.21.15.78:0/2920726244 wait complete. 2024-05-21T15:06:24.317 INFO:teuthology.orchestra.run.smithi078.stdout:Restarting the monitor... 2024-05-21T15:06:24.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:24 smithi078 systemd[1]: Stopping Ceph mon.a for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:06:25.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:24 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a[31621]: 2024-05-21T15:06:24.766+0000 7fa57718a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:06:25.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:24 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a[31621]: 2024-05-21T15:06:24.766+0000 7fa57718a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-21T15:06:25.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:24 smithi078 podman[31870]: 2024-05-21 15:06:24.95108407 +0000 UTC m=+0.477489091 container died 5727053dfd00179e5f1033cb0869fba6cf5b111c5bde45d62ac2b44dc1257625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True) 2024-05-21T15:06:25.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:25 smithi078 podman[31870]: 2024-05-21 15:06:25.291116843 +0000 UTC m=+0.817521849 container cleanup 5727053dfd00179e5f1033cb0869fba6cf5b111c5bde45d62ac2b44dc1257625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:06:25.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:25 smithi078 bash[31870]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a 2024-05-21T15:06:25.733 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:25 smithi078 podman[31882]: 2024-05-21 15:06:25.47573269 +0000 UTC m=+0.519464771 container remove 5727053dfd00179e5f1033cb0869fba6cf5b111c5bde45d62ac2b44dc1257625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-21T15:06:26.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:25 smithi078 systemd[1]: ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-21T15:06:26.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:25 smithi078 systemd[1]: Stopped Ceph mon.a for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:06:26.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:25 smithi078 systemd[1]: Starting Ceph mon.a for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:06:26.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:25 smithi078 podman[31978]: 2024-05-21 15:06:25.934865891 +0000 UTC m=+0.101193744 container create 1cd58a23b21134cfd745909164202cfacf7cf75c5970da8614312a181c4c509b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:06:26.043 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:25 smithi078 podman[31978]: 2024-05-21 15:06:25.850796599 +0000 UTC m=+0.017124456 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:06:26.187 INFO:teuthology.orchestra.run.smithi078.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-21T15:06:26.409 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 podman[31978]: 2024-05-21 15:06:26.121019797 +0000 UTC m=+0.287347649 container init 1cd58a23b21134cfd745909164202cfacf7cf75c5970da8614312a181c4c509b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514) 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 podman[31978]: 2024-05-21 15:06:26.124246703 +0000 UTC m=+0.290574555 container start 1cd58a23b21134cfd745909164202cfacf7cf75c5970da8614312a181c4c509b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: set uid:gid to 167:167 (ceph:ceph) 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev), process ceph-mon, pid 2 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: pidfile_write: ignore empty --pid-file 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: load: jerasure load: lrc 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: RocksDB version: 7.9.2 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Git sha 0 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Compile date 2024-05-17 21:21:31 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: DB SUMMARY 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: DB Session ID: 864RXQ5H81BSDJ9LCBM8 2024-05-21T15:06:26.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: CURRENT file: CURRENT 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106534 ; 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.error_if_exists: 0 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.create_if_missing: 0 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.paranoid_checks: 1 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.env: 0x55ffcaec84a0 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.info_log: 0x55ffcc553d40 2024-05-21T15:06:26.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.statistics: (nil) 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.use_fsync: 0 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_log_file_size: 0 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.allow_fallocate: 1 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T15:06:26.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.use_direct_reads: 0 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.db_log_dir: 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.wal_dir: 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T15:06:26.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.write_buffer_manager: 0x55ffcc5272c0 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T15:06:26.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.unordered_write: 0 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.row_cache: None 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.wal_filter: None 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.two_write_queues: 0 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.wal_compression: 0 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.atomic_flush: 0 2024-05-21T15:06:26.416 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.log_readahead_size: 0 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_background_jobs: 2 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_background_compactions: -1 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_subcompactions: 1 2024-05-21T15:06:26.417 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_open_files: -1 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T15:06:26.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_background_flushes: -1 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Compression algorithms supported: 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: kZSTD supported: 0 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: kXpressCompression supported: 0 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: kZlibCompression supported: 1 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: kSnappyCompression supported: 1 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: kLZ4Compression supported: 1 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: kBZip2Compression supported: 0 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T15:06:26.419 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.merge_operator: 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_filter: None 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ffcc553e40) 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_type: 0 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_index_type: 0 2024-05-21T15:06:26.420 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_shortening: 1 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: checksum: 4 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: no_block_cache: 0 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache: 0x55ffcc557090 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_options: 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: capacity : 536870912 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_shard_bits : 4 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: strict_capacity_limit : 0 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_compressed: (nil) 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: persistent_cache: (nil) 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size: 4096 2024-05-21T15:06:26.421 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size_deviation: 10 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_restart_interval: 16 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_block_restart_interval: 1 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: metadata_block_size: 4096 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: partition_filters: 0 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: use_delta_encoding: 1 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: filter_policy: bloomfilter 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: whole_key_filtering: 1 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: verify_compression: 0 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: format_version: 5 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_index_compression: 1 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_align: 0 2024-05-21T15:06:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout: max_auto_readahead_size: 262144 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout: prepopulate_block_cache: 0 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout: initial_auto_readahead_size: 8192 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression: NoCompression 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.num_levels: 7 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T15:06:26.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T15:06:26.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T15:06:26.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T15:06:26.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.table_properties_collectors: 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.inplace_update_support: 0 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.bloom_locality: 0 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.max_successive_merges: 0 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T15:06:26.427 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.ttl: 2592000 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.enable_blob_files: false 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.min_blob_size: 0 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T15:06:26.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2059f3c8-5f1c-47aa-a3aa-b4b28df1a1fd 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716303986163277, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716303986164435, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 99372, "index_size": 316, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14633, "raw_average_key_size": 49, "raw_value_size": 91446, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716303986, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2059f3c8-5f1c-47aa-a3aa-b4b28df1a1fd", "db_session_id": "864RXQ5H81BSDJ9LCBM8", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716303986164517, "job": 1, "event": "recovery_finished"} 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ffcc53dc00 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: DB pointer 0x55ffd132a000 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T15:06:26.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** DB Stats ** 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T15:06:26.430 INFO:journalctl@ceph.mon.a.smithi078.stdout: L0 2/0 100.93 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: Sum 2/0 100.93 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T15:06:26.431 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative compaction: 0.00 GB write, 30.36 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval compaction: 0.00 GB write, 30.36 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache BinnedLRUCache@0x55ffcc557090#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: starting mon.a rank 0 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a@-1(???) e1 preinit fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a@-1(???).mds e1 new map 2024-05-21T15:06:26.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a@-1(???).mds e1 print_map 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout: e1 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout: legacy client fscid: -1 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout: No filesystems configured 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 bash[31978]: 1cd58a23b21134cfd745909164202cfacf7cf75c5970da8614312a181c4c509b 2024-05-21T15:06:26.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 systemd[1]: Started Ceph mon.a for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: monmap epoch 1 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: last_changed 2024-05-21T15:06:17.120039+0000 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: min_mon_release 19 (squid) 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: election_strategy: 1 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: fsmap 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: osdmap e1: 0 total, 0 up, 0 in 2024-05-21T15:06:26.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:26 smithi078 ceph-mon[31997]: mgrmap e1: no daemons active 2024-05-21T15:06:27.253 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.619+0000 7f2b1e731700 1 Processor -- start 2024-05-21T15:06:27.253 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.619+0000 7f2b1e731700 1 -- start start 2024-05-21T15:06:27.253 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.619+0000 7f2b1e731700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b18108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:27.253 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.619+0000 7f2b1e731700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18108630 con 0x7f2b18107c30 2024-05-21T15:06:27.253 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.620+0000 7f2b17fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b18108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:27.253 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.620+0000 7f2b17fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b18108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34182/0 (socket says 172.21.15.78:34182) 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.620+0000 7f2b17fff700 1 -- 172.21.15.78:0/3449163893 learned_addr learned my addr 172.21.15.78:0/3449163893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.620+0000 7f2b17fff700 1 -- 172.21.15.78:0/3449163893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b18108ec0 con 0x7f2b18107c30 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.620+0000 7f2b17fff700 1 --2- 172.21.15.78:0/3449163893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b18108040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2b080098a0 tx=0x7f2b08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f8b9d88d02ec00 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.620+0000 7f2b16ffd700 1 -- 172.21.15.78:0/3449163893 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b08004160 con 0x7f2b18107c30 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b16ffd700 1 -- 172.21.15.78:0/3449163893 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2b08004300 con 0x7f2b18107c30 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b16ffd700 1 -- 172.21.15.78:0/3449163893 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b08011b50 con 0x7f2b18107c30 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b1e731700 1 -- 172.21.15.78:0/3449163893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 msgr2=0x7f2b18108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b1e731700 1 --2- 172.21.15.78:0/3449163893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b18108040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2b080098a0 tx=0x7f2b08009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b1e731700 1 -- 172.21.15.78:0/3449163893 shutdown_connections 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b1e731700 1 --2- 172.21.15.78:0/3449163893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b18108040 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b1e731700 1 -- 172.21.15.78:0/3449163893 >> 172.21.15.78:0/3449163893 conn(0x7f2b18103330 msgr2=0x7f2b181057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b1e731700 1 -- 172.21.15.78:0/3449163893 shutdown_connections 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.621+0000 7f2b1e731700 1 -- 172.21.15.78:0/3449163893 wait complete. 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.622+0000 7f2b1e731700 1 Processor -- start 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.622+0000 7f2b1e731700 1 -- start start 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.622+0000 7f2b1e731700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b1819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.622+0000 7f2b1e731700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1810c270 con 0x7f2b18107c30 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.622+0000 7f2b17fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b1819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.622+0000 7f2b17fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b1819c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34196/0 (socket says 172.21.15.78:34196) 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.622+0000 7f2b17fff700 1 -- 172.21.15.78:0/204581321 learned_addr learned my addr 172.21.15.78:0/204581321 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b17fff700 1 -- 172.21.15.78:0/204581321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1819c650 con 0x7f2b18107c30 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b17fff700 1 --2- 172.21.15.78:0/204581321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b1819c110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009fb0 tx=0x7f2b08011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b157fa700 1 -- 172.21.15.78:0/204581321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b080038b0 con 0x7f2b18107c30 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1819c8e0 con 0x7f2b18107c30 2024-05-21T15:06:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b157fa700 1 -- 172.21.15.78:0/204581321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2b08011e50 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b157fa700 1 -- 172.21.15.78:0/204581321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b0801a430 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1819f4f0 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b157fa700 1 -- 172.21.15.78:0/204581321 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2b0801aa80 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.623+0000 7f2b157fa700 1 -- 172.21.15.78:0/204581321 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2b08019700 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.624+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b18063ca0 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.626+0000 7f2b157fa700 1 -- 172.21.15.78:0/204581321 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2b08003a50 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.695+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f2b1819f6c0 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.697+0000 7f2b157fa700 1 -- 172.21.15.78:0/204581321 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f2b0802dcb0 con 0x7f2b18107c30 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.698+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 msgr2=0x7f2b1819c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.698+0000 7f2b1e731700 1 --2- 172.21.15.78:0/204581321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b1819c110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009fb0 tx=0x7f2b08011a00 comp rx=0 tx=0).stop 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.699+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 shutdown_connections 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.699+0000 7f2b1e731700 1 --2- 172.21.15.78:0/204581321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b18107c30 0x7f2b1819c110 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:27.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.699+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 >> 172.21.15.78:0/204581321 conn(0x7f2b18103330 msgr2=0x7f2b18103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:27.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.699+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 shutdown_connections 2024-05-21T15:06:27.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:26.699+0000 7f2b1e731700 1 -- 172.21.15.78:0/204581321 wait complete. 2024-05-21T15:06:27.257 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-21T15:06:27.271 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:06:27.271 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mgr... 2024-05-21T15:06:27.271 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-21T15:06:27.272 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-21T15:06:27.524 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T15:06:27.524 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-21T15:06:27.675 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-852c9762-1783-11ef-bc9a-c7b262605968.target.wants/ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-852c9762-1783-11ef-bc9a-c7b262605968@.service. 2024-05-21T15:06:28.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:27 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/204581321' entity='client.admin' 2024-05-21T15:06:28.189 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T15:06:28.190 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-05-21T15:06:28.197 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T15:06:28.197 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-05-21T15:06:28.197 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-05-21T15:06:28.197 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T15:06:28.205 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T15:06:28.206 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-05-21T15:06:28.213 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T15:06:28.213 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-05-21T15:06:28.213 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-05-21T15:06:28.213 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-21T15:06:28.213 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr to start... 2024-05-21T15:06:28.214 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr... 2024-05-21T15:06:28.276 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:28 smithi078 systemd[1]: Started Ceph mgr.x for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:06:28.537 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:28 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:28.273+0000 7f3ac4c4e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:06:28.537 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:28 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:28.466+0000 7f3ac4c4e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:06:28.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:28 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/336234044' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T15:06:28.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:28 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:28.570+0000 7f3ac4c4e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:06:29.307 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:28 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:28.892+0000 7f3ac4c4e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:06:29.307 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:28 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:28.978+0000 7f3ac4c4e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:29.320 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T15:06:29.321 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T15:06:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T15:06:29.323 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T15:06:20.985158+0000", 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.645+0000 7fba9bb13700 1 Processor -- start 2024-05-21T15:06:29.324 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.645+0000 7fba9bb13700 1 -- start start 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.645+0000 7fba9bb13700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba94109630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.645+0000 7fba9bb13700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba94109c00 con 0x7fba94109220 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.646+0000 7fba998af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba94109630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.646+0000 7fba998af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba94109630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35512/0 (socket says 172.21.15.78:35512) 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.646+0000 7fba998af700 1 -- 172.21.15.78:0/3579686792 learned_addr learned my addr 172.21.15.78:0/3579686792 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.646+0000 7fba998af700 1 -- 172.21.15.78:0/3579686792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba94109d80 con 0x7fba94109220 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.646+0000 7fba998af700 1 --2- 172.21.15.78:0/3579686792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba94109630 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fba840098a0 tx=0x7fba84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c406e23dafc637d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.646+0000 7fba988ad700 1 -- 172.21.15.78:0/3579686792 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba84004160 con 0x7fba94109220 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.647+0000 7fba988ad700 1 -- 172.21.15.78:0/3579686792 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba84004300 con 0x7fba94109220 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.647+0000 7fba988ad700 1 -- 172.21.15.78:0/3579686792 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba84011b50 con 0x7fba94109220 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.647+0000 7fba9bb13700 1 -- 172.21.15.78:0/3579686792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 msgr2=0x7fba94109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:29.325 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.647+0000 7fba9bb13700 1 --2- 172.21.15.78:0/3579686792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba94109630 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fba840098a0 tx=0x7fba84009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.647+0000 7fba9bb13700 1 -- 172.21.15.78:0/3579686792 shutdown_connections 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.647+0000 7fba9bb13700 1 --2- 172.21.15.78:0/3579686792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba94109630 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.647+0000 7fba9bb13700 1 -- 172.21.15.78:0/3579686792 >> 172.21.15.78:0/3579686792 conn(0x7fba94102aa0 msgr2=0x7fba94104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.648+0000 7fba9bb13700 1 -- 172.21.15.78:0/3579686792 shutdown_connections 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.648+0000 7fba9bb13700 1 -- 172.21.15.78:0/3579686792 wait complete. 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.648+0000 7fba9bb13700 1 Processor -- start 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.648+0000 7fba9bb13700 1 -- start start 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.648+0000 7fba9bb13700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba9419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba9bb13700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba9410c1a0 con 0x7fba94109220 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba998af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba9419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba998af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba9419c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35524/0 (socket says 172.21.15.78:35524) 2024-05-21T15:06:29.326 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba998af700 1 -- 172.21.15.78:0/336234044 learned_addr learned my addr 172.21.15.78:0/336234044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba998af700 1 -- 172.21.15.78:0/336234044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9419c600 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba998af700 1 --2- 172.21.15.78:0/336234044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba9419c0c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba840046e0 tx=0x7fba84011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba8affd700 1 -- 172.21.15.78:0/336234044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba840039e0 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba8affd700 1 -- 172.21.15.78:0/336234044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba84003b80 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9419c890 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba8affd700 1 -- 172.21.15.78:0/336234044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba840247c0 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.649+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9419f4a0 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.650+0000 7fba8affd700 1 -- 172.21.15.78:0/336234044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fba84022020 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.650+0000 7fba8affd700 1 -- 172.21.15.78:0/336234044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fba84011e00 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.650+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba94063ca0 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.653+0000 7fba8affd700 1 -- 172.21.15.78:0/336234044 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fba8401ab10 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.723+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fba94106e30 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.724+0000 7fba8affd700 1 -- 172.21.15.78:0/336234044 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fba8401a430 con 0x7fba94109220 2024-05-21T15:06:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.724+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 msgr2=0x7fba9419c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:29.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.724+0000 7fba9bb13700 1 --2- 172.21.15.78:0/336234044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba9419c0c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fba840046e0 tx=0x7fba84011c10 comp rx=0 tx=0).stop 2024-05-21T15:06:29.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.725+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 shutdown_connections 2024-05-21T15:06:29.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.725+0000 7fba9bb13700 1 --2- 172.21.15.78:0/336234044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba94109220 0x7fba9419c0c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:29.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.725+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 >> 172.21.15.78:0/336234044 conn(0x7fba94102aa0 msgr2=0x7fba941033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:29.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.725+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 shutdown_connections 2024-05-21T15:06:29.328 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:28.725+0000 7fba9bb13700 1 -- 172.21.15.78:0/336234044 wait complete. 2024-05-21T15:06:29.328 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (1/15)... 2024-05-21T15:06:30.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:30 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:30.211+0000 7f3ac4c4e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:06:30.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:30 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:30.387+0000 7f3ac4c4e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:06:30.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:30 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:30.561+0000 7f3ac4c4e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:06:31.158 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:30 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:30.856+0000 7f3ac4c4e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:06:31.158 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:30 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:30.951+0000 7f3ac4c4e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:06:31.158 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:31 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:31.155+0000 7f3ac4c4e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:06:31.444 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:31 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:31.353+0000 7f3ac4c4e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:06:32.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:31 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2733913875' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T15:06:32.098 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:31 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:31.980+0000 7f3ac4c4e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T15:06:32.435 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T15:06:32.436 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T15:06:32.437 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T15:06:32.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:32.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:32.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T15:06:20.985158+0000", 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.768+0000 7fa255d0d700 1 Processor -- start 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.768+0000 7fa255d0d700 1 -- start start 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.768+0000 7fa255d0d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa250107420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.768+0000 7fa255d0d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501079f0 con 0x7fa250107010 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.769+0000 7fa24f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa250107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.769+0000 7fa24f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa250107420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35528/0 (socket says 172.21.15.78:35528) 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.769+0000 7fa24f7fe700 1 -- 172.21.15.78:0/2238134423 learned_addr learned my addr 172.21.15.78:0/2238134423 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:32.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.769+0000 7fa24f7fe700 1 -- 172.21.15.78:0/2238134423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa250107b70 con 0x7fa250107010 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.769+0000 7fa24f7fe700 1 --2- 172.21.15.78:0/2238134423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa250107420 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa2440098a0 tx=0x7fa244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cab66359edaa3ad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.769+0000 7fa24e7fc700 1 -- 172.21.15.78:0/2238134423 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa244004160 con 0x7fa250107010 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.769+0000 7fa24e7fc700 1 -- 172.21.15.78:0/2238134423 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa244004300 con 0x7fa250107010 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.769+0000 7fa24e7fc700 1 -- 172.21.15.78:0/2238134423 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa244011b50 con 0x7fa250107010 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.770+0000 7fa255d0d700 1 -- 172.21.15.78:0/2238134423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 msgr2=0x7fa250107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.770+0000 7fa255d0d700 1 --2- 172.21.15.78:0/2238134423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa250107420 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa2440098a0 tx=0x7fa244009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.770+0000 7fa255d0d700 1 -- 172.21.15.78:0/2238134423 shutdown_connections 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.770+0000 7fa255d0d700 1 --2- 172.21.15.78:0/2238134423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa250107420 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.770+0000 7fa255d0d700 1 -- 172.21.15.78:0/2238134423 >> 172.21.15.78:0/2238134423 conn(0x7fa250102aa0 msgr2=0x7fa250104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.770+0000 7fa255d0d700 1 -- 172.21.15.78:0/2238134423 shutdown_connections 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.770+0000 7fa255d0d700 1 -- 172.21.15.78:0/2238134423 wait complete. 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.771+0000 7fa255d0d700 1 Processor -- start 2024-05-21T15:06:32.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.771+0000 7fa255d0d700 1 -- start start 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.771+0000 7fa255d0d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa25019d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.771+0000 7fa255d0d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa25010d300 con 0x7fa250107010 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.771+0000 7fa24f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa25019d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.771+0000 7fa24f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa25019d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35532/0 (socket says 172.21.15.78:35532) 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.771+0000 7fa24f7fe700 1 -- 172.21.15.78:0/2733913875 learned_addr learned my addr 172.21.15.78:0/2733913875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa24f7fe700 1 -- 172.21.15.78:0/2733913875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25019d700 con 0x7fa250107010 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa24f7fe700 1 --2- 172.21.15.78:0/2733913875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa25019d1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2440046e0 tx=0x7fa244011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa24cff9700 1 -- 172.21.15.78:0/2733913875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2440039e0 con 0x7fa250107010 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25019d990 con 0x7fa250107010 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa24cff9700 1 -- 172.21.15.78:0/2733913875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa244003b80 con 0x7fa250107010 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa24cff9700 1 -- 172.21.15.78:0/2733913875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2440247c0 con 0x7fa250107010 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2501a05a0 con 0x7fa250107010 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa24cff9700 1 -- 172.21.15.78:0/2733913875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa244022020 con 0x7fa250107010 2024-05-21T15:06:32.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.772+0000 7fa24cff9700 1 -- 172.21.15.78:0/2733913875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa244011e00 con 0x7fa250107010 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.773+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa250063ca0 con 0x7fa250107010 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.775+0000 7fa24cff9700 1 -- 172.21.15.78:0/2733913875 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa24401ab10 con 0x7fa250107010 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.848+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa2501a07c0 con 0x7fa250107010 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.849+0000 7fa24cff9700 1 -- 172.21.15.78:0/2733913875 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa24401a430 con 0x7fa250107010 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.850+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 msgr2=0x7fa25019d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.850+0000 7fa255d0d700 1 --2- 172.21.15.78:0/2733913875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa25019d1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2440046e0 tx=0x7fa244011c10 comp rx=0 tx=0).stop 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.850+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 shutdown_connections 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.850+0000 7fa255d0d700 1 --2- 172.21.15.78:0/2733913875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa250107010 0x7fa25019d1c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.850+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 >> 172.21.15.78:0/2733913875 conn(0x7fa250102aa0 msgr2=0x7fa250103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.850+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 shutdown_connections 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:31.850+0000 7fa255d0d700 1 -- 172.21.15.78:0/2733913875 wait complete. 2024-05-21T15:06:32.449 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (2/15)... 2024-05-21T15:06:32.506 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:32 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:32.404+0000 7f3ac4c4e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:06:32.507 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:32 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:32.503+0000 7f3ac4c4e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:06:32.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:32 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:32.702+0000 7f3ac4c4e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:06:33.549 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:33.216+0000 7f3ac4c4e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:06:33.549 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:33.306+0000 7f3ac4c4e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:06:33.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:33.545+0000 7f3ac4c4e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:06:33.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:33.678+0000 7f3ac4c4e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:06:33.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:33.772+0000 7f3ac4c4e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:06:34.188 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:33.888+0000 7f3ac4c4e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:06:34.189 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:34 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:34.185+0000 7f3ac4c4e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:06:34.835 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:34 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:34.799+0000 7f3ac4c4e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:06:35.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:34 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4162566162' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T15:06:35.098 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:35 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:35.022+0000 7f3ac4c4e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:06:35.457 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:35.457 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:35.458 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T15:06:35.459 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T15:06:35.460 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T15:06:35.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T15:06:35.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T15:06:35.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T15:06:35.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T15:06:20.985158+0000", 2024-05-21T15:06:35.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.832+0000 7fdb63025700 1 Processor -- start 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.832+0000 7fdb63025700 1 -- start start 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.833+0000 7fdb63025700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.833+0000 7fdb63025700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c108630 con 0x7fdb5c107c30 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.833+0000 7fdb60dc1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.833+0000 7fdb60dc1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35548/0 (socket says 172.21.15.78:35548) 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.833+0000 7fdb60dc1700 1 -- 172.21.15.78:0/1308865752 learned_addr learned my addr 172.21.15.78:0/1308865752 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.833+0000 7fdb60dc1700 1 -- 172.21.15.78:0/1308865752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c108ec0 con 0x7fdb5c107c30 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.833+0000 7fdb60dc1700 1 --2- 172.21.15.78:0/1308865752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c108040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb500098a0 tx=0x7fdb50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5490ecec9cf242 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.834+0000 7fdb5b7fe700 1 -- 172.21.15.78:0/1308865752 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb50004160 con 0x7fdb5c107c30 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.834+0000 7fdb5b7fe700 1 -- 172.21.15.78:0/1308865752 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdb50004300 con 0x7fdb5c107c30 2024-05-21T15:06:35.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.834+0000 7fdb5b7fe700 1 -- 172.21.15.78:0/1308865752 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb50011b50 con 0x7fdb5c107c30 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.834+0000 7fdb63025700 1 -- 172.21.15.78:0/1308865752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 msgr2=0x7fdb5c108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.834+0000 7fdb63025700 1 --2- 172.21.15.78:0/1308865752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c108040 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb500098a0 tx=0x7fdb50009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.834+0000 7fdb63025700 1 -- 172.21.15.78:0/1308865752 shutdown_connections 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.834+0000 7fdb63025700 1 --2- 172.21.15.78:0/1308865752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c108040 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.835+0000 7fdb63025700 1 -- 172.21.15.78:0/1308865752 >> 172.21.15.78:0/1308865752 conn(0x7fdb5c103330 msgr2=0x7fdb5c1057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.835+0000 7fdb63025700 1 -- 172.21.15.78:0/1308865752 shutdown_connections 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.835+0000 7fdb63025700 1 -- 172.21.15.78:0/1308865752 wait complete. 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.835+0000 7fdb63025700 1 Processor -- start 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.835+0000 7fdb63025700 1 -- start start 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.835+0000 7fdb63025700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c19d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb63025700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c10c270 con 0x7fdb5c107c30 2024-05-21T15:06:35.470 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb60dc1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c19d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb60dc1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c19d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35552/0 (socket says 172.21.15.78:35552) 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb60dc1700 1 -- 172.21.15.78:0/4162566162 learned_addr learned my addr 172.21.15.78:0/4162566162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb60dc1700 1 -- 172.21.15.78:0/4162566162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c19d790 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb60dc1700 1 --2- 172.21.15.78:0/4162566162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c19d250 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdb50004750 tx=0x7fdb50011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb59ffb700 1 -- 172.21.15.78:0/4162566162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb500039e0 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb59ffb700 1 -- 172.21.15.78:0/4162566162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdb50003b80 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c19da20 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.836+0000 7fdb59ffb700 1 -- 172.21.15.78:0/4162566162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb500246e0 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.837+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1a0630 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.837+0000 7fdb59ffb700 1 -- 172.21.15.78:0/4162566162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb50022020 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.837+0000 7fdb59ffb700 1 -- 172.21.15.78:0/4162566162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdb50011e00 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.838+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5c063ca0 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.840+0000 7fdb59ffb700 1 -- 172.21.15.78:0/4162566162 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdb50043dd0 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.911+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdb5c1a0800 con 0x7fdb5c107c30 2024-05-21T15:06:35.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.911+0000 7fdb59ffb700 1 -- 172.21.15.78:0/4162566162 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdb50032020 con 0x7fdb5c107c30 2024-05-21T15:06:35.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.912+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 msgr2=0x7fdb5c19d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:35.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.912+0000 7fdb63025700 1 --2- 172.21.15.78:0/4162566162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c19d250 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdb50004750 tx=0x7fdb50011930 comp rx=0 tx=0).stop 2024-05-21T15:06:35.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.912+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 shutdown_connections 2024-05-21T15:06:35.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.912+0000 7fdb63025700 1 --2- 172.21.15.78:0/4162566162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c107c30 0x7fdb5c19d250 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:35.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.912+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 >> 172.21.15.78:0/4162566162 conn(0x7fdb5c103330 msgr2=0x7fdb5c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:35.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.912+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 shutdown_connections 2024-05-21T15:06:35.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:34.912+0000 7fdb63025700 1 -- 172.21.15.78:0/4162566162 wait complete. 2024-05-21T15:06:35.472 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (3/15)... 2024-05-21T15:06:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: Activating manager daemon x 2024-05-21T15:06:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: mgrmap e2: x(active, starting, since 0.00357237s) 2024-05-21T15:06:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:06:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:06:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:06:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:06:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:06:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: Manager daemon x is now available 2024-05-21T15:06:36.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:06:36.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:06:36.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' 2024-05-21T15:06:36.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' 2024-05-21T15:06:36.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:35 smithi078 ceph-mon[31997]: from='mgr.14100 172.21.15.78:0/2707556783' entity='mgr.x' 2024-05-21T15:06:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:37 smithi078 ceph-mon[31997]: mgrmap e3: x(active, since 1.01009s) 2024-05-21T15:06:38.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:38 smithi078 ceph-mon[31997]: mgrmap e4: x(active, since 2s) 2024-05-21T15:06:38.788 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:38.788 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:06:38.788 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:06:38.788 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T15:06:38.788 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T15:06:38.788 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T15:06:38.788 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T15:06:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:38.790 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T15:06:38.790 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:38.790 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T15:06:38.790 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T15:06:38.790 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T15:06:38.797 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T15:06:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T15:06:38.799 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T15:06:20.985158+0000", 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.870+0000 7f6447441700 1 Processor -- start 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.870+0000 7f6447441700 1 -- start start 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.870+0000 7f6447441700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f6440108020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:38.800 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.870+0000 7f6447441700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6440108610 con 0x7f6440107c10 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.870+0000 7f64451dd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f6440108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.870+0000 7f64451dd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f6440108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35628/0 (socket says 172.21.15.78:35628) 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.870+0000 7f64451dd700 1 -- 172.21.15.78:0/1394333718 learned_addr learned my addr 172.21.15.78:0/1394333718 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.871+0000 7f64451dd700 1 -- 172.21.15.78:0/1394333718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6440108ea0 con 0x7f6440107c10 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.871+0000 7f64451dd700 1 --2- 172.21.15.78:0/1394333718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f6440108020 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f643c00bb40 tx=0x7f643c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff52ae5f1d3e3fb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.871+0000 7f6437fff700 1 -- 172.21.15.78:0/1394333718 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f643c00dbe0 con 0x7f6440107c10 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.871+0000 7f6437fff700 1 -- 172.21.15.78:0/1394333718 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f643c0044f0 con 0x7f6440107c10 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.871+0000 7f6437fff700 1 -- 172.21.15.78:0/1394333718 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f643c010470 con 0x7f6440107c10 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.872+0000 7f6447441700 1 -- 172.21.15.78:0/1394333718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 msgr2=0x7f6440108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.872+0000 7f6447441700 1 --2- 172.21.15.78:0/1394333718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f6440108020 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f643c00bb40 tx=0x7f643c00be50 comp rx=0 tx=0).stop 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.872+0000 7f6447441700 1 -- 172.21.15.78:0/1394333718 shutdown_connections 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.872+0000 7f6447441700 1 --2- 172.21.15.78:0/1394333718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f6440108020 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:38.801 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.872+0000 7f6447441700 1 -- 172.21.15.78:0/1394333718 >> 172.21.15.78:0/1394333718 conn(0x7f64401033d0 msgr2=0x7f6440105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.872+0000 7f6447441700 1 -- 172.21.15.78:0/1394333718 shutdown_connections 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.872+0000 7f6447441700 1 -- 172.21.15.78:0/1394333718 wait complete. 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f6447441700 1 Processor -- start 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f6447441700 1 -- start start 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f6447441700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f644019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f6447441700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f644007afb0 con 0x7f6440107c10 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f64451dd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f644019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f64451dd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f644019af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35634/0 (socket says 172.21.15.78:35634) 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f64451dd700 1 -- 172.21.15.78:0/3244326565 learned_addr learned my addr 172.21.15.78:0/3244326565 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f64451dd700 1 -- 172.21.15.78:0/3244326565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644019b4d0 con 0x7f6440107c10 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f64451dd700 1 --2- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f644019af90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f643c003c50 tx=0x7f643c0040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f64367fc700 1 -- 172.21.15.78:0/3244326565 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f643c027070 con 0x7f6440107c10 2024-05-21T15:06:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.873+0000 7f64367fc700 1 -- 172.21.15.78:0/3244326565 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f643c0104e0 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.874+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644019b760 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.874+0000 7f64367fc700 1 -- 172.21.15.78:0/3244326565 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f643c022c10 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.874+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644019e370 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.874+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6440063ca0 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.875+0000 7f64367fc700 1 -- 172.21.15.78:0/3244326565 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f643c02a070 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.875+0000 7f64367fc700 1 --2- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f642c039000 0x7f642c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.875+0000 7f64367fc700 1 -- 172.21.15.78:0/3244326565 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f643c01f070 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.875+0000 7f64449dc700 1 --2- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f642c039000 0x7f642c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.875+0000 7f64449dc700 1 --2- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f642c039000 0x7f642c03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6430006fd0 tx=0x7f6430006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:37.880+0000 7f64367fc700 1 -- 172.21.15.78:0/3244326565 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f643c015070 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.152+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f64401941a0 con 0x7f6440107c10 2024-05-21T15:06:38.803 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.152+0000 7f64367fc700 1 -- 172.21.15.78:0/3244326565 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f64401941a0 con 0x7f6440107c10 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f642c039000 msgr2=0x7f642c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 --2- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f642c039000 0x7f642c03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6430006fd0 tx=0x7f6430006e40 comp rx=0 tx=0).stop 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 msgr2=0x7f644019af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 --2- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f644019af90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f643c003c50 tx=0x7f643c0040b0 comp rx=0 tx=0).stop 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 shutdown_connections 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 --2- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f642c039000 0x7f642c03b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 --2- 172.21.15.78:0/3244326565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6440107c10 0x7f644019af90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 >> 172.21.15.78:0/3244326565 conn(0x7f64401033d0 msgr2=0x7f6440191260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 shutdown_connections 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:38.155+0000 7f6447441700 1 -- 172.21.15.78:0/3244326565 wait complete. 2024-05-21T15:06:38.804 INFO:teuthology.orchestra.run.smithi078.stdout:mgr is available 2024-05-21T15:06:39.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:39 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3244326565' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T15:06:40.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:40.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T15:06:40.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:40.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T15:06:40.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T15:06:40.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T15:06:40.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.219+0000 7f7fd1814700 1 Processor -- start 2024-05-21T15:06:40.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.219+0000 7f7fd1814700 1 -- start start 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.219+0000 7f7fd1814700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc10b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.219+0000 7f7fd1814700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10bf10 con 0x7f7fcc0761f0 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.219+0000 7f7fcaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc10b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.219+0000 7f7fcaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc10b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43448/0 (socket says 172.21.15.78:43448) 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.219+0000 7f7fcaffd700 1 -- 172.21.15.78:0/3234075754 learned_addr learned my addr 172.21.15.78:0/3234075754 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.219+0000 7f7fcaffd700 1 -- 172.21.15.78:0/3234075754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc10c090 con 0x7f7fcc0761f0 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.220+0000 7f7fcaffd700 1 --2- 172.21.15.78:0/3234075754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc10b940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098a0 tx=0x7f7fbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6a109c132c023c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.220+0000 7f7fc9ffb700 1 -- 172.21.15.78:0/3234075754 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fbc004160 con 0x7f7fcc0761f0 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.220+0000 7f7fc9ffb700 1 -- 172.21.15.78:0/3234075754 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fbc004300 con 0x7f7fcc0761f0 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.220+0000 7f7fc9ffb700 1 -- 172.21.15.78:0/3234075754 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fbc011b50 con 0x7f7fcc0761f0 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.220+0000 7f7fd1814700 1 -- 172.21.15.78:0/3234075754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 msgr2=0x7f7fcc10b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:40.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.220+0000 7f7fd1814700 1 --2- 172.21.15.78:0/3234075754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc10b940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098a0 tx=0x7f7fbc009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.221+0000 7f7fd1814700 1 -- 172.21.15.78:0/3234075754 shutdown_connections 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.221+0000 7f7fd1814700 1 --2- 172.21.15.78:0/3234075754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc10b940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.221+0000 7f7fd1814700 1 -- 172.21.15.78:0/3234075754 >> 172.21.15.78:0/3234075754 conn(0x7f7fcc103330 msgr2=0x7f7fcc1057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.221+0000 7f7fd1814700 1 -- 172.21.15.78:0/3234075754 shutdown_connections 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.221+0000 7f7fd1814700 1 -- 172.21.15.78:0/3234075754 wait complete. 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.221+0000 7f7fd1814700 1 Processor -- start 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.221+0000 7f7fd1814700 1 -- start start 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fd1814700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc19d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fd1814700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10d170 con 0x7f7fcc0761f0 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fcaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fcaffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc19d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43462/0 (socket says 172.21.15.78:43462) 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fcaffd700 1 -- 172.21.15.78:0/3897849819 learned_addr learned my addr 172.21.15.78:0/3897849819 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:40.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fcaffd700 1 -- 172.21.15.78:0/3897849819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc19d690 con 0x7f7fcc0761f0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fcaffd700 1 --2- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc19d150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009fb0 tx=0x7f7fbc011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fd0812700 1 -- 172.21.15.78:0/3897849819 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fbc0038b0 con 0x7f7fcc0761f0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fd0812700 1 -- 172.21.15.78:0/3897849819 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fbc011e50 con 0x7f7fcc0761f0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc19d920 con 0x7f7fcc0761f0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fd0812700 1 -- 172.21.15.78:0/3897849819 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fbc019420 con 0x7f7fcc0761f0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.222+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc1a0530 con 0x7f7fcc0761f0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.223+0000 7f7fd0812700 1 -- 172.21.15.78:0/3897849819 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f7fbc019720 con 0x7f7fcc0761f0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.223+0000 7f7fd0812700 1 --2- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f7fb4038fb0 0x7f7fb403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.223+0000 7f7fd0812700 1 -- 172.21.15.78:0/3897849819 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7fbc04e290 con 0x7f7fcc0761f0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.224+0000 7f7fca7fc700 1 --2- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f7fb4038fb0 0x7f7fb403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:40.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.224+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fcc063ca0 con 0x7f7fcc0761f0 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.224+0000 7f7fca7fc700 1 --2- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f7fb4038fb0 0x7f7fb403b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0006fd0 tx=0x7f7fc0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.229+0000 7f7fd0812700 1 -- 172.21.15.78:0/3897849819 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7fbc00bc70 con 0x7f7fcc0761f0 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.422+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7fcc1a0700 con 0x7f7fcc0761f0 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.424+0000 7f7fd0812700 1 -- 172.21.15.78:0/3897849819 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+398 (secure 0 0 0) 0x7f7fcc1a0700 con 0x7f7fcc0761f0 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f7fb4038fb0 msgr2=0x7f7fb403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 --2- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f7fb4038fb0 0x7f7fb403b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0006fd0 tx=0x7f7fc0006e40 comp rx=0 tx=0).stop 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 msgr2=0x7f7fcc19d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 --2- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc19d150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009fb0 tx=0x7f7fbc011bf0 comp rx=0 tx=0).stop 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 shutdown_connections 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 --2- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f7fb4038fb0 0x7f7fb403b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 --2- 172.21.15.78:0/3897849819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fcc0761f0 0x7f7fcc19d150 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 >> 172.21.15.78:0/3897849819 conn(0x7f7fcc103330 msgr2=0x7f7fcc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:40.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 shutdown_connections 2024-05-21T15:06:40.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:39.426+0000 7f7fd1814700 1 -- 172.21.15.78:0/3897849819 wait complete. 2024-05-21T15:06:40.056 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling cephadm module... 2024-05-21T15:06:40.294 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:40 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3897849819' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-21T15:06:41.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:41 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2879934034' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-21T15:06:41.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: ignoring --setuser ceph since I am not root 2024-05-21T15:06:41.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: ignoring --setgroup ceph since I am not root 2024-05-21T15:06:41.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:41.237+0000 7f53caf05200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:06:41.656 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:41.426+0000 7f53caf05200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:06:41.656 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:41.531+0000 7f53caf05200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.487+0000 7f6f8d50d700 1 Processor -- start 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.487+0000 7f6f8d50d700 1 -- start start 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.487+0000 7f6f8d50d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f881095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.487+0000 7f6f8d50d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f88109be0 con 0x7f6f881091a0 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.487+0000 7f6f86ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f881095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.488+0000 7f6f86ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f881095d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43472/0 (socket says 172.21.15.78:43472) 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.488+0000 7f6f86ffd700 1 -- 172.21.15.78:0/1442677448 learned_addr learned my addr 172.21.15.78:0/1442677448 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.488+0000 7f6f86ffd700 1 -- 172.21.15.78:0/1442677448 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f88109d60 con 0x7f6f881091a0 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.488+0000 7f6f86ffd700 1 --2- 172.21.15.78:0/1442677448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f881095d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6f780098a0 tx=0x7f6f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8591b18ba9fa0960 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:41.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.488+0000 7f6f867fc700 1 -- 172.21.15.78:0/1442677448 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f78004160 con 0x7f6f881091a0 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.488+0000 7f6f867fc700 1 -- 172.21.15.78:0/1442677448 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f78004300 con 0x7f6f881091a0 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.488+0000 7f6f867fc700 1 -- 172.21.15.78:0/1442677448 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f78011b50 con 0x7f6f881091a0 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.489+0000 7f6f8d50d700 1 -- 172.21.15.78:0/1442677448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 msgr2=0x7f6f881095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.489+0000 7f6f8d50d700 1 --2- 172.21.15.78:0/1442677448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f881095d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6f780098a0 tx=0x7f6f78009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.489+0000 7f6f8d50d700 1 -- 172.21.15.78:0/1442677448 shutdown_connections 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.489+0000 7f6f8d50d700 1 --2- 172.21.15.78:0/1442677448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f881095d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.489+0000 7f6f8d50d700 1 -- 172.21.15.78:0/1442677448 >> 172.21.15.78:0/1442677448 conn(0x7f6f88102ab0 msgr2=0x7f6f88104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.489+0000 7f6f8d50d700 1 -- 172.21.15.78:0/1442677448 shutdown_connections 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.489+0000 7f6f8d50d700 1 -- 172.21.15.78:0/1442677448 wait complete. 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.490+0000 7f6f8d50d700 1 Processor -- start 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.490+0000 7f6f8d50d700 1 -- start start 2024-05-21T15:06:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.490+0000 7f6f8d50d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f8819d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.490+0000 7f6f8d50d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8810c1b0 con 0x7f6f881091a0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.490+0000 7f6f86ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f8819d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.490+0000 7f6f86ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f8819d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43476/0 (socket says 172.21.15.78:43476) 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.490+0000 7f6f86ffd700 1 -- 172.21.15.78:0/2879934034 learned_addr learned my addr 172.21.15.78:0/2879934034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.490+0000 7f6f86ffd700 1 -- 172.21.15.78:0/2879934034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8819d6b0 con 0x7f6f881091a0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.491+0000 7f6f86ffd700 1 --2- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f8819d170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009fb0 tx=0x7f6f78011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.491+0000 7f6f84ff9700 1 -- 172.21.15.78:0/2879934034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f780038b0 con 0x7f6f881091a0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.491+0000 7f6f84ff9700 1 -- 172.21.15.78:0/2879934034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f78011e50 con 0x7f6f881091a0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.491+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8819d940 con 0x7f6f881091a0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.491+0000 7f6f84ff9700 1 -- 172.21.15.78:0/2879934034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f78019550 con 0x7f6f881091a0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.491+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f881a0550 con 0x7f6f881091a0 2024-05-21T15:06:41.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.492+0000 7f6f84ff9700 1 -- 172.21.15.78:0/2879934034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f6f78019800 con 0x7f6f881091a0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.492+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f88063ca0 con 0x7f6f881091a0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.492+0000 7f6f84ff9700 1 --2- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f6f74039000 0x7f6f7403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.492+0000 7f6f84ff9700 1 -- 172.21.15.78:0/2879934034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6f7804d1b0 con 0x7f6f881091a0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.492+0000 7f6f7e7fc700 1 --2- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f6f74039000 0x7f6f7403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.492+0000 7f6f7e7fc700 1 --2- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f6f74039000 0x7f6f7403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f70006fd0 tx=0x7f6f70006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.497+0000 7f6f84ff9700 1 -- 172.21.15.78:0/2879934034 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6f7800fda0 con 0x7f6f881091a0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:40.750+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f6f8802df60 con 0x7f6f881091a0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.038+0000 7f6f84ff9700 1 -- 172.21.15.78:0/2879934034 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f6f7802d960 con 0x7f6f881091a0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.040+0000 7f6f84ff9700 1 -- 172.21.15.78:0/2879934034 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6f7802d440 con 0x7f6f881091a0 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f6f74039000 msgr2=0x7f6f7403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 --2- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f6f74039000 0x7f6f7403b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f70006fd0 tx=0x7f6f70006e40 comp rx=0 tx=0).stop 2024-05-21T15:06:41.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 msgr2=0x7f6f8819d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:41.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 --2- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f8819d170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009fb0 tx=0x7f6f78011bf0 comp rx=0 tx=0).stop 2024-05-21T15:06:41.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 shutdown_connections 2024-05-21T15:06:41.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 --2- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f6f74039000 0x7f6f7403b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:41.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 --2- 172.21.15.78:0/2879934034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f881091a0 0x7f6f8819d170 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:41.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 >> 172.21.15.78:0/2879934034 conn(0x7f6f88102ab0 msgr2=0x7f6f88193740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:41.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.041+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 shutdown_connections 2024-05-21T15:06:41.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:41.042+0000 7f6f8d50d700 1 -- 172.21.15.78:0/2879934034 wait complete. 2024-05-21T15:06:41.942 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:41.848+0000 7f53caf05200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:06:41.942 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:41.939+0000 7f53caf05200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:06:42.292 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:42 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2879934034' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-21T15:06:42.292 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:42 smithi078 ceph-mon[31997]: mgrmap e5: x(active, since 6s) 2024-05-21T15:06:42.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:06:42.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-21T15:06:42.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T15:06:42.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T15:06:42.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T15:06:42.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:06:42.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.083+0000 7fb314ee3700 1 Processor -- start 2024-05-21T15:06:42.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.083+0000 7fb314ee3700 1 -- start start 2024-05-21T15:06:42.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.083+0000 7fb314ee3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:42.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.083+0000 7fb314ee3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb310108630 con 0x7fb310107c30 2024-05-21T15:06:42.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.083+0000 7fb30e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:42.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.083+0000 7fb30e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43510/0 (socket says 172.21.15.78:43510) 2024-05-21T15:06:42.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.083+0000 7fb30e59c700 1 -- 172.21.15.78:0/3492794935 learned_addr learned my addr 172.21.15.78:0/3492794935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:42.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.084+0000 7fb30e59c700 1 -- 172.21.15.78:0/3492794935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb310108ec0 con 0x7fb310107c30 2024-05-21T15:06:42.935 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.084+0000 7fb30e59c700 1 --2- 172.21.15.78:0/3492794935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310108040 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb3000098a0 tx=0x7fb300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182bdb61edf0eac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:42.935 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.084+0000 7fb30d59a700 1 -- 172.21.15.78:0/3492794935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb300004160 con 0x7fb310107c30 2024-05-21T15:06:42.935 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.084+0000 7fb30d59a700 1 -- 172.21.15.78:0/3492794935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb300004300 con 0x7fb310107c30 2024-05-21T15:06:42.935 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.084+0000 7fb30d59a700 1 -- 172.21.15.78:0/3492794935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb300011b50 con 0x7fb310107c30 2024-05-21T15:06:42.935 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.085+0000 7fb314ee3700 1 -- 172.21.15.78:0/3492794935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 msgr2=0x7fb310108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:42.935 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.085+0000 7fb314ee3700 1 --2- 172.21.15.78:0/3492794935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310108040 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb3000098a0 tx=0x7fb300009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:42.935 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.085+0000 7fb314ee3700 1 -- 172.21.15.78:0/3492794935 shutdown_connections 2024-05-21T15:06:42.936 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.085+0000 7fb314ee3700 1 --2- 172.21.15.78:0/3492794935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310108040 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:42.936 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.085+0000 7fb314ee3700 1 -- 172.21.15.78:0/3492794935 >> 172.21.15.78:0/3492794935 conn(0x7fb310103330 msgr2=0x7fb3101057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:42.936 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.085+0000 7fb314ee3700 1 -- 172.21.15.78:0/3492794935 shutdown_connections 2024-05-21T15:06:42.936 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.085+0000 7fb314ee3700 1 -- 172.21.15.78:0/3492794935 wait complete. 2024-05-21T15:06:42.936 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.086+0000 7fb314ee3700 1 Processor -- start 2024-05-21T15:06:42.936 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.086+0000 7fb314ee3700 1 -- start start 2024-05-21T15:06:42.936 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.086+0000 7fb314ee3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:42.936 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.086+0000 7fb314ee3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb31010c270 con 0x7fb310107c30 2024-05-21T15:06:42.937 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.086+0000 7fb30e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:42.937 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.086+0000 7fb30e59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43516/0 (socket says 172.21.15.78:43516) 2024-05-21T15:06:42.937 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.086+0000 7fb30e59c700 1 -- 172.21.15.78:0/2431316531 learned_addr learned my addr 172.21.15.78:0/2431316531 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:42.937 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.086+0000 7fb30e59c700 1 -- 172.21.15.78:0/2431316531 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3100763f0 con 0x7fb310107c30 2024-05-21T15:06:42.937 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.087+0000 7fb30e59c700 1 --2- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310075eb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb300009fb0 tx=0x7fb300011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:42.937 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.087+0000 7fb2ff7fe700 1 -- 172.21.15.78:0/2431316531 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3000038b0 con 0x7fb310107c30 2024-05-21T15:06:42.937 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.087+0000 7fb2ff7fe700 1 -- 172.21.15.78:0/2431316531 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb300011e50 con 0x7fb310107c30 2024-05-21T15:06:42.937 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.087+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31019ffc0 con 0x7fb310107c30 2024-05-21T15:06:42.938 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.087+0000 7fb2ff7fe700 1 -- 172.21.15.78:0/2431316531 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb30001a430 con 0x7fb310107c30 2024-05-21T15:06:42.938 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.087+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3101a0440 con 0x7fb310107c30 2024-05-21T15:06:42.938 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.088+0000 7fb2ff7fe700 1 -- 172.21.15.78:0/2431316531 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb30001a6f0 con 0x7fb310107c30 2024-05-21T15:06:42.938 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.088+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb310063ca0 con 0x7fb310107c30 2024-05-21T15:06:42.938 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.088+0000 7fb2ff7fe700 1 --2- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7fb2f8039000 0x7fb2f803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:42.938 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.088+0000 7fb2ff7fe700 1 -- 172.21.15.78:0/2431316531 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb30004d010 con 0x7fb310107c30 2024-05-21T15:06:42.938 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.088+0000 7fb30dd9b700 1 -- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7fb2f8039000 msgr2=0x7fb2f803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/888544333 2024-05-21T15:06:42.939 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.088+0000 7fb30dd9b700 1 --2- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7fb2f8039000 0x7fb2f803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:06:42.939 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.093+0000 7fb2ff7fe700 1 -- 172.21.15.78:0/2431316531 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb30000fdd0 con 0x7fb310107c30 2024-05-21T15:06:42.939 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.289+0000 7fb30dd9b700 1 -- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7fb2f8039000 msgr2=0x7fb2f803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/888544333 2024-05-21T15:06:42.939 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.289+0000 7fb30dd9b700 1 --2- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7fb2f8039000 0x7fb2f803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:06:42.939 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.381+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb3101a0d90 con 0x7fb310107c30 2024-05-21T15:06:42.939 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.382+0000 7fb2ff7fe700 1 -- 172.21.15.78:0/2431316531 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb300003a50 con 0x7fb310107c30 2024-05-21T15:06:42.939 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.389+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7fb2f8039000 msgr2=0x7fb2f803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:06:42.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.389+0000 7fb314ee3700 1 --2- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7fb2f8039000 0x7fb2f803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:42.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.389+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 msgr2=0x7fb310075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:42.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.389+0000 7fb314ee3700 1 --2- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310075eb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb300009fb0 tx=0x7fb300011a00 comp rx=0 tx=0).stop 2024-05-21T15:06:42.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.390+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 shutdown_connections 2024-05-21T15:06:42.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.390+0000 7fb314ee3700 1 --2- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7fb2f8039000 0x7fb2f803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:42.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.390+0000 7fb314ee3700 1 --2- 172.21.15.78:0/2431316531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb310107c30 0x7fb310075eb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:42.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.390+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 >> 172.21.15.78:0/2431316531 conn(0x7fb310103330 msgr2=0x7fb310103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:42.940 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.390+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 shutdown_connections 2024-05-21T15:06:42.941 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:42.391+0000 7fb314ee3700 1 -- 172.21.15.78:0/2431316531 wait complete. 2024-05-21T15:06:42.941 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-05-21T15:06:42.941 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 5... 2024-05-21T15:06:43.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:43 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2431316531' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-21T15:06:43.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:43 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:43.175+0000 7f53caf05200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:06:43.600 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:43 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:43.355+0000 7f53caf05200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:06:43.600 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:43 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:43.527+0000 7f53caf05200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:06:43.922 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:43 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:43.824+0000 7f53caf05200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:06:43.922 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:43 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:43.918+0000 7f53caf05200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:06:44.318 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:44 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:44.120+0000 7f53caf05200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:06:44.598 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:44 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:44.315+0000 7f53caf05200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:06:45.098 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:44 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:44.916+0000 7f53caf05200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:06:45.654 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:45 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:45.347+0000 7f53caf05200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:06:45.654 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:45 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:45.446+0000 7f53caf05200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:06:45.654 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:45 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:45.650+0000 7f53caf05200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:06:46.518 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:46 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:46.176+0000 7f53caf05200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:06:46.518 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:46 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:46.270+0000 7f53caf05200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:06:46.847 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:46 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:46.515+0000 7f53caf05200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:06:46.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:46 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:46.650+0000 7f53caf05200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:06:46.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:46 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:46.747+0000 7f53caf05200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:06:47.165 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:46 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:46.865+0000 7f53caf05200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:06:47.165 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:47 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:47.162+0000 7f53caf05200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:06:48.035 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:47 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:47.762+0000 7f53caf05200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:06:48.035 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:47 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:06:47.979+0000 7f53caf05200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: Active manager daemon x restarted 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: Activating manager daemon x 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: osdmap e2: 0 total, 0 up, 0 in 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: mgrmap e6: x(active, starting, since 0.00379942s) 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: Manager daemon x is now available 2024-05-21T15:06:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:06:48.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:06:48.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:06:48.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:06:48.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:48 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:06:49.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:49 smithi078 ceph-mon[31997]: Found migration_current of "None". Setting to last migration. 2024-05-21T15:06:49.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:49 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:06:49.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:49 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:06:49.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:49 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:06:49.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:49 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:06:49.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:49 smithi078 ceph-mon[31997]: mgrmap e7: x(active, since 1.00635s) 2024-05-21T15:06:49.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.391+0000 7f59aa876700 1 Processor -- start 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.391+0000 7f59aa876700 1 -- start start 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.391+0000 7f59aa876700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a4107420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.391+0000 7f59aa876700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41079f0 con 0x7f59a4107010 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.391+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a4107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.391+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a4107420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43532/0 (socket says 172.21.15.78:43532) 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.391+0000 7f59a3fff700 1 -- 172.21.15.78:0/1318837863 learned_addr learned my addr 172.21.15.78:0/1318837863 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.392+0000 7f59a3fff700 1 -- 172.21.15.78:0/1318837863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a4107b70 con 0x7f59a4107010 2024-05-21T15:06:49.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.392+0000 7f59a3fff700 1 --2- 172.21.15.78:0/1318837863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a4107420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59940098a0 tx=0x7f5994009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c57788bf437af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.392+0000 7f59a2ffd700 1 -- 172.21.15.78:0/1318837863 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5994004160 con 0x7f59a4107010 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.392+0000 7f59a2ffd700 1 -- 172.21.15.78:0/1318837863 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5994004300 con 0x7f59a4107010 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.392+0000 7f59a2ffd700 1 -- 172.21.15.78:0/1318837863 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5994011b50 con 0x7f59a4107010 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.393+0000 7f59aa876700 1 -- 172.21.15.78:0/1318837863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 msgr2=0x7f59a4107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.393+0000 7f59aa876700 1 --2- 172.21.15.78:0/1318837863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a4107420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59940098a0 tx=0x7f5994009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.393+0000 7f59aa876700 1 -- 172.21.15.78:0/1318837863 shutdown_connections 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.393+0000 7f59aa876700 1 --2- 172.21.15.78:0/1318837863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a4107420 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.393+0000 7f59aa876700 1 -- 172.21.15.78:0/1318837863 >> 172.21.15.78:0/1318837863 conn(0x7f59a4102aa0 msgr2=0x7f59a4104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.393+0000 7f59aa876700 1 -- 172.21.15.78:0/1318837863 shutdown_connections 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.393+0000 7f59aa876700 1 -- 172.21.15.78:0/1318837863 wait complete. 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.394+0000 7f59aa876700 1 Processor -- start 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.394+0000 7f59aa876700 1 -- start start 2024-05-21T15:06:49.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.394+0000 7f59aa876700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a419d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.394+0000 7f59aa876700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59a410d300 con 0x7f59a4107010 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.394+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a419d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.394+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a419d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43540/0 (socket says 172.21.15.78:43540) 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.394+0000 7f59a3fff700 1 -- 172.21.15.78:0/2720383317 learned_addr learned my addr 172.21.15.78:0/2720383317 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.395+0000 7f59a3fff700 1 -- 172.21.15.78:0/2720383317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a419d720 con 0x7f59a4107010 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.395+0000 7f59a3fff700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a419d1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5994009fb0 tx=0x7f5994011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.395+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59940038b0 con 0x7f59a4107010 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.395+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5994011e50 con 0x7f59a4107010 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.395+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a419d9b0 con 0x7f59a4107010 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.395+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f599401a5c0 con 0x7f59a4107010 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.395+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41a05c0 con 0x7f59a4107010 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.396+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f599401a8c0 con 0x7f59a4107010 2024-05-21T15:06:49.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.396+0000 7f59a17fa700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 0x7f598c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.396+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 --> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f598c03bc20 con 0x7f598c039000 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.396+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f599404d6c0 con 0x7f59a4107010 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.396+0000 7f59a37fe700 1 -- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 msgr2=0x7f598c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/888544333 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.396+0000 7f59a37fe700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 0x7f598c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.597+0000 7f59a37fe700 1 -- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 msgr2=0x7f598c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/888544333 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.597+0000 7f59a37fe700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 0x7f598c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.998+0000 7f59a37fe700 1 -- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 msgr2=0x7f598c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/888544333 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:43.998+0000 7f59a37fe700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 0x7f598c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:44.799+0000 7f59a37fe700 1 -- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 msgr2=0x7f598c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/888544333 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:44.799+0000 7f59a37fe700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 0x7f598c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:46.400+0000 7f59a37fe700 1 -- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 msgr2=0x7f598c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/888544333 2024-05-21T15:06:49.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:46.400+0000 7f59a37fe700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 0x7f598c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:47.982+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f599401ac00 con 0x7f59a4107010 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:47.983+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 msgr2=0x7f598c03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:47.983+0000 7f59a17fa700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 0x7f598c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.984+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f599404f240 con 0x7f59a4107010 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.984+0000 7f59a17fa700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f598c03c980 0x7f598c03eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.984+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f599401eea0 con 0x7f598c03c980 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.984+0000 7f59a37fe700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f598c03c980 0x7f598c03eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.985+0000 7f59a37fe700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f598c03c980 0x7f598c03eda0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5998003a10 tx=0x7f5998009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.985+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mgr.14120 v2:172.21.15.78:6800/3694112952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f599401eea0 con 0x7f598c03c980 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.992+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f59a4063ca0 con 0x7f598c03c980 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59a17fa700 1 -- 172.21.15.78:0/2720383317 <== mgr.14120 v2:172.21.15.78:6800/3694112952 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f59a4063ca0 con 0x7f598c03c980 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f598c03c980 msgr2=0x7f598c03eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:49.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f598c03c980 0x7f598c03eda0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5998003a10 tx=0x7f5998009290 comp rx=0 tx=0).stop 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 msgr2=0x7f59a419d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a419d1e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5994009fb0 tx=0x7f5994011c10 comp rx=0 tx=0).stop 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 shutdown_connections 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f598c03c980 0x7f598c03eda0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a4107010 0x7f59a419d1e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 --2- 172.21.15.78:0/2720383317 >> [v2:172.21.15.78:6800/888544333,v1:172.21.15.78:6801/888544333] conn(0x7f598c039000 0x7f598c03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 >> 172.21.15.78:0/2720383317 conn(0x7f59a4102aa0 msgr2=0x7f59a41033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 shutdown_connections 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:48.993+0000 7f59aa876700 1 -- 172.21.15.78:0/2720383317 wait complete. 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 5 is available 2024-05-21T15:06:49.608 INFO:teuthology.orchestra.run.smithi078.stdout:Setting orchestrator backend to cephadm... 2024-05-21T15:06:50.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:50 smithi078 ceph-mon[31997]: [21/May/2024:15:06:48] ENGINE Bus STARTING 2024-05-21T15:06:50.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:50 smithi078 ceph-mon[31997]: [21/May/2024:15:06:48] ENGINE Serving on https://172.21.15.78:7150 2024-05-21T15:06:50.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:50 smithi078 ceph-mon[31997]: [21/May/2024:15:06:48] ENGINE Serving on http://172.21.15.78:8765 2024-05-21T15:06:50.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:50 smithi078 ceph-mon[31997]: [21/May/2024:15:06:48] ENGINE Bus STARTED 2024-05-21T15:06:50.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:50 smithi078 ceph-mon[31997]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T15:06:50.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:50 smithi078 ceph-mon[31997]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.023+0000 7fc4db861700 1 Processor -- start 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.023+0000 7fc4db861700 1 -- start start 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4db861700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4db861700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4108630 con 0x7fc4d4107c30 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4d95fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4d95fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40986/0 (socket says 172.21.15.78:40986) 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4d95fd700 1 -- 172.21.15.78:0/3210280862 learned_addr learned my addr 172.21.15.78:0/3210280862 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4d95fd700 1 -- 172.21.15.78:0/3210280862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4108ec0 con 0x7fc4d4107c30 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4d95fd700 1 --2- 172.21.15.78:0/3210280862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4108040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00098a0 tx=0x7fc4d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976e59e33de7cd23 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:50.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4cbfff700 1 -- 172.21.15.78:0/3210280862 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d0004160 con 0x7fc4d4107c30 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4cbfff700 1 -- 172.21.15.78:0/3210280862 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4d0004300 con 0x7fc4d4107c30 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.024+0000 7fc4cbfff700 1 -- 172.21.15.78:0/3210280862 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d0011b50 con 0x7fc4d4107c30 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.025+0000 7fc4db861700 1 -- 172.21.15.78:0/3210280862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 msgr2=0x7fc4d4108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.025+0000 7fc4db861700 1 --2- 172.21.15.78:0/3210280862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4108040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00098a0 tx=0x7fc4d0009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.025+0000 7fc4db861700 1 -- 172.21.15.78:0/3210280862 shutdown_connections 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.025+0000 7fc4db861700 1 --2- 172.21.15.78:0/3210280862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4108040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.025+0000 7fc4db861700 1 -- 172.21.15.78:0/3210280862 >> 172.21.15.78:0/3210280862 conn(0x7fc4d4103330 msgr2=0x7fc4d41057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.025+0000 7fc4db861700 1 -- 172.21.15.78:0/3210280862 shutdown_connections 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.025+0000 7fc4db861700 1 -- 172.21.15.78:0/3210280862 wait complete. 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.026+0000 7fc4db861700 1 Processor -- start 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.026+0000 7fc4db861700 1 -- start start 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.026+0000 7fc4db861700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:50.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.026+0000 7fc4db861700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d410c270 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.026+0000 7fc4d95fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.026+0000 7fc4d95fd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40992/0 (socket says 172.21.15.78:40992) 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.026+0000 7fc4d95fd700 1 -- 172.21.15.78:0/3771841274 learned_addr learned my addr 172.21.15.78:0/3771841274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.026+0000 7fc4d95fd700 1 -- 172.21.15.78:0/3771841274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d40763f0 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.027+0000 7fc4d95fd700 1 --2- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4075eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009fb0 tx=0x7fc4d0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.027+0000 7fc4ca7fc700 1 -- 172.21.15.78:0/3771841274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d00038b0 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.027+0000 7fc4ca7fc700 1 -- 172.21.15.78:0/3771841274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4d0011e50 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.027+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d419ff90 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.027+0000 7fc4ca7fc700 1 -- 172.21.15.78:0/3771841274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d001a430 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.027+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d41a0410 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.028+0000 7fc4ca7fc700 1 -- 172.21.15.78:0/3771841274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fc4d001a6a0 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.028+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d4063ca0 con 0x7fc4d4107c30 2024-05-21T15:06:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.028+0000 7fc4ca7fc700 1 --2- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fc4c0038f40 0x7fc4c003b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.028+0000 7fc4ca7fc700 1 -- 172.21.15.78:0/3771841274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc4d004ce80 con 0x7fc4d4107c30 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.028+0000 7fc4d8dfc700 1 --2- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fc4c0038f40 0x7fc4c003b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.028+0000 7fc4d8dfc700 1 --2- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fc4c0038f40 0x7fc4c003b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4006fd0 tx=0x7fc4c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.033+0000 7fc4ca7fc700 1 -- 172.21.15.78:0/3771841274 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4d0020240 con 0x7fc4d4107c30 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.040+0000 7fc4ca7fc700 1 -- 172.21.15.78:0/3771841274 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc4d002d770 con 0x7fc4d4107c30 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.250+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc4d4104450 con 0x7fc4c0038f40 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.255+0000 7fc4ca7fc700 1 -- 172.21.15.78:0/3771841274 <== mgr.14120 v2:172.21.15.78:6800/3694112952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc4d4104450 con 0x7fc4c0038f40 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fc4c0038f40 msgr2=0x7fc4c003b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 --2- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fc4c0038f40 0x7fc4c003b450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4006fd0 tx=0x7fc4c4006e40 comp rx=0 tx=0).stop 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 msgr2=0x7fc4d4075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 --2- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4075eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009fb0 tx=0x7fc4d0011a00 comp rx=0 tx=0).stop 2024-05-21T15:06:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 shutdown_connections 2024-05-21T15:06:50.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 --2- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fc4c0038f40 0x7fc4c003b450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:50.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 --2- 172.21.15.78:0/3771841274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d4107c30 0x7fc4d4075eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:50.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 >> 172.21.15.78:0/3771841274 conn(0x7fc4d4103330 msgr2=0x7fc4d4103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:50.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 shutdown_connections 2024-05-21T15:06:50.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:50.258+0000 7fc4db861700 1 -- 172.21.15.78:0/3771841274 wait complete. 2024-05-21T15:06:51.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:51 smithi078 ceph-mon[31997]: mgrmap e8: x(active, since 2s) 2024-05-21T15:06:51.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:51 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:06:51.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:51 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:06:52.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-21T15:06:52.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.251+0000 7f733561b700 1 Processor -- start 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.251+0000 7f733561b700 1 -- start start 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.251+0000 7f733561b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f7330108020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.251+0000 7f733561b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7330108610 con 0x7f7330107c10 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.251+0000 7f732effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f7330108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.251+0000 7f732effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f7330108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40994/0 (socket says 172.21.15.78:40994) 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.251+0000 7f732effd700 1 -- 172.21.15.78:0/2162192858 learned_addr learned my addr 172.21.15.78:0/2162192858 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.252+0000 7f732effd700 1 -- 172.21.15.78:0/2162192858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7330108ea0 con 0x7f7330107c10 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.252+0000 7f732effd700 1 --2- 172.21.15.78:0/2162192858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f7330108020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73200098a0 tx=0x7f7320009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9397c33352fb0b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.252+0000 7f732e7fc700 1 -- 172.21.15.78:0/2162192858 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7320004160 con 0x7f7330107c10 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.252+0000 7f732e7fc700 1 -- 172.21.15.78:0/2162192858 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7320004300 con 0x7f7330107c10 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.252+0000 7f732e7fc700 1 -- 172.21.15.78:0/2162192858 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7320011b50 con 0x7f7330107c10 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.253+0000 7f733561b700 1 -- 172.21.15.78:0/2162192858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 msgr2=0x7f7330108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:52.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.253+0000 7f733561b700 1 --2- 172.21.15.78:0/2162192858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f7330108020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73200098a0 tx=0x7f7320009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.253+0000 7f733561b700 1 -- 172.21.15.78:0/2162192858 shutdown_connections 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.253+0000 7f733561b700 1 --2- 172.21.15.78:0/2162192858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f7330108020 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.253+0000 7f733561b700 1 -- 172.21.15.78:0/2162192858 >> 172.21.15.78:0/2162192858 conn(0x7f73301033d0 msgr2=0x7f7330105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.253+0000 7f733561b700 1 -- 172.21.15.78:0/2162192858 shutdown_connections 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.253+0000 7f733561b700 1 -- 172.21.15.78:0/2162192858 wait complete. 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.253+0000 7f733561b700 1 Processor -- start 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f733561b700 1 -- start start 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f733561b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f733019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f733561b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f733010c250 con 0x7f7330107c10 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f732effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f733019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f732effd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f733019c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41000/0 (socket says 172.21.15.78:41000) 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f732effd700 1 -- 172.21.15.78:0/1777370163 learned_addr learned my addr 172.21.15.78:0/1777370163 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:52.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f732effd700 1 -- 172.21.15.78:0/1777370163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733019c600 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f732effd700 1 --2- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f733019c0c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7320009fb0 tx=0x7f7320011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f732cff9700 1 -- 172.21.15.78:0/1777370163 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73200038b0 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f732cff9700 1 -- 172.21.15.78:0/1777370163 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7320011e50 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.254+0000 7f732cff9700 1 -- 172.21.15.78:0/1777370163 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f732001a430 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.255+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733019c890 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.255+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733019f4a0 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.256+0000 7f732cff9700 1 -- 172.21.15.78:0/1777370163 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f732001a6a0 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.256+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7330063ca0 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.256+0000 7f732cff9700 1 --2- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f731c039050 0x7f731c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.256+0000 7f732cff9700 1 -- 172.21.15.78:0/1777370163 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f732004d1d0 con 0x7f7330107c10 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.256+0000 7f7327fff700 1 --2- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f731c039050 0x7f731c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:52.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.256+0000 7f7327fff700 1 --2- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f731c039050 0x7f731c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7318006fd0 tx=0x7f7318006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.261+0000 7f732cff9700 1 -- 172.21.15.78:0/1777370163 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f732002d770 con 0x7f7330107c10 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.459+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7330105440 con 0x7f731c039050 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.460+0000 7f732cff9700 1 -- 172.21.15.78:0/1777370163 <== mgr.14120 v2:172.21.15.78:6800/3694112952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f7330105440 con 0x7f731c039050 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f731c039050 msgr2=0x7f731c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 --2- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f731c039050 0x7f731c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7318006fd0 tx=0x7f7318006e40 comp rx=0 tx=0).stop 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 msgr2=0x7f733019c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 --2- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f733019c0c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7320009fb0 tx=0x7f7320011a00 comp rx=0 tx=0).stop 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 shutdown_connections 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 --2- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f731c039050 0x7f731c03b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 --2- 172.21.15.78:0/1777370163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7330107c10 0x7f733019c0c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 >> 172.21.15.78:0/1777370163 conn(0x7f73301033d0 msgr2=0x7f7330104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 shutdown_connections 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:51.463+0000 7f733561b700 1 -- 172.21.15.78:0/1777370163 wait complete. 2024-05-21T15:06:52.058 INFO:teuthology.orchestra.run.smithi078.stdout:Generating ssh key... 2024-05-21T15:06:52.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:52 smithi078 ceph-mon[31997]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: Generating public/private rsa key pair. 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: Your identification has been saved in /tmp/tmp_ynv5dlv/key. 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: Your public key has been saved in /tmp/tmp_ynv5dlv/key.pub. 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: The key fingerprint is: 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: SHA256:iU9Z3qr3f1ckyIUw7vMkA5HiGpiC3W95qJ0FOEMpWFw ceph-852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: The key's randomart image is: 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: +---[RSA 3072]----+ 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: | +..E. ..o. . | 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: |. o o . .o .. . | 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: |.. * o .. o. o | 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: |o + * o. B .o . .| 2024-05-21T15:06:53.044 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: | . *.+S * o o | 2024-05-21T15:06:53.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: | . =oo B .| 2024-05-21T15:06:53.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: | + +. . . .| 2024-05-21T15:06:53.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: | . o .. o| 2024-05-21T15:06:53.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: | .. ......| 2024-05-21T15:06:53.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:06:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: +----[SHA256]-----+ 2024-05-21T15:06:53.294 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:53 smithi078 ceph-mon[31997]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:06:53.294 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:53 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:06:53.294 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:53 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:06:53.594 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.531+0000 7f1f8cf14700 1 Processor -- start 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.531+0000 7f1f8cf14700 1 -- start start 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.531+0000 7f1f8cf14700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8810a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.531+0000 7f1f8cf14700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8810ad60 con 0x7f1f8810a380 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.532+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8810a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.532+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8810a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41014/0 (socket says 172.21.15.78:41014) 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.532+0000 7f1f8659c700 1 -- 172.21.15.78:0/3747156109 learned_addr learned my addr 172.21.15.78:0/3747156109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.532+0000 7f1f8659c700 1 -- 172.21.15.78:0/3747156109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8810aee0 con 0x7f1f8810a380 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.532+0000 7f1f8659c700 1 --2- 172.21.15.78:0/3747156109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8810a790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3f6b9c5469a21f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.532+0000 7f1f8559a700 1 -- 172.21.15.78:0/3747156109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f78004160 con 0x7f1f8810a380 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.532+0000 7f1f8559a700 1 -- 172.21.15.78:0/3747156109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f78004300 con 0x7f1f8810a380 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.532+0000 7f1f8559a700 1 -- 172.21.15.78:0/3747156109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f78011b50 con 0x7f1f8810a380 2024-05-21T15:06:53.595 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.533+0000 7f1f8cf14700 1 -- 172.21.15.78:0/3747156109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 msgr2=0x7f1f8810a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.533+0000 7f1f8cf14700 1 --2- 172.21.15.78:0/3747156109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8810a790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.533+0000 7f1f8cf14700 1 -- 172.21.15.78:0/3747156109 shutdown_connections 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.533+0000 7f1f8cf14700 1 --2- 172.21.15.78:0/3747156109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8810a790 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.533+0000 7f1f8cf14700 1 -- 172.21.15.78:0/3747156109 >> 172.21.15.78:0/3747156109 conn(0x7f1f88102aa0 msgr2=0x7f1f88104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.533+0000 7f1f8cf14700 1 -- 172.21.15.78:0/3747156109 shutdown_connections 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.533+0000 7f1f8cf14700 1 -- 172.21.15.78:0/3747156109 wait complete. 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.534+0000 7f1f8cf14700 1 Processor -- start 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.534+0000 7f1f8cf14700 1 -- start start 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.534+0000 7f1f8cf14700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.534+0000 7f1f8cf14700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f88107c30 con 0x7f1f8810a380 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.534+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.534+0000 7f1f8659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41016/0 (socket says 172.21.15.78:41016) 2024-05-21T15:06:53.596 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.534+0000 7f1f8659c700 1 -- 172.21.15.78:0/2219662211 learned_addr learned my addr 172.21.15.78:0/2219662211 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.535+0000 7f1f8659c700 1 -- 172.21.15.78:0/2219662211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8819d710 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.535+0000 7f1f8659c700 1 --2- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8819d1d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009fb0 tx=0x7f1f78011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.535+0000 7f1f777fe700 1 -- 172.21.15.78:0/2219662211 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f780038b0 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.535+0000 7f1f777fe700 1 -- 172.21.15.78:0/2219662211 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f78011e50 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.535+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8819d9a0 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.535+0000 7f1f777fe700 1 -- 172.21.15.78:0/2219662211 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f7801a5c0 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.535+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f881a05b0 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.536+0000 7f1f777fe700 1 -- 172.21.15.78:0/2219662211 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1f7801a8c0 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.536+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f88063ca0 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.536+0000 7f1f777fe700 1 --2- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f70039000 0x7f1f7003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.536+0000 7f1f777fe700 1 -- 172.21.15.78:0/2219662211 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f7804d540 con 0x7f1f8810a380 2024-05-21T15:06:53.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.536+0000 7f1f85d9b700 1 --2- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f70039000 0x7f1f7003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.537+0000 7f1f85d9b700 1 --2- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f70039000 0x7f1f7003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c006fd0 tx=0x7f1f7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.541+0000 7f1f777fe700 1 -- 172.21.15.78:0/2219662211 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1f7802cc00 con 0x7f1f8810a380 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.737+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1f88103ae0 con 0x7f1f70039000 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.944+0000 7f1f777fe700 1 -- 172.21.15.78:0/2219662211 <== mgr.14120 v2:172.21.15.78:6800/3694112952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f88103ae0 con 0x7f1f70039000 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f70039000 msgr2=0x7f1f7003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 --2- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f70039000 0x7f1f7003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c006fd0 tx=0x7f1f7c006e40 comp rx=0 tx=0).stop 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 msgr2=0x7f1f8819d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 --2- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8819d1d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009fb0 tx=0x7f1f78011c10 comp rx=0 tx=0).stop 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 shutdown_connections 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 --2- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f70039000 0x7f1f7003b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 --2- 172.21.15.78:0/2219662211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f8810a380 0x7f1f8819d1d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:53.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 >> 172.21.15.78:0/2219662211 conn(0x7f1f88102aa0 msgr2=0x7f1f881033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:53.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 shutdown_connections 2024-05-21T15:06:53.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:52.947+0000 7f1f8cf14700 1 -- 172.21.15.78:0/2219662211 wait complete. 2024-05-21T15:06:54.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:54 smithi078 ceph-mon[31997]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:06:54.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:54 smithi078 ceph-mon[31997]: Generating ssh key... 2024-05-21T15:06:54.858 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuv/dbtzsSo48XCT5odeh3tIedUZawRLAVS6MycETstibL+0jS2m4Nk90hq8GecX5wxQji1i/uHYwrTWn0Um4gjzGTU5LlLUku4rHS97V9fsPgPLs6Q6p/VkQF1Ykr32dceMVnFbbu/JwHxKBhze0l4RVTzgxhsIU4n1xzs4OkXSEahyBOKvjHWlVrZlmU4TQhM/f/DMbXIDa5KlsjyTuv/2wySh3B89bJcVKTwZwj1goKyjZfJkD2kI4g5wSveH+1w309pLBnAR7C5oFioj53WqnIseVclL7JDDunGRCK0/QOqQyXYK4HGRo/p3tbWjEqVO2v6PORhYVhHFEJOpB8pLVZbYwilnMLqsIV5cpp4XkkKAgiPY4C1qNagMyFFSrDlds+9P95B2nH0njqCzh6QDsP9VOcjw9GvSFAj9lEtUN6Qmhdgo7n5e4TZYiyZ6LxmrEMthZLLkvDUlV8D10pB3VxK7gZAGTsgjAMwBIytCcAsRzq6/hvmPlzdPtrXGs= ceph-852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:06:54.858 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.014+0000 7fe97978e700 1 Processor -- start 2024-05-21T15:06:54.858 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.014+0000 7fe97978e700 1 -- start start 2024-05-21T15:06:54.858 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.014+0000 7fe97978e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe9741090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:54.858 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.014+0000 7fe97978e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741096c0 con 0x7fe974108cc0 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.014+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe9741090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.014+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe9741090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41020/0 (socket says 172.21.15.78:41020) 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.014+0000 7fe972ffd700 1 -- 172.21.15.78:0/1323820094 learned_addr learned my addr 172.21.15.78:0/1323820094 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.015+0000 7fe972ffd700 1 -- 172.21.15.78:0/1323820094 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe974109f50 con 0x7fe974108cc0 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.015+0000 7fe972ffd700 1 --2- 172.21.15.78:0/1323820094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe9741090d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9680098a0 tx=0x7fe968009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad82d0d798b163f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.015+0000 7fe971ffb700 1 -- 172.21.15.78:0/1323820094 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe968004160 con 0x7fe974108cc0 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.015+0000 7fe971ffb700 1 -- 172.21.15.78:0/1323820094 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe968004300 con 0x7fe974108cc0 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.015+0000 7fe971ffb700 1 -- 172.21.15.78:0/1323820094 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe968011b50 con 0x7fe974108cc0 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.016+0000 7fe97978e700 1 -- 172.21.15.78:0/1323820094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 msgr2=0x7fe9741090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.016+0000 7fe97978e700 1 --2- 172.21.15.78:0/1323820094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe9741090d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9680098a0 tx=0x7fe968009bb0 comp rx=0 tx=0).stop 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.016+0000 7fe97978e700 1 -- 172.21.15.78:0/1323820094 shutdown_connections 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.016+0000 7fe97978e700 1 --2- 172.21.15.78:0/1323820094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe9741090d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.016+0000 7fe97978e700 1 -- 172.21.15.78:0/1323820094 >> 172.21.15.78:0/1323820094 conn(0x7fe974104400 msgr2=0x7fe974106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:54.859 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.016+0000 7fe97978e700 1 -- 172.21.15.78:0/1323820094 shutdown_connections 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.016+0000 7fe97978e700 1 -- 172.21.15.78:0/1323820094 wait complete. 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.016+0000 7fe97978e700 1 Processor -- start 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.017+0000 7fe97978e700 1 -- start start 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.017+0000 7fe97978e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe97419d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.017+0000 7fe97978e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe97410d300 con 0x7fe974108cc0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.017+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe97419d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.017+0000 7fe972ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe97419d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41030/0 (socket says 172.21.15.78:41030) 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.017+0000 7fe972ffd700 1 -- 172.21.15.78:0/527344042 learned_addr learned my addr 172.21.15.78:0/527344042 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.017+0000 7fe972ffd700 1 -- 172.21.15.78:0/527344042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97419d6e0 con 0x7fe974108cc0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.017+0000 7fe972ffd700 1 --2- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe97419d1a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe968004750 tx=0x7fe968011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.018+0000 7fe95bfff700 1 -- 172.21.15.78:0/527344042 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9680039e0 con 0x7fe974108cc0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.018+0000 7fe95bfff700 1 -- 172.21.15.78:0/527344042 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe968003b80 con 0x7fe974108cc0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.018+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97419d970 con 0x7fe974108cc0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.018+0000 7fe95bfff700 1 -- 172.21.15.78:0/527344042 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9680246e0 con 0x7fe974108cc0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.018+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9741a0580 con 0x7fe974108cc0 2024-05-21T15:06:54.860 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.019+0000 7fe95bfff700 1 -- 172.21.15.78:0/527344042 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe968022020 con 0x7fe974108cc0 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.019+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe974063ca0 con 0x7fe974108cc0 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.019+0000 7fe95bfff700 1 --2- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe95c039050 0x7fe95c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.019+0000 7fe95bfff700 1 -- 172.21.15.78:0/527344042 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe968011e00 con 0x7fe974108cc0 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.019+0000 7fe9727fc700 1 --2- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe95c039050 0x7fe95c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.020+0000 7fe9727fc700 1 --2- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe95c039050 0x7fe95c03b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe964006fd0 tx=0x7fe964006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.024+0000 7fe95bfff700 1 -- 172.21.15.78:0/527344042 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe96802d3f0 con 0x7fe974108cc0 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.222+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe9741054b0 con 0x7fe95c039050 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.223+0000 7fe95bfff700 1 -- 172.21.15.78:0/527344042 <== mgr.14120 v2:172.21.15.78:6800/3694112952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe9741054b0 con 0x7fe95c039050 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.225+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe95c039050 msgr2=0x7fe95c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.225+0000 7fe97978e700 1 --2- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe95c039050 0x7fe95c03b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe964006fd0 tx=0x7fe964006e40 comp rx=0 tx=0).stop 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.225+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 msgr2=0x7fe97419d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.225+0000 7fe97978e700 1 --2- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe97419d1a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe968004750 tx=0x7fe968011930 comp rx=0 tx=0).stop 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.225+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 shutdown_connections 2024-05-21T15:06:54.861 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.225+0000 7fe97978e700 1 --2- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe95c039050 0x7fe95c03b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:54.862 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.225+0000 7fe97978e700 1 --2- 172.21.15.78:0/527344042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe974108cc0 0x7fe97419d1a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:06:54.862 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.225+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 >> 172.21.15.78:0/527344042 conn(0x7fe974104400 msgr2=0x7fe974104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:06:54.862 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.226+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 shutdown_connections 2024-05-21T15:06:54.862 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:54.226+0000 7fe97978e700 1 -- 172.21.15.78:0/527344042 wait complete. 2024-05-21T15:06:54.862 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-21T15:06:54.862 INFO:teuthology.orchestra.run.smithi078.stdout:Adding key to root@localhost authorized_keys... 2024-05-21T15:06:54.862 INFO:teuthology.orchestra.run.smithi078.stdout:Adding host smithi078... 2024-05-21T15:06:55.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:55 smithi078 ceph-mon[31997]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:06:56.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:56 smithi078 ceph-mon[31997]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:06:56.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:06:56 smithi078 ceph-mon[31997]: Deploying cephadm binary to smithi078 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Added host 'smithi078' with addr '172.21.15.78' 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.276+0000 7f4ddd5e5700 1 Processor -- start 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.276+0000 7f4ddd5e5700 1 -- start start 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.276+0000 7f4ddd5e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd8108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.276+0000 7f4ddd5e5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd8108630 con 0x7f4dd8107c30 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.276+0000 7f4dd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd8108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.276+0000 7f4dd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd8108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41038/0 (socket says 172.21.15.78:41038) 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.276+0000 7f4dd6ffd700 1 -- 172.21.15.78:0/2421730198 learned_addr learned my addr 172.21.15.78:0/2421730198 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.276+0000 7f4dd6ffd700 1 -- 172.21.15.78:0/2421730198 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd8108ec0 con 0x7f4dd8107c30 2024-05-21T15:07:00.265 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.277+0000 7f4dd6ffd700 1 --2- 172.21.15.78:0/2421730198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd8108040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80098a0 tx=0x7f4dc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baca21adedac9316 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.277+0000 7f4dd67fc700 1 -- 172.21.15.78:0/2421730198 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc8004160 con 0x7f4dd8107c30 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.277+0000 7f4dd67fc700 1 -- 172.21.15.78:0/2421730198 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4dc8004300 con 0x7f4dd8107c30 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.277+0000 7f4dd67fc700 1 -- 172.21.15.78:0/2421730198 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc8011b50 con 0x7f4dd8107c30 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.277+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/2421730198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 msgr2=0x7f4dd8108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.277+0000 7f4ddd5e5700 1 --2- 172.21.15.78:0/2421730198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd8108040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80098a0 tx=0x7f4dc8009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/2421730198 shutdown_connections 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 --2- 172.21.15.78:0/2421730198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd8108040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/2421730198 >> 172.21.15.78:0/2421730198 conn(0x7f4dd8103330 msgr2=0x7f4dd81057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/2421730198 shutdown_connections 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/2421730198 wait complete. 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 Processor -- start 2024-05-21T15:07:00.266 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 -- start start 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd819d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.278+0000 7f4ddd5e5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd810c270 con 0x7f4dd8107c30 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4dd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd819d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4dd6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd819d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41048/0 (socket says 172.21.15.78:41048) 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4dd6ffd700 1 -- 172.21.15.78:0/419085265 learned_addr learned my addr 172.21.15.78:0/419085265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4dd6ffd700 1 -- 172.21.15.78:0/419085265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd819d780 con 0x7f4dd8107c30 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4dd6ffd700 1 --2- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd819d240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009fb0 tx=0x7f4dc8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4dd4ff9700 1 -- 172.21.15.78:0/419085265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc80038b0 con 0x7f4dd8107c30 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd819da10 con 0x7f4dd8107c30 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4dd4ff9700 1 -- 172.21.15.78:0/419085265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4dc8011e50 con 0x7f4dd8107c30 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4dd4ff9700 1 -- 172.21.15.78:0/419085265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc801a430 con 0x7f4dd8107c30 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.279+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd81a0620 con 0x7f4dd8107c30 2024-05-21T15:07:00.267 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.280+0000 7f4dd4ff9700 1 -- 172.21.15.78:0/419085265 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4dc801a6a0 con 0x7f4dd8107c30 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.280+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd8063ca0 con 0x7f4dd8107c30 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.281+0000 7f4dd4ff9700 1 --2- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f4dc4039050 0x7f4dc403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.281+0000 7f4dd4ff9700 1 -- 172.21.15.78:0/419085265 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4dc804d070 con 0x7f4dd8107c30 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.281+0000 7f4dcffff700 1 --2- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f4dc4039050 0x7f4dc403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.281+0000 7f4dcffff700 1 --2- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f4dc4039050 0x7f4dc403b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006fd0 tx=0x7f4dc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.286+0000 7f4dd4ff9700 1 -- 172.21.15.78:0/419085265 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4dc8020240 con 0x7f4dd8107c30 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:55.486+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}) -- 0x7f4dd802dfd0 con 0x7f4dc4039050 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.598+0000 7f4dd4ff9700 1 -- 172.21.15.78:0/419085265 <== mgr.14120 v2:172.21.15.78:6800/3694112952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4dd802dfd0 con 0x7f4dc4039050 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f4dc4039050 msgr2=0x7f4dc403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 --2- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f4dc4039050 0x7f4dc403b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006fd0 tx=0x7f4dc0006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 msgr2=0x7f4dd819d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:00.268 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 --2- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd819d240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009fb0 tx=0x7f4dc8011a00 comp rx=0 tx=0).stop 2024-05-21T15:07:00.269 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 shutdown_connections 2024-05-21T15:07:00.269 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 --2- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f4dc4039050 0x7f4dc403b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:00.269 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 --2- 172.21.15.78:0/419085265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd8107c30 0x7f4dd819d240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:00.269 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 >> 172.21.15.78:0/419085265 conn(0x7f4dd8103330 msgr2=0x7f4dd81937c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:00.269 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 shutdown_connections 2024-05-21T15:07:00.269 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:06:59.601+0000 7f4ddd5e5700 1 -- 172.21.15.78:0/419085265 wait complete. 2024-05-21T15:07:00.269 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mon service... 2024-05-21T15:07:00.897 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:00 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:07:00.897 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:00 smithi078 ceph-mon[31997]: Added host smithi078 2024-05-21T15:07:00.897 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:00 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:01.633 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.755+0000 7fb020938700 1 Processor -- start 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.755+0000 7fb020938700 1 -- start start 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.755+0000 7fb020938700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c10b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.755+0000 7fb020938700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10bee0 con 0x7fb01c07bc90 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.756+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c10b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.756+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c10b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46902/0 (socket says 172.21.15.78:46902) 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.756+0000 7fb01a59c700 1 -- 172.21.15.78:0/3736028405 learned_addr learned my addr 172.21.15.78:0/3736028405 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.756+0000 7fb01a59c700 1 -- 172.21.15.78:0/3736028405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c10c060 con 0x7fb01c07bc90 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.756+0000 7fb01a59c700 1 --2- 172.21.15.78:0/3736028405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c10b8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0098a0 tx=0x7fb00c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49033925c155431 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:01.634 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.756+0000 7fb01959a700 1 -- 172.21.15.78:0/3736028405 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb00c004160 con 0x7fb01c07bc90 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.756+0000 7fb01959a700 1 -- 172.21.15.78:0/3736028405 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb00c004300 con 0x7fb01c07bc90 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.756+0000 7fb01959a700 1 -- 172.21.15.78:0/3736028405 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb00c011b50 con 0x7fb01c07bc90 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.757+0000 7fb020938700 1 -- 172.21.15.78:0/3736028405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 msgr2=0x7fb01c10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.757+0000 7fb020938700 1 --2- 172.21.15.78:0/3736028405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c10b8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0098a0 tx=0x7fb00c009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.757+0000 7fb020938700 1 -- 172.21.15.78:0/3736028405 shutdown_connections 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.757+0000 7fb020938700 1 --2- 172.21.15.78:0/3736028405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c10b8f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.757+0000 7fb020938700 1 -- 172.21.15.78:0/3736028405 >> 172.21.15.78:0/3736028405 conn(0x7fb01c1033a0 msgr2=0x7fb01c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.757+0000 7fb020938700 1 -- 172.21.15.78:0/3736028405 shutdown_connections 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.757+0000 7fb020938700 1 -- 172.21.15.78:0/3736028405 wait complete. 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.758+0000 7fb020938700 1 Processor -- start 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.758+0000 7fb020938700 1 -- start start 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.758+0000 7fb020938700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c19d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:01.635 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.758+0000 7fb020938700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10d140 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.758+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c19d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.758+0000 7fb01a59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c19d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46916/0 (socket says 172.21.15.78:46916) 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.758+0000 7fb01a59c700 1 -- 172.21.15.78:0/2434747960 learned_addr learned my addr 172.21.15.78:0/2434747960 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.758+0000 7fb01a59c700 1 -- 172.21.15.78:0/2434747960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c19d6a0 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.759+0000 7fb01a59c700 1 --2- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c19d160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009fb0 tx=0x7fb00c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.759+0000 7fb00b7fe700 1 -- 172.21.15.78:0/2434747960 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb00c0038b0 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.759+0000 7fb00b7fe700 1 -- 172.21.15.78:0/2434747960 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb00c011e50 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.759+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c19d930 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.759+0000 7fb00b7fe700 1 -- 172.21.15.78:0/2434747960 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb00c019550 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.759+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1a0540 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.760+0000 7fb00b7fe700 1 -- 172.21.15.78:0/2434747960 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb00c019850 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.760+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c063ca0 con 0x7fb01c07bc90 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.760+0000 7fb00b7fe700 1 --2- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb004039000 0x7fb00403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:01.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.760+0000 7fb00b7fe700 1 -- 172.21.15.78:0/2434747960 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb00c04d3e0 con 0x7fb01c07bc90 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.760+0000 7fb019d9b700 1 --2- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb004039000 0x7fb00403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.760+0000 7fb019d9b700 1 --2- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb004039000 0x7fb00403b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb010006fd0 tx=0x7fb010006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.765+0000 7fb00b7fe700 1 -- 172.21.15.78:0/2434747960 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb00c04cd30 con 0x7fb01c07bc90 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:00.993+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb01c105470 con 0x7fb004039000 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.001+0000 7fb00b7fe700 1 -- 172.21.15.78:0/2434747960 <== mgr.14120 v2:172.21.15.78:6800/3694112952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb01c105470 con 0x7fb004039000 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb004039000 msgr2=0x7fb00403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 --2- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb004039000 0x7fb00403b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb010006fd0 tx=0x7fb010006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 msgr2=0x7fb01c19d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 --2- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c19d160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009fb0 tx=0x7fb00c011bf0 comp rx=0 tx=0).stop 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 shutdown_connections 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 --2- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb004039000 0x7fb00403b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 --2- 172.21.15.78:0/2434747960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb01c07bc90 0x7fb01c19d160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:01.637 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 >> 172.21.15.78:0/2434747960 conn(0x7fb01c1033a0 msgr2=0x7fb01c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:01.638 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 shutdown_connections 2024-05-21T15:07:01.638 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:01.004+0000 7fb020938700 1 -- 172.21.15.78:0/2434747960 wait complete. 2024-05-21T15:07:01.638 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mgr service... 2024-05-21T15:07:02.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:02 smithi078 ceph-mon[31997]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:02.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:02 smithi078 ceph-mon[31997]: Saving service mon spec with placement count:5 2024-05-21T15:07:02.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:02 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:07:03.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:03 smithi078 ceph-mon[31997]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:03.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:03 smithi078 ceph-mon[31997]: Saving service mgr spec with placement count:2 2024-05-21T15:07:03.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:03 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:07:03.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:03 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:07:03.578 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-21T15:07:03.578 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.062+0000 7fbb58f44700 1 Processor -- start 2024-05-21T15:07:03.578 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.062+0000 7fbb58f44700 1 -- start start 2024-05-21T15:07:03.578 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.063+0000 7fbb58f44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb54108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:03.578 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.063+0000 7fbb58f44700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54108630 con 0x7fbb54107c30 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.063+0000 7fbb5259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb54108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.063+0000 7fbb5259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb54108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46928/0 (socket says 172.21.15.78:46928) 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.063+0000 7fbb5259c700 1 -- 172.21.15.78:0/152200259 learned_addr learned my addr 172.21.15.78:0/152200259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.063+0000 7fbb5259c700 1 -- 172.21.15.78:0/152200259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb54108ec0 con 0x7fbb54107c30 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.063+0000 7fbb5259c700 1 --2- 172.21.15.78:0/152200259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb54108040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb480098a0 tx=0x7fbb48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ca025f499c4ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.063+0000 7fbb5159a700 1 -- 172.21.15.78:0/152200259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb48004160 con 0x7fbb54107c30 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb5159a700 1 -- 172.21.15.78:0/152200259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb48004300 con 0x7fbb54107c30 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb5159a700 1 -- 172.21.15.78:0/152200259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb48011b50 con 0x7fbb54107c30 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb58f44700 1 -- 172.21.15.78:0/152200259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 msgr2=0x7fbb54108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:03.579 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb58f44700 1 --2- 172.21.15.78:0/152200259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb54108040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb480098a0 tx=0x7fbb48009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb58f44700 1 -- 172.21.15.78:0/152200259 shutdown_connections 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb58f44700 1 --2- 172.21.15.78:0/152200259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb54108040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb58f44700 1 -- 172.21.15.78:0/152200259 >> 172.21.15.78:0/152200259 conn(0x7fbb54103330 msgr2=0x7fbb541057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb58f44700 1 -- 172.21.15.78:0/152200259 shutdown_connections 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.064+0000 7fbb58f44700 1 -- 172.21.15.78:0/152200259 wait complete. 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.065+0000 7fbb58f44700 1 Processor -- start 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.065+0000 7fbb58f44700 1 -- start start 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb58f44700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb5419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb58f44700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5410c270 con 0x7fbb54107c30 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb5259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb5419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:03.580 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb5259c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb5419c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46932/0 (socket says 172.21.15.78:46932) 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb5259c700 1 -- 172.21.15.78:0/2065526869 learned_addr learned my addr 172.21.15.78:0/2065526869 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb5259c700 1 -- 172.21.15.78:0/2065526869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5419c620 con 0x7fbb54107c30 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb5259c700 1 --2- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb5419c0e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009fb0 tx=0x7fbb48011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb437fe700 1 -- 172.21.15.78:0/2065526869 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb480038b0 con 0x7fbb54107c30 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5419c8b0 con 0x7fbb54107c30 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb437fe700 1 -- 172.21.15.78:0/2065526869 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb48011e50 con 0x7fbb54107c30 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb437fe700 1 -- 172.21.15.78:0/2065526869 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb4801a430 con 0x7fbb54107c30 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.066+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5419f4c0 con 0x7fbb54107c30 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.067+0000 7fbb437fe700 1 -- 172.21.15.78:0/2065526869 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbb4801a6f0 con 0x7fbb54107c30 2024-05-21T15:07:03.581 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.067+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb54063ca0 con 0x7fbb54107c30 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.068+0000 7fbb437fe700 1 --2- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fbb3c039000 0x7fbb3c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.068+0000 7fbb437fe700 1 -- 172.21.15.78:0/2065526869 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb4804d0e0 con 0x7fbb54107c30 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.068+0000 7fbb51d9b700 1 --2- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fbb3c039000 0x7fbb3c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.068+0000 7fbb51d9b700 1 --2- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fbb3c039000 0x7fbb3c03b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb44006fd0 tx=0x7fbb44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.073+0000 7fbb437fe700 1 -- 172.21.15.78:0/2065526869 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbb4800fdd0 con 0x7fbb54107c30 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.288+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbb5419f690 con 0x7fbb3c039000 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.296+0000 7fbb437fe700 1 -- 172.21.15.78:0/2065526869 <== mgr.14120 v2:172.21.15.78:6800/3694112952 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbb5419f690 con 0x7fbb3c039000 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.298+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fbb3c039000 msgr2=0x7fbb3c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.298+0000 7fbb58f44700 1 --2- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fbb3c039000 0x7fbb3c03b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb44006fd0 tx=0x7fbb44006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.298+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 msgr2=0x7fbb5419c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:03.582 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.298+0000 7fbb58f44700 1 --2- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb5419c0e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009fb0 tx=0x7fbb48011a00 comp rx=0 tx=0).stop 2024-05-21T15:07:03.583 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.298+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 shutdown_connections 2024-05-21T15:07:03.583 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.298+0000 7fbb58f44700 1 --2- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fbb3c039000 0x7fbb3c03b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:03.583 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.298+0000 7fbb58f44700 1 --2- 172.21.15.78:0/2065526869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb54107c30 0x7fbb5419c0e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:03.583 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.298+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 >> 172.21.15.78:0/2065526869 conn(0x7fbb54103330 msgr2=0x7fbb54103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:03.583 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.299+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 shutdown_connections 2024-05-21T15:07:03.583 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:02.299+0000 7fbb58f44700 1 -- 172.21.15.78:0/2065526869 wait complete. 2024-05-21T15:07:05.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:04 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:07:05.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:04 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2488239492' entity='client.admin' 2024-05-21T15:07:05.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.060+0000 7f707ccb2700 1 Processor -- start 2024-05-21T15:07:05.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.060+0000 7f707ccb2700 1 -- start start 2024-05-21T15:07:05.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.060+0000 7f707ccb2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707810b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:05.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.060+0000 7f707ccb2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f707810bf10 con 0x7f7078076210 2024-05-21T15:07:05.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.061+0000 7f707659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707810b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:05.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.061+0000 7f707659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707810b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46940/0 (socket says 172.21.15.78:46940) 2024-05-21T15:07:05.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.061+0000 7f707659c700 1 -- 172.21.15.78:0/1100726726 learned_addr learned my addr 172.21.15.78:0/1100726726 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:05.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.061+0000 7f707659c700 1 -- 172.21.15.78:0/1100726726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707810c090 con 0x7f7078076210 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.061+0000 7f707659c700 1 --2- 172.21.15.78:0/1100726726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707810b920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70680098a0 tx=0x7f7068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dca3ea1fa845738 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.061+0000 7f707559a700 1 -- 172.21.15.78:0/1100726726 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7068004160 con 0x7f7078076210 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.061+0000 7f707559a700 1 -- 172.21.15.78:0/1100726726 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7068004300 con 0x7f7078076210 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.061+0000 7f707559a700 1 -- 172.21.15.78:0/1100726726 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7068011b20 con 0x7f7078076210 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.062+0000 7f707ccb2700 1 -- 172.21.15.78:0/1100726726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 msgr2=0x7f707810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.062+0000 7f707ccb2700 1 --2- 172.21.15.78:0/1100726726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707810b920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70680098a0 tx=0x7f7068009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.062+0000 7f707ccb2700 1 -- 172.21.15.78:0/1100726726 shutdown_connections 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.062+0000 7f707ccb2700 1 --2- 172.21.15.78:0/1100726726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707810b920 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.062+0000 7f707ccb2700 1 -- 172.21.15.78:0/1100726726 >> 172.21.15.78:0/1100726726 conn(0x7f7078103330 msgr2=0x7f70781057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.062+0000 7f707ccb2700 1 -- 172.21.15.78:0/1100726726 shutdown_connections 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.062+0000 7f707ccb2700 1 -- 172.21.15.78:0/1100726726 wait complete. 2024-05-21T15:07:05.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.063+0000 7f707ccb2700 1 Processor -- start 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.063+0000 7f707ccb2700 1 -- start start 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.063+0000 7f707ccb2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.063+0000 7f707ccb2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f707810d1b0 con 0x7f7078076210 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f707659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f707659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707819f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46954/0 (socket says 172.21.15.78:46954) 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f707659c700 1 -- 172.21.15.78:0/2488239492 learned_addr learned my addr 172.21.15.78:0/2488239492 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f707659c700 1 -- 172.21.15.78:0/2488239492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707819f8b0 con 0x7f7078076210 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f707659c700 1 --2- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707819f370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7068000c00 tx=0x7f7068011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f70677fe700 1 -- 172.21.15.78:0/2488239492 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7068003b60 con 0x7f7078076210 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707819fb40 con 0x7f7078076210 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f70677fe700 1 -- 172.21.15.78:0/2488239492 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f706801a910 con 0x7f7078076210 2024-05-21T15:07:05.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f70677fe700 1 -- 172.21.15.78:0/2488239492 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7068024930 con 0x7f7078076210 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.064+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70781a2750 con 0x7f7078076210 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.065+0000 7f70677fe700 1 -- 172.21.15.78:0/2488239492 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7068022020 con 0x7f7078076210 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.065+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707819ffa0 con 0x7f7078076210 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.066+0000 7f70677fe700 1 --2- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f70600390a0 0x7f706003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.066+0000 7f70677fe700 1 -- 172.21.15.78:0/2488239492 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f706804d2c0 con 0x7f7078076210 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.066+0000 7f7075d9b700 1 --2- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f70600390a0 0x7f706003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.066+0000 7f7075d9b700 1 --2- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f70600390a0 0x7f706003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f706c006fd0 tx=0x7f706c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.071+0000 7f70677fe700 1 -- 172.21.15.78:0/2488239492 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f706804cc10 con 0x7f7078076210 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.327+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f7078063ca0 con 0x7f7078076210 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.330+0000 7f70677fe700 1 -- 172.21.15.78:0/2488239492 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f706802c510 con 0x7f7078076210 2024-05-21T15:07:05.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.332+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f70600390a0 msgr2=0x7f706003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.332+0000 7f707ccb2700 1 --2- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f70600390a0 0x7f706003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f706c006fd0 tx=0x7f706c006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.332+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 msgr2=0x7f707819f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.332+0000 7f707ccb2700 1 --2- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707819f370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7068000c00 tx=0x7f7068011c10 comp rx=0 tx=0).stop 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.332+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 shutdown_connections 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.332+0000 7f707ccb2700 1 --2- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f70600390a0 0x7f706003b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.332+0000 7f707ccb2700 1 --2- 172.21.15.78:0/2488239492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7078076210 0x7f707819f370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.333+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 >> 172.21.15.78:0/2488239492 conn(0x7f7078103330 msgr2=0x7f7078107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.333+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 shutdown_connections 2024-05-21T15:07:05.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:04.333+0000 7f707ccb2700 1 -- 172.21.15.78:0/2488239492 wait complete. 2024-05-21T15:07:06.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:06 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:07:06.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.974+0000 7f7f7b6c6700 1 Processor -- start 2024-05-21T15:07:06.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.974+0000 7f7f7b6c6700 1 -- start start 2024-05-21T15:07:06.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.974+0000 7f7f7b6c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7410a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.974+0000 7f7f7b6c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7410ad10 con 0x7f7f7410a330 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.975+0000 7f7f79462700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7410a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.975+0000 7f7f79462700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7410a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46966/0 (socket says 172.21.15.78:46966) 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.975+0000 7f7f79462700 1 -- 172.21.15.78:0/1793281744 learned_addr learned my addr 172.21.15.78:0/1793281744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.975+0000 7f7f79462700 1 -- 172.21.15.78:0/1793281744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7410ae90 con 0x7f7f7410a330 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.975+0000 7f7f79462700 1 --2- 172.21.15.78:0/1793281744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7410a740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009e80 tx=0x7f7f6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dada0c4b5755a14d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.975+0000 7f7f73fff700 1 -- 172.21.15.78:0/1793281744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f64017070 con 0x7f7f7410a330 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.975+0000 7f7f73fff700 1 -- 172.21.15.78:0/1793281744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f6400b8c0 con 0x7f7f7410a330 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.975+0000 7f7f73fff700 1 -- 172.21.15.78:0/1793281744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f640045c0 con 0x7f7f7410a330 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.976+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/1793281744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 msgr2=0x7f7f7410a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.976+0000 7f7f7b6c6700 1 --2- 172.21.15.78:0/1793281744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7410a740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009e80 tx=0x7f7f6400b330 comp rx=0 tx=0).stop 2024-05-21T15:07:06.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.976+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/1793281744 shutdown_connections 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.976+0000 7f7f7b6c6700 1 --2- 172.21.15.78:0/1793281744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7410a740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.976+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/1793281744 >> 172.21.15.78:0/1793281744 conn(0x7f7f74102ab0 msgr2=0x7f7f74104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.977+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/1793281744 shutdown_connections 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.977+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/1793281744 wait complete. 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.977+0000 7f7f7b6c6700 1 Processor -- start 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.977+0000 7f7f7b6c6700 1 -- start start 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.977+0000 7f7f7b6c6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7419d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.977+0000 7f7f7b6c6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f74107c80 con 0x7f7f7410a330 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.977+0000 7f7f79462700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7419d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f79462700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7419d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46976/0 (socket says 172.21.15.78:46976) 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f79462700 1 -- 172.21.15.78:0/870885959 learned_addr learned my addr 172.21.15.78:0/870885959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f79462700 1 -- 172.21.15.78:0/870885959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7419d6e0 con 0x7f7f7410a330 2024-05-21T15:07:06.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f79462700 1 --2- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7419d1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400bfa0 tx=0x7f7f64004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f727fc700 1 -- 172.21.15.78:0/870885959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f64020020 con 0x7f7f7410a330 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7419d970 con 0x7f7f7410a330 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f727fc700 1 -- 172.21.15.78:0/870885959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f64012430 con 0x7f7f7410a330 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f727fc700 1 -- 172.21.15.78:0/870885959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f64017040 con 0x7f7f7410a330 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.978+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f741a0580 con 0x7f7f7410a330 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.979+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f74063ca0 con 0x7f7f7410a330 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.979+0000 7f7f727fc700 1 -- 172.21.15.78:0/870885959 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7f64023020 con 0x7f7f7410a330 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.980+0000 7f7f727fc700 1 --2- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f7f5c039050 0x7f7f5c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.980+0000 7f7f727fc700 1 -- 172.21.15.78:0/870885959 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f6400f050 con 0x7f7f7410a330 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.980+0000 7f7f78c61700 1 --2- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f7f5c039050 0x7f7f5c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.980+0000 7f7f78c61700 1 --2- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f7f5c039050 0x7f7f5c03b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006fd0 tx=0x7f7f68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:06.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:05.986+0000 7f7f727fc700 1 -- 172.21.15.78:0/870885959 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f6402a020 con 0x7f7f7410a330 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.192+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f7f741a0750 con 0x7f7f7410a330 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.196+0000 7f7f727fc700 1 -- 172.21.15.78:0/870885959 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7f741a0750 con 0x7f7f7410a330 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.198+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f7f5c039050 msgr2=0x7f7f5c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.198+0000 7f7f7b6c6700 1 --2- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f7f5c039050 0x7f7f5c03b560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006fd0 tx=0x7f7f68006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.198+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 msgr2=0x7f7f7419d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.198+0000 7f7f7b6c6700 1 --2- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7419d1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400bfa0 tx=0x7f7f64004980 comp rx=0 tx=0).stop 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.199+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 shutdown_connections 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.199+0000 7f7f7b6c6700 1 --2- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f7f5c039050 0x7f7f5c03b560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.199+0000 7f7f7b6c6700 1 --2- 172.21.15.78:0/870885959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7410a330 0x7f7f7419d1a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.199+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 >> 172.21.15.78:0/870885959 conn(0x7f7f74102ab0 msgr2=0x7f7f74103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.199+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 shutdown_connections 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:06.199+0000 7f7f7b6c6700 1 -- 172.21.15.78:0/870885959 wait complete. 2024-05-21T15:07:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling the dashboard module... 2024-05-21T15:07:07.276 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:07 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/870885959' entity='client.admin' 2024-05-21T15:07:07.276 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:07 smithi078 ceph-mon[31997]: from='mgr.14120 172.21.15.78:0/4110824770' entity='mgr.x' 2024-05-21T15:07:08.410 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:08 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2412396978' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-21T15:07:08.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:08 smithi078 ceph-mon[31997]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:08.411 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: ignoring --setuser ceph since I am not root 2024-05-21T15:07:08.411 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: ignoring --setgroup ceph since I am not root 2024-05-21T15:07:08.411 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:08.406+0000 7fc57514c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:07:08.702 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:08.594+0000 7fc57514c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:07:08.703 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:08.698+0000 7fc57514c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:07:08.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.257+0000 7fe644529700 1 Processor -- start 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.257+0000 7fe644529700 1 -- start start 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.257+0000 7fe644529700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c10a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.257+0000 7fe644529700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c10ad60 con 0x7fe63c10a380 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.258+0000 7fe6422c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c10a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.258+0000 7fe6422c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c10a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46990/0 (socket says 172.21.15.78:46990) 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.258+0000 7fe6422c5700 1 -- 172.21.15.78:0/622142588 learned_addr learned my addr 172.21.15.78:0/622142588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.258+0000 7fe6422c5700 1 -- 172.21.15.78:0/622142588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c10aee0 con 0x7fe63c10a380 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.258+0000 7fe6422c5700 1 --2- 172.21.15.78:0/622142588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c10a790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6380098a0 tx=0x7fe638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3480d0a51833bb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.258+0000 7fe6412c3700 1 -- 172.21.15.78:0/622142588 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe638004160 con 0x7fe63c10a380 2024-05-21T15:07:08.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.258+0000 7fe6412c3700 1 -- 172.21.15.78:0/622142588 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe638004300 con 0x7fe63c10a380 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.258+0000 7fe6412c3700 1 -- 172.21.15.78:0/622142588 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe638011b20 con 0x7fe63c10a380 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.259+0000 7fe644529700 1 -- 172.21.15.78:0/622142588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 msgr2=0x7fe63c10a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.259+0000 7fe644529700 1 --2- 172.21.15.78:0/622142588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c10a790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6380098a0 tx=0x7fe638009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.259+0000 7fe644529700 1 -- 172.21.15.78:0/622142588 shutdown_connections 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.259+0000 7fe644529700 1 --2- 172.21.15.78:0/622142588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c10a790 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.259+0000 7fe644529700 1 -- 172.21.15.78:0/622142588 >> 172.21.15.78:0/622142588 conn(0x7fe63c102aa0 msgr2=0x7fe63c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.259+0000 7fe644529700 1 -- 172.21.15.78:0/622142588 shutdown_connections 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.259+0000 7fe644529700 1 -- 172.21.15.78:0/622142588 wait complete. 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.260+0000 7fe644529700 1 Processor -- start 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.260+0000 7fe644529700 1 -- start start 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.260+0000 7fe644529700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c19d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.260+0000 7fe644529700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c107c70 con 0x7fe63c10a380 2024-05-21T15:07:08.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.260+0000 7fe6422c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c19d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.260+0000 7fe6422c5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c19d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46998/0 (socket says 172.21.15.78:46998) 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.260+0000 7fe6422c5700 1 -- 172.21.15.78:0/2412396978 learned_addr learned my addr 172.21.15.78:0/2412396978 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.261+0000 7fe6422c5700 1 -- 172.21.15.78:0/2412396978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c19d6b0 con 0x7fe63c10a380 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.261+0000 7fe6422c5700 1 --2- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c19d170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe638011ad0 tx=0x7fe638011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.261+0000 7fe6337fe700 1 -- 172.21.15.78:0/2412396978 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe638003a40 con 0x7fe63c10a380 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.261+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63c19d940 con 0x7fe63c10a380 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.261+0000 7fe6337fe700 1 -- 172.21.15.78:0/2412396978 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe638003be0 con 0x7fe63c10a380 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.261+0000 7fe6337fe700 1 -- 172.21.15.78:0/2412396978 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe638024740 con 0x7fe63c10a380 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.261+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63c1a0550 con 0x7fe63c10a380 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.262+0000 7fe6337fe700 1 -- 172.21.15.78:0/2412396978 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe6380249f0 con 0x7fe63c10a380 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.262+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63c063ca0 con 0x7fe63c10a380 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.262+0000 7fe6337fe700 1 --2- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe628039050 0x7fe62803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:08.932 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.263+0000 7fe6337fe700 1 -- 172.21.15.78:0/2412396978 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe63804d340 con 0x7fe63c10a380 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.263+0000 7fe641ac4700 1 --2- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe628039050 0x7fe62803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.263+0000 7fe641ac4700 1 --2- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe628039050 0x7fe62803b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006fd0 tx=0x7fe62c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.272+0000 7fe6337fe700 1 -- 172.21.15.78:0/2412396978 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe63800fdd0 con 0x7fe63c10a380 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:07.540+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fe63c1a0720 con 0x7fe63c10a380 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.200+0000 7fe6337fe700 1 -- 172.21.15.78:0/2412396978 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fe63802d770 con 0x7fe63c10a380 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.202+0000 7fe6337fe700 1 -- 172.21.15.78:0/2412396978 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fe63801ab30 con 0x7fe63c10a380 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.203+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe628039050 msgr2=0x7fe62803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.203+0000 7fe644529700 1 --2- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe628039050 0x7fe62803b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006fd0 tx=0x7fe62c006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.204+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 msgr2=0x7fe63c19d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.204+0000 7fe644529700 1 --2- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c19d170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe638011ad0 tx=0x7fe638011bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.204+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 shutdown_connections 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.204+0000 7fe644529700 1 --2- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fe628039050 0x7fe62803b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:08.933 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.204+0000 7fe644529700 1 --2- 172.21.15.78:0/2412396978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe63c10a380 0x7fe63c19d170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:08.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.204+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 >> 172.21.15.78:0/2412396978 conn(0x7fe63c102aa0 msgr2=0x7fe63c103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:08.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.204+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 shutdown_connections 2024-05-21T15:07:08.934 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:08.204+0000 7fe644529700 1 -- 172.21.15.78:0/2412396978 wait complete. 2024-05-21T15:07:09.275 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:09 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2412396978' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-21T15:07:09.275 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:09 smithi078 ceph-mon[31997]: mgrmap e9: x(active, since 20s) 2024-05-21T15:07:09.275 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:09 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:09.013+0000 7fc57514c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:07:09.275 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:09 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:09.102+0000 7fc57514c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:07:10.195 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:07:10.195 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-21T15:07:10.195 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T15:07:10.195 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T15:07:10.195 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T15:07:10.195 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.326+0000 7fb0a828c700 1 Processor -- start 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.326+0000 7fb0a828c700 1 -- start start 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.326+0000 7fb0a828c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a010b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.326+0000 7fb0a828c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a010bf10 con 0x7fb0a00761f0 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.326+0000 7fb0a6028700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a010b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.326+0000 7fb0a6028700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a010b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54002/0 (socket says 172.21.15.78:54002) 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.326+0000 7fb0a6028700 1 -- 172.21.15.78:0/952386213 learned_addr learned my addr 172.21.15.78:0/952386213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.327+0000 7fb0a6028700 1 -- 172.21.15.78:0/952386213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a010c090 con 0x7fb0a00761f0 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.327+0000 7fb0a6028700 1 --2- 172.21.15.78:0/952386213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a010b940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0098a0 tx=0x7fb09c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e4ed7c07b4736b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.327+0000 7fb0a5026700 1 -- 172.21.15.78:0/952386213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb09c004160 con 0x7fb0a00761f0 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.327+0000 7fb0a5026700 1 -- 172.21.15.78:0/952386213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c004300 con 0x7fb0a00761f0 2024-05-21T15:07:10.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.327+0000 7fb0a5026700 1 -- 172.21.15.78:0/952386213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb09c011b20 con 0x7fb0a00761f0 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.328+0000 7fb0a828c700 1 -- 172.21.15.78:0/952386213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 msgr2=0x7fb0a010b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.328+0000 7fb0a828c700 1 --2- 172.21.15.78:0/952386213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a010b940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0098a0 tx=0x7fb09c009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.328+0000 7fb0a828c700 1 -- 172.21.15.78:0/952386213 shutdown_connections 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.328+0000 7fb0a828c700 1 --2- 172.21.15.78:0/952386213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a010b940 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.328+0000 7fb0a828c700 1 -- 172.21.15.78:0/952386213 >> 172.21.15.78:0/952386213 conn(0x7fb0a0103330 msgr2=0x7fb0a01057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.328+0000 7fb0a828c700 1 -- 172.21.15.78:0/952386213 shutdown_connections 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.328+0000 7fb0a828c700 1 -- 172.21.15.78:0/952386213 wait complete. 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.329+0000 7fb0a828c700 1 Processor -- start 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.329+0000 7fb0a828c700 1 -- start start 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.329+0000 7fb0a828c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a019d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.329+0000 7fb0a828c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a010d1b0 con 0x7fb0a00761f0 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.329+0000 7fb0a6028700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a019d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:10.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.329+0000 7fb0a6028700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a019d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54014/0 (socket says 172.21.15.78:54014) 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.329+0000 7fb0a6028700 1 -- 172.21.15.78:0/1084701199 learned_addr learned my addr 172.21.15.78:0/1084701199 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.329+0000 7fb0a6028700 1 -- 172.21.15.78:0/1084701199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a019d640 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.330+0000 7fb0a6028700 1 --2- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a019d100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb09c011740 tx=0x7fb09c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.330+0000 7fb0977fe700 1 -- 172.21.15.78:0/1084701199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb09c003a40 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.330+0000 7fb0977fe700 1 -- 172.21.15.78:0/1084701199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c003be0 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.330+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a019d8d0 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.330+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a01a04e0 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.330+0000 7fb0977fe700 1 -- 172.21.15.78:0/1084701199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb09c01a750 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.331+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a019dd30 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.331+0000 7fb0977fe700 1 -- 172.21.15.78:0/1084701199 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb09c01ab10 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.331+0000 7fb0977fe700 1 --2- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb08c039050 0x7fb08c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.331+0000 7fb0977fe700 1 -- 172.21.15.78:0/1084701199 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb09c04d7b0 con 0x7fb0a00761f0 2024-05-21T15:07:10.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.331+0000 7fb0a5827700 1 -- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb08c039050 msgr2=0x7fb08c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3694112952 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.331+0000 7fb0a5827700 1 --2- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb08c039050 0x7fb08c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.336+0000 7fb0977fe700 1 -- 172.21.15.78:0/1084701199 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb09c011e20 con 0x7fb0a00761f0 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.532+0000 7fb0a5827700 1 -- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb08c039050 msgr2=0x7fb08c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3694112952 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.532+0000 7fb0a5827700 1 --2- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb08c039050 0x7fb08c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.631+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb0a01a06b0 con 0x7fb0a00761f0 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.631+0000 7fb0977fe700 1 -- 172.21.15.78:0/1084701199 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fb09c00bc40 con 0x7fb0a00761f0 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb08c039050 msgr2=0x7fb08c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 --2- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb08c039050 0x7fb08c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 msgr2=0x7fb0a019d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 --2- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a019d100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb09c011740 tx=0x7fb09c011ba0 comp rx=0 tx=0).stop 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 shutdown_connections 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 --2- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7fb08c039050 0x7fb08c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:10.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 --2- 172.21.15.78:0/1084701199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0a00761f0 0x7fb0a019d100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:10.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 >> 172.21.15.78:0/1084701199 conn(0x7fb0a0103330 msgr2=0x7fb0a0103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:10.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 shutdown_connections 2024-05-21T15:07:10.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:09.633+0000 7fb0a828c700 1 -- 172.21.15.78:0/1084701199 wait complete. 2024-05-21T15:07:10.200 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-05-21T15:07:10.200 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 9... 2024-05-21T15:07:10.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:10 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1084701199' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-21T15:07:10.498 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:10.354+0000 7fc57514c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:07:10.819 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:10.532+0000 7fc57514c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:07:10.819 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:10.712+0000 7fc57514c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:07:11.098 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:11 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:11.005+0000 7fc57514c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:07:11.502 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:11 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:11.098+0000 7fc57514c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:07:11.502 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:11 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:11.300+0000 7fc57514c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:07:11.847 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:11 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:11.495+0000 7fc57514c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:07:12.347 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:12 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:12.098+0000 7fc57514c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:07:12.833 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:12 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:12.526+0000 7fc57514c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:07:12.833 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:12 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:12.624+0000 7fc57514c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:07:12.833 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:12 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:12.828+0000 7fc57514c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:07:13.622 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:13 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:13.347+0000 7fc57514c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:07:13.622 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:13 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:13.441+0000 7fc57514c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:07:13.921 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:13 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:13.687+0000 7fc57514c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:07:13.921 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:13 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:13.821+0000 7fc57514c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:07:13.922 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:13 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:13.917+0000 7fc57514c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:07:14.335 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:14 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:14.035+0000 7fc57514c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:07:14.598 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:14 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:14.331+0000 7fc57514c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:07:15.214 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:14 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:14.934+0000 7fc57514c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:07:15.215 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:07:15 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:07:15.154+0000 7fc57514c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: Active manager daemon x restarted 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: Activating manager daemon x 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: osdmap e3: 0 total, 0 up, 0 in 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: mgrmap e10: x(active, starting, since 0.00381515s) 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: Manager daemon x is now available 2024-05-21T15:07:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:15.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:07:16.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:07:16.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:16 smithi078 ceph-mon[31997]: mgrmap e11: x(active, since 1.00726s) 2024-05-21T15:07:17.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-05-21T15:07:17.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-21T15:07:17.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T15:07:17.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-05-21T15:07:17.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.608+0000 7f1f782e3700 1 Processor -- start 2024-05-21T15:07:17.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.608+0000 7f1f782e3700 1 -- start start 2024-05-21T15:07:17.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f782e3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70107420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f782e3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f701079f0 con 0x7f1f70107010 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f7607f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f7607f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70107420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54026/0 (socket says 172.21.15.78:54026) 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f7607f700 1 -- 172.21.15.78:0/3110111176 learned_addr learned my addr 172.21.15.78:0/3110111176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f7607f700 1 -- 172.21.15.78:0/3110111176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f70107b70 con 0x7f1f70107010 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f7607f700 1 --2- 172.21.15.78:0/3110111176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70107420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f600098a0 tx=0x7f1f60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9edaa565cf9afa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f7507d700 1 -- 172.21.15.78:0/3110111176 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f60004160 con 0x7f1f70107010 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f7507d700 1 -- 172.21.15.78:0/3110111176 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f60004300 con 0x7f1f70107010 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.609+0000 7f1f7507d700 1 -- 172.21.15.78:0/3110111176 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f60011b20 con 0x7f1f70107010 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.610+0000 7f1f782e3700 1 -- 172.21.15.78:0/3110111176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 msgr2=0x7f1f70107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:17.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.610+0000 7f1f782e3700 1 --2- 172.21.15.78:0/3110111176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70107420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f600098a0 tx=0x7f1f60009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.610+0000 7f1f782e3700 1 -- 172.21.15.78:0/3110111176 shutdown_connections 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.610+0000 7f1f782e3700 1 --2- 172.21.15.78:0/3110111176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70107420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.610+0000 7f1f782e3700 1 -- 172.21.15.78:0/3110111176 >> 172.21.15.78:0/3110111176 conn(0x7f1f70102aa0 msgr2=0x7f1f70104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.610+0000 7f1f782e3700 1 -- 172.21.15.78:0/3110111176 shutdown_connections 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.610+0000 7f1f782e3700 1 -- 172.21.15.78:0/3110111176 wait complete. 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.611+0000 7f1f782e3700 1 Processor -- start 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.611+0000 7f1f782e3700 1 -- start start 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.611+0000 7f1f782e3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70109850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.611+0000 7f1f782e3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7010d300 con 0x7f1f70107010 2024-05-21T15:07:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f7607f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70109850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f7607f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70109850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54040/0 (socket says 172.21.15.78:54040) 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f7607f700 1 -- 172.21.15.78:0/3490475221 learned_addr learned my addr 172.21.15.78:0/3490475221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f7607f700 1 -- 172.21.15.78:0/3490475221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f70109d90 con 0x7f1f70107010 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f7607f700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70109850 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009fb0 tx=0x7f1f60011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f60003930 con 0x7f1f70107010 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f60003ad0 con 0x7f1f70107010 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f782e3700 1 -- 172.21.15.78:0/3490475221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f70107fe0 con 0x7f1f70107010 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f60019670 con 0x7f1f70107010 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.612+0000 7f1f782e3700 1 -- 172.21.15.78:0/3490475221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f70108440 con 0x7f1f70107010 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.614+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1f60019970 con 0x7f1f70107010 2024-05-21T15:07:17.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.614+0000 7f1f677fe700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 0x7f1f5c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.614+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 --> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1f5c03bc70 con 0x7f1f5c039050 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.614+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f6004d640 con 0x7f1f70107010 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.614+0000 7f1f7587e700 1 -- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 msgr2=0x7f1f5c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3694112952 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.614+0000 7f1f7587e700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 0x7f1f5c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.815+0000 7f1f7587e700 1 -- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 msgr2=0x7f1f5c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3694112952 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:10.815+0000 7f1f7587e700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 0x7f1f5c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:11.216+0000 7f1f7587e700 1 -- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 msgr2=0x7f1f5c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3694112952 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:11.216+0000 7f1f7587e700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 0x7f1f5c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:12.017+0000 7f1f7587e700 1 -- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 msgr2=0x7f1f5c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3694112952 2024-05-21T15:07:17.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:12.017+0000 7f1f7587e700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 0x7f1f5c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:13.618+0000 7f1f7587e700 1 -- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 msgr2=0x7f1f5c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/3694112952 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:13.618+0000 7f1f7587e700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 0x7f1f5c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:15.158+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f1f60019d00 con 0x7f1f70107010 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:15.159+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 msgr2=0x7f1f5c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:15.159+0000 7f1f677fe700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 0x7f1f5c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.160+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f1f6004f2f0 con 0x7f1f70107010 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.160+0000 7f1f677fe700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1f6004d1d0 0x7f1f5c03eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.160+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1f6002be50 con 0x7f1f6004d1d0 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.161+0000 7f1f7587e700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1f6004d1d0 0x7f1f5c03eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:17.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.161+0000 7f1f7587e700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1f6004d1d0 0x7f1f5c03eb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c003a10 tx=0x7f1f6c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.162+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f1f6002be50 con 0x7f1f6004d1d0 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.169+0000 7f1f657fa700 1 -- 172.21.15.78:0/3490475221 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1f70063ca0 con 0x7f1f6004d1d0 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.169+0000 7f1f677fe700 1 -- 172.21.15.78:0/3490475221 <== mgr.14152 v2:172.21.15.78:6800/2228258510 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1f70063ca0 con 0x7f1f6004d1d0 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 -- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1f6004d1d0 msgr2=0x7f1f5c03eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1f6004d1d0 0x7f1f5c03eb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c003a10 tx=0x7f1f6c0092b0 comp rx=0 tx=0).stop 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 -- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 msgr2=0x7f1f70109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70109850 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009fb0 tx=0x7f1f60011c10 comp rx=0 tx=0).stop 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 -- 172.21.15.78:0/3490475221 shutdown_connections 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1f6004d1d0 0x7f1f5c03eb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f70107010 0x7f1f70109850 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 --2- 172.21.15.78:0/3490475221 >> [v2:172.21.15.78:6800/3694112952,v1:172.21.15.78:6801/3694112952] conn(0x7f1f5c039050 0x7f1f5c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 -- 172.21.15.78:0/3490475221 >> 172.21.15.78:0/3490475221 conn(0x7f1f70102aa0 msgr2=0x7f1f70103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:17.087 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 -- 172.21.15.78:0/3490475221 shutdown_connections 2024-05-21T15:07:17.088 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:16.170+0000 7f1f657fa700 1 -- 172.21.15.78:0/3490475221 wait complete. 2024-05-21T15:07:17.088 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 9 is available 2024-05-21T15:07:17.088 INFO:teuthology.orchestra.run.smithi078.stdout:Generating a dashboard self-signed certificate... 2024-05-21T15:07:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:17 smithi078 ceph-mon[31997]: [21/May/2024:15:07:15] ENGINE Bus STARTING 2024-05-21T15:07:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:17 smithi078 ceph-mon[31997]: [21/May/2024:15:07:16] ENGINE Serving on http://172.21.15.78:8765 2024-05-21T15:07:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:17 smithi078 ceph-mon[31997]: [21/May/2024:15:07:16] ENGINE Serving on https://172.21.15.78:7150 2024-05-21T15:07:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:17 smithi078 ceph-mon[31997]: [21/May/2024:15:07:16] ENGINE Bus STARTED 2024-05-21T15:07:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:17 smithi078 ceph-mon[31997]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T15:07:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:17 smithi078 ceph-mon[31997]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T15:07:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:17 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:18.393 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.499+0000 7f02dd63b700 1 Processor -- start 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.500+0000 7f02dd63b700 1 -- start start 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.500+0000 7f02dd63b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d8107790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.500+0000 7f02dd63b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8107d80 con 0x7f02d8107380 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.500+0000 7f02d6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d8107790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.500+0000 7f02d6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d8107790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54118/0 (socket says 172.21.15.78:54118) 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.500+0000 7f02d6ffd700 1 -- 172.21.15.78:0/1187033217 learned_addr learned my addr 172.21.15.78:0/1187033217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.500+0000 7f02d6ffd700 1 -- 172.21.15.78:0/1187033217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d8108610 con 0x7f02d8107380 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.500+0000 7f02d6ffd700 1 --2- 172.21.15.78:0/1187033217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d8107790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00bb40 tx=0x7f02cc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be190c9d44b50e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:18.394 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.501+0000 7f02d5ffb700 1 -- 172.21.15.78:0/1187033217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02cc00dad0 con 0x7f02d8107380 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.501+0000 7f02d5ffb700 1 -- 172.21.15.78:0/1187033217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc004510 con 0x7f02d8107380 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.501+0000 7f02d5ffb700 1 -- 172.21.15.78:0/1187033217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02cc003e60 con 0x7f02d8107380 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.502+0000 7f02dd63b700 1 -- 172.21.15.78:0/1187033217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 msgr2=0x7f02d8107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.502+0000 7f02dd63b700 1 --2- 172.21.15.78:0/1187033217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d8107790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00bb40 tx=0x7f02cc00be50 comp rx=0 tx=0).stop 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.502+0000 7f02dd63b700 1 -- 172.21.15.78:0/1187033217 shutdown_connections 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.502+0000 7f02dd63b700 1 --2- 172.21.15.78:0/1187033217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d8107790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.502+0000 7f02dd63b700 1 -- 172.21.15.78:0/1187033217 >> 172.21.15.78:0/1187033217 conn(0x7f02d8102aa0 msgr2=0x7f02d8104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.502+0000 7f02dd63b700 1 -- 172.21.15.78:0/1187033217 shutdown_connections 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.502+0000 7f02dd63b700 1 -- 172.21.15.78:0/1187033217 wait complete. 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.503+0000 7f02dd63b700 1 Processor -- start 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.503+0000 7f02dd63b700 1 -- start start 2024-05-21T15:07:18.395 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.503+0000 7f02dd63b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.503+0000 7f02dd63b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8108d80 con 0x7f02d8107380 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.503+0000 7f02d6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.503+0000 7f02d6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d819afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54132/0 (socket says 172.21.15.78:54132) 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.504+0000 7f02d6ffd700 1 -- 172.21.15.78:0/1737351391 learned_addr learned my addr 172.21.15.78:0/1737351391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.504+0000 7f02d6ffd700 1 -- 172.21.15.78:0/1737351391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d819b510 con 0x7f02d8107380 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.504+0000 7f02d6ffd700 1 --2- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d819afd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02cc003bf0 tx=0x7f02cc003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.504+0000 7f02bffff700 1 -- 172.21.15.78:0/1737351391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02cc0175d0 con 0x7f02d8107380 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.504+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d819b7a0 con 0x7f02d8107380 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.504+0000 7f02bffff700 1 -- 172.21.15.78:0/1737351391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc017c50 con 0x7f02d8107380 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.504+0000 7f02bffff700 1 -- 172.21.15.78:0/1737351391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02cc021a40 con 0x7f02d8107380 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.504+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d819e3b0 con 0x7f02d8107380 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.505+0000 7f02bffff700 1 -- 172.21.15.78:0/1737351391 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f02cc02a400 con 0x7f02d8107380 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.505+0000 7f02bffff700 1 --2- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f02c0038f20 0x7f02c003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:18.396 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.505+0000 7f02bffff700 1 -- 172.21.15.78:0/1737351391 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f02cc050d30 con 0x7f02d8107380 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.505+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d8063ca0 con 0x7f02d8107380 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.505+0000 7f02d67fc700 1 --2- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f02c0038f20 0x7f02c003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.506+0000 7f02d67fc700 1 --2- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f02c0038f20 0x7f02c003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02c8006fd0 tx=0x7f02c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.511+0000 7f02bffff700 1 -- 172.21.15.78:0/1737351391 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f02cc02f070 con 0x7f02d8107380 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.719+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f02d802dfd0 con 0x7f02c0038f20 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.766+0000 7f02bffff700 1 -- 172.21.15.78:0/1737351391 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f02d802dfd0 con 0x7f02c0038f20 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.769+0000 7f02bffff700 1 -- 172.21.15.78:0/1737351391 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f02cc02ad00 con 0x7f02d8107380 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.769+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f02c0038f20 msgr2=0x7f02c003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.769+0000 7f02dd63b700 1 --2- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f02c0038f20 0x7f02c003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02c8006fd0 tx=0x7f02c8006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.770+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 msgr2=0x7f02d819afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.770+0000 7f02dd63b700 1 --2- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d819afd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02cc003bf0 tx=0x7f02cc003fb0 comp rx=0 tx=0).stop 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.770+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 shutdown_connections 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.770+0000 7f02dd63b700 1 --2- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f02c0038f20 0x7f02c003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:18.397 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.770+0000 7f02dd63b700 1 --2- 172.21.15.78:0/1737351391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02d8107380 0x7f02d819afd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:18.398 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.770+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 >> 172.21.15.78:0/1737351391 conn(0x7f02d8102aa0 msgr2=0x7f02d8191630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:18.398 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.770+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 shutdown_connections 2024-05-21T15:07:18.398 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:17.770+0000 7f02dd63b700 1 -- 172.21.15.78:0/1737351391 wait complete. 2024-05-21T15:07:18.398 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial admin user... 2024-05-21T15:07:19.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:18 smithi078 ceph-mon[31997]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:19.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:19.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:19.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:18 smithi078 ceph-mon[31997]: mgrmap e12: x(active, since 2s) 2024-05-21T15:07:19.024 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$XjpyFfvaqChjQ7kOUniK7OrtKtHMI05sv7r2yAZyM77La9OwjWN4G", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716304039, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.774+0000 7feedad2a700 1 Processor -- start 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.774+0000 7feedad2a700 1 -- start start 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.774+0000 7feedad2a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed4109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.774+0000 7feedad2a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feed4109710 con 0x7feed4108d10 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.774+0000 7feed8ac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed4109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.774+0000 7feed8ac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed4109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41640/0 (socket says 172.21.15.78:41640) 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.774+0000 7feed8ac6700 1 -- 172.21.15.78:0/3060491328 learned_addr learned my addr 172.21.15.78:0/3060491328 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.775+0000 7feed8ac6700 1 -- 172.21.15.78:0/3060491328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4109fa0 con 0x7feed4108d10 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.775+0000 7feed8ac6700 1 --2- 172.21.15.78:0/3060491328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed4109120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feec80098a0 tx=0x7feec8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18156f6aee365049 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.775+0000 7feed37fe700 1 -- 172.21.15.78:0/3060491328 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feec8004160 con 0x7feed4108d10 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.775+0000 7feed37fe700 1 -- 172.21.15.78:0/3060491328 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec8004300 con 0x7feed4108d10 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.775+0000 7feed37fe700 1 -- 172.21.15.78:0/3060491328 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feec8011b20 con 0x7feed4108d10 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 -- 172.21.15.78:0/3060491328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 msgr2=0x7feed4109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 --2- 172.21.15.78:0/3060491328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed4109120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feec80098a0 tx=0x7feec8009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 -- 172.21.15.78:0/3060491328 shutdown_connections 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 --2- 172.21.15.78:0/3060491328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed4109120 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 -- 172.21.15.78:0/3060491328 >> 172.21.15.78:0/3060491328 conn(0x7feed4104410 msgr2=0x7feed41068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 -- 172.21.15.78:0/3060491328 shutdown_connections 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 -- 172.21.15.78:0/3060491328 wait complete. 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 Processor -- start 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 -- start start 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed419d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:19.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.776+0000 7feedad2a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feed410d350 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feed8ac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed419d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feed8ac6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed419d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41656/0 (socket says 172.21.15.78:41656) 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feed8ac6700 1 -- 172.21.15.78:0/885819613 learned_addr learned my addr 172.21.15.78:0/885819613 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feed8ac6700 1 -- 172.21.15.78:0/885819613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed419d660 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feed8ac6700 1 --2- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed419d120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feec8011ad0 tx=0x7feec8011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feed1ffb700 1 -- 172.21.15.78:0/885819613 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feec8003a40 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feed1ffb700 1 -- 172.21.15.78:0/885819613 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec8003be0 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed419d8f0 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feed1ffb700 1 -- 172.21.15.78:0/885819613 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feec8024740 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.777+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41a0500 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.778+0000 7feed1ffb700 1 -- 172.21.15.78:0/885819613 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7feec80249f0 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.778+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed419dd50 con 0x7feed4108d10 2024-05-21T15:07:19.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.778+0000 7feed1ffb700 1 --2- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feec40390a0 0x7feec403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.778+0000 7feed1ffb700 1 -- 172.21.15.78:0/885819613 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feec804d480 con 0x7feed4108d10 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.779+0000 7feed3fff700 1 --2- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feec40390a0 0x7feec403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.779+0000 7feed3fff700 1 --2- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feec40390a0 0x7feec403b5b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feec0006fd0 tx=0x7feec0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:18.783+0000 7feed1ffb700 1 -- 172.21.15.78:0/885819613 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feec80094f0 con 0x7feed4108d10 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.020+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7feed4063ca0 con 0x7feec40390a0 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.246+0000 7feed1ffb700 1 -- 172.21.15.78:0/885819613 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7feed4063ca0 con 0x7feec40390a0 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feec40390a0 msgr2=0x7feec403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 --2- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feec40390a0 0x7feec403b5b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feec0006fd0 tx=0x7feec0006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 msgr2=0x7feed419d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 --2- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed419d120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feec8011ad0 tx=0x7feec8011bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 shutdown_connections 2024-05-21T15:07:19.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 --2- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feec40390a0 0x7feec403b5b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:19.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 --2- 172.21.15.78:0/885819613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feed4108d10 0x7feed419d120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:19.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 >> 172.21.15.78:0/885819613 conn(0x7feed4104410 msgr2=0x7feed4104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:19.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 shutdown_connections 2024-05-21T15:07:19.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:19.249+0000 7feedad2a700 1 -- 172.21.15.78:0/885819613 wait complete. 2024-05-21T15:07:19.983 INFO:teuthology.orchestra.run.smithi078.stdout:Fetching dashboard port number... 2024-05-21T15:07:20.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:20 smithi078 ceph-mon[31997]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:21.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 8443 2024-05-21T15:07:21.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.416+0000 7ff8aeb0b700 1 Processor -- start 2024-05-21T15:07:21.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.416+0000 7ff8aeb0b700 1 -- start start 2024-05-21T15:07:21.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.417+0000 7ff8aeb0b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a810a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:21.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.417+0000 7ff8aeb0b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a810ad60 con 0x7ff8a810a380 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.417+0000 7ff8ac8a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a810a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.417+0000 7ff8ac8a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a810a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41668/0 (socket says 172.21.15.78:41668) 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.417+0000 7ff8ac8a7700 1 -- 172.21.15.78:0/800562359 learned_addr learned my addr 172.21.15.78:0/800562359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.417+0000 7ff8ac8a7700 1 -- 172.21.15.78:0/800562359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a810aee0 con 0x7ff8a810a380 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.417+0000 7ff8ac8a7700 1 --2- 172.21.15.78:0/800562359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a810a790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0098a0 tx=0x7ff89c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86e41b5bb84fed server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.417+0000 7ff8a77fe700 1 -- 172.21.15.78:0/800562359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff89c004160 con 0x7ff8a810a380 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8a77fe700 1 -- 172.21.15.78:0/800562359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89c004300 con 0x7ff8a810a380 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8a77fe700 1 -- 172.21.15.78:0/800562359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff89c011b20 con 0x7ff8a810a380 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/800562359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 msgr2=0x7ff8a810a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8aeb0b700 1 --2- 172.21.15.78:0/800562359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a810a790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0098a0 tx=0x7ff89c009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/800562359 shutdown_connections 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8aeb0b700 1 --2- 172.21.15.78:0/800562359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a810a790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:21.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/800562359 >> 172.21.15.78:0/800562359 conn(0x7ff8a8102aa0 msgr2=0x7ff8a8104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/800562359 shutdown_connections 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.418+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/800562359 wait complete. 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.419+0000 7ff8aeb0b700 1 Processor -- start 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.419+0000 7ff8aeb0b700 1 -- start start 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.419+0000 7ff8aeb0b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a819d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.419+0000 7ff8aeb0b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a8107c70 con 0x7ff8a810a380 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.419+0000 7ff8ac8a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a819d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.419+0000 7ff8ac8a7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a819d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41672/0 (socket says 172.21.15.78:41672) 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.419+0000 7ff8ac8a7700 1 -- 172.21.15.78:0/1257768529 learned_addr learned my addr 172.21.15.78:0/1257768529 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.420+0000 7ff8ac8a7700 1 -- 172.21.15.78:0/1257768529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a819d6b0 con 0x7ff8a810a380 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.420+0000 7ff8ac8a7700 1 --2- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a819d170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fb0 tx=0x7ff89c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.420+0000 7ff8a5ffb700 1 -- 172.21.15.78:0/1257768529 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff89c003930 con 0x7ff8a810a380 2024-05-21T15:07:21.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.420+0000 7ff8a5ffb700 1 -- 172.21.15.78:0/1257768529 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89c003ad0 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.420+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a819d940 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.420+0000 7ff8a5ffb700 1 -- 172.21.15.78:0/1257768529 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff89c019670 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.420+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a81a0550 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.421+0000 7ff8a5ffb700 1 -- 172.21.15.78:0/1257768529 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff89c019920 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.421+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a819dda0 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.421+0000 7ff8a5ffb700 1 --2- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8980390b0 0x7ff89803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.422+0000 7ff8a5ffb700 1 -- 172.21.15.78:0/1257768529 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff89c04d770 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.422+0000 7ff8a7fff700 1 --2- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8980390b0 0x7ff89803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.422+0000 7ff8a7fff700 1 --2- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8980390b0 0x7ff89803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff894006fd0 tx=0x7ff894006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.427+0000 7ff8a5ffb700 1 -- 172.21.15.78:0/1257768529 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff89c011e20 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.628+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff8a81a0720 con 0x7ff8a810a380 2024-05-21T15:07:21.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.629+0000 7ff8a5ffb700 1 -- 172.21.15.78:0/1257768529 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff89c019d60 con 0x7ff8a810a380 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8980390b0 msgr2=0x7ff89803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 --2- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8980390b0 0x7ff89803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff894006fd0 tx=0x7ff894006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 msgr2=0x7ff8a819d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 --2- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a819d170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fb0 tx=0x7ff89c011c10 comp rx=0 tx=0).stop 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 shutdown_connections 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 --2- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8980390b0 0x7ff89803b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 --2- 172.21.15.78:0/1257768529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8a810a380 0x7ff8a819d170 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 >> 172.21.15.78:0/1257768529 conn(0x7ff8a8102aa0 msgr2=0x7ff8a8103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 shutdown_connections 2024-05-21T15:07:21.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:20.631+0000 7ff8aeb0b700 1 -- 172.21.15.78:0/1257768529 wait complete. 2024-05-21T15:07:21.387 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T15:07:21.387 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-05-21T15:07:21.395 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T15:07:21.395 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-05-21T15:07:21.395 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-05-21T15:07:21.395 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-21T15:07:21.396 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph Dashboard is now available at: 2024-05-21T15:07:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: URL: https://smithi078.front.sepia.ceph.com:8443/ 2024-05-21T15:07:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: User: admin 2024-05-21T15:07:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: Password: t75ftx7qvv 2024-05-21T15:07:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:21.397 INFO:teuthology.orchestra.run.smithi078.stdout:Saving cluster configuration to /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config directory 2024-05-21T15:07:21.397 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling autotune for osd_memory_target 2024-05-21T15:07:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:21 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1257768529' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-21T15:07:22.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:22 smithi078 ceph-mon[31997]: mgrmap e13: x(active, since 6s) 2024-05-21T15:07:22.660 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.818+0000 7fd2afd30700 1 Processor -- start 2024-05-21T15:07:22.660 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.818+0000 7fd2afd30700 1 -- start start 2024-05-21T15:07:22.660 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.818+0000 7fd2afd30700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a8109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:22.660 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2afd30700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8109710 con 0x7fd2a8108d10 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2adacc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a8109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2adacc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a8109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41684/0 (socket says 172.21.15.78:41684) 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2adacc700 1 -- 172.21.15.78:0/344491915 learned_addr learned my addr 172.21.15.78:0/344491915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2adacc700 1 -- 172.21.15.78:0/344491915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a8109fa0 con 0x7fd2a8108d10 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2adacc700 1 --2- 172.21.15.78:0/344491915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a8109120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40098a0 tx=0x7fd2a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6af3fa9bc3fd1d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2acaca700 1 -- 172.21.15.78:0/344491915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2a4004160 con 0x7fd2a8108d10 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2acaca700 1 -- 172.21.15.78:0/344491915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4004300 con 0x7fd2a8108d10 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.819+0000 7fd2acaca700 1 -- 172.21.15.78:0/344491915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2a4011b20 con 0x7fd2a8108d10 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.820+0000 7fd2afd30700 1 -- 172.21.15.78:0/344491915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 msgr2=0x7fd2a8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.820+0000 7fd2afd30700 1 --2- 172.21.15.78:0/344491915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a8109120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40098a0 tx=0x7fd2a4009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.820+0000 7fd2afd30700 1 -- 172.21.15.78:0/344491915 shutdown_connections 2024-05-21T15:07:22.661 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.820+0000 7fd2afd30700 1 --2- 172.21.15.78:0/344491915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a8109120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.820+0000 7fd2afd30700 1 -- 172.21.15.78:0/344491915 >> 172.21.15.78:0/344491915 conn(0x7fd2a8104410 msgr2=0x7fd2a81068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.820+0000 7fd2afd30700 1 -- 172.21.15.78:0/344491915 shutdown_connections 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.820+0000 7fd2afd30700 1 -- 172.21.15.78:0/344491915 wait complete. 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.821+0000 7fd2afd30700 1 Processor -- start 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.821+0000 7fd2afd30700 1 -- start start 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.821+0000 7fd2afd30700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a819d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.821+0000 7fd2afd30700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a810d350 con 0x7fd2a8108d10 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.821+0000 7fd2adacc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a819d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.821+0000 7fd2adacc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a819d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41688/0 (socket says 172.21.15.78:41688) 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.821+0000 7fd2adacc700 1 -- 172.21.15.78:0/3493335489 learned_addr learned my addr 172.21.15.78:0/3493335489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.821+0000 7fd2adacc700 1 -- 172.21.15.78:0/3493335489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a819d660 con 0x7fd2a8108d10 2024-05-21T15:07:22.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.822+0000 7fd2adacc700 1 --2- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a819d120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4011ad0 tx=0x7fd2a4011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.822+0000 7fd29affd700 1 -- 172.21.15.78:0/3493335489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2a4003a40 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.822+0000 7fd29affd700 1 -- 172.21.15.78:0/3493335489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4003be0 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.822+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a819d8f0 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.822+0000 7fd29affd700 1 -- 172.21.15.78:0/3493335489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2a4024740 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.822+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81a0500 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.823+0000 7fd29affd700 1 -- 172.21.15.78:0/3493335489 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd2a40249f0 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.823+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a819dd50 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.823+0000 7fd29affd700 1 --2- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd294039140 0x7fd29403b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.823+0000 7fd29affd700 1 -- 172.21.15.78:0/3493335489 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd2a404d7d0 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.823+0000 7fd2ad2cb700 1 --2- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd294039140 0x7fd29403b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.824+0000 7fd2ad2cb700 1 --2- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd294039140 0x7fd29403b650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd29c006fd0 tx=0x7fd29c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:21.828+0000 7fd29affd700 1 -- 172.21.15.78:0/3493335489 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd2a40094f0 con 0x7fd2a8108d10 2024-05-21T15:07:22.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.023+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fd2a81a06d0 con 0x7fd2a8108d10 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.023+0000 7fd29affd700 1 -- 172.21.15.78:0/3493335489 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fd2a4016df0 con 0x7fd2a8108d10 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd294039140 msgr2=0x7fd29403b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 --2- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd294039140 0x7fd29403b650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd29c006fd0 tx=0x7fd29c006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 msgr2=0x7fd2a819d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 --2- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a819d120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4011ad0 tx=0x7fd2a4011bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 shutdown_connections 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 --2- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd294039140 0x7fd29403b650 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 --2- 172.21.15.78:0/3493335489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8108d10 0x7fd2a819d120 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 >> 172.21.15.78:0/3493335489 conn(0x7fd2a8104410 msgr2=0x7fd2a8104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 shutdown_connections 2024-05-21T15:07:22.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:22.026+0000 7fd2afd30700 1 -- 172.21.15.78:0/3493335489 wait complete. 2024-05-21T15:07:23.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.066+0000 7fb8d7d8a700 1 Processor -- start 2024-05-21T15:07:23.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.066+0000 7fb8d7d8a700 1 -- start start 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.067+0000 7fb8d7d8a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d010b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.067+0000 7fb8d7d8a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d010bec0 con 0x7fb8d007bc70 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.067+0000 7fb8d5b26700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d010b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.067+0000 7fb8d5b26700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d010b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41700/0 (socket says 172.21.15.78:41700) 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.067+0000 7fb8d5b26700 1 -- 172.21.15.78:0/309806438 learned_addr learned my addr 172.21.15.78:0/309806438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.067+0000 7fb8d5b26700 1 -- 172.21.15.78:0/309806438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d010c040 con 0x7fb8d007bc70 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.067+0000 7fb8d5b26700 1 --2- 172.21.15.78:0/309806438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d010b8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0098a0 tx=0x7fb8cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2eeced3ccf6c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.067+0000 7fb8d4b24700 1 -- 172.21.15.78:0/309806438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8cc004160 con 0x7fb8d007bc70 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d4b24700 1 -- 172.21.15.78:0/309806438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc004300 con 0x7fb8d007bc70 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d4b24700 1 -- 172.21.15.78:0/309806438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8cc011b20 con 0x7fb8d007bc70 2024-05-21T15:07:23.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/309806438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 msgr2=0x7fb8d010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d7d8a700 1 --2- 172.21.15.78:0/309806438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d010b8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0098a0 tx=0x7fb8cc009bb0 comp rx=0 tx=0).stop 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/309806438 shutdown_connections 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d7d8a700 1 --2- 172.21.15.78:0/309806438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d010b8f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/309806438 >> 172.21.15.78:0/309806438 conn(0x7fb8d01033a0 msgr2=0x7fb8d01057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/309806438 shutdown_connections 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.068+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/309806438 wait complete. 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.069+0000 7fb8d7d8a700 1 Processor -- start 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.069+0000 7fb8d7d8a700 1 -- start start 2024-05-21T15:07:23.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.069+0000 7fb8d7d8a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d019d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:23.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.069+0000 7fb8d7d8a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d010d160 con 0x7fb8d007bc70 2024-05-21T15:07:23.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.069+0000 7fb8d5b26700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d019d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:23.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.069+0000 7fb8d5b26700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d019d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41714/0 (socket says 172.21.15.78:41714) 2024-05-21T15:07:23.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.069+0000 7fb8d5b26700 1 -- 172.21.15.78:0/3393488875 learned_addr learned my addr 172.21.15.78:0/3393488875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:23.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.070+0000 7fb8d5b26700 1 -- 172.21.15.78:0/3393488875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d019d620 con 0x7fb8d007bc70 2024-05-21T15:07:23.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.070+0000 7fb8d5b26700 1 --2- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d019d0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc011740 tx=0x7fb8cc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:23.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.070+0000 7fb8c6ffd700 1 -- 172.21.15.78:0/3393488875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8cc003a40 con 0x7fb8d007bc70 2024-05-21T15:07:23.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.070+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d019d8b0 con 0x7fb8d007bc70 2024-05-21T15:07:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.070+0000 7fb8c6ffd700 1 -- 172.21.15.78:0/3393488875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc003be0 con 0x7fb8d007bc70 2024-05-21T15:07:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.070+0000 7fb8c6ffd700 1 -- 172.21.15.78:0/3393488875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8cc01a750 con 0x7fb8d007bc70 2024-05-21T15:07:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.070+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a04c0 con 0x7fb8d007bc70 2024-05-21T15:07:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.071+0000 7fb8c6ffd700 1 -- 172.21.15.78:0/3393488875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb8cc01aa50 con 0x7fb8d007bc70 2024-05-21T15:07:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.071+0000 7fb8c6ffd700 1 --2- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8bc0390f0 0x7fb8bc03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.071+0000 7fb8c6ffd700 1 -- 172.21.15.78:0/3393488875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb8cc04e860 con 0x7fb8d007bc70 2024-05-21T15:07:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.071+0000 7fb8d5325700 1 --2- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8bc0390f0 0x7fb8bc03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:23.997 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.071+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d0063ca0 con 0x7fb8d007bc70 2024-05-21T15:07:23.997 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.071+0000 7fb8d5325700 1 --2- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8bc0390f0 0x7fb8bc03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0006fd0 tx=0x7fb8c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:23.997 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.076+0000 7fb8c6ffd700 1 -- 172.21.15.78:0/3393488875 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8cc011e20 con 0x7fb8d007bc70 2024-05-21T15:07:23.997 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.384+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb8d01a0690 con 0x7fb8d007bc70 2024-05-21T15:07:23.997 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.386+0000 7fb8c6ffd700 1 -- 172.21.15.78:0/3393488875 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb8cc032020 con 0x7fb8d007bc70 2024-05-21T15:07:23.997 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8bc0390f0 msgr2=0x7fb8bc03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 --2- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8bc0390f0 0x7fb8bc03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0006fd0 tx=0x7fb8c0006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 msgr2=0x7fb8d019d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 --2- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d019d0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc011740 tx=0x7fb8cc011ba0 comp rx=0 tx=0).stop 2024-05-21T15:07:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 shutdown_connections 2024-05-21T15:07:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 --2- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8bc0390f0 0x7fb8bc03b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 --2- 172.21.15.78:0/3393488875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8d007bc70 0x7fb8d019d0e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 >> 172.21.15.78:0/3393488875 conn(0x7fb8d01033a0 msgr2=0x7fb8d0104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 shutdown_connections 2024-05-21T15:07:23.999 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-05-21T15:07:23.389+0000 7fb8d7d8a700 1 -- 172.21.15.78:0/3393488875 wait complete. 2024-05-21T15:07:23.999 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-21T15:07:23.999 INFO:teuthology.orchestra.run.smithi078.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-21T15:07:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:07:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:23.999 INFO:teuthology.orchestra.run.smithi078.stdout:Or, if you are only running a single cluster on this host: 2024-05-21T15:07:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-21T15:07:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:24.000 INFO:teuthology.orchestra.run.smithi078.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-21T15:07:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: ceph telemetry on 2024-05-21T15:07:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:24.001 INFO:teuthology.orchestra.run.smithi078.stdout:For more information see: 2024-05-21T15:07:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-21T15:07:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:24.001 INFO:teuthology.orchestra.run.smithi078.stdout:Bootstrap complete. 2024-05-21T15:07:24.033 INFO:tasks.cephadm:Fetching config... 2024-05-21T15:07:24.034 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:07:24.034 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-21T15:07:24.051 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-21T15:07:24.051 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:07:24.051 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-21T15:07:24.107 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-21T15:07:24.107 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:07:24.107 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-21T15:07:24.173 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-21T15:07:24.173 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:07:24.173 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-21T15:07:24.228 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-21T15:07:24.228 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuv/dbtzsSo48XCT5odeh3tIedUZawRLAVS6MycETstibL+0jS2m4Nk90hq8GecX5wxQji1i/uHYwrTWn0Um4gjzGTU5LlLUku4rHS97V9fsPgPLs6Q6p/VkQF1Ykr32dceMVnFbbu/JwHxKBhze0l4RVTzgxhsIU4n1xzs4OkXSEahyBOKvjHWlVrZlmU4TQhM/f/DMbXIDa5KlsjyTuv/2wySh3B89bJcVKTwZwj1goKyjZfJkD2kI4g5wSveH+1w309pLBnAR7C5oFioj53WqnIseVclL7JDDunGRCK0/QOqQyXYK4HGRo/p3tbWjEqVO2v6PORhYVhHFEJOpB8pLVZbYwilnMLqsIV5cpp4XkkKAgiPY4C1qNagMyFFSrDlds+9P95B2nH0njqCzh6QDsP9VOcjw9GvSFAj9lEtUN6Qmhdgo7n5e4TZYiyZ6LxmrEMthZLLkvDUlV8D10pB3VxK7gZAGTsgjAMwBIytCcAsRzq6/hvmPlzdPtrXGs= ceph-852c9762-1783-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T15:07:24.303 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuv/dbtzsSo48XCT5odeh3tIedUZawRLAVS6MycETstibL+0jS2m4Nk90hq8GecX5wxQji1i/uHYwrTWn0Um4gjzGTU5LlLUku4rHS97V9fsPgPLs6Q6p/VkQF1Ykr32dceMVnFbbu/JwHxKBhze0l4RVTzgxhsIU4n1xzs4OkXSEahyBOKvjHWlVrZlmU4TQhM/f/DMbXIDa5KlsjyTuv/2wySh3B89bJcVKTwZwj1goKyjZfJkD2kI4g5wSveH+1w309pLBnAR7C5oFioj53WqnIseVclL7JDDunGRCK0/QOqQyXYK4HGRo/p3tbWjEqVO2v6PORhYVhHFEJOpB8pLVZbYwilnMLqsIV5cpp4XkkKAgiPY4C1qNagMyFFSrDlds+9P95B2nH0njqCzh6QDsP9VOcjw9GvSFAj9lEtUN6Qmhdgo7n5e4TZYiyZ6LxmrEMthZLLkvDUlV8D10pB3VxK7gZAGTsgjAMwBIytCcAsRzq6/hvmPlzdPtrXGs= ceph-852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:07:24.321 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuv/dbtzsSo48XCT5odeh3tIedUZawRLAVS6MycETstibL+0jS2m4Nk90hq8GecX5wxQji1i/uHYwrTWn0Um4gjzGTU5LlLUku4rHS97V9fsPgPLs6Q6p/VkQF1Ykr32dceMVnFbbu/JwHxKBhze0l4RVTzgxhsIU4n1xzs4OkXSEahyBOKvjHWlVrZlmU4TQhM/f/DMbXIDa5KlsjyTuv/2wySh3B89bJcVKTwZwj1goKyjZfJkD2kI4g5wSveH+1w309pLBnAR7C5oFioj53WqnIseVclL7JDDunGRCK0/QOqQyXYK4HGRo/p3tbWjEqVO2v6PORhYVhHFEJOpB8pLVZbYwilnMLqsIV5cpp4XkkKAgiPY4C1qNagMyFFSrDlds+9P95B2nH0njqCzh6QDsP9VOcjw9GvSFAj9lEtUN6Qmhdgo7n5e4TZYiyZ6LxmrEMthZLLkvDUlV8D10pB3VxK7gZAGTsgjAMwBIytCcAsRzq6/hvmPlzdPtrXGs= ceph-852c9762-1783-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T15:07:24.361 INFO:teuthology.orchestra.run.smithi084.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuv/dbtzsSo48XCT5odeh3tIedUZawRLAVS6MycETstibL+0jS2m4Nk90hq8GecX5wxQji1i/uHYwrTWn0Um4gjzGTU5LlLUku4rHS97V9fsPgPLs6Q6p/VkQF1Ykr32dceMVnFbbu/JwHxKBhze0l4RVTzgxhsIU4n1xzs4OkXSEahyBOKvjHWlVrZlmU4TQhM/f/DMbXIDa5KlsjyTuv/2wySh3B89bJcVKTwZwj1goKyjZfJkD2kI4g5wSveH+1w309pLBnAR7C5oFioj53WqnIseVclL7JDDunGRCK0/QOqQyXYK4HGRo/p3tbWjEqVO2v6PORhYVhHFEJOpB8pLVZbYwilnMLqsIV5cpp4XkkKAgiPY4C1qNagMyFFSrDlds+9P95B2nH0njqCzh6QDsP9VOcjw9GvSFAj9lEtUN6Qmhdgo7n5e4TZYiyZ6LxmrEMthZLLkvDUlV8D10pB3VxK7gZAGTsgjAMwBIytCcAsRzq6/hvmPlzdPtrXGs= ceph-852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:07:24.373 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuv/dbtzsSo48XCT5odeh3tIedUZawRLAVS6MycETstibL+0jS2m4Nk90hq8GecX5wxQji1i/uHYwrTWn0Um4gjzGTU5LlLUku4rHS97V9fsPgPLs6Q6p/VkQF1Ykr32dceMVnFbbu/JwHxKBhze0l4RVTzgxhsIU4n1xzs4OkXSEahyBOKvjHWlVrZlmU4TQhM/f/DMbXIDa5KlsjyTuv/2wySh3B89bJcVKTwZwj1goKyjZfJkD2kI4g5wSveH+1w309pLBnAR7C5oFioj53WqnIseVclL7JDDunGRCK0/QOqQyXYK4HGRo/p3tbWjEqVO2v6PORhYVhHFEJOpB8pLVZbYwilnMLqsIV5cpp4XkkKAgiPY4C1qNagMyFFSrDlds+9P95B2nH0njqCzh6QDsP9VOcjw9GvSFAj9lEtUN6Qmhdgo7n5e4TZYiyZ6LxmrEMthZLLkvDUlV8D10pB3VxK7gZAGTsgjAMwBIytCcAsRzq6/hvmPlzdPtrXGs= ceph-852c9762-1783-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T15:07:24.410 INFO:teuthology.orchestra.run.smithi172.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuv/dbtzsSo48XCT5odeh3tIedUZawRLAVS6MycETstibL+0jS2m4Nk90hq8GecX5wxQji1i/uHYwrTWn0Um4gjzGTU5LlLUku4rHS97V9fsPgPLs6Q6p/VkQF1Ykr32dceMVnFbbu/JwHxKBhze0l4RVTzgxhsIU4n1xzs4OkXSEahyBOKvjHWlVrZlmU4TQhM/f/DMbXIDa5KlsjyTuv/2wySh3B89bJcVKTwZwj1goKyjZfJkD2kI4g5wSveH+1w309pLBnAR7C5oFioj53WqnIseVclL7JDDunGRCK0/QOqQyXYK4HGRo/p3tbWjEqVO2v6PORhYVhHFEJOpB8pLVZbYwilnMLqsIV5cpp4XkkKAgiPY4C1qNagMyFFSrDlds+9P95B2nH0njqCzh6QDsP9VOcjw9GvSFAj9lEtUN6Qmhdgo7n5e4TZYiyZ6LxmrEMthZLLkvDUlV8D10pB3VxK7gZAGTsgjAMwBIytCcAsRzq6/hvmPlzdPtrXGs= ceph-852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:07:24.421 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-21T15:07:24.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:24 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3393488875' entity='client.admin' 2024-05-21T15:07:24.665 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.831+0000 7f0f4254c700 1 -- 172.21.15.78:0/2989307659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 msgr2=0x7f0f3c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.831+0000 7f0f4254c700 1 --2- 172.21.15.78:0/2989307659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 0x7f0f3c074a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009910 tx=0x7f0f2c009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.831+0000 7f0f4254c700 1 -- 172.21.15.78:0/2989307659 shutdown_connections 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.831+0000 7f0f4254c700 1 --2- 172.21.15.78:0/2989307659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 0x7f0f3c074a30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.831+0000 7f0f4254c700 1 -- 172.21.15.78:0/2989307659 >> 172.21.15.78:0/2989307659 conn(0x7f0f3c0fe0a0 msgr2=0x7f0f3c100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.831+0000 7f0f4254c700 1 -- 172.21.15.78:0/2989307659 shutdown_connections 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.831+0000 7f0f4254c700 1 -- 172.21.15.78:0/2989307659 wait complete. 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.832+0000 7f0f4254c700 1 Processor -- start 2024-05-21T15:07:26.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.832+0000 7f0f4254c700 1 -- start start 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.832+0000 7f0f4254c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 0x7f0f3c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.832+0000 7f0f4254c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c10f230 con 0x7f0f3c076600 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.832+0000 7f0f3bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 0x7f0f3c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.832+0000 7f0f3bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 0x7f0f3c19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41738/0 (socket says 172.21.15.78:41738) 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.832+0000 7f0f3bfff700 1 -- 172.21.15.78:0/516042485 learned_addr learned my addr 172.21.15.78:0/516042485 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.833+0000 7f0f3bfff700 1 -- 172.21.15.78:0/516042485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c19f170 con 0x7f0f3c076600 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.833+0000 7f0f3bfff700 1 --2- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 0x7f0f3c19ec30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0052c0 tx=0x7f0f2c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.833+0000 7f0f397fa700 1 -- 172.21.15.78:0/516042485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f2c026070 con 0x7f0f3c076600 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.833+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c19f400 con 0x7f0f3c076600 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.833+0000 7f0f397fa700 1 -- 172.21.15.78:0/516042485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c0044a0 con 0x7f0f3c076600 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.833+0000 7f0f397fa700 1 -- 172.21.15.78:0/516042485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f2c003ba0 con 0x7f0f3c076600 2024-05-21T15:07:26.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.833+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c1a2010 con 0x7f0f3c076600 2024-05-21T15:07:26.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.834+0000 7f0f397fa700 1 -- 172.21.15.78:0/516042485 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0f2c029070 con 0x7f0f3c076600 2024-05-21T15:07:26.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.834+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3c10f8e0 con 0x7f0f3c076600 2024-05-21T15:07:26.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.834+0000 7f0f397fa700 1 --2- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f0f240390f0 0x7f0f2403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:26.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.834+0000 7f0f397fa700 1 -- 172.21.15.78:0/516042485 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0f2c01e070 con 0x7f0f3c076600 2024-05-21T15:07:26.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.834+0000 7f0f3b7fe700 1 --2- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f0f240390f0 0x7f0f2403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:26.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.835+0000 7f0f3b7fe700 1 --2- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f0f240390f0 0x7f0f2403b600 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f30006fd0 tx=0x7f0f30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:26.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:26.839+0000 7f0f397fa700 1 -- 172.21.15.78:0/516042485 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0f2c013070 con 0x7f0f3c076600 2024-05-21T15:07:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.036+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f0f3c02dd70 con 0x7f0f3c076600 2024-05-21T15:07:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.039+0000 7f0f397fa700 1 -- 172.21.15.78:0/516042485 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0f2c030020 con 0x7f0f3c076600 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.041+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f0f240390f0 msgr2=0x7f0f2403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.041+0000 7f0f4254c700 1 --2- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f0f240390f0 0x7f0f2403b600 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f30006fd0 tx=0x7f0f30006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.041+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 msgr2=0x7f0f3c19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.041+0000 7f0f4254c700 1 --2- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 0x7f0f3c19ec30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0052c0 tx=0x7f0f2c005030 comp rx=0 tx=0).stop 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.042+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 shutdown_connections 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.042+0000 7f0f4254c700 1 --2- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f0f240390f0 0x7f0f2403b600 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.042+0000 7f0f4254c700 1 --2- 172.21.15.78:0/516042485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f3c076600 0x7f0f3c19ec30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.042+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 >> 172.21.15.78:0/516042485 conn(0x7f0f3c0fe0a0 msgr2=0x7f0f3c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.042+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 shutdown_connections 2024-05-21T15:07:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:27.042+0000 7f0f4254c700 1 -- 172.21.15.78:0/516042485 wait complete. 2024-05-21T15:07:27.650 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-21T15:07:27.650 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-21T15:07:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:07:28.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:28 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/516042485' entity='client.admin' 2024-05-21T15:07:30.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.149+0000 7ff32ef72700 1 -- 172.21.15.78:0/1109364319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 msgr2=0x7ff328105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:30.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.149+0000 7ff32ef72700 1 --2- 172.21.15.78:0/1109364319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 0x7ff328105980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009910 tx=0x7ff31c009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:30.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.149+0000 7ff32ef72700 1 -- 172.21.15.78:0/1109364319 shutdown_connections 2024-05-21T15:07:30.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.149+0000 7ff32ef72700 1 --2- 172.21.15.78:0/1109364319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 0x7ff328105980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:30.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.149+0000 7ff32ef72700 1 -- 172.21.15.78:0/1109364319 >> 172.21.15.78:0/1109364319 conn(0x7ff3280fcfc0 msgr2=0x7ff3280ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:30.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32ef72700 1 -- 172.21.15.78:0/1109364319 shutdown_connections 2024-05-21T15:07:30.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32ef72700 1 -- 172.21.15.78:0/1109364319 wait complete. 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32ef72700 1 Processor -- start 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32ef72700 1 -- start start 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32ef72700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 0x7ff328198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32ef72700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff328108b70 con 0x7ff328103560 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32cd0e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 0x7ff328198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32cd0e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 0x7ff328198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43058/0 (socket says 172.21.15.78:43058) 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.150+0000 7ff32cd0e700 1 -- 172.21.15.78:0/4008921261 learned_addr learned my addr 172.21.15.78:0/4008921261 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.151+0000 7ff32cd0e700 1 -- 172.21.15.78:0/4008921261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff328198bb0 con 0x7ff328103560 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.151+0000 7ff32cd0e700 1 --2- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 0x7ff328198670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff31c004fb0 tx=0x7ff31c005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.151+0000 7ff325ffb700 1 -- 172.21.15.78:0/4008921261 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff31c027070 con 0x7ff328103560 2024-05-21T15:07:30.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.151+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff328198e40 con 0x7ff328103560 2024-05-21T15:07:30.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.151+0000 7ff325ffb700 1 -- 172.21.15.78:0/4008921261 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff31c022470 con 0x7ff328103560 2024-05-21T15:07:30.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.151+0000 7ff325ffb700 1 -- 172.21.15.78:0/4008921261 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff31c00f5b0 con 0x7ff328103560 2024-05-21T15:07:30.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.151+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32819ba50 con 0x7ff328103560 2024-05-21T15:07:30.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.152+0000 7ff325ffb700 1 -- 172.21.15.78:0/4008921261 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff31c02a070 con 0x7ff328103560 2024-05-21T15:07:30.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.152+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3281092f0 con 0x7ff328103560 2024-05-21T15:07:30.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.152+0000 7ff325ffb700 1 --2- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3180390f0 0x7ff31803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:30.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.152+0000 7ff325ffb700 1 -- 172.21.15.78:0/4008921261 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff31c01e030 con 0x7ff328103560 2024-05-21T15:07:30.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.152+0000 7ff327fff700 1 --2- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3180390f0 0x7ff31803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:30.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.153+0000 7ff327fff700 1 --2- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3180390f0 0x7ff31803b600 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff314006fd0 tx=0x7ff314006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:30.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.157+0000 7ff325ffb700 1 -- 172.21.15.78:0/4008921261 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff31c013070 con 0x7ff328103560 2024-05-21T15:07:30.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.359+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff32802dd70 con 0x7ff3180390f0 2024-05-21T15:07:30.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.363+0000 7ff325ffb700 1 -- 172.21.15.78:0/4008921261 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff32802dd70 con 0x7ff3180390f0 2024-05-21T15:07:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3180390f0 msgr2=0x7ff31803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 --2- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3180390f0 0x7ff31803b600 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff314006fd0 tx=0x7ff314006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 msgr2=0x7ff328198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 --2- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 0x7ff328198670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff31c004fb0 tx=0x7ff31c005090 comp rx=0 tx=0).stop 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 shutdown_connections 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 --2- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3180390f0 0x7ff31803b600 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 --2- 172.21.15.78:0/4008921261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff328103560 0x7ff328198670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 >> 172.21.15.78:0/4008921261 conn(0x7ff3280fcfc0 msgr2=0x7ff3280fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 shutdown_connections 2024-05-21T15:07:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:30.365+0000 7ff32ef72700 1 -- 172.21.15.78:0/4008921261 wait complete. 2024-05-21T15:07:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:07:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:31.070 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi084 2024-05-21T15:07:31.071 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:07:31.071 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.conf 2024-05-21T15:07:31.089 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:07:31.089 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:07:31.146 INFO:tasks.cephadm:Adding host smithi084 to orchestrator... 2024-05-21T15:07:31.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch host add smithi084 2024-05-21T15:07:31.389 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:07:31.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: Updating smithi078:/etc/ceph/ceph.conf 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:31.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.928+0000 7f77bbfe4700 1 -- 172.21.15.78:0/4117660128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 msgr2=0x7f77b4108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.928+0000 7f77bbfe4700 1 --2- 172.21.15.78:0/4117660128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 0x7f77b4108e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009910 tx=0x7f77b0009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.928+0000 7f77bbfe4700 1 -- 172.21.15.78:0/4117660128 shutdown_connections 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.928+0000 7f77bbfe4700 1 --2- 172.21.15.78:0/4117660128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 0x7f77b4108e00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.928+0000 7f77bbfe4700 1 -- 172.21.15.78:0/4117660128 >> 172.21.15.78:0/4117660128 conn(0x7f77b40fe4e0 msgr2=0x7f77b4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.928+0000 7f77bbfe4700 1 -- 172.21.15.78:0/4117660128 shutdown_connections 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.928+0000 7f77bbfe4700 1 -- 172.21.15.78:0/4117660128 wait complete. 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.929+0000 7f77bbfe4700 1 Processor -- start 2024-05-21T15:07:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.929+0000 7f77bbfe4700 1 -- start start 2024-05-21T15:07:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.929+0000 7f77bbfe4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 0x7f77b419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.929+0000 7f77bbfe4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77b410d090 con 0x7f77b41089f0 2024-05-21T15:07:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.929+0000 7f77b9d80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 0x7f77b419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.929+0000 7f77b9d80700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 0x7f77b419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43080/0 (socket says 172.21.15.78:43080) 2024-05-21T15:07:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.929+0000 7f77b9d80700 1 -- 172.21.15.78:0/3044797889 learned_addr learned my addr 172.21.15.78:0/3044797889 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.930+0000 7f77b9d80700 1 -- 172.21.15.78:0/3044797889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b419eac0 con 0x7f77b41089f0 2024-05-21T15:07:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.930+0000 7f77b9d80700 1 --2- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 0x7f77b419e580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77b0004f80 tx=0x7f77b0005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.930+0000 7f77aaffd700 1 -- 172.21.15.78:0/3044797889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b0026070 con 0x7f77b41089f0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.930+0000 7f77aaffd700 1 -- 172.21.15.78:0/3044797889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b000b6f0 con 0x7f77b41089f0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.930+0000 7f77aaffd700 1 -- 172.21.15.78:0/3044797889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b0003910 con 0x7f77b41089f0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.930+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b419edb0 con 0x7f77b41089f0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.930+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b4198930 con 0x7f77b41089f0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.931+0000 7f77aaffd700 1 -- 172.21.15.78:0/3044797889 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f77b0029070 con 0x7f77b41089f0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.931+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b4104100 con 0x7f77b41089f0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.931+0000 7f77aaffd700 1 --2- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f77a0039150 0x7f77a003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.931+0000 7f77aaffd700 1 -- 172.21.15.78:0/3044797889 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f77b001e070 con 0x7f77b41089f0 2024-05-21T15:07:32.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.931+0000 7f77b957f700 1 --2- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f77a0039150 0x7f77a003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:32.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.932+0000 7f77b957f700 1 --2- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f77a0039150 0x7f77a003b660 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006fd0 tx=0x7f77a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:32.936+0000 7f77aaffd700 1 -- 172.21.15.78:0/3044797889 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f77b0013070 con 0x7f77b41089f0 2024-05-21T15:07:33.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:33.135+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}) -- 0x7f77b4199c50 con 0x7f77a0039150 2024-05-21T15:07:33.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:33 smithi078 ceph-mon[31997]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:34.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:34 smithi078 ceph-mon[31997]: Deploying cephadm binary to smithi084 2024-05-21T15:07:35.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:35 smithi078 ceph-mon[31997]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:37.346 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi084' with addr '172.21.15.84' 2024-05-21T15:07:37.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.342+0000 7f77aaffd700 1 -- 172.21.15.78:0/3044797889 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f77b4199c50 con 0x7f77a0039150 2024-05-21T15:07:37.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f77a0039150 msgr2=0x7f77a003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:37.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 --2- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f77a0039150 0x7f77a003b660 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006fd0 tx=0x7f77a4006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:37.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 msgr2=0x7f77b419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:37.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 --2- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 0x7f77b419e580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77b0004f80 tx=0x7f77b0005060 comp rx=0 tx=0).stop 2024-05-21T15:07:37.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 shutdown_connections 2024-05-21T15:07:37.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 --2- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f77a0039150 0x7f77a003b660 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:37.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 --2- 172.21.15.78:0/3044797889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b41089f0 0x7f77b419e580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:37.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 >> 172.21.15.78:0/3044797889 conn(0x7f77b40fe4e0 msgr2=0x7f77b40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:37.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 shutdown_connections 2024-05-21T15:07:37.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:37.345+0000 7f77bbfe4700 1 -- 172.21.15.78:0/3044797889 wait complete. 2024-05-21T15:07:37.937 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T15:07:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:07:38.514 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:38 smithi078 ceph-mon[31997]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:38.514 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:38 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:38.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:38 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:38.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:38 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:39.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:39 smithi078 ceph-mon[31997]: Added host smithi084 2024-05-21T15:07:39.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:39.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.772+0000 7f3056f65700 1 -- 172.21.15.78:0/3204763870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 msgr2=0x7f3050107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:39.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.772+0000 7f3056f65700 1 --2- 172.21.15.78:0/3204763870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 0x7f3050107e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3040009910 tx=0x7f3040009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.772+0000 7f3056f65700 1 -- 172.21.15.78:0/3204763870 shutdown_connections 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.772+0000 7f3056f65700 1 --2- 172.21.15.78:0/3204763870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 0x7f3050107e10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.772+0000 7f3056f65700 1 -- 172.21.15.78:0/3204763870 >> 172.21.15.78:0/3204763870 conn(0x7f3050077bf0 msgr2=0x7f3050078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.772+0000 7f3056f65700 1 -- 172.21.15.78:0/3204763870 shutdown_connections 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.772+0000 7f3056f65700 1 -- 172.21.15.78:0/3204763870 wait complete. 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.773+0000 7f3056f65700 1 Processor -- start 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.773+0000 7f3056f65700 1 -- start start 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.773+0000 7f3056f65700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 0x7f30501020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.773+0000 7f3056f65700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f305010d1f0 con 0x7f3050107a00 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.773+0000 7f3054d01700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 0x7f30501020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.773+0000 7f3054d01700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 0x7f30501020e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44668/0 (socket says 172.21.15.78:44668) 2024-05-21T15:07:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.773+0000 7f3054d01700 1 -- 172.21.15.78:0/3688931535 learned_addr learned my addr 172.21.15.78:0/3688931535 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.773+0000 7f3054d01700 1 -- 172.21.15.78:0/3688931535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3050103f30 con 0x7f3050107a00 2024-05-21T15:07:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.774+0000 7f3054d01700 1 --2- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 0x7f30501020e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f30400092e0 tx=0x7f3040005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.774+0000 7f304dffb700 1 -- 172.21.15.78:0/3688931535 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3040026070 con 0x7f3050107a00 2024-05-21T15:07:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.774+0000 7f304dffb700 1 -- 172.21.15.78:0/3688931535 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304000b750 con 0x7f3050107a00 2024-05-21T15:07:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.774+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3050102620 con 0x7f3050107a00 2024-05-21T15:07:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.774+0000 7f304dffb700 1 -- 172.21.15.78:0/3688931535 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30400038c0 con 0x7f3050107a00 2024-05-21T15:07:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.774+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3050102a20 con 0x7f3050107a00 2024-05-21T15:07:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.775+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3050063ca0 con 0x7f3050107a00 2024-05-21T15:07:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.775+0000 7f304dffb700 1 -- 172.21.15.78:0/3688931535 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3040029070 con 0x7f3050107a00 2024-05-21T15:07:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.775+0000 7f304dffb700 1 --2- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3038039150 0x7f303803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.775+0000 7f304dffb700 1 -- 172.21.15.78:0/3688931535 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f304001e070 con 0x7f3050107a00 2024-05-21T15:07:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.776+0000 7f304ffff700 1 --2- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3038039150 0x7f303803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:39.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.776+0000 7f304ffff700 1 --2- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3038039150 0x7f303803b660 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3044006fd0 tx=0x7f3044006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.781+0000 7f304dffb700 1 -- 172.21.15.78:0/3688931535 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3040013070 con 0x7f3050107a00 2024-05-21T15:07:39.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.985+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3050102ca0 con 0x7f3038039150 2024-05-21T15:07:39.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.986+0000 7f304dffb700 1 -- 172.21.15.78:0/3688931535 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f3050102ca0 con 0x7f3038039150 2024-05-21T15:07:39.989 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:39.990 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}] 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3038039150 msgr2=0x7f303803b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 --2- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3038039150 0x7f303803b660 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3044006fd0 tx=0x7f3044006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 msgr2=0x7f30501020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 --2- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 0x7f30501020e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f30400092e0 tx=0x7f3040005030 comp rx=0 tx=0).stop 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 shutdown_connections 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 --2- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3038039150 0x7f303803b660 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 --2- 172.21.15.78:0/3688931535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3050107a00 0x7f30501020e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 >> 172.21.15.78:0/3688931535 conn(0x7f3050077bf0 msgr2=0x7f3050104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 shutdown_connections 2024-05-21T15:07:39.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:39.989+0000 7f3056f65700 1 -- 172.21.15.78:0/3688931535 wait complete. 2024-05-21T15:07:40.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:40 smithi078 ceph-mon[31997]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:40.634 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi172 2024-05-21T15:07:40.634 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:07:40.634 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.conf 2024-05-21T15:07:40.653 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:07:40.653 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:07:40.710 INFO:tasks.cephadm:Adding host smithi172 to orchestrator... 2024-05-21T15:07:40.711 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch host add smithi172 2024-05-21T15:07:40.944 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:07:41.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:41 smithi078 ceph-mon[31997]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.352+0000 7fb59bda2700 1 -- 172.21.15.78:0/4292644070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 msgr2=0x7fb594107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.352+0000 7fb59bda2700 1 --2- 172.21.15.78:0/4292644070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 0x7fb594107d00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb590009910 tx=0x7fb590009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.352+0000 7fb59bda2700 1 -- 172.21.15.78:0/4292644070 shutdown_connections 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.352+0000 7fb59bda2700 1 --2- 172.21.15.78:0/4292644070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 0x7fb594107d00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.352+0000 7fb59bda2700 1 -- 172.21.15.78:0/4292644070 >> 172.21.15.78:0/4292644070 conn(0x7fb5940fd3e0 msgr2=0x7fb5940ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.352+0000 7fb59bda2700 1 -- 172.21.15.78:0/4292644070 shutdown_connections 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.352+0000 7fb59bda2700 1 -- 172.21.15.78:0/4292644070 wait complete. 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.353+0000 7fb59bda2700 1 Processor -- start 2024-05-21T15:07:42.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.353+0000 7fb59bda2700 1 -- start start 2024-05-21T15:07:42.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.353+0000 7fb59bda2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 0x7fb59419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:42.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.353+0000 7fb59bda2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb59410bf90 con 0x7fb5941078f0 2024-05-21T15:07:42.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb599b3e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 0x7fb59419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:42.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb599b3e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 0x7fb59419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44682/0 (socket says 172.21.15.78:44682) 2024-05-21T15:07:42.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb599b3e700 1 -- 172.21.15.78:0/1199428637 learned_addr learned my addr 172.21.15.78:0/1199428637 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:42.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb599b3e700 1 -- 172.21.15.78:0/1199428637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59419eb00 con 0x7fb5941078f0 2024-05-21T15:07:42.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb599b3e700 1 --2- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 0x7fb59419e5c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb590004f80 tx=0x7fb590005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:42.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb58affd700 1 -- 172.21.15.78:0/1199428637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb590026070 con 0x7fb5941078f0 2024-05-21T15:07:42.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb58affd700 1 -- 172.21.15.78:0/1199428637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59000b6f0 con 0x7fb5941078f0 2024-05-21T15:07:42.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59419edf0 con 0x7fb5941078f0 2024-05-21T15:07:42.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb58affd700 1 -- 172.21.15.78:0/1199428637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb590003910 con 0x7fb5941078f0 2024-05-21T15:07:42.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.354+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb594198920 con 0x7fb5941078f0 2024-05-21T15:07:42.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.355+0000 7fb58affd700 1 -- 172.21.15.78:0/1199428637 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb590029070 con 0x7fb5941078f0 2024-05-21T15:07:42.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.355+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb594103000 con 0x7fb5941078f0 2024-05-21T15:07:42.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.355+0000 7fb58affd700 1 --2- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb580039150 0x7fb58003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:42.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.355+0000 7fb58affd700 1 -- 172.21.15.78:0/1199428637 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb59001e070 con 0x7fb5941078f0 2024-05-21T15:07:42.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.356+0000 7fb59933d700 1 --2- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb580039150 0x7fb58003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:42.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.356+0000 7fb59933d700 1 --2- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb580039150 0x7fb58003b660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb584006fd0 tx=0x7fb584006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:42.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.361+0000 7fb58affd700 1 -- 172.21.15.78:0/1199428637 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb590013070 con 0x7fb5941078f0 2024-05-21T15:07:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:42.561+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}) -- 0x7fb594199c60 con 0x7fb580039150 2024-05-21T15:07:42.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:42 smithi078 ceph-mon[31997]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: Deploying cephadm binary to smithi172 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:07:45.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:45 smithi078 ceph-mon[31997]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:07:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:45 smithi078 ceph-mon[31997]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:07:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:46 smithi078 ceph-mon[31997]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:46.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.708+0000 7fb58affd700 1 -- 172.21.15.78:0/1199428637 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb594199c60 con 0x7fb580039150 2024-05-21T15:07:46.712 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi172' with addr '172.21.15.172' 2024-05-21T15:07:46.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb580039150 msgr2=0x7fb58003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:46.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 --2- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb580039150 0x7fb58003b660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb584006fd0 tx=0x7fb584006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:46.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 msgr2=0x7fb59419e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:46.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 --2- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 0x7fb59419e5c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb590004f80 tx=0x7fb590005060 comp rx=0 tx=0).stop 2024-05-21T15:07:46.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 shutdown_connections 2024-05-21T15:07:46.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 --2- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb580039150 0x7fb58003b660 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:46.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 --2- 172.21.15.78:0/1199428637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5941078f0 0x7fb59419e5c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:46.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 >> 172.21.15.78:0/1199428637 conn(0x7fb5940fd3e0 msgr2=0x7fb5941053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:46.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 shutdown_connections 2024-05-21T15:07:46.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:46.711+0000 7fb59bda2700 1 -- 172.21.15.78:0/1199428637 wait complete. 2024-05-21T15:07:47.296 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:47 smithi078 ceph-mon[31997]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:07:47.296 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:47 smithi078 ceph-mon[31997]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:07:47.296 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:47 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:47.296 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:47 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:47.296 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:47 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:47.296 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:47 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:47.296 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:47 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:47.323 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T15:07:47.557 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:07:48.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:48 smithi078 ceph-mon[31997]: Added host smithi172 2024-05-21T15:07:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:48 smithi078 ceph-mon[31997]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:49.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.123+0000 7f70f1cac700 1 -- 172.21.15.78:0/478886578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 msgr2=0x7f70ec107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:49.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.123+0000 7f70f1cac700 1 --2- 172.21.15.78:0/478886578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 0x7f70ec107e30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009910 tx=0x7f70dc009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:49.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.123+0000 7f70f1cac700 1 -- 172.21.15.78:0/478886578 shutdown_connections 2024-05-21T15:07:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.123+0000 7f70f1cac700 1 --2- 172.21.15.78:0/478886578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 0x7f70ec107e30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.123+0000 7f70f1cac700 1 -- 172.21.15.78:0/478886578 >> 172.21.15.78:0/478886578 conn(0x7f70ec0fccb0 msgr2=0x7f70ec0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.123+0000 7f70f1cac700 1 -- 172.21.15.78:0/478886578 shutdown_connections 2024-05-21T15:07:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.123+0000 7f70f1cac700 1 -- 172.21.15.78:0/478886578 wait complete. 2024-05-21T15:07:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.123+0000 7f70f1cac700 1 Processor -- start 2024-05-21T15:07:49.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.124+0000 7f70f1cac700 1 -- start start 2024-05-21T15:07:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.124+0000 7f70f1cac700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 0x7f70ec19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.124+0000 7f70f1cac700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec10d1f0 con 0x7f70ec107a20 2024-05-21T15:07:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.124+0000 7f70eb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 0x7f70ec19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.124+0000 7f70eb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 0x7f70ec19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38266/0 (socket says 172.21.15.78:38266) 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.124+0000 7f70eb7fe700 1 -- 172.21.15.78:0/1140174184 learned_addr learned my addr 172.21.15.78:0/1140174184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.124+0000 7f70eb7fe700 1 -- 172.21.15.78:0/1140174184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ec19eb00 con 0x7f70ec107a20 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.124+0000 7f70eb7fe700 1 --2- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 0x7f70ec19e5c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00b5c0 tx=0x7f70dc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.125+0000 7f70e97fa700 1 -- 172.21.15.78:0/1140174184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70dc026070 con 0x7f70ec107a20 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.125+0000 7f70e97fa700 1 -- 172.21.15.78:0/1140174184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70dc00b750 con 0x7f70ec107a20 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.125+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ec19ed90 con 0x7f70ec107a20 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.125+0000 7f70e97fa700 1 -- 172.21.15.78:0/1140174184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70dc0038e0 con 0x7f70ec107a20 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.125+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ec198800 con 0x7f70ec107a20 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.126+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ec104130 con 0x7f70ec107a20 2024-05-21T15:07:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.126+0000 7f70e97fa700 1 -- 172.21.15.78:0/1140174184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f70dc029070 con 0x7f70ec107a20 2024-05-21T15:07:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.126+0000 7f70e97fa700 1 --2- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f70d80390f0 0x7f70d803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.126+0000 7f70e97fa700 1 -- 172.21.15.78:0/1140174184 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f70dc01e070 con 0x7f70ec107a20 2024-05-21T15:07:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.126+0000 7f70e3fff700 1 --2- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f70d80390f0 0x7f70d803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.126+0000 7f70e3fff700 1 --2- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f70d80390f0 0x7f70d803b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006fd0 tx=0x7f70d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.131+0000 7f70e97fa700 1 -- 172.21.15.78:0/1140174184 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70dc013070 con 0x7f70ec107a20 2024-05-21T15:07:49.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.334+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70ec199d10 con 0x7f70d80390f0 2024-05-21T15:07:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.335+0000 7f70e97fa700 1 -- 172.21.15.78:0/1140174184 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f70ec199d10 con 0x7f70d80390f0 2024-05-21T15:07:49.338 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:07:49.338 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}, {"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}] 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.337+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f70d80390f0 msgr2=0x7f70d803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.337+0000 7f70f1cac700 1 --2- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f70d80390f0 0x7f70d803b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006fd0 tx=0x7f70d4006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.337+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 msgr2=0x7f70ec19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.337+0000 7f70f1cac700 1 --2- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 0x7f70ec19e5c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70dc00b5c0 tx=0x7f70dc005030 comp rx=0 tx=0).stop 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.337+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 shutdown_connections 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.337+0000 7f70f1cac700 1 --2- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f70d80390f0 0x7f70d803b600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.337+0000 7f70f1cac700 1 --2- 172.21.15.78:0/1140174184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70ec107a20 0x7f70ec19e5c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.338+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 >> 172.21.15.78:0/1140174184 conn(0x7f70ec0fccb0 msgr2=0x7f70ec1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.338+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 shutdown_connections 2024-05-21T15:07:49.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:49.338+0000 7f70f1cac700 1 -- 172.21.15.78:0/1140174184 wait complete. 2024-05-21T15:07:49.929 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-21T15:07:49.929 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-21T15:07:50.163 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:07:50.595 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:50 smithi078 ceph-mon[31997]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:50.595 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:51.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:51 smithi078 ceph-mon[31997]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:07:51.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.712+0000 7f7cd78fe700 1 -- 172.21.15.78:0/764689429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 msgr2=0x7f7cd0107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:51.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.712+0000 7f7cd78fe700 1 --2- 172.21.15.78:0/764689429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 0x7f7cd0107d00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009910 tx=0x7f7cc4009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:51.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.712+0000 7f7cd78fe700 1 -- 172.21.15.78:0/764689429 shutdown_connections 2024-05-21T15:07:51.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.712+0000 7f7cd78fe700 1 --2- 172.21.15.78:0/764689429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 0x7f7cd0107d00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.712+0000 7f7cd78fe700 1 -- 172.21.15.78:0/764689429 >> 172.21.15.78:0/764689429 conn(0x7f7cd00fd440 msgr2=0x7f7cd00ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.712+0000 7f7cd78fe700 1 -- 172.21.15.78:0/764689429 shutdown_connections 2024-05-21T15:07:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.712+0000 7f7cd78fe700 1 -- 172.21.15.78:0/764689429 wait complete. 2024-05-21T15:07:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.712+0000 7f7cd78fe700 1 Processor -- start 2024-05-21T15:07:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd78fe700 1 -- start start 2024-05-21T15:07:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd78fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 0x7f7cd01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd78fe700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd010bf90 con 0x7f7cd01078f0 2024-05-21T15:07:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd569a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 0x7f7cd01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd569a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 0x7f7cd01996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38292/0 (socket says 172.21.15.78:38292) 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd569a700 1 -- 172.21.15.78:0/1814729035 learned_addr learned my addr 172.21.15.78:0/1814729035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd569a700 1 -- 172.21.15.78:0/1814729035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd006b8f0 con 0x7f7cd01078f0 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd569a700 1 --2- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 0x7f7cd01996e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4004f80 tx=0x7f7cc4005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cc27fc700 1 -- 172.21.15.78:0/1814729035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cc4026070 con 0x7f7cd01078f0 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.713+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd0199ce0 con 0x7f7cd01078f0 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.714+0000 7f7cc27fc700 1 -- 172.21.15.78:0/1814729035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc400b750 con 0x7f7cd01078f0 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.714+0000 7f7cc27fc700 1 -- 172.21.15.78:0/1814729035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cc40038d0 con 0x7f7cd01078f0 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.714+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd019a1a0 con 0x7f7cd01078f0 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.714+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd019a4e0 con 0x7f7cd01078f0 2024-05-21T15:07:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.715+0000 7f7cc27fc700 1 -- 172.21.15.78:0/1814729035 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7cc4029070 con 0x7f7cd01078f0 2024-05-21T15:07:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.715+0000 7f7cc27fc700 1 --2- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7cbc0390f0 0x7f7cbc03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.715+0000 7f7cc27fc700 1 -- 172.21.15.78:0/1814729035 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7cc401e070 con 0x7f7cd01078f0 2024-05-21T15:07:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.715+0000 7f7cd4e99700 1 --2- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7cbc0390f0 0x7f7cbc03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.715+0000 7f7cd4e99700 1 --2- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7cbc0390f0 0x7f7cbc03b600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006fd0 tx=0x7f7ccc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:51.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.720+0000 7f7cc27fc700 1 -- 172.21.15.78:0/1814729035 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7cc4013070 con 0x7f7cd01078f0 2024-05-21T15:07:51.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:51.923+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7cd0063ca0 con 0x7f7cd01078f0 2024-05-21T15:07:52.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.290+0000 7f7cc27fc700 1 -- 172.21.15.78:0/1814729035 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f7cc4021410 con 0x7f7cd01078f0 2024-05-21T15:07:52.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.292+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7cbc0390f0 msgr2=0x7f7cbc03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.292+0000 7f7cd78fe700 1 --2- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7cbc0390f0 0x7f7cbc03b600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006fd0 tx=0x7f7ccc006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.293+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 msgr2=0x7f7cd01996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.293+0000 7f7cd78fe700 1 --2- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 0x7f7cd01996e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4004f80 tx=0x7f7cc4005060 comp rx=0 tx=0).stop 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.293+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 shutdown_connections 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.293+0000 7f7cd78fe700 1 --2- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7cbc0390f0 0x7f7cbc03b600 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.293+0000 7f7cd78fe700 1 --2- 172.21.15.78:0/1814729035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cd01078f0 0x7f7cd01996e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.293+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 >> 172.21.15.78:0/1814729035 conn(0x7f7cd00fd440 msgr2=0x7f7cd00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.293+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 shutdown_connections 2024-05-21T15:07:52.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:07:52.293+0000 7f7cd78fe700 1 -- 172.21.15.78:0/1814729035 wait complete. 2024-05-21T15:07:52.297 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2024-05-21T15:07:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:52 smithi078 ceph-mon[31997]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:52 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1814729035' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-21T15:07:52.974 INFO:tasks.cephadm:Adding mon.a on smithi078 2024-05-21T15:07:52.974 INFO:tasks.cephadm:Adding mon.b on smithi084 2024-05-21T15:07:52.974 INFO:tasks.cephadm:Adding mon.c on smithi172 2024-05-21T15:07:52.974 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c' 2024-05-21T15:07:53.213 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:07:53.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1814729035' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T15:07:53.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:07:53.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:53.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:53.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:53.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:53.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:07:53.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:53.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:07:53.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:53.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.582+0000 7fa6a179e700 1 -- 172.21.15.172:0/2185828168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 msgr2=0x7fa69c107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.582+0000 7fa6a179e700 1 --2- 172.21.15.172:0/2185828168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 0x7fa69c107b60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009910 tx=0x7fa68c009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.582+0000 7fa6a179e700 1 -- 172.21.15.172:0/2185828168 shutdown_connections 2024-05-21T15:07:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.582+0000 7fa6a179e700 1 --2- 172.21.15.172:0/2185828168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 0x7fa69c107b60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.582+0000 7fa6a179e700 1 -- 172.21.15.172:0/2185828168 >> 172.21.15.172:0/2185828168 conn(0x7fa69c0fd260 msgr2=0x7fa69c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.582+0000 7fa6a179e700 1 -- 172.21.15.172:0/2185828168 shutdown_connections 2024-05-21T15:07:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.582+0000 7fa6a179e700 1 -- 172.21.15.172:0/2185828168 wait complete. 2024-05-21T15:07:54.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.583+0000 7fa6a179e700 1 Processor -- start 2024-05-21T15:07:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.583+0000 7fa6a179e700 1 -- start start 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.583+0000 7fa6a179e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 0x7fa69c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.583+0000 7fa6a179e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c10bdf0 con 0x7fa69c107750 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.583+0000 7fa69bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 0x7fa69c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa69bfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 0x7fa69c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.172:42644/0 (socket says 172.21.15.172:42644) 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa69bfff700 1 -- 172.21.15.172:0/1836918790 learned_addr learned my addr 172.21.15.172:0/1836918790 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa69bfff700 1 -- 172.21.15.172:0/1836918790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c198c50 con 0x7fa69c107750 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa69bfff700 1 --2- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 0x7fa69c198710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa68c004f80 tx=0x7fa68c005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa6997fa700 1 -- 172.21.15.172:0/1836918790 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa68c026070 con 0x7fa69c107750 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69c198f40 con 0x7fa69c107750 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa6997fa700 1 -- 172.21.15.172:0/1836918790 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa68c00b6f0 con 0x7fa69c107750 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa6997fa700 1 -- 172.21.15.172:0/1836918790 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa68c003910 con 0x7fa69c107750 2024-05-21T15:07:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.584+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69c19cf40 con 0x7fa69c107750 2024-05-21T15:07:54.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.585+0000 7fa6997fa700 1 -- 172.21.15.172:0/1836918790 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa68c029070 con 0x7fa69c107750 2024-05-21T15:07:54.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.585+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69c102e60 con 0x7fa69c107750 2024-05-21T15:07:54.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.585+0000 7fa6997fa700 1 --2- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa6840390f0 0x7fa68403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:54.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.586+0000 7fa6997fa700 1 -- 172.21.15.172:0/1836918790 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa68c01e070 con 0x7fa69c107750 2024-05-21T15:07:54.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.586+0000 7fa69b7fe700 1 --2- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa6840390f0 0x7fa68403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:54.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.586+0000 7fa69b7fe700 1 --2- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa6840390f0 0x7fa68403b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa690006fd0 tx=0x7fa690006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.591+0000 7fa6997fa700 1 -- 172.21.15.172:0/1836918790 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa68c013070 con 0x7fa69c107750 2024-05-21T15:07:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:54 smithi078 ceph-mon[31997]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:07:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:54 smithi078 ceph-mon[31997]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:07:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:54 smithi078 ceph-mon[31997]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:07:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:54 smithi078 ceph-mon[31997]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:07:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:54 smithi078 ceph-mon[31997]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:54 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:54.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.801+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}) -- 0x7fa69c199a70 con 0x7fa6840390f0 2024-05-21T15:07:54.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.809+0000 7fa6997fa700 1 -- 172.21.15.172:0/1836918790 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa69c199a70 con 0x7fa6840390f0 2024-05-21T15:07:54.812 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mon update... 2024-05-21T15:07:54.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa6840390f0 msgr2=0x7fa68403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:54.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 --2- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa6840390f0 0x7fa68403b600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa690006fd0 tx=0x7fa690006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:54.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 msgr2=0x7fa69c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:54.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 --2- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 0x7fa69c198710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa68c004f80 tx=0x7fa68c005060 comp rx=0 tx=0).stop 2024-05-21T15:07:54.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 shutdown_connections 2024-05-21T15:07:54.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 --2- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa6840390f0 0x7fa68403b600 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:54.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 --2- 172.21.15.172:0/1836918790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa69c107750 0x7fa69c198710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:54.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 >> 172.21.15.172:0/1836918790 conn(0x7fa69c0fd260 msgr2=0x7fa69c105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:54.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 shutdown_connections 2024-05-21T15:07:54.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:54.811+0000 7fa6a179e700 1 -- 172.21.15.172:0/1836918790 wait complete. 2024-05-21T15:07:55.387 DEBUG:teuthology.orchestra.run.smithi084:mon.b> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.b.service 2024-05-21T15:07:55.390 DEBUG:teuthology.orchestra.run.smithi172:mon.c> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.c.service 2024-05-21T15:07:55.392 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T15:07:55.392 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T15:07:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c;count:3 2024-05-21T15:07:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:07:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:07:56.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:56.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: Deploying daemon mon.c on smithi172 2024-05-21T15:07:56.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:55 smithi078 ceph-mon[31997]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:56.376 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:07:58.585 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:58 smithi172 systemd[1]: Starting Ceph mon.c for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:07:58.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:07:58 smithi078 ceph-mon[31997]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.045+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/3603874355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 msgr2=0x7fb0f8103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:59.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.045+0000 7fb0fe1f5700 1 --2- 172.21.15.172:0/3603874355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 0x7fb0f8103f60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009910 tx=0x7fb0ec009c20 comp rx=0 tx=0).stop 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.045+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/3603874355 shutdown_connections 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.045+0000 7fb0fe1f5700 1 --2- 172.21.15.172:0/3603874355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 0x7fb0f8103f60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.045+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/3603874355 >> 172.21.15.172:0/3603874355 conn(0x7fb0f80fd400 msgr2=0x7fb0f80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.045+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/3603874355 shutdown_connections 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.046+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/3603874355 wait complete. 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.046+0000 7fb0fe1f5700 1 Processor -- start 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.046+0000 7fb0fe1f5700 1 -- start start 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.047+0000 7fb0fe1f5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 0x7fb0f8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.047+0000 7fb0fe1f5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f810d100 con 0x7fb0f8103b50 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.047+0000 7fb0f77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 0x7fb0f8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.047+0000 7fb0f77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 0x7fb0f8101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.172:42670/0 (socket says 172.21.15.172:42670) 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.047+0000 7fb0f77fe700 1 -- 172.21.15.172:0/2395195846 learned_addr learned my addr 172.21.15.172:0/2395195846 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:07:59.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.047+0000 7fb0f77fe700 1 -- 172.21.15.172:0/2395195846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f8103690 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.048+0000 7fb0f77fe700 1 --2- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 0x7fb0f8101840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.048+0000 7fb0f4ff9700 1 -- 172.21.15.172:0/2395195846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0ec026070 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.048+0000 7fb0f4ff9700 1 -- 172.21.15.172:0/2395195846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ec00bc30 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.048+0000 7fb0f4ff9700 1 -- 172.21.15.172:0/2395195846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0ec0039a0 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.048+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f8101d80 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.048+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f8102180 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.049+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f806b2a0 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.049+0000 7fb0f4ff9700 1 -- 172.21.15.172:0/2395195846 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb0ec029070 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.049+0000 7fb0f4ff9700 1 --2- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0e00390f0 0x7fb0e003b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.049+0000 7fb0f4ff9700 1 -- 172.21.15.172:0/2395195846 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb0ec01e070 con 0x7fb0f8103b50 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.049+0000 7fb0f6ffd700 1 --2- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0e00390f0 0x7fb0e003b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:07:59.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.050+0000 7fb0f6ffd700 1 --2- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0e00390f0 0x7fb0e003b600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8006fd0 tx=0x7fb0e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:07:59.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.055+0000 7fb0f4ff9700 1 -- 172.21.15.172:0/2395195846 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb0ec013070 con 0x7fb0f8103b50 2024-05-21T15:07:59.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.336+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb0f8102400 con 0x7fb0f8103b50 2024-05-21T15:07:59.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.337+0000 7fb0f4ff9700 1 -- 172.21.15.172:0/2395195846 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fb0ec030090 con 0x7fb0f8103b50 2024-05-21T15:07:59.341 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:07:59.341 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":1,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","modified":"2024-05-21T15:06:17.120039Z","created":"2024-05-21T15:06:17.120039Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.339+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0e00390f0 msgr2=0x7fb0e003b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.339+0000 7fb0fe1f5700 1 --2- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0e00390f0 0x7fb0e003b600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8006fd0 tx=0x7fb0e8006e40 comp rx=0 tx=0).stop 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.339+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 msgr2=0x7fb0f8101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.339+0000 7fb0fe1f5700 1 --2- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 0x7fb0f8101840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec005030 comp rx=0 tx=0).stop 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.339+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 shutdown_connections 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.339+0000 7fb0fe1f5700 1 --2- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0e00390f0 0x7fb0e003b600 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.339+0000 7fb0fe1f5700 1 --2- 172.21.15.172:0/2395195846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0f8103b50 0x7fb0f8101840 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec005030 comp rx=0 tx=0).stop 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.339+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 >> 172.21.15.172:0/2395195846 conn(0x7fb0f80fd400 msgr2=0x7fb0f8104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.340+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 shutdown_connections 2024-05-21T15:07:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:07:59.340+0000 7fb0fe1f5700 1 -- 172.21.15.172:0/2395195846 wait complete. 2024-05-21T15:07:59.344 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 1 2024-05-21T15:07:59.412 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 podman[36347]: 2024-05-21 15:07:59.135893432 +0000 UTC m=+0.282361569 container create 14e48ae402610e5ce2c57bc349f15268681567c72498128f6af9cf3b3aaea333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True) 2024-05-21T15:07:59.412 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 podman[36347]: 2024-05-21 15:07:59.052973882 +0000 UTC m=+0.199442027 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:07:59.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 podman[36347]: 2024-05-21 15:07:59.371909564 +0000 UTC m=+0.518377702 container init 14e48ae402610e5ce2c57bc349f15268681567c72498128f6af9cf3b3aaea333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:07:59.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 podman[36347]: 2024-05-21 15:07:59.376167805 +0000 UTC m=+0.522635943 container start 14e48ae402610e5ce2c57bc349f15268681567c72498128f6af9cf3b3aaea333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-21T15:07:59.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: set uid:gid to 167:167 (ceph:ceph) 2024-05-21T15:07:59.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev), process ceph-mon, pid 2 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pidfile_write: ignore empty --pid-file 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: load: jerasure load: lrc 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: RocksDB version: 7.9.2 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Git sha 0 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Compile date 2024-05-17 21:21:31 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: DB SUMMARY 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: DB Session ID: M9JB6G1297WQW6X1X3MS 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: CURRENT file: CURRENT 2024-05-21T15:07:59.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.error_if_exists: 0 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.create_if_missing: 0 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.paranoid_checks: 1 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.env: 0x561f1016f4a0 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.info_log: 0x561f1231fd40 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T15:07:59.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.statistics: (nil) 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.use_fsync: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_log_file_size: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.allow_fallocate: 1 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.use_direct_reads: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T15:07:59.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.db_log_dir: 2024-05-21T15:07:59.687 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.wal_dir: 2024-05-21T15:07:59.687 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T15:07:59.687 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T15:07:59.687 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T15:07:59.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T15:07:59.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T15:07:59.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.write_buffer_manager: 0x561f122f32c0 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T15:07:59.689 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.unordered_write: 0 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.row_cache: None 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.wal_filter: None 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.two_write_queues: 0 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.wal_compression: 0 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.atomic_flush: 0 2024-05-21T15:07:59.690 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.log_readahead_size: 0 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_background_jobs: 2 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_background_compactions: -1 2024-05-21T15:07:59.691 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_subcompactions: 1 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_open_files: -1 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T15:07:59.692 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_background_flushes: -1 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Compression algorithms supported: 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: kZSTD supported: 0 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: kXpressCompression supported: 0 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: kZlibCompression supported: 1 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: kSnappyCompression supported: 1 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: kLZ4Compression supported: 1 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: kBZip2Compression supported: 0 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T15:07:59.693 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.merge_operator: 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_filter: None 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561f1231fe40) 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout: cache_index_and_filter_blocks: 1 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T15:07:59.694 INFO:journalctl@ceph.mon.c.smithi172.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: pin_top_level_index_and_filter: 1 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_type: 0 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: data_block_index_type: 0 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_shortening: 1 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: checksum: 4 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: no_block_cache: 0 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache: 0x561f12323090 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_name: BinnedLRUCache 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_options: 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: capacity : 536870912 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: num_shard_bits : 4 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: strict_capacity_limit : 0 2024-05-21T15:07:59.695 INFO:journalctl@ceph.mon.c.smithi172.stdout: high_pri_pool_ratio: 0.000 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_cache_compressed: (nil) 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: persistent_cache: (nil) 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_size: 4096 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_size_deviation: 10 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_restart_interval: 16 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: index_block_restart_interval: 1 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: metadata_block_size: 4096 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: partition_filters: 0 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: use_delta_encoding: 1 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: filter_policy: bloomfilter 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: whole_key_filtering: 1 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: verify_compression: 0 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: read_amp_bytes_per_bit: 0 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: format_version: 5 2024-05-21T15:07:59.696 INFO:journalctl@ceph.mon.c.smithi172.stdout: enable_index_compression: 1 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout: block_align: 0 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout: max_auto_readahead_size: 262144 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout: prepopulate_block_cache: 0 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout: initial_auto_readahead_size: 8192 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression: NoCompression 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T15:07:59.697 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.num_levels: 7 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T15:07:59.698 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T15:07:59.699 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T15:07:59.700 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T15:07:59.701 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.table_properties_collectors: 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.inplace_update_support: 0 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.bloom_locality: 0 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.max_successive_merges: 0 2024-05-21T15:07:59.702 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T15:07:59.703 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T15:07:59.703 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T15:07:59.703 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T15:07:59.703 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.ttl: 2592000 2024-05-21T15:07:59.703 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T15:07:59.703 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T15:07:59.704 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T15:07:59.704 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.enable_blob_files: false 2024-05-21T15:07:59.704 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.min_blob_size: 0 2024-05-21T15:07:59.704 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T15:07:59.704 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T15:07:59.704 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T15:07:59.704 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 94aba65a-742f-4c59-80d7-9dad24e3284c 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716304079417209, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716304079417727, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716304079, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "94aba65a-742f-4c59-80d7-9dad24e3284c", "db_session_id": "M9JB6G1297WQW6X1X3MS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716304079417825, "job": 1, "event": "recovery_finished"} 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T15:07:59.705 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561f12309c00 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: DB pointer 0x561f170f6000 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** DB Stats ** 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T15:07:59.706 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** Compaction Stats [default] ** 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** Compaction Stats [default] ** 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-21T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T15:07:59.708 INFO:journalctl@ceph.mon.c.smithi172.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout: Block cache BinnedLRUCache@0x561f12323090#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: using public_addr v2:172.21.15.172:0/0 -> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: starting mon.c rank -1 at public addrs [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] at bind addrs [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(???) e0 preinit fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).mds e1 new map 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout: e1 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T15:07:59.709 INFO:journalctl@ceph.mon.c.smithi172.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout: legacy client fscid: -1 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout: 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout: No filesystems configured 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:07:59.710 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Added host smithi084 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Deploying cephadm binary to smithi172 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.711 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:59.712 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Added host smithi172 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1814729035' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1814729035' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.713 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:07:59.714 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c;count:3 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: Deploying daemon mon.c on smithi172 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 ceph-mon[36402]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 bash[36347]: 14e48ae402610e5ce2c57bc349f15268681567c72498128f6af9cf3b3aaea333 2024-05-21T15:07:59.715 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:07:59 smithi172 systemd[1]: Started Ceph mon.c for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:08:00.987 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T15:08:00.987 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T15:08:01.225 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:08:04.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:08:04.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: mon.a calling monitor election 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: mon.c calling monitor election 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: monmap epoch 2 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:04.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: last_changed 2024-05-21T15:07:59.443711+0000 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: min_mon_release 19 (squid) 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: election_strategy: 1 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: fsmap 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: mgrmap e13: x(active, since 49s) 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: overall HEALTH_OK 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:04.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:08:04.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: mon.a calling monitor election 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: mon.c calling monitor election 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:08:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: monmap epoch 2 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: last_changed 2024-05-21T15:07:59.443711+0000 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: min_mon_release 19 (squid) 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: election_strategy: 1 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: fsmap 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: mgrmap e13: x(active, since 49s) 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: overall HEALTH_OK 2024-05-21T15:08:04.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:04.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:05.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.704+0000 7f249ad38700 1 -- 172.21.15.172:0/4150441052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 msgr2=0x7f247c005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:05.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.704+0000 7f249ad38700 1 --2- 172.21.15.172:0/4150441052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 0x7f247c005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2494101900 tx=0x7f248c009e80 comp rx=0 tx=0).stop 2024-05-21T15:08:05.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.704+0000 7f249ad38700 1 -- 172.21.15.172:0/4150441052 shutdown_connections 2024-05-21T15:08:05.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.704+0000 7f249ad38700 1 --2- 172.21.15.172:0/4150441052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 0x7f247c005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:05.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.704+0000 7f249ad38700 1 -- 172.21.15.172:0/4150441052 >> 172.21.15.172:0/4150441052 conn(0x7f24940fd530 msgr2=0x7f24940ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:05.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.704+0000 7f249ad38700 1 -- 172.21.15.172:0/4150441052 shutdown_connections 2024-05-21T15:08:05.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.704+0000 7f249ad38700 1 -- 172.21.15.172:0/4150441052 wait complete. 2024-05-21T15:08:05.708 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:05.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:05.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:05.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:05 smithi172 ceph-mon[36402]: Deploying daemon mon.b on smithi084 2024-05-21T15:08:05.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:05 smithi172 ceph-mon[36402]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:05.709 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:05.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.705+0000 7f249ad38700 1 Processor -- start 2024-05-21T15:08:05.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.705+0000 7f249ad38700 1 -- start start 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.705+0000 7f249ad38700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 0x7f2494197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.705+0000 7f249ad38700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 0x7f2494197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.705+0000 7f249ad38700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2494109b00 con 0x7f247c0035c0 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.705+0000 7f249ad38700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2494109980 con 0x7f2494197bb0 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2493fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 0x7f2494197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2493fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 0x7f2494197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39820/0 (socket says 172.21.15.172:39820) 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2493fff700 1 -- 172.21.15.172:0/650923943 learned_addr learned my addr 172.21.15.172:0/650923943 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2498ad4700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 0x7f2494197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2493fff700 1 -- 172.21.15.172:0/650923943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 msgr2=0x7f2494197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-21T15:08:05.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2493fff700 1 -- 172.21.15.172:0/650923943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 msgr2=0x7f2494197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2493fff700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 0x7f2494197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2493fff700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 0x7f2494197fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2498ad4700 1 -- 172.21.15.172:0/650923943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 msgr2=0x7f2494197fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2498ad4700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 0x7f2494197fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2498ad4700 1 -- 172.21.15.172:0/650923943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249406b8c0 con 0x7f247c0035c0 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2498ad4700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 0x7f2494197670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f248c01b000 tx=0x7f248c007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2491ffb700 1 -- 172.21.15.172:0/650923943 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f248c02a470 con 0x7f247c0035c0 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2491ffb700 1 -- 172.21.15.172:0/650923943 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f248c025070 con 0x7f247c0035c0 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24941a4ce0 con 0x7f247c0035c0 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.706+0000 7f2491ffb700 1 -- 172.21.15.172:0/650923943 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f248c021b10 con 0x7f247c0035c0 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.707+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941a5190 con 0x7f247c0035c0 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.708+0000 7f2491ffb700 1 -- 172.21.15.172:0/650923943 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f248c01e070 con 0x7f247c0035c0 2024-05-21T15:08:05.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.708+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2494103110 con 0x7f247c0035c0 2024-05-21T15:08:05.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.708+0000 7f2491ffb700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2480038f00 0x7f248003b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:05.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.708+0000 7f2491ffb700 1 -- 172.21.15.172:0/650923943 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f248c01c070 con 0x7f247c0035c0 2024-05-21T15:08:05.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.708+0000 7f2493fff700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2480038f00 0x7f248003b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:05.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.709+0000 7f2493fff700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2480038f00 0x7f248003b410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2494100e70 tx=0x7f2488008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:05.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.714+0000 7f2491ffb700 1 -- 172.21.15.172:0/650923943 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f248c016c90 con 0x7f247c0035c0 2024-05-21T15:08:05.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:05.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:05.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:05.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:05 smithi078 ceph-mon[31997]: Deploying daemon mon.b on smithi084 2024-05-21T15:08:05.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:05 smithi078 ceph-mon[31997]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:05.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:05.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.988+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f249406b1f0 con 0x7f247c0035c0 2024-05-21T15:08:05.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.989+0000 7f2491ffb700 1 -- 172.21.15.172:0/650923943 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f248c01f3d0 con 0x7f247c0035c0 2024-05-21T15:08:05.992 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:08:05.992 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","modified":"2024-05-21T15:07:59.443711Z","created":"2024-05-21T15:06:17.120039Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2480038f00 msgr2=0x7f248003b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2480038f00 0x7f248003b410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2494100e70 tx=0x7f2488008040 comp rx=0 tx=0).stop 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 msgr2=0x7f2494197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 0x7f2494197670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f248c01b000 tx=0x7f248c007da0 comp rx=0 tx=0).stop 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 shutdown_connections 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2480038f00 0x7f248003b410 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247c0035c0 0x7f2494197670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 --2- 172.21.15.172:0/650923943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2494197bb0 0x7f2494197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 >> 172.21.15.172:0/650923943 conn(0x7f24940fd530 msgr2=0x7f2494104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:05.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 shutdown_connections 2024-05-21T15:08:05.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:05.991+0000 7f249ad38700 1 -- 172.21.15.172:0/650923943 wait complete. 2024-05-21T15:08:05.996 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 2 2024-05-21T15:08:06.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:06 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/650923943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:08:06.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:06 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/650923943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:08:07.569 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T15:08:07.569 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T15:08:07.814 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:08:07.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:07 smithi078 ceph-mon[31997]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:07.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:07 smithi172 ceph-mon[36402]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.074 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: load: jerasure load: lrc 2024-05-21T15:08:08.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: RocksDB version: 7.9.2 2024-05-21T15:08:08.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Git sha 0 2024-05-21T15:08:08.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Compile date 2024-05-17 21:21:31 2024-05-21T15:08:08.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: DB SUMMARY 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: DB Session ID: OLSHDBEXCBCXPOX79U3Y 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: CURRENT file: CURRENT 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.error_if_exists: 0 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.create_if_missing: 0 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.paranoid_checks: 1 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T15:08:08.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.env: 0x55ff21a314a0 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.info_log: 0x55ff228dfd40 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.statistics: (nil) 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.use_fsync: 0 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_log_file_size: 0 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T15:08:08.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T15:08:08.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T15:08:08.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.allow_fallocate: 1 2024-05-21T15:08:08.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T15:08:08.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T15:08:08.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.use_direct_reads: 0 2024-05-21T15:08:08.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T15:08:08.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T15:08:08.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.db_log_dir: 2024-05-21T15:08:08.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.wal_dir: 2024-05-21T15:08:08.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T15:08:08.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T15:08:08.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T15:08:08.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T15:08:08.403 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T15:08:08.403 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T15:08:08.403 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.write_buffer_manager: 0x55ff228b32c0 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.unordered_write: 0 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T15:08:08.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.row_cache: None 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.wal_filter: None 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.two_write_queues: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.wal_compression: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.atomic_flush: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T15:08:08.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.log_readahead_size: 0 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_background_jobs: 2 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_background_compactions: -1 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_subcompactions: 1 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T15:08:08.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_open_files: -1 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_background_flushes: -1 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Compression algorithms supported: 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T15:08:08.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: kZSTD supported: 0 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: kXpressCompression supported: 0 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: kZlibCompression supported: 1 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: kSnappyCompression supported: 1 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: kLZ4Compression supported: 1 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: kBZip2Compression supported: 0 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.merge_operator: 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_filter: None 2024-05-21T15:08:08.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ff228dfe40) 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: cache_index_and_filter_blocks: 1 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: pin_top_level_index_and_filter: 1 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_type: 0 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: data_block_index_type: 0 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_shortening: 1 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T15:08:08.409 INFO:journalctl@ceph.mon.b.smithi084.stdout: checksum: 4 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: no_block_cache: 0 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache: 0x55ff228e3090 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_name: BinnedLRUCache 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_options: 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: capacity : 536870912 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: num_shard_bits : 4 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: strict_capacity_limit : 0 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: high_pri_pool_ratio: 0.000 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_compressed: (nil) 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: persistent_cache: (nil) 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_size: 4096 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_size_deviation: 10 2024-05-21T15:08:08.410 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_restart_interval: 16 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_block_restart_interval: 1 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: metadata_block_size: 4096 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: partition_filters: 0 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: use_delta_encoding: 1 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: filter_policy: bloomfilter 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: whole_key_filtering: 1 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: verify_compression: 0 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: read_amp_bytes_per_bit: 0 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: format_version: 5 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: enable_index_compression: 1 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_align: 0 2024-05-21T15:08:08.411 INFO:journalctl@ceph.mon.b.smithi084.stdout: max_auto_readahead_size: 262144 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout: prepopulate_block_cache: 0 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout: initial_auto_readahead_size: 8192 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression: NoCompression 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.num_levels: 7 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T15:08:08.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T15:08:08.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T15:08:08.414 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T15:08:08.415 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.table_properties_collectors: 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.inplace_update_support: 0 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T15:08:08.416 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.bloom_locality: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.max_successive_merges: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.ttl: 2592000 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.enable_blob_files: false 2024-05-21T15:08:08.417 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.min_blob_size: 0 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 37b322e8-c49a-4a9a-891c-9fc85cd79a2d 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716304088073428, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-21T15:08:08.418 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716304088074079, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716304088, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "37b322e8-c49a-4a9a-891c-9fc85cd79a2d", "db_session_id": "OLSHDBEXCBCXPOX79U3Y", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716304088074185, "job": 1, "event": "recovery_finished"} 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ff228c9c00 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: DB pointer 0x55ff276b6000 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** DB Stats ** 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T15:08:08.419 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** Compaction Stats [default] ** 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** Compaction Stats [default] ** 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T15:08:08.420 INFO:journalctl@ceph.mon.b.smithi084.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T15:08:08.421 INFO:journalctl@ceph.mon.b.smithi084.stdout: Block cache BinnedLRUCache@0x55ff228e3090#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: using public_addr v2:172.21.15.84:0/0 -> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: starting mon.b rank -1 at public addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] at bind addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(???) e0 preinit fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).mds e1 new map 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout: e1 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T15:08:08.422 INFO:journalctl@ceph.mon.b.smithi084.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout: legacy client fscid: -1 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout: No filesystems configured 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T15:08:08.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Added host smithi084 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Deploying cephadm binary to smithi172 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.424 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Added host smithi172 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1814729035' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1814729035' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.426 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:08:08.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c;count:3 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Deploying daemon mon.c on smithi172 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.a calling monitor election 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.c calling monitor election 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:08.429 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: monmap epoch 2 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: last_changed 2024-05-21T15:07:59.443711+0000 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: min_mon_release 19 (squid) 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: election_strategy: 1 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: fsmap 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mgrmap e13: x(active, since 49s) 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: overall HEALTH_OK 2024-05-21T15:08:08.430 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: Deploying daemon mon.b on smithi084 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/650923943' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 ceph-mon[36334]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 bash[36313]: 85123a736b782adda8df75cfdbcc90ae68b04a8ac362501d2781819fb07a4f59 2024-05-21T15:08:08.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:08 smithi084 systemd[1]: Started Ceph mon.b for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:08:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:08:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: mon.c calling monitor election 2024-05-21T15:08:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: mon.a calling monitor election 2024-05-21T15:08:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: monmap epoch 3 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: last_changed 2024-05-21T15:08:08.101753+0000 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: min_mon_release 19 (squid) 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: election_strategy: 1 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: 2: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-05-21T15:08:13.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: fsmap 2024-05-21T15:08:13.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:13.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: mgrmap e13: x(active, since 57s) 2024-05-21T15:08:13.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: overall HEALTH_OK 2024-05-21T15:08:13.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:13.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:08:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: mon.c calling monitor election 2024-05-21T15:08:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: mon.a calling monitor election 2024-05-21T15:08:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: monmap epoch 3 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: last_changed 2024-05-21T15:08:08.101753+0000 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: min_mon_release 19 (squid) 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: election_strategy: 1 2024-05-21T15:08:13.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:13.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:13.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: 2: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-05-21T15:08:13.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: fsmap 2024-05-21T15:08:13.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:13.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: mgrmap e13: x(active, since 57s) 2024-05-21T15:08:13.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: overall HEALTH_OK 2024-05-21T15:08:13.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:13.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:13.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.694+0000 7ff751247700 1 -- 172.21.15.172:0/2595646960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 msgr2=0x7ff734005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:13.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.694+0000 7ff751247700 1 --2- 172.21.15.172:0/2595646960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 0x7ff734005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff74c1029f0 tx=0x7ff74000fe60 comp rx=0 tx=0).stop 2024-05-21T15:08:13.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.694+0000 7ff751247700 1 -- 172.21.15.172:0/2595646960 shutdown_connections 2024-05-21T15:08:13.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.694+0000 7ff751247700 1 --2- 172.21.15.172:0/2595646960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 0x7ff734005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:13.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.694+0000 7ff751247700 1 -- 172.21.15.172:0/2595646960 >> 172.21.15.172:0/2595646960 conn(0x7ff74c0fe620 msgr2=0x7ff74c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:13.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.694+0000 7ff751247700 1 -- 172.21.15.172:0/2595646960 shutdown_connections 2024-05-21T15:08:13.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.694+0000 7ff751247700 1 -- 172.21.15.172:0/2595646960 wait complete. 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.695+0000 7ff751247700 1 Processor -- start 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.695+0000 7ff751247700 1 -- start start 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.695+0000 7ff751247700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 0x7ff74c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.695+0000 7ff751247700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 0x7ff74c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.695+0000 7ff751247700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 0x7ff74c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.695+0000 7ff751247700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c10aba0 con 0x7ff7340035c0 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.695+0000 7ff751247700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c10aa20 con 0x7ff74c198f00 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.695+0000 7ff751247700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c10ad20 con 0x7ff74c19ece0 2024-05-21T15:08:13.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74a59c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 0x7ff74c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74a59c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 0x7ff74c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40812/0 (socket says 172.21.15.172:40812) 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74a59c700 1 -- 172.21.15.172:0/2911619279 learned_addr learned my addr 172.21.15.172:0/2911619279 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74ad9d700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 0x7ff74c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74a59c700 1 -- 172.21.15.172:0/2911619279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 msgr2=0x7ff74c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74b59e700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 0x7ff74c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74a59c700 1 -- 172.21.15.172:0/2911619279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 msgr2=0x7ff74c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74a59c700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 0x7ff74c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74a59c700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 0x7ff74c1989c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74b59e700 1 -- 172.21.15.172:0/2911619279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 msgr2=0x7ff74c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74b59e700 1 -- 172.21.15.172:0/2911619279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 msgr2=0x7ff74c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74b59e700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 0x7ff74c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74b59e700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 0x7ff74c199310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:08:13.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74ad9d700 1 -- 172.21.15.172:0/2911619279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 msgr2=0x7ff74c1989c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74ad9d700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 0x7ff74c1989c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74ad9d700 1 -- 172.21.15.172:0/2911619279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 msgr2=0x7ff74c199310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74ad9d700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 0x7ff74c199310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74ad9d700 1 -- 172.21.15.172:0/2911619279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c1a5630 con 0x7ff7340035c0 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff74ad9d700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 0x7ff74c19e7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff74c1029f0 tx=0x7ff740005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff733fff700 1 -- 172.21.15.172:0/2911619279 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74002b020 con 0x7ff7340035c0 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff733fff700 1 -- 172.21.15.172:0/2911619279 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff740014550 con 0x7ff7340035c0 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.696+0000 7ff733fff700 1 -- 172.21.15.172:0/2911619279 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740022040 con 0x7ff7340035c0 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.697+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c1a5860 con 0x7ff7340035c0 2024-05-21T15:08:13.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.697+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1a5dc0 con 0x7ff7340035c0 2024-05-21T15:08:13.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.698+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74c063ca0 con 0x7ff7340035c0 2024-05-21T15:08:13.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.698+0000 7ff733fff700 1 -- 172.21.15.172:0/2911619279 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff74002e020 con 0x7ff7340035c0 2024-05-21T15:08:13.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.698+0000 7ff733fff700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff738039580 0x7ff73803ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:13.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.698+0000 7ff733fff700 1 -- 172.21.15.172:0/2911619279 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff74004eb90 con 0x7ff7340035c0 2024-05-21T15:08:13.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.698+0000 7ff74a59c700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff738039580 0x7ff73803ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:13.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.699+0000 7ff74a59c700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff738039580 0x7ff73803ba90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0fefe0 tx=0x7ff73c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:13.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:13.704+0000 7ff733fff700 1 -- 172.21.15.172:0/2911619279 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff740035020 con 0x7ff7340035c0 2024-05-21T15:08:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.003+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff74c02dc60 con 0x7ff7340035c0 2024-05-21T15:08:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.003+0000 7ff733fff700 1 -- 172.21.15.172:0/2911619279 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7ff740026370 con 0x7ff7340035c0 2024-05-21T15:08:14.008 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-05-21T15:08:14.008 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":3,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","modified":"2024-05-21T15:08:08.101753Z","created":"2024-05-21T15:06:17.120039Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T15:08:14.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff738039580 msgr2=0x7ff73803ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:14.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff738039580 0x7ff73803ba90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0fefe0 tx=0x7ff73c008040 comp rx=0 tx=0).stop 2024-05-21T15:08:14.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 msgr2=0x7ff74c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:14.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 0x7ff74c19e7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff74c1029f0 tx=0x7ff740005720 comp rx=0 tx=0).stop 2024-05-21T15:08:14.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 shutdown_connections 2024-05-21T15:08:14.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff738039580 0x7ff73803ba90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:14.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c198f00 0x7ff74c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:14.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7340035c0 0x7ff74c19e7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:14.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 --2- 172.21.15.172:0/2911619279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c19ece0 0x7ff74c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:14.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 >> 172.21.15.172:0/2911619279 conn(0x7ff74c0fe620 msgr2=0x7ff74c105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:14.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 shutdown_connections 2024-05-21T15:08:14.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:14.006+0000 7ff751247700 1 -- 172.21.15.172:0/2911619279 wait complete. 2024-05-21T15:08:14.011 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 3 2024-05-21T15:08:14.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:14 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:14.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:14 smithi172 ceph-mon[36402]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:14.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:14 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:14.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:14 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:14.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:14 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/2911619279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:08:14.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:14 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:14.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:14 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:14 smithi078 ceph-mon[31997]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:14 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:14 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:14 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/2911619279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:08:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:14 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:14.772 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-21T15:08:14.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-21T15:08:15.022 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:08:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:08:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: mon.c calling monitor election 2024-05-21T15:08:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: mon.a calling monitor election 2024-05-21T15:08:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:08:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: monmap epoch 3 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: last_changed 2024-05-21T15:08:08.101753+0000 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: min_mon_release 19 (squid) 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: election_strategy: 1 2024-05-21T15:08:15.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: 2: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: fsmap 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: mgrmap e13: x(active, since 57s) 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: overall HEALTH_OK 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/2911619279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T15:08:15.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: mon.b calling monitor election 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: Updating smithi078:/etc/ceph/ceph.conf 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: mon.b calling monitor election 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: mon.c calling monitor election 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: mon.a calling monitor election 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: monmap epoch 3 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: last_changed 2024-05-21T15:08:08.101753+0000 2024-05-21T15:08:16.375 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: min_mon_release 19 (squid) 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: election_strategy: 1 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: 2: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: fsmap 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: mgrmap e13: x(active, since 59s) 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: overall HEALTH_OK 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:16.376 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:16.377 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: mon.b calling monitor election 2024-05-21T15:08:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: Updating smithi078:/etc/ceph/ceph.conf 2024-05-21T15:08:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:08:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:08:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: mon.b calling monitor election 2024-05-21T15:08:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: mon.c calling monitor election 2024-05-21T15:08:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: mon.a calling monitor election 2024-05-21T15:08:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: monmap epoch 3 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: last_changed 2024-05-21T15:08:08.101753+0000 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: min_mon_release 19 (squid) 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: election_strategy: 1 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: 2: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: fsmap 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: mgrmap e13: x(active, since 59s) 2024-05-21T15:08:16.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: overall HEALTH_OK 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:16.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:16.399 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: mon.b calling monitor election 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: Updating smithi078:/etc/ceph/ceph.conf 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: mon.b calling monitor election 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: mon.c calling monitor election 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: mon.a calling monitor election 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T15:08:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: monmap epoch 3 2024-05-21T15:08:16.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:16.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: last_changed 2024-05-21T15:08:08.101753+0000 2024-05-21T15:08:16.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: created 2024-05-21T15:06:17.120039+0000 2024-05-21T15:08:16.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: min_mon_release 19 (squid) 2024-05-21T15:08:16.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: election_strategy: 1 2024-05-21T15:08:16.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: 2: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: fsmap 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: mgrmap e13: x(active, since 59s) 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: overall HEALTH_OK 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:16.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:16.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:16.438 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 -- 172.21.15.78:0/2778698691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e81089f0 msgr2=0x7f81e8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 --2- 172.21.15.78:0/2778698691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e81089f0 0x7f81e8108e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009910 tx=0x7f81d8009c20 comp rx=0 tx=0).stop 2024-05-21T15:08:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 -- 172.21.15.78:0/2778698691 shutdown_connections 2024-05-21T15:08:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 --2- 172.21.15.78:0/2778698691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e81089f0 0x7f81e8108e00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 -- 172.21.15.78:0/2778698691 >> 172.21.15.78:0/2778698691 conn(0x7f81e80fe4e0 msgr2=0x7f81e8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 -- 172.21.15.78:0/2778698691 shutdown_connections 2024-05-21T15:08:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 -- 172.21.15.78:0/2778698691 wait complete. 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 Processor -- start 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.722+0000 7f81ee3af700 1 -- start start 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81ee3af700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e81089f0 0x7f81e819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81ee3af700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e819ecd0 0x7f81e81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81ee3af700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e8198ef0 0x7f81e8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81ee3af700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81e810d090 con 0x7f81e819ecd0 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81ee3af700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f81e810cf10 con 0x7f81e81089f0 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81ee3af700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81e810d210 con 0x7f81e8198ef0 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e819ecd0 0x7f81e81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e819ecd0 0x7f81e81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35566/0 (socket says 172.21.15.78:35566) 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 -- 172.21.15.78:0/3069089388 learned_addr learned my addr 172.21.15.78:0/3069089388 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:08:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e7fff700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e81089f0 0x7f81e819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:17.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 -- 172.21.15.78:0/3069089388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e8198ef0 msgr2=0x7f81e8199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:17.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e8198ef0 0x7f81e8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:17.323 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:17 smithi078 ceph-mon[31997]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.323 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:17 smithi078 ceph-mon[31997]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.323 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:17 smithi078 ceph-mon[31997]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.323 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:17 smithi078 ceph-mon[31997]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T15:08:17.323 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:17 smithi078 ceph-mon[31997]: Reconfiguring daemon mon.a on smithi078 2024-05-21T15:08:17.323 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:17 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:17.323 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:17 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3069089388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:17.324 INFO:teuthology.orchestra.run.smithi078.stdout:# minimal ceph.conf for 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:17.324 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-05-21T15:08:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:08:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: mon_host = [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] 2024-05-21T15:08:17.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 -- 172.21.15.78:0/3069089388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e81089f0 msgr2=0x7f81e819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:17.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e81089f0 0x7f81e819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:17.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 -- 172.21.15.78:0/3069089388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e81a5620 con 0x7f81e819ecd0 2024-05-21T15:08:17.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e7fff700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e81089f0 0x7f81e819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.723+0000 7f81e77fe700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e819ecd0 0x7f81e81989b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00b780 tx=0x7f81dc00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.724+0000 7f81e57fa700 1 -- 172.21.15.78:0/3069089388 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc004230 con 0x7f81e819ecd0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.724+0000 7f81e57fa700 1 -- 172.21.15.78:0/3069089388 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81dc0043d0 con 0x7f81e819ecd0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.724+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e81a58b0 con 0x7f81e819ecd0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.724+0000 7f81e57fa700 1 -- 172.21.15.78:0/3069089388 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc011c60 con 0x7f81e819ecd0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.724+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e81a5dc0 con 0x7f81e819ecd0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.725+0000 7f81e57fa700 1 -- 172.21.15.78:0/3069089388 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f81dc01a400 con 0x7f81e819ecd0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.725+0000 7f81e57fa700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f81d0039580 0x7f81d003ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.725+0000 7f81e57fa700 1 -- 172.21.15.78:0/3069089388 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f81dc04ce20 con 0x7f81e819ecd0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.725+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e8063ca0 con 0x7f81e819ecd0 2024-05-21T15:08:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.725+0000 7f81e7fff700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f81d0039580 0x7f81d003ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.725+0000 7f81e7fff700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f81d0039580 0x7f81d003ba90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81d8006010 tx=0x7f81d801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.730+0000 7f81e57fa700 1 -- 172.21.15.78:0/3069089388 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f81dc020500 con 0x7f81e819ecd0 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.934+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f81e81a6120 con 0x7f81e819ecd0 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.935+0000 7f81e57fa700 1 -- 172.21.15.78:0/3069089388 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f81dc0202a0 con 0x7f81e819ecd0 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f81d0039580 msgr2=0x7f81d003ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f81d0039580 0x7f81d003ba90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81d8006010 tx=0x7f81d801a040 comp rx=0 tx=0).stop 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e819ecd0 msgr2=0x7f81e81989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e819ecd0 0x7f81e81989b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00b780 tx=0x7f81dc00bb40 comp rx=0 tx=0).stop 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 shutdown_connections 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f81d0039580 0x7f81d003ba90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e8198ef0 0x7f81e8199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:17.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81e81089f0 0x7f81e819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:17.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 --2- 172.21.15.78:0/3069089388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81e819ecd0 0x7f81e81989b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:17.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 >> 172.21.15.78:0/3069089388 conn(0x7f81e80fe4e0 msgr2=0x7f81e80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:17.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 shutdown_connections 2024-05-21T15:08:17.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:16.937+0000 7f81ee3af700 1 -- 172.21.15.78:0/3069089388 wait complete. 2024-05-21T15:08:17.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:17 smithi084 ceph-mon[36334]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:17 smithi084 ceph-mon[36334]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:17 smithi084 ceph-mon[36334]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:17 smithi084 ceph-mon[36334]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T15:08:17.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:17 smithi084 ceph-mon[36334]: Reconfiguring daemon mon.a on smithi078 2024-05-21T15:08:17.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:17 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:17.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:17 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3069089388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:17.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:17 smithi172 ceph-mon[36402]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:17 smithi172 ceph-mon[36402]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:17 smithi172 ceph-mon[36402]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:08:17.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:17 smithi172 ceph-mon[36402]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T15:08:17.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:17 smithi172 ceph-mon[36402]: Reconfiguring daemon mon.a on smithi078 2024-05-21T15:08:17.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:17 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:08:17.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:17 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3069089388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:17.870 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-21T15:08:17.870 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:08:17.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T15:08:17.899 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:08:17.899 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:08:17.964 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:08:17.964 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T15:08:17.994 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:08:17.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:08:18.059 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:08:18.059 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T15:08:18.090 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:08:18.091 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:08:18.159 INFO:tasks.cephadm:Adding mgr.x on smithi078 2024-05-21T15:08:18.159 INFO:tasks.cephadm:Adding mgr.y on smithi084 2024-05-21T15:08:18.159 INFO:tasks.cephadm:Adding mgr.z on smithi172 2024-05-21T15:08:18.160 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi078=x;smithi084=y;smithi172=z' 2024-05-21T15:08:18.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:18 smithi084 ceph-mon[36334]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:18.433 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:08:18.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:18 smithi172 ceph-mon[36402]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:18.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:18 smithi078 ceph-mon[31997]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:19.605 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:19.606 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:19.606 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:19 smithi084 ceph-mon[36334]: Reconfiguring mon.b (monmap changed)... 2024-05-21T15:08:19.606 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:19.606 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:19.606 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:19.606 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:19 smithi084 ceph-mon[36334]: Reconfiguring daemon mon.b on smithi084 2024-05-21T15:08:19.606 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:19 smithi084 ceph-mon[36334]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:19.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:19.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:19.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:19 smithi172 ceph-mon[36402]: Reconfiguring mon.b (monmap changed)... 2024-05-21T15:08:19.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:19.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:19.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:19.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:19 smithi172 ceph-mon[36402]: Reconfiguring daemon mon.b on smithi084 2024-05-21T15:08:19.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:19 smithi172 ceph-mon[36402]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:19.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:19.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:19.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:19 smithi078 ceph-mon[31997]: Reconfiguring mon.b (monmap changed)... 2024-05-21T15:08:19.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:19.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:19.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:19.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:19 smithi078 ceph-mon[31997]: Reconfiguring daemon mon.b on smithi084 2024-05-21T15:08:19.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:19 smithi078 ceph-mon[31997]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:19.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 -- 172.21.15.172:0/2202973999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce541036d0 msgr2=0x7fce54105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:19.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 --2- 172.21.15.172:0/2202973999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce541036d0 0x7fce54105af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fce48009910 tx=0x7fce48009c20 comp rx=0 tx=0).stop 2024-05-21T15:08:19.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 -- 172.21.15.172:0/2202973999 shutdown_connections 2024-05-21T15:08:19.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 --2- 172.21.15.172:0/2202973999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce541036d0 0x7fce54105af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:19.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 -- 172.21.15.172:0/2202973999 >> 172.21.15.172:0/2202973999 conn(0x7fce540fd0d0 msgr2=0x7fce540ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:19.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 -- 172.21.15.172:0/2202973999 shutdown_connections 2024-05-21T15:08:19.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 -- 172.21.15.172:0/2202973999 wait complete. 2024-05-21T15:08:19.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 Processor -- start 2024-05-21T15:08:19.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.936+0000 7fce5ace2700 1 -- start start 2024-05-21T15:08:19.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce5ace2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce541036d0 0x7fce541988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:19.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce5ace2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce54198e20 0x7fce5419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:19.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce5ace2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5419e2f0 0x7fce541a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:19.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce5ace2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce54108c30 con 0x7fce5419e2f0 2024-05-21T15:08:19.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce5ace2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fce54108ab0 con 0x7fce541036d0 2024-05-21T15:08:19.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce5ace2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fce54108db0 con 0x7fce54198e20 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce53fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce54198e20 0x7fce5419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce541036d0 0x7fce541988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce541036d0 0x7fce541988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.172:47168/0 (socket says 172.21.15.172:47168) 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce5927f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5419e2f0 0x7fce541a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 -- 172.21.15.172:0/4273529543 learned_addr learned my addr 172.21.15.172:0/4273529543 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce53fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce54198e20 0x7fce5419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40822/0 (socket says 172.21.15.172:40822) 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 -- 172.21.15.172:0/4273529543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce54198e20 msgr2=0x7fce5419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce54198e20 0x7fce5419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 -- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5419e2f0 msgr2=0x7fce541a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5419e2f0 0x7fce541a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 -- 172.21.15.172:0/4273529543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce541a0c80 con 0x7fce541036d0 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce5927f700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5419e2f0 0x7fce541a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:08:19.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.937+0000 7fce58a7e700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce541036d0 0x7fce541988e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fce48004f80 tx=0x7fce48004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.938+0000 7fce51ffb700 1 -- 172.21.15.172:0/4273529543 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48026070 con 0x7fce541036d0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.938+0000 7fce51ffb700 1 -- 172.21.15.172:0/4273529543 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce4800b820 con 0x7fce541036d0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.938+0000 7fce51ffb700 1 -- 172.21.15.172:0/4273529543 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48003990 con 0x7fce541036d0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.938+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce541a0f10 con 0x7fce541036d0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.939+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce541a13e0 con 0x7fce541036d0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.940+0000 7fce51ffb700 1 -- 172.21.15.172:0/4273529543 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fce48029070 con 0x7fce541036d0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.940+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce541a1640 con 0x7fce541036d0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.940+0000 7fce51ffb700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fce44039580 0x7fce4403ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.940+0000 7fce51ffb700 1 -- 172.21.15.172:0/4273529543 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fce4804d0c0 con 0x7fce541036d0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.940+0000 7fce53fff700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fce44039580 0x7fce4403ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:19.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.941+0000 7fce53fff700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fce44039580 0x7fce4403ba90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fce54199990 tx=0x7fce40009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:19.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:19.946+0000 7fce51ffb700 1 -- 172.21.15.172:0/4273529543 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce48013070 con 0x7fce541036d0 2024-05-21T15:08:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.167+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi084=y;smithi172=z", "target": ["mon-mgr", ""]}) -- 0x7fce540fee80 con 0x7fce44039580 2024-05-21T15:08:20.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.174+0000 7fce51ffb700 1 -- 172.21.15.172:0/4273529543 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fce540fee80 con 0x7fce44039580 2024-05-21T15:08:20.177 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mgr update... 2024-05-21T15:08:20.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fce44039580 msgr2=0x7fce4403ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fce44039580 0x7fce4403ba90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fce54199990 tx=0x7fce40009210 comp rx=0 tx=0).stop 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce541036d0 msgr2=0x7fce541988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce541036d0 0x7fce541988e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fce48004f80 tx=0x7fce48004fb0 comp rx=0 tx=0).stop 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 shutdown_connections 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fce44039580 0x7fce4403ba90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce5419e2f0 0x7fce541a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce541036d0 0x7fce541988e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 --2- 172.21.15.172:0/4273529543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce54198e20 0x7fce5419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 >> 172.21.15.172:0/4273529543 conn(0x7fce540fd0d0 msgr2=0x7fce540fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 shutdown_connections 2024-05-21T15:08:20.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:08:20.176+0000 7fce5ace2700 1 -- 172.21.15.172:0/4273529543 wait complete. 2024-05-21T15:08:20.797 DEBUG:teuthology.orchestra.run.smithi084:mgr.y> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-21T15:08:20.802 DEBUG:teuthology.orchestra.run.smithi172:mgr.z> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-21T15:08:20.804 INFO:tasks.cephadm:Deploying OSDs... 2024-05-21T15:08:20.804 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:08:20.804 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T15:08:20.823 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T15:08:20.823 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-05-21 15:07:28.291642769 +0000 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-05-21 15:01:27.799790614 +0000 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-05-21 15:01:27.799790614 +0000 2024-05-21T15:08:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-05-21 15:01:27.799790614 +0000 2024-05-21T15:08:20.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T15:08:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-05-21T15:08:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-05-21T15:08:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000256601 s, 2.0 MB/s 2024-05-21T15:08:20.950 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T15:08:21.007 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-05-21T15:08:21.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: Reconfiguring mon.c (monmap changed)... 2024-05-21T15:08:21.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:21.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:21.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:21.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: Reconfiguring daemon mon.c on smithi172 2024-05-21T15:08:21.060 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi084=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:21.061 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: Saving service mgr spec with placement smithi078=x;smithi084=y;smithi172=z;count:3 2024-05-21T15:08:21.061 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T15:08:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:21.064 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-21T15:08:21.064 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:21.064 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:21.064 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-05-21 15:07:28.323642302 +0000 2024-05-21T15:08:21.065 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-05-21 15:01:28.160785458 +0000 2024-05-21T15:08:21.065 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-05-21 15:01:28.160785458 +0000 2024-05-21T15:08:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-05-21 15:01:28.160785458 +0000 2024-05-21T15:08:21.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T15:08:21.130 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-05-21T15:08:21.130 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-05-21T15:08:21.130 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000222947 s, 2.3 MB/s 2024-05-21T15:08:21.131 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T15:08:21.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: Reconfiguring mon.c (monmap changed)... 2024-05-21T15:08:21.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:21.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:21.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:21.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: Reconfiguring daemon mon.c on smithi172 2024-05-21T15:08:21.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi084=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:21.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: Saving service mgr spec with placement smithi078=x;smithi084=y;smithi172=z;count:3 2024-05-21T15:08:21.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.189 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-05-21 15:07:28.355641835 +0000 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-05-21 15:01:28.497780645 +0000 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-05-21 15:01:28.497780645 +0000 2024-05-21T15:08:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-05-21 15:01:28.497780645 +0000 2024-05-21T15:08:21.246 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: Reconfiguring mon.c (monmap changed)... 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: Reconfiguring daemon mon.c on smithi172 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: from='client.24104 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi084=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:21.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: Saving service mgr spec with placement smithi078=x;smithi084=y;smithi172=z;count:3 2024-05-21T15:08:21.308 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:21.309 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-05-21T15:08:21.309 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-05-21T15:08:21.309 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000275666 s, 1.9 MB/s 2024-05-21T15:08:21.310 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T15:08:21.367 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-05-21T15:08:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T15:08:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:21.424 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-21T15:08:21.424 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:21.424 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:21.424 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-05-21 15:07:28.386641382 +0000 2024-05-21T15:08:21.424 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-05-21 15:01:28.830775889 +0000 2024-05-21T15:08:21.424 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-05-21 15:01:28.830775889 +0000 2024-05-21T15:08:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-05-21 15:01:28.830775889 +0000 2024-05-21T15:08:21.424 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T15:08:21.487 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-05-21T15:08:21.487 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-05-21T15:08:21.487 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000253743 s, 2.0 MB/s 2024-05-21T15:08:21.488 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T15:08:21.545 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-05-21T15:08:21.545 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T15:08:21.564 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T15:08:21.564 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-21 15:08:07.528418915 +0000 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-21 15:01:46.901086804 +0000 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-21 15:01:46.901086804 +0000 2024-05-21T15:08:21.622 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-21 15:01:46.901086804 +0000 2024-05-21T15:08:21.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T15:08:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-05-21T15:08:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-05-21T15:08:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000355199 s, 1.4 MB/s 2024-05-21T15:08:21.689 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T15:08:21.747 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-05-21T15:08:21.804 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T15:08:21.805 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:21.805 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-21T15:08:21.805 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:21.805 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:21.805 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-21 15:08:07.528418915 +0000 2024-05-21T15:08:21.805 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-21 15:01:47.242080726 +0000 2024-05-21T15:08:21.805 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-21 15:01:47.242080726 +0000 2024-05-21T15:08:21.805 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-21 15:01:47.242080726 +0000 2024-05-21T15:08:21.805 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T15:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-05-21T15:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-05-21T15:08:21.869 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000235483 s, 2.2 MB/s 2024-05-21T15:08:21.870 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T15:08:21.927 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-21 15:08:07.529418898 +0000 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-21 15:01:47.579074719 +0000 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-21 15:01:47.579074719 +0000 2024-05-21T15:08:21.984 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-21 15:01:47.579074719 +0000 2024-05-21T15:08:21.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T15:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-05-21T15:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-05-21T15:08:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000206307 s, 2.5 MB/s 2024-05-21T15:08:22.048 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T15:08:22.105 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-05-21T15:08:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T15:08:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:22.160 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-21T15:08:22.160 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:22.160 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:22.160 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-05-21 15:08:07.529418898 +0000 2024-05-21T15:08:22.160 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-05-21 15:01:47.903068945 +0000 2024-05-21T15:08:22.160 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-05-21 15:01:47.903068945 +0000 2024-05-21T15:08:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-05-21 15:01:47.903068945 +0000 2024-05-21T15:08:22.161 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T15:08:22.220 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:21 smithi084 ceph-mon[36334]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:22.221 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:21 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:22.221 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:21 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:22.221 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:21 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:22.223 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-05-21T15:08:22.223 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-05-21T15:08:22.223 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000197137 s, 2.6 MB/s 2024-05-21T15:08:22.224 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T15:08:22.281 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-05-21T15:08:22.281 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T15:08:22.298 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T15:08:22.299 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-05-21T15:08:22.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:21 smithi078 ceph-mon[31997]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:22.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:21 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:22.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:21 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:22.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:21 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:22.355 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T15:08:22.355 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:22.355 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 659 Links: 1 2024-05-21T15:08:22.355 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:22.355 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:22.355 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-05-21 15:07:58.391309064 +0000 2024-05-21T15:08:22.355 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-05-21 15:02:10.032429661 +0000 2024-05-21T15:08:22.355 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-05-21 15:02:10.032429661 +0000 2024-05-21T15:08:22.356 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-05-21 15:02:10.032429661 +0000 2024-05-21T15:08:22.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T15:08:22.417 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:21 smithi172 ceph-mon[36402]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:22.417 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:21 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:22.417 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:21 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:22.418 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:21 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:22.420 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-05-21T15:08:22.420 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-05-21T15:08:22.420 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000250351 s, 2.0 MB/s 2024-05-21T15:08:22.422 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T15:08:22.477 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-05-21T15:08:22.534 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T15:08:22.534 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:22.534 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 676 Links: 1 2024-05-21T15:08:22.534 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:22.534 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:22.534 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-05-21 15:07:58.391309064 +0000 2024-05-21T15:08:22.534 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-05-21 15:02:10.376421349 +0000 2024-05-21T15:08:22.534 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-05-21 15:02:10.376421349 +0000 2024-05-21T15:08:22.535 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-05-21 15:02:10.376421349 +0000 2024-05-21T15:08:22.535 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T15:08:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-05-21T15:08:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-05-21T15:08:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00024526 s, 2.1 MB/s 2024-05-21T15:08:22.598 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T15:08:22.654 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-05-21T15:08:22.709 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T15:08:22.709 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:22.710 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 695 Links: 1 2024-05-21T15:08:22.710 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:22.710 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:22.711 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-05-21 15:07:58.391309064 +0000 2024-05-21T15:08:22.711 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-05-21 15:02:10.704413423 +0000 2024-05-21T15:08:22.711 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-05-21 15:02:10.704413423 +0000 2024-05-21T15:08:22.711 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-05-21 15:02:10.704413423 +0000 2024-05-21T15:08:22.712 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T15:08:22.772 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-05-21T15:08:22.772 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-05-21T15:08:22.772 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000196128 s, 2.6 MB/s 2024-05-21T15:08:22.773 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T15:08:22.832 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-05-21T15:08:22.888 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T15:08:22.888 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T15:08:22.888 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 714 Links: 1 2024-05-21T15:08:22.888 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:08:22.888 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T15:08:22.889 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-05-21 15:07:58.391309064 +0000 2024-05-21T15:08:22.889 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-05-21 15:02:11.031405522 +0000 2024-05-21T15:08:22.889 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-05-21 15:02:11.031405522 +0000 2024-05-21T15:08:22.889 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-05-21 15:02:11.031405522 +0000 2024-05-21T15:08:22.889 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T15:08:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-05-21T15:08:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-05-21T15:08:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000262895 s, 1.9 MB/s 2024-05-21T15:08:22.953 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T15:08:23.008 INFO:tasks.cephadm:Deploying osd.0 on smithi078 with /dev/vg_nvme/lv_4... 2024-05-21T15:08:23.008 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T15:08:23.020 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:23.020 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:23.020 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:23.020 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:23.020 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:08:23.021 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:23.021 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:08:23.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:23.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:23.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:23.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:23.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:08:23.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:23.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:23.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:23.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:23.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:23.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:23.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:08:23.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:23.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:24.152 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:23 smithi172 ceph-mon[36402]: Deploying daemon mgr.z on smithi172 2024-05-21T15:08:24.152 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:23 smithi172 ceph-mon[36402]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:24.315 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:23 smithi078 ceph-mon[31997]: Deploying daemon mgr.z on smithi172 2024-05-21T15:08:24.315 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:23 smithi078 ceph-mon[31997]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:24.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:23 smithi084 ceph-mon[36334]: Deploying daemon mgr.z on smithi172 2024-05-21T15:08:24.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:23 smithi084 ceph-mon[36334]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:24.434 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 systemd[1]: Starting Ceph mgr.z for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:08:24.807 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 podman[38240]: 2024-05-21 15:08:24.443974758 +0000 UTC m=+0.017305745 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:08:24.807 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 podman[38240]: 2024-05-21 15:08:24.548412342 +0000 UTC m=+0.121743307 container create 3e525d029db1c8e458786783b28cc7fb1e53304ecd736d620c5b575d4e9ce51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-21T15:08:24.807 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 podman[38240]: 2024-05-21 15:08:24.767197238 +0000 UTC m=+0.340528206 container init 3e525d029db1c8e458786783b28cc7fb1e53304ecd736d620c5b575d4e9ce51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:08:24.807 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 podman[38240]: 2024-05-21 15:08:24.769822737 +0000 UTC m=+0.343153711 container start 3e525d029db1c8e458786783b28cc7fb1e53304ecd736d620c5b575d4e9ce51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-21T15:08:25.030 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T15:08:25.030 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T15:08:25.030 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-05-21T15:08:25.030 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-05-21T15:08:25.030 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115483 s, 908 MB/s 2024-05-21T15:08:25.030 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-05-21T15:08:25.122 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 ceph-mgr[38259]: -- 172.21.15.172:0/2131591382 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e948de05a0 con 0x55e948f1e480 2024-05-21T15:08:25.123 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 bash[38240]: 3e525d029db1c8e458786783b28cc7fb1e53304ecd736d620c5b575d4e9ce51a 2024-05-21T15:08:25.123 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 systemd[1]: Started Ceph mgr.z for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:08:25.123 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:24 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:24.934+0000 7f413b50e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:08:25.123 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:25 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:25.119+0000 7f413b50e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:08:25.433 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:25 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:25.221+0000 7f413b50e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:08:25.650 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-05-21T15:08:25.884 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:08:25.908 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:25 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:25.532+0000 7f413b50e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:08:25.908 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:25 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:25.616+0000 7f413b50e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: Deploying daemon mgr.y on smithi084 2024-05-21T15:08:26.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:25 smithi084 ceph-mon[36334]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:26.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:26.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:08:26.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:26.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:26.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: Deploying daemon mgr.y on smithi084 2024-05-21T15:08:26.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:25 smithi172 ceph-mon[36402]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: Deploying daemon mgr.y on smithi084 2024-05-21T15:08:26.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:25 smithi078 ceph-mon[31997]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:27.166 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:26 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:26.822+0000 7f413b50e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:08:27.166 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:26 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:26.991+0000 7f413b50e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:08:27.166 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:27.162+0000 7f413b50e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:08:27.288 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 systemd[1]: Starting Ceph mgr.y for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 -- 172.21.15.78:0/2585541060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac1020d0 msgr2=0x7f9dac1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/2585541060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac1020d0 0x7f9dac1024e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009990 tx=0x7f9d9c009ca0 comp rx=0 tx=0).stop 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 -- 172.21.15.78:0/2585541060 shutdown_connections 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/2585541060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac103ba0 0x7f9dac10a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/2585541060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac1020d0 0x7f9dac1024e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/2585541060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dac103200 0x7f9dac103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 -- 172.21.15.78:0/2585541060 >> 172.21.15.78:0/2585541060 conn(0x7f9dac0fd710 msgr2=0x7f9dac0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 -- 172.21.15.78:0/2585541060 shutdown_connections 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.502+0000 7f9db0e0c700 1 -- 172.21.15.78:0/2585541060 wait complete. 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9db0e0c700 1 Processor -- start 2024-05-21T15:08:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9db0e0c700 1 -- start start 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9db0e0c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dac1020d0 0x7f9dac105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9db0e0c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac103200 0x7f9dac104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9db0e0c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac103ba0 0x7f9dac104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9db0e0c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac10dd00 con 0x7f9dac103ba0 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9db0e0c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac10db80 con 0x7f9dac1020d0 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9db0e0c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac10de80 con 0x7f9dac103200 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9daad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac103ba0 0x7f9dac104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9daa59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dac1020d0 0x7f9dac105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9daad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac103ba0 0x7f9dac104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47640/0 (socket says 172.21.15.78:47640) 2024-05-21T15:08:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9daad9d700 1 -- 172.21.15.78:0/629621463 learned_addr learned my addr 172.21.15.78:0/629621463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.503+0000 7f9da3fff700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac103200 0x7f9dac104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da3fff700 1 -- 172.21.15.78:0/629621463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dac1020d0 msgr2=0x7f9dac105e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da3fff700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dac1020d0 0x7f9dac105e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da3fff700 1 -- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac103ba0 msgr2=0x7f9dac104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da3fff700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac103ba0 0x7f9dac104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da3fff700 1 -- 172.21.15.78:0/629621463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac1051e0 con 0x7f9dac103200 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9daa59c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dac1020d0 0x7f9dac105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9daad9d700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac103ba0 0x7f9dac104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da3fff700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac103200 0x7f9dac104540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b450 tx=0x7f9d9400b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94015020 con 0x7f9dac103200 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac1a39e0 con 0x7f9dac103200 2024-05-21T15:08:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d94011d10 con 0x7f9dac103200 2024-05-21T15:08:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94010d20 con 0x7f9dac103200 2024-05-21T15:08:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.504+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1a3f10 con 0x7f9dac103200 2024-05-21T15:08:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.505+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac050170 con 0x7f9dac103200 2024-05-21T15:08:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.505+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9d94018020 con 0x7f9dac103200 2024-05-21T15:08:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.505+0000 7f9da37fe700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9d98039330 0x7f9d9803b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.506+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9d9404b8d0 con 0x7f9dac103200 2024-05-21T15:08:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.506+0000 7f9daa59c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9d98039330 0x7f9d9803b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.506+0000 7f9daa59c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9d98039330 0x7f9d9803b840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0098e0 tx=0x7f9d9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.512+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d9401f020 con 0x7f9dac103200 2024-05-21T15:08:27.550 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 podman[37702]: 2024-05-21 15:08:27.187828526 +0000 UTC m=+0.016892710 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:08:27.550 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 podman[37702]: 2024-05-21 15:08:27.309450074 +0000 UTC m=+0.138514298 container create 472ee36adf996ddabcb20e79f69dd35086a3d5d0128e5c834c94862d177830b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-21T15:08:27.550 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 podman[37702]: 2024-05-21 15:08:27.511590725 +0000 UTC m=+0.340654900 container init 472ee36adf996ddabcb20e79f69dd35086a3d5d0128e5c834c94862d177830b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-21T15:08:27.550 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 podman[37702]: 2024-05-21 15:08:27.514109327 +0000 UTC m=+0.343173511 container start 472ee36adf996ddabcb20e79f69dd35086a3d5d0128e5c834c94862d177830b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-21T15:08:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:27.721+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9dac0ff6d0 con 0x7f9d98039330 2024-05-21T15:08:27.751 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:27.450+0000 7f413b50e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:08:27.751 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:27.542+0000 7f413b50e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:08:27.751 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:27.747+0000 7f413b50e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:08:27.871 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 ceph-mgr[37721]: -- 172.21.15.84:0/1027480407 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd261505a0 con 0x55fd2628e480 2024-05-21T15:08:27.871 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 bash[37702]: 472ee36adf996ddabcb20e79f69dd35086a3d5d0128e5c834c94862d177830b1 2024-05-21T15:08:27.871 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 systemd[1]: Started Ceph mgr.y for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:08:27.871 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:27.677+0000 7f6c6f2ac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:08:27.871 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:27.868+0000 7f6c6f2ac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:08:28.146 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:27 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:27.976+0000 7f6c6f2ac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:08:28.183 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:27.947+0000 7f413b50e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:08:28.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:28.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:08:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:08:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:28.575 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:28.576 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.576 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.576 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.576 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.576 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:08:28.576 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:08:28.576 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:28.576 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:28.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:28.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:28.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:08:28.595 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:08:28.596 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:28.596 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:28.596 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:28 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:28.303+0000 7f6c6f2ac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:08:28.596 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:28 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:28.412+0000 7f6c6f2ac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:08:28.933 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:28 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:28.584+0000 7f413b50e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:08:29.331 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:29 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:29.024+0000 7f413b50e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:08:29.331 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:29 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:29.125+0000 7f413b50e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:08:29.331 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:29 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:29.327+0000 7f413b50e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:08:29.331 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:29 smithi172 ceph-mon[36402]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:29.331 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.331 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.332 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.332 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.332 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:29.332 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:29.332 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:29 smithi078 ceph-mon[31997]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:29.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:29.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:29.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:29 smithi084 ceph-mon[36334]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:29.989 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:29 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:29.650+0000 7f6c6f2ac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:08:29.989 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:29 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:29.818+0000 7f6c6f2ac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:08:29.989 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:29 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:29.986+0000 7f6c6f2ac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:08:30.183 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:29 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:29.851+0000 7f413b50e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:08:30.184 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:29 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:29.943+0000 7f413b50e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:08:30.273 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:30 smithi084 ceph-mon[36334]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:30.274 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:30 smithi084 ceph-mon[36334]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T15:08:30.274 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:30.274 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:30.274 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:30.274 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:30 smithi084 ceph-mon[36334]: Reconfiguring daemon mgr.x on smithi078 2024-05-21T15:08:30.274 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:30.534 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:30 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:30.186+0000 7f413b50e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:08:30.534 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:30 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:30.322+0000 7f413b50e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:08:30.534 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:30 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:30.416+0000 7f413b50e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:08:30.534 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:30 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:30.530+0000 7f413b50e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:08:30.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:30 smithi172 ceph-mon[36402]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:30.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:30 smithi172 ceph-mon[36402]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T15:08:30.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:30.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:30.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:30.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:30 smithi172 ceph-mon[36402]: Reconfiguring daemon mgr.x on smithi078 2024-05-21T15:08:30.535 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:30.564 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:30 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:30.270+0000 7f6c6f2ac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:08:30.565 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:30 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:30.361+0000 7f6c6f2ac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:08:30.565 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:30 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:30.561+0000 7f6c6f2ac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:08:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:30 smithi078 ceph-mon[31997]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:30 smithi078 ceph-mon[31997]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T15:08:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T15:08:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T15:08:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:30 smithi078 ceph-mon[31997]: Reconfiguring daemon mgr.x on smithi078 2024-05-21T15:08:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:30.896 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:30 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:30.753+0000 7f6c6f2ac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:08:31.183 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:30 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:30.836+0000 7f413b50e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:08:31.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:31 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1636218972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3fb7e10-ca25-4585-becc-d4ff4784cb9b"}]: dispatch 2024-05-21T15:08:31.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:31 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1636218972' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3fb7e10-ca25-4585-becc-d4ff4784cb9b"}]': finished 2024-05-21T15:08:31.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:31 smithi078 ceph-mon[31997]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T15:08:31.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:31.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:31 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3132071732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:08:31.646 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:31 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:31.350+0000 7f6c6f2ac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:08:31.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:31 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1636218972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3fb7e10-ca25-4585-becc-d4ff4784cb9b"}]: dispatch 2024-05-21T15:08:31.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:31 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1636218972' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3fb7e10-ca25-4585-becc-d4ff4784cb9b"}]': finished 2024-05-21T15:08:31.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:31 smithi084 ceph-mon[36334]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T15:08:31.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:31 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:31.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:31 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3132071732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:08:31.655 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:31 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:31.438+0000 7f413b50e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:08:31.656 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:08:31 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:08:31.652+0000 7f413b50e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:08:31.656 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:31 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1636218972' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3fb7e10-ca25-4585-becc-d4ff4784cb9b"}]: dispatch 2024-05-21T15:08:31.656 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:31 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1636218972' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3fb7e10-ca25-4585-becc-d4ff4784cb9b"}]': finished 2024-05-21T15:08:31.656 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:31 smithi172 ceph-mon[36402]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T15:08:31.656 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:31 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:31.656 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:31 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3132071732' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:08:32.061 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:31 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:31.765+0000 7f6c6f2ac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:08:32.062 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:31 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:31.861+0000 7f6c6f2ac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:08:32.062 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:32 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:32.058+0000 7f6c6f2ac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:08:32.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:32.382+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f9d9400fa30 con 0x7f9dac103200 2024-05-21T15:08:32.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:32 smithi084 ceph-mon[36334]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:32.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:32 smithi084 ceph-mon[36334]: Standby manager daemon z started 2024-05-21T15:08:32.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:32 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:08:32.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:32 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:08:32.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:32 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:08:32.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:32 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:08:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:32 smithi078 ceph-mon[31997]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:32 smithi078 ceph-mon[31997]: Standby manager daemon z started 2024-05-21T15:08:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:32 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:08:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:32 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:08:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:32 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:08:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:32 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:08:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:32 smithi172 ceph-mon[36402]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:32 smithi172 ceph-mon[36402]: Standby manager daemon z started 2024-05-21T15:08:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:32 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:08:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:32 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:08:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:32 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:08:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:32 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.172:0/2470373348' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:08:32.896 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:32 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:32.574+0000 7f6c6f2ac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:08:32.896 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:32 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:32.664+0000 7f6c6f2ac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:08:33.245 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:32 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:32.904+0000 7f6c6f2ac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:08:33.246 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:33 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:33.036+0000 7f6c6f2ac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:08:33.246 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:33 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:33.129+0000 7f6c6f2ac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:08:33.246 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:33 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:33.241+0000 7f6c6f2ac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:08:33.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:33 smithi084 ceph-mon[36334]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T15:08:33.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:08:33.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:33 smithi084 ceph-mon[36334]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:33.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:33 smithi172 ceph-mon[36402]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T15:08:33.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:08:33.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:33 smithi172 ceph-mon[36402]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:33.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:33 smithi078 ceph-mon[31997]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-21T15:08:33.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:08:33.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:33 smithi078 ceph-mon[31997]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:33.896 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:33 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:33.539+0000 7f6c6f2ac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:08:34.391 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:34 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:34.135+0000 7f6c6f2ac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:08:34.391 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:08:34 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:08:34.346+0000 7f6c6f2ac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:08:34.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:34.398+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mon.1 v2:172.21.15.172:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f9d9405e7f0 con 0x7f9dac103200 2024-05-21T15:08:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:34 smithi084 ceph-mon[36334]: Standby manager daemon y started 2024-05-21T15:08:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:34 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:08:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:34 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:08:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:34 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:08:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:34 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:08:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:34 smithi172 ceph-mon[36402]: Standby manager daemon y started 2024-05-21T15:08:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:34 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:08:34.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:34 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:08:34.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:34 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:08:34.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:34 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:08:34.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:34 smithi078 ceph-mon[31997]: Standby manager daemon y started 2024-05-21T15:08:34.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:34 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:08:34.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:34 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:08:34.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:34 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:08:34.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:34 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.84:0/436262596' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:08:35.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:35 smithi078 ceph-mon[31997]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-21T15:08:35.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:08:35.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:35 smithi078 ceph-mon[31997]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:35.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:35 smithi172 ceph-mon[36402]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-21T15:08:35.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:08:35.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:35 smithi172 ceph-mon[36402]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:35 smithi084 ceph-mon[36334]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-21T15:08:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:08:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:35 smithi084 ceph-mon[36334]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:37 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:37 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:37 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:37 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:37 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:37 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:37.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:37 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:37.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:37 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:37.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:37 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:37.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:37 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:37.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:37 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:37.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:37 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:37.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:37 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:37.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:37 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:37.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:37 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:37.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:37 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:37.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:37 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:37.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:37 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:38.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:38 smithi078 ceph-mon[31997]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:38.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:38 smithi084 ceph-mon[36334]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:38 smithi172 ceph-mon[36402]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:39.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T15:08:39.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:39.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T15:08:39.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:39.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T15:08:39.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:40.579 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:40 smithi078 ceph-mon[31997]: Deploying daemon osd.0 on smithi078 2024-05-21T15:08:40.579 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:40 smithi078 ceph-mon[31997]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:40 smithi084 ceph-mon[36334]: Deploying daemon osd.0 on smithi078 2024-05-21T15:08:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:40 smithi084 ceph-mon[36334]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:40 smithi172 ceph-mon[36402]: Deploying daemon osd.0 on smithi078 2024-05-21T15:08:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:40 smithi172 ceph-mon[36402]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:41.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:41 smithi078 ceph-mon[31997]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:41 smithi084 ceph-mon[36334]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:41.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:41 smithi172 ceph-mon[36402]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:44.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:44 smithi078 ceph-mon[31997]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:44.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:44.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:44.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:44 smithi084 ceph-mon[36334]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:44 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:44 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:44 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:44 smithi172 ceph-mon[36402]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:44.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:44 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:44.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:44 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:44.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:44 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:46.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:46 smithi078 ceph-mon[31997]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:46 smithi084 ceph-mon[36334]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:46 smithi172 ceph-mon[36402]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:48.086 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 0 on host 'smithi078' 2024-05-21T15:08:48.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.081+0000 7f9da37fe700 1 -- 172.21.15.78:0/629621463 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9dac0ff6d0 con 0x7f9d98039330 2024-05-21T15:08:48.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9d98039330 msgr2=0x7f9d9803b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:48.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9d98039330 0x7f9d9803b840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0098e0 tx=0x7f9d9c01a040 comp rx=0 tx=0).stop 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac103200 msgr2=0x7f9dac104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac103200 0x7f9dac104540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b450 tx=0x7f9d9400b760 comp rx=0 tx=0).stop 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 shutdown_connections 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9d98039330 0x7f9d9803b840 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9dac103ba0 0x7f9dac104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dac1020d0 0x7f9dac105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 --2- 172.21.15.78:0/629621463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac103200 0x7f9dac104540 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 >> 172.21.15.78:0/629621463 conn(0x7f9dac0fd710 msgr2=0x7f9dac0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 shutdown_connections 2024-05-21T15:08:48.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:08:48.084+0000 7f9db0e0c700 1 -- 172.21.15.78:0/629621463 wait complete. 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:48.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:48.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:48.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T15:08:48.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:08:48.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:48.716 DEBUG:teuthology.orchestra.run.smithi078:osd.0> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.0.service 2024-05-21T15:08:48.718 INFO:tasks.cephadm:Deploying osd.1 on smithi084 with /dev/vg_nvme/lv_4... 2024-05-21T15:08:48.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T15:08:48.961 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:08:49.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:49 smithi078 ceph-mon[31997]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T15:08:49.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:49 smithi078 ceph-mon[31997]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T15:08:49.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:49 smithi078 ceph-mon[31997]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:08:49.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:49 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:49.348 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:08:49 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0[47099]: 2024-05-21T15:08:49.122+0000 7fdd72538700 -1 osd.0 0 waiting for initial osdmap 2024-05-21T15:08:49.348 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:08:49 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0[47099]: 2024-05-21T15:08:49.125+0000 7fdd6eaee700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:08:49.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:49 smithi084 ceph-mon[36334]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T15:08:49.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:49 smithi084 ceph-mon[36334]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T15:08:49.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:49 smithi084 ceph-mon[36334]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:08:49.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:49 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:49.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:49 smithi172 ceph-mon[36402]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T15:08:49.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:49 smithi172 ceph-mon[36402]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T15:08:49.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:49 smithi172 ceph-mon[36402]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:08:49.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:49 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:50.198 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:50 smithi084 ceph-mon[36334]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:08:50.199 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:50 smithi084 ceph-mon[36334]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T15:08:50.199 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:50.199 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:50.199 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:50 smithi084 ceph-mon[36334]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:50.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:50 smithi172 ceph-mon[36402]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:08:50.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:50 smithi172 ceph-mon[36402]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T15:08:50.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:50.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:50.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:50 smithi172 ceph-mon[36402]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:50 smithi078 ceph-mon[31997]: from='osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:08:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:50 smithi078 ceph-mon[31997]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T15:08:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:50 smithi078 ceph-mon[31997]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T15:08:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T15:08:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T15:08:50.750 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-05-21T15:08:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-05-21T15:08:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120287 s, 872 MB/s 2024-05-21T15:08:50.751 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-05-21T15:08:51.328 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:51 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:08:51.328 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:51 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:08:51.328 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:51 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:51.328 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:51 smithi078 ceph-mon[31997]: osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] boot 2024-05-21T15:08:51.328 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:51 smithi078 ceph-mon[31997]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T15:08:51.328 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:51 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:51.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_4 2024-05-21T15:08:51.361 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:51 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:08:51.361 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:51 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:08:51.361 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:51 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:51.361 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:51 smithi084 ceph-mon[36334]: osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] boot 2024-05-21T15:08:51.361 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:51 smithi084 ceph-mon[36334]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T15:08:51.361 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:51 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:51.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:51 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:08:51.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:51 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:08:51.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:51 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:51.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:51 smithi172 ceph-mon[36402]: osd.0 [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] boot 2024-05-21T15:08:51.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:51 smithi172 ceph-mon[36402]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T15:08:51.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:51 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:08:51.590 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:08:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T15:08:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:08:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:52.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T15:08:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:08:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T15:08:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:08:52.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:52.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:52.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:08:52.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:08:53.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 -- 172.21.15.84:0/3610968847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc103860 msgr2=0x7f61bc10b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:53.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 --2- 172.21.15.84:0/3610968847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc103860 0x7f61bc10b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009990 tx=0x7f61b4009ca0 comp rx=0 tx=0).stop 2024-05-21T15:08:53.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 -- 172.21.15.84:0/3610968847 shutdown_connections 2024-05-21T15:08:53.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 --2- 172.21.15.84:0/3610968847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc103860 0x7f61bc10b570 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 --2- 172.21.15.84:0/3610968847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61bc104830 0x7f61bc104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 --2- 172.21.15.84:0/3610968847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc102de0 0x7f61bc1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 -- 172.21.15.84:0/3610968847 >> 172.21.15.84:0/3610968847 conn(0x7f61bc0fe850 msgr2=0x7f61bc100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 -- 172.21.15.84:0/3610968847 shutdown_connections 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.165+0000 7f61c4150700 1 -- 172.21.15.84:0/3610968847 wait complete. 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c4150700 1 Processor -- start 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c4150700 1 -- start start 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c4150700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61bc102de0 0x7f61bc1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c4150700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc103860 0x7f61bc199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c4150700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc104830 0x7f61bc1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c4150700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc10eec0 con 0x7f61bc102de0 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c4150700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc10ed40 con 0x7f61bc104830 2024-05-21T15:08:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c4150700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc10f040 con 0x7f61bc103860 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c26ed700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc104830 0x7f61bc1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c26ed700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc104830 0x7f61bc1a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38306/0 (socket says 172.21.15.84:38306) 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c26ed700 1 -- 172.21.15.84:0/3210892075 learned_addr learned my addr 172.21.15.84:0/3210892075 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c16eb700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc103860 0x7f61bc199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c1eec700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61bc102de0 0x7f61bc1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c26ed700 1 -- 172.21.15.84:0/3210892075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc103860 msgr2=0x7f61bc199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c26ed700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc103860 0x7f61bc199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c26ed700 1 -- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61bc102de0 msgr2=0x7f61bc1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c26ed700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61bc102de0 0x7f61bc1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.166+0000 7f61c26ed700 1 -- 172.21.15.84:0/3210892075 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc1a1380 con 0x7f61bc104830 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.167+0000 7f61c26ed700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc104830 0x7f61bc1a0d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f61b400b2e0 tx=0x7f61b400bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.167+0000 7f61b2ffd700 1 -- 172.21.15.84:0/3210892075 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4026070 con 0x7f61bc104830 2024-05-21T15:08:53.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.167+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc1a1610 con 0x7f61bc104830 2024-05-21T15:08:53.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.167+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc1a1ac0 con 0x7f61bc104830 2024-05-21T15:08:53.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.167+0000 7f61b2ffd700 1 -- 172.21.15.84:0/3210892075 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b4003d70 con 0x7f61bc104830 2024-05-21T15:08:53.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.167+0000 7f61b2ffd700 1 -- 172.21.15.84:0/3210892075 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4021410 con 0x7f61bc104830 2024-05-21T15:08:53.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.168+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61bc050170 con 0x7f61bc104830 2024-05-21T15:08:53.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.169+0000 7f61b2ffd700 1 -- 172.21.15.84:0/3210892075 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f61b4029070 con 0x7f61bc104830 2024-05-21T15:08:53.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.170+0000 7f61b2ffd700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f61a80a2c10 0x7f61a80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:08:53.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.170+0000 7f61b2ffd700 1 -- 172.21.15.84:0/3210892075 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f61b40ce580 con 0x7f61bc104830 2024-05-21T15:08:53.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.170+0000 7f61c1eec700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f61a80a2c10 0x7f61a80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:08:53.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.171+0000 7f61c1eec700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f61a80a2c10 0x7f61a80a5120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61bc104220 tx=0x7f61b8007a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:08:53.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.174+0000 7f61b2ffd700 1 -- 172.21.15.84:0/3210892075 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f61b4013070 con 0x7f61bc104830 2024-05-21T15:08:53.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:08:53.375+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f61bc02df90 con 0x7f61a80a2c10 2024-05-21T15:08:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:53 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi078 2024-05-21T15:08:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:53 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi078 to 17088M 2024-05-21T15:08:53.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:53 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi078 2024-05-21T15:08:53.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:53 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi078 to 17088M 2024-05-21T15:08:53.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:53 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi078 2024-05-21T15:08:53.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:53 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi078 to 17088M 2024-05-21T15:08:54.266 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:54 smithi084 ceph-mon[36334]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:54.266 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:54 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:08:54.266 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:54 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:08:54.266 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:54 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:54.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:54 smithi172 ceph-mon[36402]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:54 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:08:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:54 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:08:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:54 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:54 smithi078 ceph-mon[31997]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:54 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:08:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:54 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:08:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:54 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:08:55.242 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:55 smithi084 ceph-mon[36334]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:55.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:55 smithi172 ceph-mon[36402]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:55.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:55 smithi078 ceph-mon[31997]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:08:56.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:56 smithi172 ceph-mon[36402]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:56.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:56 smithi172 ceph-mon[36402]: from='client.? 172.21.15.84:0/4069124514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]: dispatch 2024-05-21T15:08:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:56 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]: dispatch 2024-05-21T15:08:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:56 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]': finished 2024-05-21T15:08:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:56 smithi172 ceph-mon[36402]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T15:08:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:56 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:08:56.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:56 smithi078 ceph-mon[31997]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:56.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:56 smithi078 ceph-mon[31997]: from='client.? 172.21.15.84:0/4069124514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]: dispatch 2024-05-21T15:08:56.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:56 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]: dispatch 2024-05-21T15:08:56.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:56 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]': finished 2024-05-21T15:08:56.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:56 smithi078 ceph-mon[31997]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T15:08:56.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:56 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:08:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:56 smithi084 ceph-mon[36334]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:56 smithi084 ceph-mon[36334]: from='client.? 172.21.15.84:0/4069124514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]: dispatch 2024-05-21T15:08:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:56 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]: dispatch 2024-05-21T15:08:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:56 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "549cfd32-c874-4e14-9489-c7fb33b2ee68"}]': finished 2024-05-21T15:08:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:56 smithi084 ceph-mon[36334]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T15:08:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:56 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:08:57.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:57 smithi172 ceph-mon[36402]: from='client.? 172.21.15.84:0/950221784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:08:57.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:57 smithi078 ceph-mon[31997]: from='client.? 172.21.15.84:0/950221784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:08:57.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:57 smithi084 ceph-mon[36334]: from='client.? 172.21.15.84:0/950221784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:08:58.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:08:58 smithi078 ceph-mon[31997]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:08:58 smithi172 ceph-mon[36402]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:08:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:08:58 smithi084 ceph-mon[36334]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:00.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:00 smithi084 ceph-mon[36334]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:00.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:00 smithi078 ceph-mon[31997]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:00.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:00 smithi172 ceph-mon[36402]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:02.353 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:02 smithi084 ceph-mon[36334]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:02.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:02 smithi078 ceph-mon[31997]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:02.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:02 smithi172 ceph-mon[36402]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:04.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:04 smithi078 ceph-mon[31997]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:04.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T15:09:04.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:04.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:04 smithi084 ceph-mon[36334]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:04.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T15:09:04.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:04.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:04 smithi172 ceph-mon[36402]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:04.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T15:09:04.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:05.340 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:05 smithi084 ceph-mon[36334]: Deploying daemon osd.1 on smithi084 2024-05-21T15:09:05.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:05 smithi078 ceph-mon[31997]: Deploying daemon osd.1 on smithi084 2024-05-21T15:09:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:05 smithi172 ceph-mon[36402]: Deploying daemon osd.1 on smithi084 2024-05-21T15:09:06.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:06 smithi084 ceph-mon[36334]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:06.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:06 smithi078 ceph-mon[31997]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:06 smithi172 ceph-mon[36402]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:08.348 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:08 smithi084 ceph-mon[36334]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:08.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:08 smithi078 ceph-mon[31997]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:08.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:08 smithi172 ceph-mon[36402]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:09 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:09 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:09 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:09.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:09 smithi084 ceph-mon[36334]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:09.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:09 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:09.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:09 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:09.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:09 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:09.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:09 smithi078 ceph-mon[31997]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:09.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:09 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:09.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:09 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:09.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:09 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:09.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:09 smithi172 ceph-mon[36402]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:12.136 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.136 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.136 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:12.136 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:12.136 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.136 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:11 smithi084 ceph-mon[36334]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:11 smithi172 ceph-mon[36402]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:12.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.193+0000 7f61b2ffd700 1 -- 172.21.15.84:0/3210892075 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f61bc02df90 con 0x7f61a80a2c10 2024-05-21T15:09:12.196 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 1 on host 'smithi084' 2024-05-21T15:09:12.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f61a80a2c10 msgr2=0x7f61a80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:12.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f61a80a2c10 0x7f61a80a5120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61bc104220 tx=0x7f61b8007a30 comp rx=0 tx=0).stop 2024-05-21T15:09:12.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc104830 msgr2=0x7f61bc1a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc104830 0x7f61bc1a0d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f61b400b2e0 tx=0x7f61b400bbb0 comp rx=0 tx=0).stop 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 shutdown_connections 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f61a80a2c10 0x7f61a80a5120 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61bc104830 0x7f61bc1a0d90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61bc102de0 0x7f61bc1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 --2- 172.21.15.84:0/3210892075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc103860 0x7f61bc199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 >> 172.21.15.84:0/3210892075 conn(0x7f61bc0fe850 msgr2=0x7f61bc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 shutdown_connections 2024-05-21T15:09:12.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:09:12.195+0000 7f61c4150700 1 -- 172.21.15.84:0/3210892075 wait complete. 2024-05-21T15:09:12.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:12.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:12.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:12.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:11 smithi078 ceph-mon[31997]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:12.807 DEBUG:teuthology.orchestra.run.smithi084:osd.1> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.1.service 2024-05-21T15:09:12.849 INFO:tasks.cephadm:Deploying osd.2 on smithi172 with /dev/vg_nvme/lv_4... 2024-05-21T15:09:12.849 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T15:09:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:09:13.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:13 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:13.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:13 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:13.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:13 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:13.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:13 smithi084 ceph-mon[36334]: from='osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:09:13.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:13 smithi084 ceph-mon[36334]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:09:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:13 smithi172 ceph-mon[36402]: from='osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:09:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:13 smithi172 ceph-mon[36402]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:09:13.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:13 smithi078 ceph-mon[31997]: from='osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:09:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:13 smithi078 ceph-mon[31997]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T15:09:14.321 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:14 smithi172 ceph-mon[36402]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T15:09:14.322 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:14 smithi172 ceph-mon[36402]: from='osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:09:14.322 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:14 smithi172 ceph-mon[36402]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T15:09:14.322 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:14 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:14.322 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:14 smithi172 ceph-mon[36402]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:09:14.322 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:14 smithi172 ceph-mon[36402]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:14 smithi078 ceph-mon[31997]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T15:09:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:14 smithi078 ceph-mon[31997]: from='osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:09:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:14 smithi078 ceph-mon[31997]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T15:09:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:14 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:14 smithi078 ceph-mon[31997]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:09:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:14 smithi078 ceph-mon[31997]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:14 smithi084 ceph-mon[36334]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T15:09:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:14 smithi084 ceph-mon[36334]: from='osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:09:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:14 smithi084 ceph-mon[36334]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T15:09:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:14 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:14 smithi084 ceph-mon[36334]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:09:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:14 smithi084 ceph-mon[36334]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:14.647 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:14 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1[40938]: 2024-05-21T15:09:14.181+0000 7f309c000700 -1 osd.1 0 waiting for initial osdmap 2024-05-21T15:09:14.647 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:14 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1[40938]: 2024-05-21T15:09:14.184+0000 7f30985b6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:09:14.889 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T15:09:14.889 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T15:09:14.889 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-05-21T15:09:14.889 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-05-21T15:09:14.889 INFO:teuthology.orchestra.run.smithi172.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124169 s, 844 MB/s 2024-05-21T15:09:14.889 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 2024-05-21T15:09:14.889 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-05-21T15:09:15.342 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:15 smithi084 ceph-mon[36334]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:09:15.342 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:15 smithi084 ceph-mon[36334]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T15:09:15.343 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:15.343 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:15.448 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:15 smithi172 ceph-mon[36402]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:09:15.448 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:15 smithi172 ceph-mon[36402]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T15:09:15.448 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:15.449 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:15.475 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_4 2024-05-21T15:09:15.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:15 smithi078 ceph-mon[31997]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:09:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:15 smithi078 ceph-mon[31997]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T15:09:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:09:16.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] boot 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] boot 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:09:16.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:16.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:16.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:09:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:09:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: osd.1 [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] boot 2024-05-21T15:09:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T15:09:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:09:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-21T15:09:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:09:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:17.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 -- 172.21.15.172:0/166630954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104320 msgr2=0x7ff3a4104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:17.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 --2- 172.21.15.172:0/166630954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104320 0x7ff3a4104730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff394009990 tx=0x7ff394009ca0 comp rx=0 tx=0).stop 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 -- 172.21.15.172:0/166630954 shutdown_connections 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 --2- 172.21.15.172:0/166630954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3a4104cc0 0x7ff3a410b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 --2- 172.21.15.172:0/166630954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3a41031f0 0x7ff3a4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 --2- 172.21.15.172:0/166630954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104320 0x7ff3a4104730 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 -- 172.21.15.172:0/166630954 >> 172.21.15.172:0/166630954 conn(0x7ff3a40fe850 msgr2=0x7ff3a4100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 -- 172.21.15.172:0/166630954 shutdown_connections 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.303+0000 7ff3ac657700 1 -- 172.21.15.172:0/166630954 wait complete. 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3ac657700 1 Processor -- start 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3ac657700 1 -- start start 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3ac657700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3a41031f0 0x7ff3a4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3ac657700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3a4104320 0x7ff3a4199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3ac657700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104cc0 0x7ff3a41a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:09:17.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3ac657700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a410ee70 con 0x7ff3a41031f0 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3ac657700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a410ecf0 con 0x7ff3a4104320 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3ac657700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a410eff0 con 0x7ff3a4104cc0 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3aabf4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104cc0 0x7ff3a41a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3aabf4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104cc0 0x7ff3a41a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44718/0 (socket says 172.21.15.172:44718) 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.304+0000 7ff3aabf4700 1 -- 172.21.15.172:0/3937047975 learned_addr learned my addr 172.21.15.172:0/3937047975 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3a9bf2700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3a4104320 0x7ff3a4199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3aa3f3700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3a41031f0 0x7ff3a4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3aabf4700 1 -- 172.21.15.172:0/3937047975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3a4104320 msgr2=0x7ff3a4199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3aabf4700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3a4104320 0x7ff3a4199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3aabf4700 1 -- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3a41031f0 msgr2=0x7ff3a4199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:17.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3aabf4700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3a41031f0 0x7ff3a4199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3aabf4700 1 -- 172.21.15.172:0/3937047975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a41a1460 con 0x7ff3a4104cc0 2024-05-21T15:09:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3aabf4700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104cc0 0x7ff3a41a0de0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00b970 tx=0x7ff39c00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:09:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff39b7fe700 1 -- 172.21.15.172:0/3937047975 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c019070 con 0x7ff3a4104cc0 2024-05-21T15:09:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff39b7fe700 1 -- 172.21.15.172:0/3937047975 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff39c014440 con 0x7ff3a4104cc0 2024-05-21T15:09:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a41a1750 con 0x7ff3a4104cc0 2024-05-21T15:09:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff39b7fe700 1 -- 172.21.15.172:0/3937047975 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0075d0 con 0x7ff3a4104cc0 2024-05-21T15:09:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.305+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a41a1cc0 con 0x7ff3a4104cc0 2024-05-21T15:09:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.306+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a4050170 con 0x7ff3a4104cc0 2024-05-21T15:09:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.307+0000 7ff39b7fe700 1 -- 172.21.15.172:0/3937047975 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7ff39c01c070 con 0x7ff3a4104cc0 2024-05-21T15:09:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.308+0000 7ff39b7fe700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3900a2b90 0x7ff3900a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:09:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.309+0000 7ff39b7fe700 1 -- 172.21.15.172:0/3937047975 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7ff39c0cd190 con 0x7ff3a4104cc0 2024-05-21T15:09:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.309+0000 7ff3aa3f3700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3900a2b90 0x7ff3900a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:09:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.309+0000 7ff3aa3f3700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3900a2b90 0x7ff3900a50a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0005950 tx=0x7ff3a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:09:17.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.313+0000 7ff39b7fe700 1 -- 172.21.15.172:0/3937047975 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff39c09b8a0 con 0x7ff3a4104cc0 2024-05-21T15:09:17.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:17 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi084 2024-05-21T15:09:17.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:17 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi084 to 17088M 2024-05-21T15:09:17.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:17 smithi172 ceph-mon[36402]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T15:09:17.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:17.517+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff3a4062a80 con 0x7ff3900a2b90 2024-05-21T15:09:17.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:17 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi084 2024-05-21T15:09:17.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:17 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi084 to 17088M 2024-05-21T15:09:17.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:17 smithi078 ceph-mon[31997]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T15:09:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:17 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi084 2024-05-21T15:09:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:17 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi084 to 17088M 2024-05-21T15:09:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:17 smithi084 ceph-mon[36334]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T15:09:18.395 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:18 smithi172 ceph-mon[36402]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:18.395 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:18 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:09:18.395 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:18 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:09:18.395 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:18 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:18.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:18 smithi078 ceph-mon[31997]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:09:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:09:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:18 smithi084 ceph-mon[36334]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:18 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:09:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:18 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:09:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:18 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:19.284 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:19 smithi172 ceph-mon[36402]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:09:19.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:19 smithi078 ceph-mon[31997]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:09:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:19 smithi084 ceph-mon[36334]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:09:20.502 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:20 smithi172 ceph-mon[36402]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:20.502 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:20 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/1487857602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]: dispatch 2024-05-21T15:09:20.502 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:20 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]: dispatch 2024-05-21T15:09:20.502 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:20 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]': finished 2024-05-21T15:09:20.503 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:20 smithi172 ceph-mon[36402]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T15:09:20.503 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:20 smithi078 ceph-mon[31997]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:20 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/1487857602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]: dispatch 2024-05-21T15:09:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:20 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]: dispatch 2024-05-21T15:09:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:20 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]': finished 2024-05-21T15:09:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:20 smithi078 ceph-mon[31997]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T15:09:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:20 smithi084 ceph-mon[36334]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:20 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/1487857602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]: dispatch 2024-05-21T15:09:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:20 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]: dispatch 2024-05-21T15:09:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:20 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "233933e5-4dde-47f6-b007-015683591374"}]': finished 2024-05-21T15:09:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:20 smithi084 ceph-mon[36334]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T15:09:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:21 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/281296661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:09:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:21 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/281296661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:09:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:21 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/281296661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:09:22.585 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:22 smithi172 ceph-mon[36402]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:22.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:22 smithi078 ceph-mon[31997]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:22 smithi084 ceph-mon[36334]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:24.355 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:24 smithi172 ceph-mon[36402]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:24.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:24 smithi078 ceph-mon[31997]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:24 smithi084 ceph-mon[36334]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:26.352 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:26 smithi172 ceph-mon[36402]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:26.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:26 smithi078 ceph-mon[31997]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:26 smithi084 ceph-mon[36334]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:28.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:28 smithi078 ceph-mon[31997]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:28.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T15:09:28.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:28 smithi084 ceph-mon[36334]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T15:09:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:28 smithi172 ceph-mon[36402]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T15:09:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:29.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:29 smithi172 ceph-mon[36402]: Deploying daemon osd.2 on smithi172 2024-05-21T15:09:29.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:29 smithi078 ceph-mon[31997]: Deploying daemon osd.2 on smithi172 2024-05-21T15:09:29.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:29 smithi084 ceph-mon[36334]: Deploying daemon osd.2 on smithi172 2024-05-21T15:09:30.384 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:30 smithi172 ceph-mon[36402]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:30 smithi078 ceph-mon[31997]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:30 smithi084 ceph-mon[36334]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:32.430 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:32 smithi172 ceph-mon[36402]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:32.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:32 smithi078 ceph-mon[31997]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:32.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:32 smithi084 ceph-mon[36334]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:33.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:33.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:33.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:33.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:33 smithi084 ceph-mon[36334]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:33.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:33.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:33.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:33.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:33 smithi172 ceph-mon[36402]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:34.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:34.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:34.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:34.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:33 smithi078 ceph-mon[31997]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:35.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.833+0000 7ff39b7fe700 1 -- 172.21.15.172:0/3937047975 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff3a4062a80 con 0x7ff3900a2b90 2024-05-21T15:09:35.837 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 2 on host 'smithi172' 2024-05-21T15:09:35.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3900a2b90 msgr2=0x7ff3900a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:35.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3900a2b90 0x7ff3900a50a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0005950 tx=0x7ff3a00058e0 comp rx=0 tx=0).stop 2024-05-21T15:09:35.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104cc0 msgr2=0x7ff3a41a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:35.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104cc0 0x7ff3a41a0de0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00b970 tx=0x7ff39c00bd30 comp rx=0 tx=0).stop 2024-05-21T15:09:35.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 shutdown_connections 2024-05-21T15:09:35.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff3900a2b90 0x7ff3900a50a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:35.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3a4104cc0 0x7ff3a41a0de0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:35.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3a41031f0 0x7ff3a4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:35.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 --2- 172.21.15.172:0/3937047975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3a4104320 0x7ff3a4199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:35.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 >> 172.21.15.172:0/3937047975 conn(0x7ff3a40fe850 msgr2=0x7ff3a4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:09:35.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 shutdown_connections 2024-05-21T15:09:35.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:09:35.836+0000 7ff3ac657700 1 -- 172.21.15.172:0/3937047975 wait complete. 2024-05-21T15:09:36.387 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:36 smithi172 ceph-mon[36402]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:36.387 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:36.387 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:36.413 DEBUG:teuthology.orchestra.run.smithi172:osd.2> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.2.service 2024-05-21T15:09:36.415 INFO:tasks.cephadm:Deploying osd.3 on smithi078 with /dev/vg_nvme/lv_3... 2024-05-21T15:09:36.415 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T15:09:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:36 smithi078 ceph-mon[31997]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:36 smithi084 ceph-mon[36334]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:36.665 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:09:36.683 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:36 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2[41522]: 2024-05-21T15:09:36.603+0000 7fc2517aa7c0 -1 osd.2 0 log_to_monitors true 2024-05-21T15:09:37.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:37 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:37.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:37 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:37.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:37 smithi172 ceph-mon[36402]: from='osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:09:37.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:37 smithi172 ceph-mon[36402]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:09:37.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:37 smithi172 ceph-mon[36402]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:37.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:37 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:37.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:37 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:37.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:37 smithi078 ceph-mon[31997]: from='osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:09:37.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:37 smithi078 ceph-mon[31997]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:09:37.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:37 smithi078 ceph-mon[31997]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:37 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:37 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:37 smithi084 ceph-mon[36334]: from='osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:09:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:37 smithi084 ceph-mon[36334]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T15:09:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:37 smithi084 ceph-mon[36334]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T15:09:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T15:09:38.535 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-05-21T15:09:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-05-21T15:09:38.535 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114641 s, 915 MB/s 2024-05-21T15:09:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-05-21T15:09:38.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:38 smithi078 ceph-mon[31997]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T15:09:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:38 smithi078 ceph-mon[31997]: from='osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:09:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:38 smithi078 ceph-mon[31997]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T15:09:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:38 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:38 smithi078 ceph-mon[31997]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:09:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:38 smithi084 ceph-mon[36334]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T15:09:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:38 smithi084 ceph-mon[36334]: from='osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:09:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:38 smithi084 ceph-mon[36334]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T15:09:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:38 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:38 smithi084 ceph-mon[36334]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:09:38.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:38 smithi172 ceph-mon[36402]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T15:09:38.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:38 smithi172 ceph-mon[36402]: from='osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:09:38.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:38 smithi172 ceph-mon[36402]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T15:09:38.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:38 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:38.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:38 smithi172 ceph-mon[36402]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:09:39.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-05-21T15:09:39.369 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:09:39.509 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:39 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2[41522]: 2024-05-21T15:09:39.362+0000 7fc24660d700 -1 osd.2 0 waiting for initial osdmap 2024-05-21T15:09:39.509 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:39 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2[41522]: 2024-05-21T15:09:39.373+0000 7fc23ebbb700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:09:39.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:39 smithi078 ceph-mon[31997]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:09:39.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:39 smithi078 ceph-mon[31997]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T15:09:39.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:39.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:39.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:39 smithi078 ceph-mon[31997]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:39.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:39 smithi078 ceph-mon[31997]: from='osd.2 ' entity='osd.2' 2024-05-21T15:09:39.827 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:39.842 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:39 smithi172 ceph-mon[36402]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:09:39.842 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:39 smithi172 ceph-mon[36402]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T15:09:39.842 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:39.842 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:39.842 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:39 smithi172 ceph-mon[36402]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:39.842 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:39 smithi172 ceph-mon[36402]: from='osd.2 ' entity='osd.2' 2024-05-21T15:09:39.842 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:39.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:39 smithi084 ceph-mon[36334]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:09:39.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:39 smithi084 ceph-mon[36334]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T15:09:39.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:39.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:39.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:39 smithi084 ceph-mon[36334]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-21T15:09:39.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:39 smithi084 ceph-mon[36334]: from='osd.2 ' entity='osd.2' 2024-05-21T15:09:39.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:40.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:09:40.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:09:40.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi172 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi172 to 17088M 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] boot 2024-05-21T15:09:40.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T15:09:40.698 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:09:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:09:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi172 2024-05-21T15:09:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:09:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi172 to 17088M 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] boot 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T15:09:40.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 -- 172.21.15.78:0/21760758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350106630 msgr2=0x7fe350108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 --2- 172.21.15.78:0/21760758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350106630 0x7fe350108a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009990 tx=0x7fe34c009ca0 comp rx=0 tx=0).stop 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 -- 172.21.15.78:0/21760758 shutdown_connections 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 --2- 172.21.15.78:0/21760758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe350108f90 0x7fe35010b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 --2- 172.21.15.78:0/21760758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe350103cd0 0x7fe3501060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 --2- 172.21.15.78:0/21760758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350106630 0x7fe350108a50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 -- 172.21.15.78:0/21760758 >> 172.21.15.78:0/21760758 conn(0x7fe3500fd710 msgr2=0x7fe3500ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 -- 172.21.15.78:0/21760758 shutdown_connections 2024-05-21T15:09:40.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.920+0000 7fe357f7d700 1 -- 172.21.15.78:0/21760758 wait complete. 2024-05-21T15:09:40.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe357f7d700 1 Processor -- start 2024-05-21T15:09:40.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe357f7d700 1 -- start start 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe357f7d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe350103cd0 0x7fe350199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe357f7d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe350106630 0x7fe350199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe357f7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350108f90 0x7fe3501a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe357f7d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe35010ee70 con 0x7fe350108f90 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe357f7d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe35010ecf0 con 0x7fe350106630 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe357f7d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe35010eff0 con 0x7fe350103cd0 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe35651a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350108f90 0x7fe3501a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe35651a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350108f90 0x7fe3501a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60468/0 (socket says 172.21.15.78:60468) 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe35651a700 1 -- 172.21.15.78:0/3826325247 learned_addr learned my addr 172.21.15.78:0/3826325247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe355d19700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe350103cd0 0x7fe350199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe355518700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe350106630 0x7fe350199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:09:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe35651a700 1 -- 172.21.15.78:0/3826325247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe350103cd0 msgr2=0x7fe350199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe35651a700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe350103cd0 0x7fe350199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe35651a700 1 -- 172.21.15.78:0/3826325247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe350106630 msgr2=0x7fe350199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe35651a700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe350106630 0x7fe350199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.921+0000 7fe35651a700 1 -- 172.21.15.78:0/3826325247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3501a14e0 con 0x7fe350108f90 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.922+0000 7fe35651a700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350108f90 0x7fe3501a0e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe34800b970 tx=0x7fe34800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.922+0000 7fe342ffd700 1 -- 172.21.15.78:0/3826325247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34800d860 con 0x7fe350108f90 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.922+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3501a17d0 con 0x7fe350108f90 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.922+0000 7fe342ffd700 1 -- 172.21.15.78:0/3826325247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34800e430 con 0x7fe350108f90 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.922+0000 7fe342ffd700 1 -- 172.21.15.78:0/3826325247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348017660 con 0x7fe350108f90 2024-05-21T15:09:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.922+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3501a1d90 con 0x7fe350108f90 2024-05-21T15:09:40.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.923+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe350050170 con 0x7fe350108f90 2024-05-21T15:09:40.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.924+0000 7fe342ffd700 1 -- 172.21.15.78:0/3826325247 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fe34800e5e0 con 0x7fe350108f90 2024-05-21T15:09:40.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.925+0000 7fe342ffd700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe33c0a2b90 0x7fe33c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:09:40.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.925+0000 7fe355d19700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe33c0a2b90 0x7fe33c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:09:40.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.925+0000 7fe342ffd700 1 -- 172.21.15.78:0/3826325247 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fe3480cd570 con 0x7fe350108f90 2024-05-21T15:09:40.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.925+0000 7fe355d19700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe33c0a2b90 0x7fe33c0a50a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe350075290 tx=0x7fe344009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:09:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:40.929+0000 7fe342ffd700 1 -- 172.21.15.78:0/3826325247 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe348013030 con 0x7fe350108f90 2024-05-21T15:09:40.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi172 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi172 to 17088M 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:40.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:40.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:09:40.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:40.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: osd.2 [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] boot 2024-05-21T15:09:40.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T15:09:40.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:09:41.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:41.139+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe350062a80 con 0x7fe33c0a2b90 2024-05-21T15:09:41.793 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:41 smithi078 ceph-mon[31997]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:09:41.793 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:41 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:09:41.793 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:41 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:09:41.794 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:41 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:41.794 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:41 smithi078 ceph-mon[31997]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:41.794 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:41 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:09:41.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:41 smithi084 ceph-mon[36334]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:09:41.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:41 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:09:41.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:41 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:09:41.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:41 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:41.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:41 smithi084 ceph-mon[36334]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:41.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:41 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:09:41.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:41 smithi172 ceph-mon[36402]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:09:41.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:41 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:09:41.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:41 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:09:41.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:41 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:41.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:41 smithi172 ceph-mon[36402]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:41.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:41 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:09:42.574 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T15:09:42.574 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:42 smithi078 ceph-mon[31997]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T15:09:42.574 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:09:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T15:09:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:42 smithi084 ceph-mon[36334]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T15:09:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:09:42.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T15:09:42.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:42 smithi172 ceph-mon[36402]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T15:09:42.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:09:43.134 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:09:42 smithi078 sudo[51484]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:09:43.134 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:09:42 smithi078 sudo[51484]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:43.134 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:09:42 smithi078 sudo[51484]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:43.134 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:09:42 smithi078 sudo[51484]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:43.523 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:09:43 smithi078 sudo[51489]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:09:43.523 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:09:43 smithi078 sudo[51489]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:43.523 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:09:43 smithi078 sudo[51489]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:43.523 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:09:43 smithi078 sudo[51489]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:43.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:09:43.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:43 smithi084 ceph-mon[36334]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T15:09:43.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:43.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:43 smithi084 ceph-mon[36334]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:43.657 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:43 smithi084 sudo[45074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:09:43.657 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:43 smithi084 sudo[45074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:43.657 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:43 smithi084 sudo[45074]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:43.657 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:43 smithi084 sudo[45074]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:43.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:09:43.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:43 smithi078 ceph-mon[31997]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T15:09:43.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:43.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:43 smithi078 ceph-mon[31997]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:43.925 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:09:43.926 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:43 smithi172 ceph-mon[36402]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T15:09:43.926 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:43.926 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:43 smithi172 ceph-mon[36402]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:44.146 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:43 smithi084 sudo[45077]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:09:44.146 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:43 smithi084 sudo[45077]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:44.146 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:43 smithi084 sudo[45077]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:44.146 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:09:43 smithi084 sudo[45077]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:44.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:09:44.172+0000 7fe342ffd700 1 -- 172.21.15.78:0/3826325247 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe348097a70 con 0x7fe350108f90 2024-05-21T15:09:44.181 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:43 smithi172 sudo[45170]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:09:44.181 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:43 smithi172 sudo[45170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:44.181 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:43 smithi172 sudo[45170]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:44.181 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:43 smithi172 sudo[45170]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:44.433 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:44 smithi172 sudo[45173]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:09:44.433 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:44 smithi172 sudo[45173]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:44.433 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:44 smithi172 sudo[45173]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:44.433 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:09:44 smithi172 sudo[45173]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:44.706 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 sudo[51570]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:09:44.706 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 sudo[51570]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:44.706 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 sudo[51570]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:44.706 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 sudo[51570]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 sudo[51574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 sudo[51574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 sudo[51574]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 sudo[51574]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3574936994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]: dispatch 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]: dispatch 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]': finished 2024-05-21T15:09:45.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T15:09:45.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:09:45.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T15:09:45.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:45.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:44 smithi078 ceph-mon[31997]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:45.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3574936994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]: dispatch 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]: dispatch 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]': finished 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:45.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:44 smithi084 ceph-mon[36334]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:45.199 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:44 smithi172 sudo[45176]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:09:45.199 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:44 smithi172 sudo[45176]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:45.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:44 smithi172 sudo[45176]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:45.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:44 smithi172 sudo[45176]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:45.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 sudo[45179]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:09:45.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 sudo[45179]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:45.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 sudo[45179]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:45.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 sudo[45179]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T15:09:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3574936994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]: dispatch 2024-05-21T15:09:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]: dispatch 2024-05-21T15:09:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bb0ba8c-de4c-4334-a4cf-dda60352271a"}]': finished 2024-05-21T15:09:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T15:09:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:09:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T15:09:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:45.695 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 sudo[45080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T15:09:45.696 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 sudo[45080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:45.696 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 sudo[45080]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:45.696 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 sudo[45080]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:46.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:46.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2982965426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:09:46.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:46.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:45 smithi078 ceph-mon[31997]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:46.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 sudo[45083]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T15:09:46.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 sudo[45083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T15:09:46.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 sudo[45083]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T15:09:46.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 sudo[45083]: pam_unix(sudo:session): session closed for user root 2024-05-21T15:09:46.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:46.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2982965426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:09:46.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:45 smithi084 ceph-mon[36334]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:46.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:46.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2982965426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:09:46.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:45 smithi172 ceph-mon[36402]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T15:09:48.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:48 smithi078 ceph-mon[31997]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:48 smithi084 ceph-mon[36334]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:48 smithi172 ceph-mon[36402]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:50.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:50 smithi078 ceph-mon[31997]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:50 smithi084 ceph-mon[36334]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:50.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:50 smithi172 ceph-mon[36402]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:09:52.503 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:52 smithi172 ceph-mon[36402]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-05-21T15:09:52.504 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T15:09:52.504 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:52.535 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:52 smithi078 ceph-mon[31997]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-05-21T15:09:52.535 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T15:09:52.535 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:52 smithi084 ceph-mon[36334]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-05-21T15:09:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T15:09:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:09:53.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:53 smithi078 ceph-mon[31997]: Deploying daemon osd.3 on smithi078 2024-05-21T15:09:53.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:53 smithi084 ceph-mon[36334]: Deploying daemon osd.3 on smithi078 2024-05-21T15:09:53.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:53 smithi172 ceph-mon[36402]: Deploying daemon osd.3 on smithi078 2024-05-21T15:09:54.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:54 smithi078 ceph-mon[31997]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-21T15:09:54.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:54 smithi084 ceph-mon[36334]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-21T15:09:54.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:54 smithi172 ceph-mon[36402]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-05-21T15:09:56.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:56 smithi078 ceph-mon[31997]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-21T15:09:56.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:56 smithi084 ceph-mon[36334]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-21T15:09:56.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:56 smithi172 ceph-mon[36402]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-05-21T15:09:58.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:58 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:58 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:58 smithi078 ceph-mon[31997]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-21T15:09:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:09:58 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:58 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:58 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:58 smithi172 ceph-mon[36402]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-21T15:09:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:09:58 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:09:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:58 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:58 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:09:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:58 smithi084 ceph-mon[36334]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-21T15:09:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:09:58 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:00 smithi078 ceph-mon[31997]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-05-21T15:10:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:00 smithi078 ceph-mon[31997]: overall HEALTH_OK 2024-05-21T15:10:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:00 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:00 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:00 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:00 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:00 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:00 smithi084 ceph-mon[36334]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-05-21T15:10:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:00 smithi084 ceph-mon[36334]: overall HEALTH_OK 2024-05-21T15:10:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:00 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:00 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:00 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:00 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:00 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:00.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:00 smithi172 ceph-mon[36402]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-05-21T15:10:00.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:00 smithi172 ceph-mon[36402]: overall HEALTH_OK 2024-05-21T15:10:00.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:00 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:00.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:00 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:00.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:00 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:00.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:00 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:00.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:00 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:01.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.044+0000 7fe342ffd700 1 -- 172.21.15.78:0/3826325247 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe350062a80 con 0x7fe33c0a2b90 2024-05-21T15:10:01.048 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 3 on host 'smithi078' 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe33c0a2b90 msgr2=0x7fe33c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe33c0a2b90 0x7fe33c0a50a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe350075290 tx=0x7fe344009190 comp rx=0 tx=0).stop 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350108f90 msgr2=0x7fe3501a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350108f90 0x7fe3501a0e60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe34800b970 tx=0x7fe34800bc80 comp rx=0 tx=0).stop 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 shutdown_connections 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe33c0a2b90 0x7fe33c0a50a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe350108f90 0x7fe3501a0e60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe350103cd0 0x7fe350199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:01.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 --2- 172.21.15.78:0/3826325247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe350106630 0x7fe350199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:01.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 >> 172.21.15.78:0/3826325247 conn(0x7fe3500fd710 msgr2=0x7fe350104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:10:01.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 shutdown_connections 2024-05-21T15:10:01.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:01.047+0000 7fe357f7d700 1 -- 172.21.15.78:0/3826325247 wait complete. 2024-05-21T15:10:01.684 DEBUG:teuthology.orchestra.run.smithi078:osd.3> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.3.service 2024-05-21T15:10:01.687 INFO:tasks.cephadm:Deploying osd.4 on smithi084 with /dev/vg_nvme/lv_3... 2024-05-21T15:10:01.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T15:10:01.934 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:10:02.290 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:01 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:02.290 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:01 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:02.291 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:01 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:02.291 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:01 smithi084 ceph-mon[36334]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T15:10:02.291 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:01 smithi084 ceph-mon[36334]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-05-21T15:10:02.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:01 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:02.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:01 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:02.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:01 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:02.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:01 smithi078 ceph-mon[31997]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T15:10:02.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:01 smithi078 ceph-mon[31997]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-05-21T15:10:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:01 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:01 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:01 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:02.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:01 smithi172 ceph-mon[36402]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T15:10:02.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:01 smithi172 ceph-mon[36402]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-05-21T15:10:03.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:03 smithi078 ceph-mon[31997]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T15:10:03.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:03 smithi078 ceph-mon[31997]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T15:10:03.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:03 smithi078 ceph-mon[31997]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:10:03.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:03.098 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:10:03 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:10:03.014+0000 7fee0d900700 -1 osd.3 0 waiting for initial osdmap 2024-05-21T15:10:03.099 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:10:03 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:10:03.020+0000 7fee09eb6700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:10:03.264 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:03 smithi084 ceph-mon[36334]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T15:10:03.264 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:03 smithi084 ceph-mon[36334]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T15:10:03.264 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:03 smithi084 ceph-mon[36334]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:10:03.265 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:03 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:03.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:03 smithi172 ceph-mon[36402]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T15:10:03.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:03 smithi172 ceph-mon[36402]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T15:10:03.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:03 smithi172 ceph-mon[36402]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:10:03.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:03 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:03.807 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T15:10:03.807 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T15:10:03.807 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-05-21T15:10:03.807 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-05-21T15:10:03.807 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011448 s, 916 MB/s 2024-05-21T15:10:03.807 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 2024-05-21T15:10:03.808 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-05-21T15:10:04.263 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:04 smithi078 ceph-mon[31997]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:10:04.263 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:04 smithi078 ceph-mon[31997]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T15:10:04.263 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:04.263 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:04.263 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:04 smithi078 ceph-mon[31997]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:10:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:04 smithi084 ceph-mon[36334]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:10:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:04 smithi084 ceph-mon[36334]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T15:10:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:04 smithi084 ceph-mon[36334]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:10:04.427 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_3 2024-05-21T15:10:04.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:04 smithi172 ceph-mon[36402]: from='osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:10:04.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:04 smithi172 ceph-mon[36402]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T15:10:04.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:04.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:04.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:04 smithi172 ceph-mon[36402]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-05-21T15:10:04.661 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:10:05.103 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] boot 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: osdmap e25: 4 total, 4 up, 4 in 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.104 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:05.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:05.105 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:10:05.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:10:05.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:05.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] boot 2024-05-21T15:10:05.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: osdmap e25: 4 total, 4 up, 4 in 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:05.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:10:05.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:10:05.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: osd.3 [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] boot 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: osdmap e25: 4 total, 4 up, 4 in 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:06.258 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 -- 172.21.15.84:0/672916969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6648104780 msgr2=0x7f6648104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:06.258 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 --2- 172.21.15.84:0/672916969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6648104780 0x7f6648104b90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6638009990 tx=0x7f6638009ca0 comp rx=0 tx=0).stop 2024-05-21T15:10:06.258 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 -- 172.21.15.84:0/672916969 shutdown_connections 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 --2- 172.21.15.84:0/672916969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6648101b20 0x7f664810b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 --2- 172.21.15.84:0/672916969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6648104780 0x7f6648104b90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 --2- 172.21.15.84:0/672916969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66481011d0 0x7f66481015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 -- 172.21.15.84:0/672916969 >> 172.21.15.84:0/672916969 conn(0x7f66480fcea0 msgr2=0x7f66480ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 -- 172.21.15.84:0/672916969 shutdown_connections 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.253+0000 7f664cbc6700 1 -- 172.21.15.84:0/672916969 wait complete. 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664cbc6700 1 Processor -- start 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664cbc6700 1 -- start start 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664cbc6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66481011d0 0x7f6648199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664cbc6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6648101b20 0x7f6648199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664cbc6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6648104780 0x7f66481a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:06.259 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664cbc6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f664810ed80 con 0x7f6648101b20 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664cbc6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f664810ec00 con 0x7f66481011d0 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664cbc6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f664810ef00 con 0x7f6648104780 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66481011d0 0x7f6648199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66481011d0 0x7f6648199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35892/0 (socket says 172.21.15.84:35892) 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664659c700 1 -- 172.21.15.84:0/3128306061 learned_addr learned my addr 172.21.15.84:0/3128306061 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f6645d9b700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6648101b20 0x7f6648199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f6646d9d700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6648104780 0x7f66481a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664659c700 1 -- 172.21.15.84:0/3128306061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6648104780 msgr2=0x7f66481a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664659c700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6648104780 0x7f66481a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664659c700 1 -- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6648101b20 msgr2=0x7f6648199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664659c700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6648101b20 0x7f6648199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:06.260 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.254+0000 7f664659c700 1 -- 172.21.15.84:0/3128306061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66481a13a0 con 0x7f66481011d0 2024-05-21T15:10:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.255+0000 7f664659c700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66481011d0 0x7f6648199840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6638006010 tx=0x7f663800bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:10:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.255+0000 7f66377fe700 1 -- 172.21.15.84:0/3128306061 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638026070 con 0x7f66481011d0 2024-05-21T15:10:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.255+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66481a1630 con 0x7f66481011d0 2024-05-21T15:10:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.255+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66481a1ae0 con 0x7f66481011d0 2024-05-21T15:10:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.255+0000 7f66377fe700 1 -- 172.21.15.84:0/3128306061 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663800f460 con 0x7f66481011d0 2024-05-21T15:10:06.261 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.255+0000 7f66377fe700 1 -- 172.21.15.84:0/3128306061 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638021650 con 0x7f66481011d0 2024-05-21T15:10:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.256+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6648050170 con 0x7f66481011d0 2024-05-21T15:10:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.257+0000 7f66377fe700 1 -- 172.21.15.84:0/3128306061 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6638029070 con 0x7f66481011d0 2024-05-21T15:10:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.258+0000 7f66377fe700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f66300a2c20 0x7f66300a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.258+0000 7f66377fe700 1 -- 172.21.15.84:0/3128306061 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f66380cf440 con 0x7f66481011d0 2024-05-21T15:10:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.258+0000 7f6645d9b700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f66300a2c20 0x7f66300a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:06.262 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.258+0000 7f6645d9b700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f66300a2c20 0x7f66300a5130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f664819a8f0 tx=0x7f663c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:10:06.265 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.262+0000 7f66377fe700 1 -- 172.21.15.84:0/3128306061 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6638013070 con 0x7f66481011d0 2024-05-21T15:10:06.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:06 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi078 2024-05-21T15:10:06.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:06 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi078 to 8544M 2024-05-21T15:10:06.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:06 smithi078 ceph-mon[31997]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T15:10:06.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:06 smithi078 ceph-mon[31997]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:06.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:06 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi078 2024-05-21T15:10:06.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:06 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi078 to 8544M 2024-05-21T15:10:06.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:06 smithi084 ceph-mon[36334]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T15:10:06.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:06 smithi084 ceph-mon[36334]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:06.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:06 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi078 2024-05-21T15:10:06.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:06 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi078 to 8544M 2024-05-21T15:10:06.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:06 smithi172 ceph-mon[36402]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T15:10:06.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:06 smithi172 ceph-mon[36402]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:06.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:06.466+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6648062a80 con 0x7f66300a2c20 2024-05-21T15:10:07.295 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:07 smithi084 ceph-mon[36334]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T15:10:07.295 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:07.296 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:07.296 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:07.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:07 smithi078 ceph-mon[31997]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T15:10:07.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:07.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:07.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:07.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:07 smithi172 ceph-mon[36402]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T15:10:07.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:07.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:07.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:08.240 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:08 smithi084 ceph-mon[36334]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:08.240 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:08 smithi084 ceph-mon[36334]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T15:10:08.240 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:08 smithi084 ceph-mon[36334]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:08.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:08 smithi078 ceph-mon[31997]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:08.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:08 smithi078 ceph-mon[31997]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T15:10:08.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:08 smithi078 ceph-mon[31997]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:08 smithi172 ceph-mon[36402]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:08 smithi172 ceph-mon[36402]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T15:10:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:08 smithi172 ceph-mon[36402]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:09.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:09 smithi078 ceph-mon[31997]: osdmap e29: 4 total, 4 up, 4 in 2024-05-21T15:10:09.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:09 smithi084 ceph-mon[36334]: osdmap e29: 4 total, 4 up, 4 in 2024-05-21T15:10:09.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:09 smithi172 ceph-mon[36402]: osdmap e29: 4 total, 4 up, 4 in 2024-05-21T15:10:10.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:10 smithi078 ceph-mon[31997]: from='client.? 172.21.15.84:0/2836856663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]: dispatch 2024-05-21T15:10:10.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:10 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]: dispatch 2024-05-21T15:10:10.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:10 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]': finished 2024-05-21T15:10:10.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:10 smithi078 ceph-mon[31997]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T15:10:10.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:10.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:10 smithi078 ceph-mon[31997]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:10.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:10 smithi078 ceph-mon[31997]: from='client.? 172.21.15.84:0/3364512420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:10:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:10 smithi084 ceph-mon[36334]: from='client.? 172.21.15.84:0/2836856663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]: dispatch 2024-05-21T15:10:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:10 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]: dispatch 2024-05-21T15:10:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:10 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]': finished 2024-05-21T15:10:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:10 smithi084 ceph-mon[36334]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T15:10:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:10 smithi084 ceph-mon[36334]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:10 smithi084 ceph-mon[36334]: from='client.? 172.21.15.84:0/3364512420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:10:10.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:10 smithi172 ceph-mon[36402]: from='client.? 172.21.15.84:0/2836856663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]: dispatch 2024-05-21T15:10:10.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:10 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]: dispatch 2024-05-21T15:10:10.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:10 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "727cecb4-86c9-438c-9c09-7468e39aad27"}]': finished 2024-05-21T15:10:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:10 smithi172 ceph-mon[36402]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T15:10:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:10 smithi172 ceph-mon[36402]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:10 smithi172 ceph-mon[36402]: from='client.? 172.21.15.84:0/3364512420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:10:12.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:12 smithi078 ceph-mon[31997]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-05-21T15:10:12.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:12 smithi084 ceph-mon[36334]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-05-21T15:10:12.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:12 smithi172 ceph-mon[36402]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-05-21T15:10:14.584 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:14 smithi084 ceph-mon[36334]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-21T15:10:14.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:14 smithi078 ceph-mon[31997]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-21T15:10:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:14 smithi172 ceph-mon[36402]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-21T15:10:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:16 smithi078 ceph-mon[31997]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:10:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:16 smithi084 ceph-mon[36334]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:10:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:16 smithi172 ceph-mon[36402]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:10:17.371 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:17 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T15:10:17.371 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:17 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:17.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:17 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T15:10:17.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:17 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:17 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T15:10:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:17 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:18.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:18 smithi078 ceph-mon[31997]: Deploying daemon osd.4 on smithi084 2024-05-21T15:10:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:18 smithi078 ceph-mon[31997]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-21T15:10:18.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:18 smithi084 ceph-mon[36334]: Deploying daemon osd.4 on smithi084 2024-05-21T15:10:18.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:18 smithi084 ceph-mon[36334]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-21T15:10:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:18 smithi172 ceph-mon[36402]: Deploying daemon osd.4 on smithi084 2024-05-21T15:10:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:18 smithi172 ceph-mon[36402]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-21T15:10:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:20 smithi078 ceph-mon[31997]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-21T15:10:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:20 smithi084 ceph-mon[36334]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-21T15:10:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:20 smithi172 ceph-mon[36402]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-21T15:10:22.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:22 smithi078 ceph-mon[31997]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T15:10:22.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:22.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:22.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:22.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:22 smithi084 ceph-mon[36334]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T15:10:22.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:22.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:22.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:22 smithi172 ceph-mon[36402]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T15:10:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:24.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:24 smithi078 ceph-mon[31997]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:24.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:24 smithi084 ceph-mon[36334]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:24 smithi172 ceph-mon[36402]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:25.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.528+0000 7f66377fe700 1 -- 172.21.15.84:0/3128306061 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6648062a80 con 0x7f66300a2c20 2024-05-21T15:10:25.531 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 4 on host 'smithi084' 2024-05-21T15:10:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f66300a2c20 msgr2=0x7f66300a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f66300a2c20 0x7f66300a5130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f664819a8f0 tx=0x7f663c00b410 comp rx=0 tx=0).stop 2024-05-21T15:10:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66481011d0 msgr2=0x7f6648199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66481011d0 0x7f6648199840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6638006010 tx=0x7f663800bbb0 comp rx=0 tx=0).stop 2024-05-21T15:10:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 shutdown_connections 2024-05-21T15:10:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f66300a2c20 0x7f66300a5130 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6648104780 0x7f66481a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:25.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66481011d0 0x7f6648199840 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 --2- 172.21.15.84:0/3128306061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6648101b20 0x7f6648199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 >> 172.21.15.84:0/3128306061 conn(0x7f66480fcea0 msgr2=0x7f6648107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:10:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 shutdown_connections 2024-05-21T15:10:25.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:10:25.530+0000 7f664cbc6700 1 -- 172.21.15.84:0/3128306061 wait complete. 2024-05-21T15:10:25.810 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:25.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:25.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:25.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:26.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:26.113 DEBUG:teuthology.orchestra.run.smithi084:osd.4> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.4.service 2024-05-21T15:10:26.115 INFO:tasks.cephadm:Deploying osd.5 on smithi172 with /dev/vg_nvme/lv_3... 2024-05-21T15:10:26.115 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T15:10:26.358 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:10:26.792 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:26 smithi172 ceph-mon[36402]: from='osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:10:26.792 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:26 smithi172 ceph-mon[36402]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:10:26.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:26 smithi084 ceph-mon[36334]: from='osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:10:26.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:26 smithi084 ceph-mon[36334]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:10:27.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:26 smithi078 ceph-mon[31997]: from='osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:10:27.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:26 smithi078 ceph-mon[31997]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T15:10:27.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:27 smithi084 ceph-mon[36334]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T15:10:27.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:27 smithi084 ceph-mon[36334]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T15:10:27.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:27 smithi084 ceph-mon[36334]: from='osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:10:27.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:27 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:27.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:27 smithi084 ceph-mon[36334]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:10:27.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:27 smithi084 ceph-mon[36334]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:27.897 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:10:27 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4[47864]: 2024-05-21T15:10:27.637+0000 7f522737b700 -1 osd.4 0 waiting for initial osdmap 2024-05-21T15:10:27.897 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:10:27 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4[47864]: 2024-05-21T15:10:27.647+0000 7f5223931700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:10:27.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:27 smithi172 ceph-mon[36402]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T15:10:27.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:27 smithi172 ceph-mon[36402]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T15:10:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:27 smithi172 ceph-mon[36402]: from='osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:10:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:27 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:27 smithi172 ceph-mon[36402]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:10:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:27 smithi172 ceph-mon[36402]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:28.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:27 smithi078 ceph-mon[31997]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T15:10:28.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:27 smithi078 ceph-mon[31997]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T15:10:28.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:27 smithi078 ceph-mon[31997]: from='osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:10:28.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:27 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:28.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:27 smithi078 ceph-mon[31997]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:10:28.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:27 smithi078 ceph-mon[31997]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:28.208 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T15:10:28.208 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T15:10:28.208 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-05-21T15:10:28.208 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-05-21T15:10:28.208 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123785 s, 847 MB/s 2024-05-21T15:10:28.208 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-05-21T15:10:28.731 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:28 smithi084 ceph-mon[36334]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:10:28.732 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:28 smithi084 ceph-mon[36334]: osdmap e32: 5 total, 4 up, 5 in 2024-05-21T15:10:28.732 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:28.732 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:28 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:28.777 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:28 smithi172 ceph-mon[36402]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:10:28.777 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:28 smithi172 ceph-mon[36402]: osdmap e32: 5 total, 4 up, 5 in 2024-05-21T15:10:28.777 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:28.777 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:28 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:28.804 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_3 2024-05-21T15:10:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:10:29.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:28 smithi078 ceph-mon[31997]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:10:29.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:28 smithi078 ceph-mon[31997]: osdmap e32: 5 total, 4 up, 5 in 2024-05-21T15:10:29.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:29.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:28 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:29.808 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:10:29.808 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:10:29.808 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] boot 2024-05-21T15:10:29.808 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T15:10:29.808 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:29.808 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi084 2024-05-21T15:10:29.808 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi084 to 8544M 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:29.809 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:29 smithi172 ceph-mon[36402]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] boot 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi084 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:30.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi084 to 8544M 2024-05-21T15:10:30.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:30.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:30.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:30.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:30.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:29 smithi078 ceph-mon[31997]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:30.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:10:30.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:10:30.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: osd.4 [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] boot 2024-05-21T15:10:30.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T15:10:30.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi084 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi084 to 8544M 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:30.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:29 smithi084 ceph-mon[36334]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-21T15:10:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.496+0000 7ff84697e700 1 -- 172.21.15.172:0/3794889130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104cc0 msgr2=0x7ff84010b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.496+0000 7ff84697e700 1 --2- 172.21.15.172:0/3794889130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104cc0 0x7ff84010b570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff83800b0a0 tx=0x7ff83800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 -- 172.21.15.172:0/3794889130 shutdown_connections 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 --2- 172.21.15.172:0/3794889130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104cc0 0x7ff84010b570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 --2- 172.21.15.172:0/3794889130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8401031f0 0x7ff840103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 --2- 172.21.15.172:0/3794889130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff840104320 0x7ff840104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 -- 172.21.15.172:0/3794889130 >> 172.21.15.172:0/3794889130 conn(0x7ff8400fe850 msgr2=0x7ff840100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 -- 172.21.15.172:0/3794889130 shutdown_connections 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 -- 172.21.15.172:0/3794889130 wait complete. 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 Processor -- start 2024-05-21T15:10:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.497+0000 7ff84697e700 1 -- start start 2024-05-21T15:10:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff84697e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8401031f0 0x7ff840199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff84697e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104320 0x7ff840199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff84697e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff840104cc0 0x7ff8401a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff84697e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010eda0 con 0x7ff840104cc0 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff84697e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010ec20 con 0x7ff8401031f0 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff84697e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010ef20 con 0x7ff840104320 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104320 0x7ff840199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104320 0x7ff840199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47168/0 (socket says 172.21.15.172:47168) 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 -- 172.21.15.172:0/4035050652 learned_addr learned my addr 172.21.15.172:0/4035050652 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83ffff700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8401031f0 0x7ff840199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff844f1b700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff840104cc0 0x7ff8401a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 -- 172.21.15.172:0/4035050652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8401031f0 msgr2=0x7ff840199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8401031f0 0x7ff840199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 -- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff840104cc0 msgr2=0x7ff8401a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff840104cc0 0x7ff8401a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 -- 172.21.15.172:0/4035050652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8401a1380 con 0x7ff840104320 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.498+0000 7ff83f7fe700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104320 0x7ff840199d60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff83000c920 tx=0x7ff83000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.499+0000 7ff83d7fa700 1 -- 172.21.15.172:0/4035050652 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830004a30 con 0x7ff840104320 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.499+0000 7ff83d7fa700 1 -- 172.21.15.172:0/4035050652 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff830005500 con 0x7ff840104320 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.499+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8401a1670 con 0x7ff840104320 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.499+0000 7ff83d7fa700 1 -- 172.21.15.172:0/4035050652 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830007d50 con 0x7ff840104320 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.499+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401a1bb0 con 0x7ff840104320 2024-05-21T15:10:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.500+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff840050170 con 0x7ff840104320 2024-05-21T15:10:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.501+0000 7ff83d7fa700 1 -- 172.21.15.172:0/4035050652 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff830005020 con 0x7ff840104320 2024-05-21T15:10:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.501+0000 7ff83d7fa700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8280a2c20 0x7ff8280a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.502+0000 7ff83d7fa700 1 -- 172.21.15.172:0/4035050652 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7ff8300cdb70 con 0x7ff840104320 2024-05-21T15:10:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.502+0000 7ff83ffff700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8280a2c20 0x7ff8280a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.502+0000 7ff83ffff700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8280a2c20 0x7ff8280a5130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff840104220 tx=0x7ff83400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:10:30.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.506+0000 7ff83d7fa700 1 -- 172.21.15.172:0/4035050652 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff830014030 con 0x7ff840104320 2024-05-21T15:10:30.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:30.711+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff840062a80 con 0x7ff8280a2c20 2024-05-21T15:10:31.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:30 smithi078 ceph-mon[31997]: osdmap e34: 5 total, 5 up, 5 in 2024-05-21T15:10:31.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:31.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:31.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:31.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:30 smithi084 ceph-mon[36334]: osdmap e34: 5 total, 5 up, 5 in 2024-05-21T15:10:31.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:31.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:31.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:31.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:30 smithi172 ceph-mon[36402]: osdmap e34: 5 total, 5 up, 5 in 2024-05-21T15:10:31.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:31.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:31.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:31.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:31 smithi172 ceph-mon[36402]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:31.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:31 smithi172 ceph-mon[36402]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:32.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:31 smithi078 ceph-mon[31997]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:32.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:31 smithi078 ceph-mon[31997]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:32.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:31 smithi084 ceph-mon[36334]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:32.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:31 smithi084 ceph-mon[36334]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:34 smithi078 ceph-mon[31997]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:34 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/3449944945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]: dispatch 2024-05-21T15:10:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:34 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]: dispatch 2024-05-21T15:10:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:34 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]': finished 2024-05-21T15:10:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:34 smithi078 ceph-mon[31997]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T15:10:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:34 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:34 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/1636909474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:10:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:34 smithi084 ceph-mon[36334]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:34 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/3449944945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]: dispatch 2024-05-21T15:10:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:34 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]: dispatch 2024-05-21T15:10:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:34 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]': finished 2024-05-21T15:10:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:34 smithi084 ceph-mon[36334]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T15:10:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:34 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:34 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/1636909474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:10:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:34 smithi172 ceph-mon[36402]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:34 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/3449944945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]: dispatch 2024-05-21T15:10:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:34 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]: dispatch 2024-05-21T15:10:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:34 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cc5c54e-742b-4406-80d2-e84e0ba50b8a"}]': finished 2024-05-21T15:10:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:34 smithi172 ceph-mon[36402]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T15:10:34.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:34 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:34.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:34 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/1636909474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:10:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:36 smithi078 ceph-mon[31997]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:36 smithi084 ceph-mon[36334]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:36 smithi172 ceph-mon[36402]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:38.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:38 smithi172 ceph-mon[36402]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:38.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:38 smithi078 ceph-mon[31997]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:38.598 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:38 smithi084 ceph-mon[36334]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:40.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:40 smithi172 ceph-mon[36402]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:40.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:40 smithi078 ceph-mon[31997]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:40 smithi084 ceph-mon[36334]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:41 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T15:10:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:41 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:41.617 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:41 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T15:10:41.617 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:41 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:41.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:41 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T15:10:41.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:41 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:42.548 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:42 smithi172 ceph-mon[36402]: Deploying daemon osd.5 on smithi172 2024-05-21T15:10:42.548 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:42 smithi172 ceph-mon[36402]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:42 smithi078 ceph-mon[31997]: Deploying daemon osd.5 on smithi172 2024-05-21T15:10:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:42 smithi078 ceph-mon[31997]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:42.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:42 smithi084 ceph-mon[36334]: Deploying daemon osd.5 on smithi172 2024-05-21T15:10:42.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:42 smithi084 ceph-mon[36334]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:44 smithi078 ceph-mon[31997]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:44 smithi084 ceph-mon[36334]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:44 smithi172 ceph-mon[36402]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:46 smithi078 ceph-mon[31997]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:46 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:46.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:46 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:46.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:46 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:46 smithi084 ceph-mon[36334]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:46 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:46 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:46 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:46 smithi172 ceph-mon[36402]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:46 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:46 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:46 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:48.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:48 smithi078 ceph-mon[31997]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:48.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:48 smithi084 ceph-mon[36334]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:48 smithi172 ceph-mon[36402]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:49.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.226+0000 7ff83d7fa700 1 -- 172.21.15.172:0/4035050652 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff840062a80 con 0x7ff8280a2c20 2024-05-21T15:10:49.230 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 5 on host 'smithi172' 2024-05-21T15:10:49.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8280a2c20 msgr2=0x7ff8280a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:49.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8280a2c20 0x7ff8280a5130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff840104220 tx=0x7ff83400b410 comp rx=0 tx=0).stop 2024-05-21T15:10:49.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104320 msgr2=0x7ff840199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:49.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104320 0x7ff840199d60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff83000c920 tx=0x7ff83000cc30 comp rx=0 tx=0).stop 2024-05-21T15:10:49.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 shutdown_connections 2024-05-21T15:10:49.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff8280a2c20 0x7ff8280a5130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:49.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff840104cc0 0x7ff8401a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:49.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8401031f0 0x7ff840199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:49.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 --2- 172.21.15.172:0/4035050652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff840104320 0x7ff840199d60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:49.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 >> 172.21.15.172:0/4035050652 conn(0x7ff8400fe850 msgr2=0x7ff840107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:10:49.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 shutdown_connections 2024-05-21T15:10:49.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:10:49.229+0000 7ff84697e700 1 -- 172.21.15.172:0/4035050652 wait complete. 2024-05-21T15:10:49.868 DEBUG:teuthology.orchestra.run.smithi172:osd.5> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.5.service 2024-05-21T15:10:49.870 INFO:tasks.cephadm:Deploying osd.6 on smithi078 with /dev/vg_nvme/lv_2... 2024-05-21T15:10:49.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T15:10:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:10:50.183 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:10:49 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:10:49.976+0000 7fa9f21087c0 -1 osd.5 0 log_to_monitors true 2024-05-21T15:10:50.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:50 smithi078 ceph-mon[31997]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:50.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:50 smithi078 ceph-mon[31997]: from='osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:10:50.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:50 smithi078 ceph-mon[31997]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:10:50.577 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:50 smithi172 ceph-mon[36402]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:50.577 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.577 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.577 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.577 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.577 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:50 smithi172 ceph-mon[36402]: from='osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:10:50.577 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:50 smithi172 ceph-mon[36402]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:10:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:50 smithi084 ceph-mon[36334]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:50 smithi084 ceph-mon[36334]: from='osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:10:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:50 smithi084 ceph-mon[36334]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T15:10:52.281 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:51 smithi078 ceph-mon[31997]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T15:10:52.281 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:51 smithi078 ceph-mon[31997]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T15:10:52.281 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:51 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:52.281 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:51 smithi078 ceph-mon[31997]: from='osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:10:52.281 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:51 smithi078 ceph-mon[31997]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:10:52.281 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:51 smithi078 ceph-mon[31997]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:52.282 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T15:10:52.282 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T15:10:52.282 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-05-21T15:10:52.282 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-05-21T15:10:52.282 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115425 s, 908 MB/s 2024-05-21T15:10:52.282 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-05-21T15:10:52.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:51 smithi084 ceph-mon[36334]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T15:10:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:51 smithi084 ceph-mon[36334]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T15:10:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:51 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:51 smithi084 ceph-mon[36334]: from='osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:10:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:51 smithi084 ceph-mon[36334]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:10:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:51 smithi084 ceph-mon[36334]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:52.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:51 smithi172 ceph-mon[36402]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T15:10:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:51 smithi172 ceph-mon[36402]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T15:10:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:51 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:51 smithi172 ceph-mon[36402]: from='osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:10:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:51 smithi172 ceph-mon[36402]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:10:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:51 smithi172 ceph-mon[36402]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:52.977 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-05-21T15:10:53.126 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:10:52 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:10:52.842+0000 7fa9e876e700 -1 osd.5 0 waiting for initial osdmap 2024-05-21T15:10:53.126 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:10:52 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:10:52.852+0000 7fa9df519700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:10:53.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:52 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:10:53.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:52 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:10:53.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:52 smithi172 ceph-mon[36402]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:10:53.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:52 smithi172 ceph-mon[36402]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T15:10:53.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:53.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:53.127 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:52 smithi172 ceph-mon[36402]: from='osd.5 ' entity='osd.5' 2024-05-21T15:10:53.239 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:10:53.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:52 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:10:53.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:52 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:10:53.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:52 smithi078 ceph-mon[31997]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:10:53.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:52 smithi078 ceph-mon[31997]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T15:10:53.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:53.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:53.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:52 smithi078 ceph-mon[31997]: from='osd.5 ' entity='osd.5' 2024-05-21T15:10:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:52 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:10:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:52 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:10:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:52 smithi084 ceph-mon[36334]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:10:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:52 smithi084 ceph-mon[36334]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T15:10:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:53.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:52 smithi084 ceph-mon[36334]: from='osd.5 ' entity='osd.5' 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] boot 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:54.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:54.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:54.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:54.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:54.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:53 smithi078 ceph-mon[31997]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] boot 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:54.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:54.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:54.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:54.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:53 smithi084 ceph-mon[36334]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T15:10:54.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: osd.5 [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] boot 2024-05-21T15:10:54.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T15:10:54.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:10:54.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:10:54.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:53 smithi172 ceph-mon[36402]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T15:10:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 -- 172.21.15.78:0/80034262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 msgr2=0x7f640c10b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 --2- 172.21.15.78:0/80034262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 0x7f640c10b570 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f640400b0a0 tx=0x7f640400b3b0 comp rx=0 tx=0).stop 2024-05-21T15:10:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 -- 172.21.15.78:0/80034262 shutdown_connections 2024-05-21T15:10:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 --2- 172.21.15.78:0/80034262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 0x7f640c10b570 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 --2- 172.21.15.78:0/80034262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f640c1031f0 0x7f640c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 --2- 172.21.15.78:0/80034262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640c104320 0x7f640c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 -- 172.21.15.78:0/80034262 >> 172.21.15.78:0/80034262 conn(0x7f640c0fe850 msgr2=0x7f640c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:10:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 -- 172.21.15.78:0/80034262 shutdown_connections 2024-05-21T15:10:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.870+0000 7f6410c07700 1 -- 172.21.15.78:0/80034262 wait complete. 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6410c07700 1 Processor -- start 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6410c07700 1 -- start start 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6410c07700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640c1031f0 0x7f640c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6410c07700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f640c104320 0x7f640c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6410c07700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 0x7f640c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6410c07700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10ee30 con 0x7f640c104cc0 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6410c07700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10ecb0 con 0x7f640c1031f0 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6410c07700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10efb0 con 0x7f640c104320 2024-05-21T15:10:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 0x7f640c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 0x7f640c1a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34978/0 (socket says 172.21.15.78:34978) 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640ad9d700 1 -- 172.21.15.78:0/281952828 learned_addr learned my addr 172.21.15.78:0/281952828 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640a59c700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640c1031f0 0x7f640c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6409d9b700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f640c104320 0x7f640c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640ad9d700 1 -- 172.21.15.78:0/281952828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f640c104320 msgr2=0x7f640c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640ad9d700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f640c104320 0x7f640c199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640ad9d700 1 -- 172.21.15.78:0/281952828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640c1031f0 msgr2=0x7f640c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640ad9d700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640c1031f0 0x7f640c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f640ad9d700 1 -- 172.21.15.78:0/281952828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1a13f0 con 0x7f640c104cc0 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.871+0000 7f6409d9b700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f640c104320 0x7f640c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.872+0000 7f640ad9d700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 0x7f640c1a0d70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64040062a0 tx=0x7f6404004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.872+0000 7f64037fe700 1 -- 172.21.15.78:0/281952828 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404021070 con 0x7f640c104cc0 2024-05-21T15:10:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.872+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c1a1680 con 0x7f640c104cc0 2024-05-21T15:10:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.872+0000 7f64037fe700 1 -- 172.21.15.78:0/281952828 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6404004020 con 0x7f640c104cc0 2024-05-21T15:10:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.872+0000 7f64037fe700 1 -- 172.21.15.78:0/281952828 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640401d6b0 con 0x7f640c104cc0 2024-05-21T15:10:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.872+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1a1b30 con 0x7f640c104cc0 2024-05-21T15:10:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.873+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640c050170 con 0x7f640c104cc0 2024-05-21T15:10:54.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.874+0000 7f64037fe700 1 -- 172.21.15.78:0/281952828 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6404004540 con 0x7f640c104cc0 2024-05-21T15:10:54.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.875+0000 7f64037fe700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f63f80a2c20 0x7f63f80a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:10:54.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.875+0000 7f640a59c700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f63f80a2c20 0x7f63f80a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:10:54.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.875+0000 7f64037fe700 1 -- 172.21.15.78:0/281952828 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4562+0+0 (secure 0 0 0) 0x7f640400f020 con 0x7f640c104cc0 2024-05-21T15:10:54.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.876+0000 7f640a59c700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f63f80a2c20 0x7f63f80a5130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63fc005950 tx=0x7f63fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:10:54.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:54.879+0000 7f64037fe700 1 -- 172.21.15.78:0/281952828 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f640402a020 con 0x7f640c104cc0 2024-05-21T15:10:55.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:10:55.087+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f640c062a80 con 0x7f63f80a2c20 2024-05-21T15:10:55.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:54 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi172 2024-05-21T15:10:55.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:54 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi172 to 8544M 2024-05-21T15:10:55.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:54 smithi078 ceph-mon[31997]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T15:10:55.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:54 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi172 2024-05-21T15:10:55.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:54 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi172 to 8544M 2024-05-21T15:10:55.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:54 smithi084 ceph-mon[36334]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T15:10:55.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:54 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi172 2024-05-21T15:10:55.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:54 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi172 to 8544M 2024-05-21T15:10:55.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:54 smithi172 ceph-mon[36402]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T15:10:56.297 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:56 smithi084 ceph-mon[36334]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:56.297 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:56 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:56.297 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:56 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:56.297 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:56 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:56.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:56 smithi084 ceph-mon[36334]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:10:56.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:56 smithi078 ceph-mon[31997]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:56.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:56 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:56.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:56 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:56.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:56 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:56.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:56 smithi078 ceph-mon[31997]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:10:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:56 smithi172 ceph-mon[36402]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:10:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:56 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:10:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:56 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:10:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:56 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:10:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:56 smithi172 ceph-mon[36402]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:10:57.309 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:57 smithi084 ceph-mon[36334]: osdmap e41: 6 total, 6 up, 6 in 2024-05-21T15:10:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:57 smithi078 ceph-mon[31997]: osdmap e41: 6 total, 6 up, 6 in 2024-05-21T15:10:57.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:57 smithi172 ceph-mon[36402]: osdmap e41: 6 total, 6 up, 6 in 2024-05-21T15:10:58.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:58 smithi078 ceph-mon[31997]: osdmap e42: 6 total, 6 up, 6 in 2024-05-21T15:10:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:58 smithi078 ceph-mon[31997]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-21T15:10:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:58 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4132724436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a3eb06b-7754-4293-bb69-301f02709e7d"}]: dispatch 2024-05-21T15:10:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:58 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4132724436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a3eb06b-7754-4293-bb69-301f02709e7d"}]': finished 2024-05-21T15:10:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:58 smithi078 ceph-mon[31997]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T15:10:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:58 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:10:58.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:58 smithi084 ceph-mon[36334]: osdmap e42: 6 total, 6 up, 6 in 2024-05-21T15:10:58.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:58 smithi084 ceph-mon[36334]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-21T15:10:58.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:58 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4132724436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a3eb06b-7754-4293-bb69-301f02709e7d"}]: dispatch 2024-05-21T15:10:58.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:58 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4132724436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a3eb06b-7754-4293-bb69-301f02709e7d"}]': finished 2024-05-21T15:10:58.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:58 smithi084 ceph-mon[36334]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T15:10:58.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:58 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:10:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:58 smithi172 ceph-mon[36402]: osdmap e42: 6 total, 6 up, 6 in 2024-05-21T15:10:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:58 smithi172 ceph-mon[36402]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-21T15:10:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:58 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4132724436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a3eb06b-7754-4293-bb69-301f02709e7d"}]: dispatch 2024-05-21T15:10:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:58 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4132724436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a3eb06b-7754-4293-bb69-301f02709e7d"}]': finished 2024-05-21T15:10:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:58 smithi172 ceph-mon[36402]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T15:10:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:58 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:10:59.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:59 smithi078 ceph-mon[31997]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-21T15:10:59.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:10:59 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/640951539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:10:59.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:59 smithi084 ceph-mon[36334]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-21T15:10:59.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:10:59 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/640951539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:10:59.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:59 smithi172 ceph-mon[36402]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-21T15:10:59.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:10:59 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/640951539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:11:00.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:00 smithi078 ceph-mon[31997]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-21T15:11:00.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:00 smithi084 ceph-mon[36334]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-21T15:11:00.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:00 smithi172 ceph-mon[36402]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-21T15:11:01.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:01 smithi078 ceph-mon[31997]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-21T15:11:01.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:01 smithi078 ceph-mon[31997]: Cluster is now healthy 2024-05-21T15:11:01.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:01 smithi084 ceph-mon[36334]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-21T15:11:01.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:01 smithi084 ceph-mon[36334]: Cluster is now healthy 2024-05-21T15:11:01.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:01 smithi172 ceph-mon[36402]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-21T15:11:01.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:01 smithi172 ceph-mon[36402]: Cluster is now healthy 2024-05-21T15:11:02.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:02 smithi078 ceph-mon[31997]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:11:02.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:02 smithi084 ceph-mon[36334]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:11:02.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:02 smithi172 ceph-mon[36402]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:11:04.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:04 smithi172 ceph-mon[36402]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-21T15:11:04.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:04 smithi078 ceph-mon[31997]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-21T15:11:04.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:04 smithi084 ceph-mon[36334]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-21T15:11:05.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T15:11:05.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T15:11:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T15:11:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:06.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:06 smithi078 ceph-mon[31997]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-21T15:11:06.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:06 smithi078 ceph-mon[31997]: Deploying daemon osd.6 on smithi078 2024-05-21T15:11:06.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:06 smithi084 ceph-mon[36334]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-21T15:11:06.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:06 smithi084 ceph-mon[36334]: Deploying daemon osd.6 on smithi078 2024-05-21T15:11:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:06 smithi172 ceph-mon[36402]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-21T15:11:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:06 smithi172 ceph-mon[36402]: Deploying daemon osd.6 on smithi078 2024-05-21T15:11:08.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:08 smithi078 ceph-mon[31997]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:11:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:08 smithi084 ceph-mon[36334]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:11:08.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:08 smithi172 ceph-mon[36402]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:11:10.436 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:10 smithi078 ceph-mon[31997]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-21T15:11:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:10 smithi084 ceph-mon[36334]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-21T15:11:10.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:10 smithi172 ceph-mon[36402]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-21T15:11:12.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:12.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:12.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:12.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:11 smithi078 ceph-mon[31997]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T15:11:12.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:12.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:12.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:12.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:11 smithi084 ceph-mon[36334]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T15:11:12.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:12.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:12.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:12.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:11 smithi172 ceph-mon[36402]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-21T15:11:14.343 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 6 on host 'smithi078' 2024-05-21T15:11:14.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.340+0000 7f64037fe700 1 -- 172.21.15.78:0/281952828 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f640c062a80 con 0x7f63f80a2c20 2024-05-21T15:11:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f63f80a2c20 msgr2=0x7f63f80a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f63f80a2c20 0x7f63f80a5130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63fc005950 tx=0x7f63fc0058e0 comp rx=0 tx=0).stop 2024-05-21T15:11:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 msgr2=0x7f640c1a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 0x7f640c1a0d70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64040062a0 tx=0x7f6404004e20 comp rx=0 tx=0).stop 2024-05-21T15:11:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 shutdown_connections 2024-05-21T15:11:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f63f80a2c20 0x7f63f80a5130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f640c104cc0 0x7f640c1a0d70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:14.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f640c1031f0 0x7f640c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 --2- 172.21.15.78:0/281952828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f640c104320 0x7f640c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 >> 172.21.15.78:0/281952828 conn(0x7f640c0fe850 msgr2=0x7f640c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:11:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 shutdown_connections 2024-05-21T15:11:14.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:11:14.342+0000 7f6410c07700 1 -- 172.21.15.78:0/281952828 wait complete. 2024-05-21T15:11:14.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:14 smithi078 ceph-mon[31997]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:14.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:14 smithi084 ceph-mon[36334]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:14 smithi172 ceph-mon[36402]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:15.086 DEBUG:teuthology.orchestra.run.smithi078:osd.6> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.6.service 2024-05-21T15:11:15.089 INFO:tasks.cephadm:Deploying osd.7 on smithi084 with /dev/vg_nvme/lv_2... 2024-05-21T15:11:15.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T15:11:15.335 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:11:15.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:15 smithi078 ceph-mon[31997]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T15:11:15.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:15 smithi084 ceph-mon[36334]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T15:11:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:15 smithi172 ceph-mon[36402]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T15:11:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:16 smithi084 ceph-mon[36334]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:16 smithi084 ceph-mon[36334]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T15:11:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:16 smithi084 ceph-mon[36334]: osdmap e44: 7 total, 6 up, 7 in 2024-05-21T15:11:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:16 smithi084 ceph-mon[36334]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:11:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:16.593 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:16 smithi078 ceph-mon[31997]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:16.593 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:16 smithi078 ceph-mon[31997]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T15:11:16.593 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:16 smithi078 ceph-mon[31997]: osdmap e44: 7 total, 6 up, 7 in 2024-05-21T15:11:16.593 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:16 smithi078 ceph-mon[31997]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:11:16.593 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:16.594 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:11:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:11:16.327+0000 7f92d4d62700 -1 osd.6 0 waiting for initial osdmap 2024-05-21T15:11:16.594 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:11:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:11:16.338+0000 7f92d1318700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:11:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:16 smithi172 ceph-mon[36402]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:16 smithi172 ceph-mon[36402]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T15:11:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:16 smithi172 ceph-mon[36402]: osdmap e44: 7 total, 6 up, 7 in 2024-05-21T15:11:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:16 smithi172 ceph-mon[36402]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:11:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T15:11:17.381 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T15:11:17.381 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-05-21T15:11:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-05-21T15:11:17.382 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012788 s, 820 MB/s 2024-05-21T15:11:17.382 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-05-21T15:11:17.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:17 smithi084 ceph-mon[36334]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:11:17.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:17 smithi084 ceph-mon[36334]: osdmap e45: 7 total, 6 up, 7 in 2024-05-21T15:11:17.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:17 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:17.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:17 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:17.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:17 smithi084 ceph-mon[36334]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:17.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:17 smithi078 ceph-mon[31997]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:11:17.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:17 smithi078 ceph-mon[31997]: osdmap e45: 7 total, 6 up, 7 in 2024-05-21T15:11:17.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:17 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:17.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:17 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:17.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:17 smithi078 ceph-mon[31997]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:17 smithi172 ceph-mon[36402]: from='osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:11:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:17 smithi172 ceph-mon[36402]: osdmap e45: 7 total, 6 up, 7 in 2024-05-21T15:11:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:17 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:17 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:17 smithi172 ceph-mon[36402]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-21T15:11:17.991 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_2 2024-05-21T15:11:18.232 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:11:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:18 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:11:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:18 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:11:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:18 smithi078 ceph-mon[31997]: osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] boot 2024-05-21T15:11:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:18 smithi078 ceph-mon[31997]: osdmap e46: 7 total, 7 up, 7 in 2024-05-21T15:11:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:18.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:18 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:11:18.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:18 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:11:18.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:18 smithi084 ceph-mon[36334]: osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] boot 2024-05-21T15:11:18.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:18 smithi084 ceph-mon[36334]: osdmap e46: 7 total, 7 up, 7 in 2024-05-21T15:11:18.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:18 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:18 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:11:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:18 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:11:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:18 smithi172 ceph-mon[36402]: osd.6 [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] boot 2024-05-21T15:11:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:18 smithi172 ceph-mon[36402]: osdmap e46: 7 total, 7 up, 7 in 2024-05-21T15:11:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:18 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:11:19.515 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: osdmap e47: 7 total, 7 up, 7 in 2024-05-21T15:11:19.515 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi078 2024-05-21T15:11:19.515 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.515 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.515 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi078 to 5696M 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.516 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:19 smithi084 ceph-mon[36334]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:19.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: osdmap e47: 7 total, 7 up, 7 in 2024-05-21T15:11:19.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi078 2024-05-21T15:11:19.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi078 to 5696M 2024-05-21T15:11:19.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:19.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:19.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:19.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:19.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:19.601 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.601 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:19 smithi078 ceph-mon[31997]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: osdmap e47: 7 total, 7 up, 7 in 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi078 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi078 to 5696M 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:19.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:19.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:19.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:19.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:19.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:19 smithi172 ceph-mon[36402]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:19.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 -- 172.21.15.84:0/531052255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4434103b90 msgr2=0x7f443410a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:19.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 --2- 172.21.15.84:0/531052255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4434103b90 0x7f443410a440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f442c00b0a0 tx=0x7f442c00b3b0 comp rx=0 tx=0).stop 2024-05-21T15:11:19.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 -- 172.21.15.84:0/531052255 shutdown_connections 2024-05-21T15:11:19.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 --2- 172.21.15.84:0/531052255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4434103b90 0x7f443410a440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:19.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 --2- 172.21.15.84:0/531052255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44341020c0 0x7f44341024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:19.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 --2- 172.21.15.84:0/531052255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44341031f0 0x7f4434103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:19.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 -- 172.21.15.84:0/531052255 >> 172.21.15.84:0/531052255 conn(0x7f44340fd700 msgr2=0x7f44340ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 -- 172.21.15.84:0/531052255 shutdown_connections 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.768+0000 7f44397a8700 1 -- 172.21.15.84:0/531052255 wait complete. 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44397a8700 1 Processor -- start 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44397a8700 1 -- start start 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44397a8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44341020c0 0x7f443406af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44397a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44341031f0 0x7f443419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44397a8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4434103b90 0x7f44341a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44397a8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f443410dc90 con 0x7f44341031f0 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44397a8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f443410db10 con 0x7f44341020c0 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44397a8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f443410de10 con 0x7f4434103b90 2024-05-21T15:11:19.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f4432ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44341020c0 0x7f443406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f4432ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44341020c0 0x7f443406af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48192/0 (socket says 172.21.15.84:48192) 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f4432ffd700 1 -- 172.21.15.84:0/1424209949 learned_addr learned my addr 172.21.15.84:0/1424209949 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f442bfff700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44341031f0 0x7f443419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f44337fe700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4434103b90 0x7f44341a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f4432ffd700 1 -- 172.21.15.84:0/1424209949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4434103b90 msgr2=0x7f44341a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f4432ffd700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4434103b90 0x7f44341a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f4432ffd700 1 -- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44341031f0 msgr2=0x7f443419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f4432ffd700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44341031f0 0x7f443419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.769+0000 7f4432ffd700 1 -- 172.21.15.84:0/1424209949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44341a1330 con 0x7f44341020c0 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.770+0000 7f4432ffd700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44341020c0 0x7f443406af50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f442400d950 tx=0x7f442400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.770+0000 7f4430ff9700 1 -- 172.21.15.84:0/1424209949 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424017070 con 0x7f44341020c0 2024-05-21T15:11:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.770+0000 7f4430ff9700 1 -- 172.21.15.84:0/1424209949 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4424009c10 con 0x7f44341020c0 2024-05-21T15:11:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.770+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44341a1620 con 0x7f44341020c0 2024-05-21T15:11:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.770+0000 7f4430ff9700 1 -- 172.21.15.84:0/1424209949 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424003be0 con 0x7f44341020c0 2024-05-21T15:11:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.770+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341a1b60 con 0x7f44341020c0 2024-05-21T15:11:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.771+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4434050170 con 0x7f44341020c0 2024-05-21T15:11:19.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.773+0000 7f4430ff9700 1 -- 172.21.15.84:0/1424209949 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4424004220 con 0x7f44341020c0 2024-05-21T15:11:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.774+0000 7f4430ff9700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f44200a2c20 0x7f44200a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:11:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.774+0000 7f4430ff9700 1 -- 172.21.15.84:0/1424209949 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f44240ce5d0 con 0x7f44341020c0 2024-05-21T15:11:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.774+0000 7f442bfff700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f44200a2c20 0x7f44200a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:11:19.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.774+0000 7f442bfff700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f44200a2c20 0x7f44200a5130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f443406a900 tx=0x7f441c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:11:19.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.777+0000 7f4430ff9700 1 -- 172.21.15.84:0/1424209949 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4424020030 con 0x7f44341020c0 2024-05-21T15:11:19.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:19.982+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4434062a80 con 0x7f44200a2c20 2024-05-21T15:11:20.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:20 smithi078 ceph-mon[31997]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:11:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:11:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:11:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:20.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:20 smithi084 ceph-mon[36334]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:11:20.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:11:20.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:11:20.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:20 smithi172 ceph-mon[36402]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:11:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:11:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:11:20.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:21.546 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:21 smithi084 ceph-mon[36334]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:21 smithi172 ceph-mon[36402]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:21.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:21 smithi078 ceph-mon[31997]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:23.048 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:22 smithi084 ceph-mon[36334]: from='client.? 172.21.15.84:0/257140809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]: dispatch 2024-05-21T15:11:23.048 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:22 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]: dispatch 2024-05-21T15:11:23.049 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:22 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]': finished 2024-05-21T15:11:23.049 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:22 smithi084 ceph-mon[36334]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T15:11:23.049 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:23.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:22 smithi078 ceph-mon[31997]: from='client.? 172.21.15.84:0/257140809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]: dispatch 2024-05-21T15:11:23.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:22 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]: dispatch 2024-05-21T15:11:23.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:22 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]': finished 2024-05-21T15:11:23.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:22 smithi078 ceph-mon[31997]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T15:11:23.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:23.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:22 smithi172 ceph-mon[36402]: from='client.? 172.21.15.84:0/257140809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]: dispatch 2024-05-21T15:11:23.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:22 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]: dispatch 2024-05-21T15:11:23.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:22 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9155447f-a10d-4e00-8f4f-a891a095b823"}]': finished 2024-05-21T15:11:23.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:22 smithi172 ceph-mon[36402]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T15:11:23.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:24.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:23 smithi078 ceph-mon[31997]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:24.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:23 smithi078 ceph-mon[31997]: from='client.? 172.21.15.84:0/1170795012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:11:24.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:23 smithi084 ceph-mon[36334]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:24.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:23 smithi084 ceph-mon[36334]: from='client.? 172.21.15.84:0/1170795012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:11:24.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:23 smithi172 ceph-mon[36402]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:24.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:23 smithi172 ceph-mon[36402]: from='client.? 172.21.15.84:0/1170795012' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:11:26.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:26 smithi078 ceph-mon[31997]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:26.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:26 smithi084 ceph-mon[36334]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:26.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:26 smithi172 ceph-mon[36402]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:28.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:28 smithi078 ceph-mon[31997]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:28.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:28 smithi084 ceph-mon[36334]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:28 smithi172 ceph-mon[36402]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:30.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:30 smithi078 ceph-mon[31997]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T15:11:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:30 smithi084 ceph-mon[36334]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T15:11:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:30 smithi172 ceph-mon[36402]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T15:11:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:31.562 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:31 smithi084 ceph-mon[36334]: Deploying daemon osd.7 on smithi084 2024-05-21T15:11:31.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:31 smithi078 ceph-mon[31997]: Deploying daemon osd.7 on smithi084 2024-05-21T15:11:31.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:31 smithi172 ceph-mon[36402]: Deploying daemon osd.7 on smithi084 2024-05-21T15:11:32.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:32 smithi084 ceph-mon[36334]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:32.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:32 smithi078 ceph-mon[31997]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:32 smithi172 ceph-mon[36402]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:34 smithi078 ceph-mon[31997]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:34 smithi084 ceph-mon[36334]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:34 smithi172 ceph-mon[36402]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:36.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:36 smithi084 ceph-mon[36334]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:36.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:36.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:36.584 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:36 smithi078 ceph-mon[31997]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:36.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:36 smithi172 ceph-mon[36402]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:36.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:36.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:38.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:38 smithi078 ceph-mon[31997]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:38.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:38 smithi084 ceph-mon[36334]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:38 smithi172 ceph-mon[36402]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:39.153 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 7 on host 'smithi084' 2024-05-21T15:11:39.154 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.149+0000 7f4430ff9700 1 -- 172.21.15.84:0/1424209949 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4434062a80 con 0x7f44200a2c20 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f44200a2c20 msgr2=0x7f44200a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f44200a2c20 0x7f44200a5130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f443406a900 tx=0x7f441c00b410 comp rx=0 tx=0).stop 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44341020c0 msgr2=0x7f443406af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44341020c0 0x7f443406af50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f442400d950 tx=0x7f442400dc60 comp rx=0 tx=0).stop 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 shutdown_connections 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f44200a2c20 0x7f44200a5130 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4434103b90 0x7f44341a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44341020c0 0x7f443406af50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:39.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 --2- 172.21.15.84:0/1424209949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44341031f0 0x7f443419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:39.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 >> 172.21.15.84:0/1424209949 conn(0x7f44340fd700 msgr2=0x7f4434106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:11:39.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 shutdown_connections 2024-05-21T15:11:39.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:11:39.153+0000 7f44397a8700 1 -- 172.21.15.84:0/1424209949 wait complete. 2024-05-21T15:11:39.759 DEBUG:teuthology.orchestra.run.smithi084:osd.7> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.7.service 2024-05-21T15:11:39.761 INFO:tasks.cephadm:Deploying osd.8 on smithi172 with /dev/vg_nvme/lv_2... 2024-05-21T15:11:39.761 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T15:11:40.007 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:11:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:40 smithi084 ceph-mon[36334]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:40 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:40 smithi084 ceph-mon[36334]: from='osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:11:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:40 smithi084 ceph-mon[36334]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:11:40.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:40 smithi172 ceph-mon[36402]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:40.413 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.414 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:40 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.414 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:40 smithi172 ceph-mon[36402]: from='osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:11:40.414 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:40 smithi172 ceph-mon[36402]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:11:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:40 smithi078 ceph-mon[31997]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:40 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:40 smithi078 ceph-mon[31997]: from='osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:11:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:40 smithi078 ceph-mon[31997]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T15:11:41.379 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:41 smithi172 ceph-mon[36402]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T15:11:41.380 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:41 smithi172 ceph-mon[36402]: osdmap e49: 8 total, 7 up, 8 in 2024-05-21T15:11:41.380 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:41 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:41.380 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:41 smithi172 ceph-mon[36402]: from='osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:11:41.380 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:41 smithi172 ceph-mon[36402]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:11:41.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:41 smithi078 ceph-mon[31997]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T15:11:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:41 smithi078 ceph-mon[31997]: osdmap e49: 8 total, 7 up, 8 in 2024-05-21T15:11:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:41 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:41 smithi078 ceph-mon[31997]: from='osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:11:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:41 smithi078 ceph-mon[31997]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:11:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:41 smithi084 ceph-mon[36334]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T15:11:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:41 smithi084 ceph-mon[36334]: osdmap e49: 8 total, 7 up, 8 in 2024-05-21T15:11:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:41 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:41 smithi084 ceph-mon[36334]: from='osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:11:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:41 smithi084 ceph-mon[36334]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:11:41.944 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T15:11:41.944 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T15:11:41.944 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-05-21T15:11:41.944 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-05-21T15:11:41.944 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139748 s, 750 MB/s 2024-05-21T15:11:41.944 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-05-21T15:11:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:42 smithi084 ceph-mon[36334]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:42 smithi084 ceph-mon[36334]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:11:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:42 smithi084 ceph-mon[36334]: osdmap e50: 8 total, 7 up, 8 in 2024-05-21T15:11:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:42 smithi084 ceph-mon[36334]: from='osd.7 ' entity='osd.7' 2024-05-21T15:11:42.397 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:11:42 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:11:42.135+0000 7f8df5aa5700 -1 osd.7 0 waiting for initial osdmap 2024-05-21T15:11:42.397 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:11:42 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:11:42.152+0000 7f8dee053700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:11:42.491 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:42 smithi172 ceph-mon[36402]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:42.491 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:42 smithi172 ceph-mon[36402]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:11:42.491 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:42 smithi172 ceph-mon[36402]: osdmap e50: 8 total, 7 up, 8 in 2024-05-21T15:11:42.491 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:42.491 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:42.491 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:42 smithi172 ceph-mon[36402]: from='osd.7 ' entity='osd.7' 2024-05-21T15:11:42.518 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_2 2024-05-21T15:11:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:42 smithi078 ceph-mon[31997]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-05-21T15:11:42.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:42 smithi078 ceph-mon[31997]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:11:42.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:42 smithi078 ceph-mon[31997]: osdmap e50: 8 total, 7 up, 8 in 2024-05-21T15:11:42.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:42.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:42.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:42 smithi078 ceph-mon[31997]: from='osd.7 ' entity='osd.7' 2024-05-21T15:11:42.762 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:11:43.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:11:43.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:11:43.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] boot 2024-05-21T15:11:43.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T15:11:43.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:43.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:43.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:43.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:43.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:43.436 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:43.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.437 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:43 smithi172 ceph-mon[36402]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] boot 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:43 smithi078 ceph-mon[31997]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: osd.7 [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] boot 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:11:43.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:11:43.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:43 smithi084 ceph-mon[36334]: osdmap e52: 8 total, 8 up, 8 in 2024-05-21T15:11:44.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 -- 172.21.15.172:0/1916313197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c8103ba0 msgr2=0x7fe8c810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:44.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1916313197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c8103ba0 0x7fe8c810a450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009990 tx=0x7fe8c0009ca0 comp rx=0 tx=0).stop 2024-05-21T15:11:44.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 -- 172.21.15.172:0/1916313197 shutdown_connections 2024-05-21T15:11:44.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1916313197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c8103ba0 0x7fe8c810a450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:44.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1916313197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c81020d0 0x7fe8c81024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:44.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1916313197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c8103200 0x7fe8c8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:44.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 -- 172.21.15.172:0/1916313197 >> 172.21.15.172:0/1916313197 conn(0x7fe8c80fd710 msgr2=0x7fe8c80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:11:44.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 -- 172.21.15.172:0/1916313197 shutdown_connections 2024-05-21T15:11:44.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.377+0000 7fe8d054f700 1 -- 172.21.15.172:0/1916313197 wait complete. 2024-05-21T15:11:44.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8d054f700 1 Processor -- start 2024-05-21T15:11:44.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8d054f700 1 -- start start 2024-05-21T15:11:44.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8d054f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c81020d0 0x7fe8c8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:11:44.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8d054f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c8103200 0x7fe8c8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:11:44.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8d054f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c8103ba0 0x7fe8c81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:11:44.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8d054f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c810dca0 con 0x7fe8c8103200 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8d054f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c810db20 con 0x7fe8c8103ba0 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8d054f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c810de20 con 0x7fe8c81020d0 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8ce2eb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c81020d0 0x7fe8c8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8ce2eb700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c81020d0 0x7fe8c8199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47626/0 (socket says 172.21.15.172:47626) 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8ce2eb700 1 -- 172.21.15.172:0/1437134835 learned_addr learned my addr 172.21.15.172:0/1437134835 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.378+0000 7fe8cdaea700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c8103200 0x7fe8c8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8ceaec700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c8103ba0 0x7fe8c81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8ce2eb700 1 -- 172.21.15.172:0/1437134835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c8103ba0 msgr2=0x7fe8c81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8ce2eb700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c8103ba0 0x7fe8c81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8ce2eb700 1 -- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c8103200 msgr2=0x7fe8c8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8ce2eb700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c8103200 0x7fe8c8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8ce2eb700 1 -- 172.21.15.172:0/1437134835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c81a12d0 con 0x7fe8c81020d0 2024-05-21T15:11:44.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8cdaea700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c8103200 0x7fe8c8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:11:44.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8ce2eb700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c81020d0 0x7fe8c8199800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4007ee0 tx=0x7fe8c400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:11:44.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8bf7fe700 1 -- 172.21.15.172:0/1437134835 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c400dc90 con 0x7fe8c81020d0 2024-05-21T15:11:44.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c81a15c0 con 0x7fe8c81020d0 2024-05-21T15:11:44.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8bf7fe700 1 -- 172.21.15.172:0/1437134835 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c400de30 con 0x7fe8c81020d0 2024-05-21T15:11:44.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8bf7fe700 1 -- 172.21.15.172:0/1437134835 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40133f0 con 0x7fe8c81020d0 2024-05-21T15:11:44.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.379+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c81a1b00 con 0x7fe8c81020d0 2024-05-21T15:11:44.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.380+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c8050170 con 0x7fe8c81020d0 2024-05-21T15:11:44.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.381+0000 7fe8bf7fe700 1 -- 172.21.15.172:0/1437134835 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe8c401a020 con 0x7fe8c81020d0 2024-05-21T15:11:44.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.382+0000 7fe8bf7fe700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8b40a2c20 0x7fe8b40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:11:44.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.382+0000 7fe8bf7fe700 1 -- 172.21.15.172:0/1437134835 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7fe8c40ce3f0 con 0x7fe8c81020d0 2024-05-21T15:11:44.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.382+0000 7fe8cdaea700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8b40a2c20 0x7fe8b40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:11:44.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.383+0000 7fe8cdaea700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8b40a2c20 0x7fe8b40a5130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe8c819a8b0 tx=0x7fe8b800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:11:44.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.386+0000 7fe8bf7fe700 1 -- 172.21.15.172:0/1437134835 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8c4010070 con 0x7fe8c81020d0 2024-05-21T15:11:44.390 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:44 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi084 2024-05-21T15:11:44.391 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:44 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi084 to 5696M 2024-05-21T15:11:44.391 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:44 smithi172 ceph-mon[36402]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:44.391 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:44 smithi172 ceph-mon[36402]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T15:11:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:11:44.594+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe8c81a1e20 con 0x7fe8b40a2c20 2024-05-21T15:11:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:44 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi084 2024-05-21T15:11:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:44 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi084 to 5696M 2024-05-21T15:11:44.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:44 smithi078 ceph-mon[31997]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:44.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:44 smithi078 ceph-mon[31997]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T15:11:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:44 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi084 2024-05-21T15:11:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:44 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi084 to 5696M 2024-05-21T15:11:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:44 smithi084 ceph-mon[36334]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:44 smithi084 ceph-mon[36334]: osdmap e53: 8 total, 8 up, 8 in 2024-05-21T15:11:45.510 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:11:45.510 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:11:45.510 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:11:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:11:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:45.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:11:45.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:11:45.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:46.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:46 smithi172 ceph-mon[36402]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:11:46.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:46 smithi172 ceph-mon[36402]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:46.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:46 smithi172 ceph-mon[36402]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T15:11:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:46 smithi078 ceph-mon[31997]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:11:46.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:46 smithi078 ceph-mon[31997]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:46.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:46 smithi078 ceph-mon[31997]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T15:11:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:46 smithi084 ceph-mon[36334]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:11:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:46 smithi084 ceph-mon[36334]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:46 smithi084 ceph-mon[36334]: osdmap e54: 8 total, 8 up, 8 in 2024-05-21T15:11:47.579 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:47 smithi172 ceph-mon[36402]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T15:11:47.579 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:47 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/431210824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]: dispatch 2024-05-21T15:11:47.579 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:47 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]: dispatch 2024-05-21T15:11:47.579 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:47 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]': finished 2024-05-21T15:11:47.579 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:47 smithi172 ceph-mon[36402]: osdmap e56: 9 total, 8 up, 9 in 2024-05-21T15:11:47.579 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:47 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:11:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:47 smithi078 ceph-mon[31997]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T15:11:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:47 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/431210824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]: dispatch 2024-05-21T15:11:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:47 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]: dispatch 2024-05-21T15:11:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:47 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]': finished 2024-05-21T15:11:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:47 smithi078 ceph-mon[31997]: osdmap e56: 9 total, 8 up, 9 in 2024-05-21T15:11:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:47 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:11:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:47 smithi084 ceph-mon[36334]: osdmap e55: 8 total, 8 up, 8 in 2024-05-21T15:11:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:47 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/431210824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]: dispatch 2024-05-21T15:11:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:47 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]: dispatch 2024-05-21T15:11:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:47 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26bbfd7f-c916-46e3-8908-19f65e5350a3"}]': finished 2024-05-21T15:11:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:47 smithi084 ceph-mon[36334]: osdmap e56: 9 total, 8 up, 9 in 2024-05-21T15:11:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:47 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:11:48.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:48 smithi078 ceph-mon[31997]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:48 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/1806943932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:11:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:48 smithi084 ceph-mon[36334]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:48 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/1806943932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:11:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:48 smithi172 ceph-mon[36402]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:48 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/1806943932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:11:50.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:50 smithi078 ceph-mon[31997]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:50 smithi084 ceph-mon[36334]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:50.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:50 smithi172 ceph-mon[36402]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:52 smithi078 ceph-mon[31997]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:52.601 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:52 smithi172 ceph-mon[36402]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:52 smithi084 ceph-mon[36334]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:54.586 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:54 smithi172 ceph-mon[36402]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:54.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:54 smithi078 ceph-mon[31997]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:54.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:54 smithi084 ceph-mon[36334]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:55.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T15:11:55.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:55.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T15:11:55.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:55.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T15:11:55.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:11:56.520 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:56 smithi172 ceph-mon[36402]: Deploying daemon osd.8 on smithi172 2024-05-21T15:11:56.520 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:56 smithi172 ceph-mon[36402]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:56.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:56 smithi078 ceph-mon[31997]: Deploying daemon osd.8 on smithi172 2024-05-21T15:11:56.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:56 smithi078 ceph-mon[31997]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:56 smithi084 ceph-mon[36334]: Deploying daemon osd.8 on smithi172 2024-05-21T15:11:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:56 smithi084 ceph-mon[36334]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:58.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:11:58 smithi078 ceph-mon[31997]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:11:58 smithi084 ceph-mon[36334]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:11:58.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:11:58 smithi172 ceph-mon[36402]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:00.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:00 smithi078 ceph-mon[31997]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:00 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:00 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:00 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:00 smithi084 ceph-mon[36334]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:00 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:00 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:00 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:00.653 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:00 smithi172 ceph-mon[36402]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:00.653 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:00 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:00.653 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:00 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:00.653 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:00 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:02.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:02 smithi078 ceph-mon[31997]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:02.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:02 smithi084 ceph-mon[36334]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:02.649 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:02 smithi172 ceph-mon[36402]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.812+0000 7fe8bf7fe700 1 -- 172.21.15.172:0/1437134835 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe8c81a1e20 con 0x7fe8b40a2c20 2024-05-21T15:12:03.816 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 8 on host 'smithi172' 2024-05-21T15:12:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8b40a2c20 msgr2=0x7fe8b40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8b40a2c20 0x7fe8b40a5130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe8c819a8b0 tx=0x7fe8b800a680 comp rx=0 tx=0).stop 2024-05-21T15:12:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c81020d0 msgr2=0x7fe8c8199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c81020d0 0x7fe8c8199800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4007ee0 tx=0x7fe8c400d3b0 comp rx=0 tx=0).stop 2024-05-21T15:12:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 shutdown_connections 2024-05-21T15:12:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8b40a2c20 0x7fe8b40a5130 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8c8103ba0 0x7fe8c81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8c81020d0 0x7fe8c8199800 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 --2- 172.21.15.172:0/1437134835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8c8103200 0x7fe8c8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.814+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 >> 172.21.15.172:0/1437134835 conn(0x7fe8c80fd710 msgr2=0x7fe8c8074870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:12:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.815+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 shutdown_connections 2024-05-21T15:12:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:12:03.815+0000 7fe8d054f700 1 -- 172.21.15.172:0/1437134835 wait complete. 2024-05-21T15:12:04.173 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.173 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.173 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:04.173 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:04.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:04.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:04.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:12:04.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:04 smithi172 ceph-mon[36402]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:04.351 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:03 smithi078 ceph-mon[31997]: from='osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:12:04.351 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:04 smithi078 ceph-mon[31997]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:12:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:04 smithi084 ceph-mon[36334]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T15:12:04.498 DEBUG:teuthology.orchestra.run.smithi172:osd.8> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.8.service 2024-05-21T15:12:04.500 INFO:tasks.cephadm:Deploying osd.9 on smithi078 with /dev/vg_nvme/lv_1... 2024-05-21T15:12:04.500 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T15:12:04.746 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:12:05.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:05 smithi172 ceph-mon[36402]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T15:12:05.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:05 smithi172 ceph-mon[36402]: from='osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:12:05.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:05 smithi172 ceph-mon[36402]: osdmap e57: 9 total, 8 up, 9 in 2024-05-21T15:12:05.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:05.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:05 smithi172 ceph-mon[36402]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:12:05.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:05 smithi172 ceph-mon[36402]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:05.935 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:12:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:12:05.818+0000 7f5d020f2700 -1 osd.8 0 waiting for initial osdmap 2024-05-21T15:12:05.935 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:12:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:12:05.833+0000 7f5cfe6a8700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:12:06.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:05 smithi078 ceph-mon[31997]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T15:12:06.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:05 smithi078 ceph-mon[31997]: from='osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:12:06.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:05 smithi078 ceph-mon[31997]: osdmap e57: 9 total, 8 up, 9 in 2024-05-21T15:12:06.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:06.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:05 smithi078 ceph-mon[31997]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:12:06.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:05 smithi078 ceph-mon[31997]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:06.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:05 smithi084 ceph-mon[36334]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T15:12:06.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:05 smithi084 ceph-mon[36334]: from='osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:12:06.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:05 smithi084 ceph-mon[36334]: osdmap e57: 9 total, 8 up, 9 in 2024-05-21T15:12:06.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:06.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:05 smithi084 ceph-mon[36334]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:12:06.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:05 smithi084 ceph-mon[36334]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-21T15:12:06.699 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T15:12:06.699 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T15:12:06.699 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-05-21T15:12:06.699 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-05-21T15:12:06.699 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115301 s, 909 MB/s 2024-05-21T15:12:06.699 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-05-21T15:12:06.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:06 smithi172 ceph-mon[36402]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:12:06.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:06 smithi172 ceph-mon[36402]: osdmap e58: 9 total, 8 up, 9 in 2024-05-21T15:12:06.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:06 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:06.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:06 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:06.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:06 smithi172 ceph-mon[36402]: osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] boot 2024-05-21T15:12:06.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:06 smithi172 ceph-mon[36402]: osdmap e59: 9 total, 9 up, 9 in 2024-05-21T15:12:06.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:06 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:07.078 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:06 smithi078 ceph-mon[31997]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:12:07.078 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:06 smithi078 ceph-mon[31997]: osdmap e58: 9 total, 8 up, 9 in 2024-05-21T15:12:07.078 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:06 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:07.078 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:06 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:07.079 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:06 smithi078 ceph-mon[31997]: osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] boot 2024-05-21T15:12:07.079 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:06 smithi078 ceph-mon[31997]: osdmap e59: 9 total, 9 up, 9 in 2024-05-21T15:12:07.079 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:06 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:07.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:06 smithi084 ceph-mon[36334]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:12:07.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:06 smithi084 ceph-mon[36334]: osdmap e58: 9 total, 8 up, 9 in 2024-05-21T15:12:07.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:06 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:07.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:06 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:07.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:06 smithi084 ceph-mon[36334]: osd.8 [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] boot 2024-05-21T15:12:07.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:06 smithi084 ceph-mon[36334]: osdmap e59: 9 total, 9 up, 9 in 2024-05-21T15:12:07.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:06 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:12:07.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-05-21T15:12:07.750 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:12:08.009 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: osdmap e60: 9 total, 9 up, 9 in 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.010 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:08.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:08.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:07 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:12:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:12:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: osdmap e60: 9 total, 9 up, 9 in 2024-05-21T15:12:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:08.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:08.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:07 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: osdmap e60: 9 total, 9 up, 9 in 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:08.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:07 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:09.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:09 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi172 2024-05-21T15:12:09.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:09 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi172 to 5696M 2024-05-21T15:12:09.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:09 smithi078 ceph-mon[31997]: osdmap e61: 9 total, 9 up, 9 in 2024-05-21T15:12:09.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 -- 172.21.15.78:0/2420009998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4101520 msgr2=0x7f21f4101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:09.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 --2- 172.21.15.78:0/2420009998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4101520 0x7f21f4101930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f21f000b0a0 tx=0x7f21f000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:12:09.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 -- 172.21.15.78:0/2420009998 shutdown_connections 2024-05-21T15:12:09.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 --2- 172.21.15.78:0/2420009998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f4109000 0x7f21f410b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:09.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 --2- 172.21.15.78:0/2420009998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4101520 0x7f21f4101930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:09.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 --2- 172.21.15.78:0/2420009998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f41066a0 0x7f21f4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:09.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 -- 172.21.15.78:0/2420009998 >> 172.21.15.78:0/2420009998 conn(0x7f21f40fce80 msgr2=0x7f21f40ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:12:09.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 -- 172.21.15.78:0/2420009998 shutdown_connections 2024-05-21T15:12:09.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.536+0000 7f21fb583700 1 -- 172.21.15.78:0/2420009998 wait complete. 2024-05-21T15:12:09.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21fb583700 1 Processor -- start 2024-05-21T15:12:09.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21fb583700 1 -- start start 2024-05-21T15:12:09.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21fb583700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f4101520 0x7f21f4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:12:09.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21fb583700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f41066a0 0x7f21f4199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:12:09.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21fb583700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4109000 0x7f21f41a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21fb583700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21f410ed70 con 0x7f21f4109000 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21fb583700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f21f410ebf0 con 0x7f21f41066a0 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21fb583700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21f410eef0 con 0x7f21f4101520 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21f9b20700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4109000 0x7f21f41a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21f9b20700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4109000 0x7f21f41a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40920/0 (socket says 172.21.15.78:40920) 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21f9b20700 1 -- 172.21.15.78:0/3694444506 learned_addr learned my addr 172.21.15.78:0/3694444506 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.537+0000 7f21f931f700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f4101520 0x7f21f4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21f8b1e700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f41066a0 0x7f21f4199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21f9b20700 1 -- 172.21.15.78:0/3694444506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f4101520 msgr2=0x7f21f4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21f9b20700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f4101520 0x7f21f4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21f9b20700 1 -- 172.21.15.78:0/3694444506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f41066a0 msgr2=0x7f21f4199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21f9b20700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f41066a0 0x7f21f4199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21f9b20700 1 -- 172.21.15.78:0/3694444506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f41a13b0 con 0x7f21f4109000 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21f8b1e700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f41066a0 0x7f21f4199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21f9b20700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4109000 0x7f21f41a0d30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005fd0 tx=0x7f21ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21ea7fc700 1 -- 172.21.15.78:0/3694444506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec016070 con 0x7f21f4109000 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21ea7fc700 1 -- 172.21.15.78:0/3694444506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec009e10 con 0x7f21f4109000 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f41a16a0 con 0x7f21f4109000 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21ea7fc700 1 -- 172.21.15.78:0/3694444506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00f3c0 con 0x7f21f4109000 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.538+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a1be0 con 0x7f21f4109000 2024-05-21T15:12:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.539+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f4050170 con 0x7f21f4109000 2024-05-21T15:12:09.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.540+0000 7f21ea7fc700 1 -- 172.21.15.78:0/3694444506 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f21ec019070 con 0x7f21f4109000 2024-05-21T15:12:09.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.541+0000 7f21ea7fc700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f21e00a2c20 0x7f21e00a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:12:09.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.541+0000 7f21f931f700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f21e00a2c20 0x7f21e00a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:12:09.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.541+0000 7f21ea7fc700 1 -- 172.21.15.78:0/3694444506 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7f21ec0ce140 con 0x7f21f4109000 2024-05-21T15:12:09.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.542+0000 7f21f931f700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f21e00a2c20 0x7f21e00a5130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009fd0 tx=0x7f21f000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:12:09.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.545+0000 7f21ea7fc700 1 -- 172.21.15.78:0/3694444506 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f21ec09ba50 con 0x7f21f4109000 2024-05-21T15:12:09.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:09 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi172 2024-05-21T15:12:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:09 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi172 to 5696M 2024-05-21T15:12:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:09 smithi084 ceph-mon[36334]: osdmap e61: 9 total, 9 up, 9 in 2024-05-21T15:12:09.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:09 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi172 2024-05-21T15:12:09.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:09 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi172 to 5696M 2024-05-21T15:12:09.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:09 smithi172 ceph-mon[36402]: osdmap e61: 9 total, 9 up, 9 in 2024-05-21T15:12:09.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:09.749+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f21f4062a80 con 0x7f21e00a2c20 2024-05-21T15:12:10.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:10 smithi078 ceph-mon[31997]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:10.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:10 smithi078 ceph-mon[31997]: osdmap e62: 9 total, 9 up, 9 in 2024-05-21T15:12:10.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:12:10.588 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:12:10.588 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:10 smithi084 ceph-mon[36334]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:10 smithi084 ceph-mon[36334]: osdmap e62: 9 total, 9 up, 9 in 2024-05-21T15:12:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:12:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:12:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:10.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:10 smithi172 ceph-mon[36402]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:10.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:10 smithi172 ceph-mon[36402]: osdmap e62: 9 total, 9 up, 9 in 2024-05-21T15:12:10.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:12:10.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:12:10.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:11.596 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:11 smithi078 ceph-mon[31997]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:12:11.596 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:11 smithi078 ceph-mon[31997]: osdmap e63: 9 total, 9 up, 9 in 2024-05-21T15:12:11.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:11 smithi084 ceph-mon[36334]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:12:11.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:11 smithi084 ceph-mon[36334]: osdmap e63: 9 total, 9 up, 9 in 2024-05-21T15:12:11.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:11 smithi172 ceph-mon[36402]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:12:11.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:11 smithi172 ceph-mon[36402]: osdmap e63: 9 total, 9 up, 9 in 2024-05-21T15:12:12.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:12 smithi078 ceph-mon[31997]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:12 smithi084 ceph-mon[36334]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:12.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:12 smithi172 ceph-mon[36402]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:13.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:13 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3462819185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92534fe2-a86c-45e6-b3f1-b436d49fb37d"}]: dispatch 2024-05-21T15:12:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:13 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3462819185' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92534fe2-a86c-45e6-b3f1-b436d49fb37d"}]': finished 2024-05-21T15:12:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:13 smithi078 ceph-mon[31997]: osdmap e64: 10 total, 9 up, 10 in 2024-05-21T15:12:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:13.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:13 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/871028296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:12:13.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:13 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3462819185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92534fe2-a86c-45e6-b3f1-b436d49fb37d"}]: dispatch 2024-05-21T15:12:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:13 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3462819185' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92534fe2-a86c-45e6-b3f1-b436d49fb37d"}]': finished 2024-05-21T15:12:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:13 smithi084 ceph-mon[36334]: osdmap e64: 10 total, 9 up, 10 in 2024-05-21T15:12:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:13 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:13 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/871028296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:12:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:13 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3462819185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92534fe2-a86c-45e6-b3f1-b436d49fb37d"}]: dispatch 2024-05-21T15:12:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:13 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3462819185' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92534fe2-a86c-45e6-b3f1-b436d49fb37d"}]': finished 2024-05-21T15:12:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:13 smithi172 ceph-mon[36402]: osdmap e64: 10 total, 9 up, 10 in 2024-05-21T15:12:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:13 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/871028296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:12:14.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:14 smithi078 ceph-mon[31997]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-21T15:12:14.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:14 smithi084 ceph-mon[36334]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-21T15:12:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:14 smithi172 ceph-mon[36402]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-21T15:12:16.555 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:16 smithi078 ceph-mon[31997]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:12:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:16 smithi084 ceph-mon[36334]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:12:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:16 smithi172 ceph-mon[36402]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:12:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:18 smithi078 ceph-mon[31997]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:12:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:18 smithi084 ceph-mon[36334]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:12:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:18 smithi172 ceph-mon[36402]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:12:20.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:20 smithi078 ceph-mon[31997]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T15:12:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:20 smithi084 ceph-mon[36334]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T15:12:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:20 smithi172 ceph-mon[36402]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T15:12:21.480 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:21 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T15:12:21.480 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:21 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:21.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:21 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T15:12:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:21 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:21 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T15:12:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:21 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:22.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:22 smithi078 ceph-mon[31997]: Deploying daemon osd.9 on smithi078 2024-05-21T15:12:22.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:22 smithi078 ceph-mon[31997]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:22 smithi084 ceph-mon[36334]: Deploying daemon osd.9 on smithi078 2024-05-21T15:12:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:22 smithi084 ceph-mon[36334]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:12:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:22 smithi172 ceph-mon[36402]: Deploying daemon osd.9 on smithi078 2024-05-21T15:12:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:22 smithi172 ceph-mon[36402]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T15:12:24.353 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:24 smithi078 ceph-mon[31997]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:24 smithi084 ceph-mon[36334]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:24 smithi172 ceph-mon[36402]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:26.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:26 smithi078 ceph-mon[31997]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:26.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:26 smithi084 ceph-mon[36334]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:26.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:26 smithi172 ceph-mon[36402]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:27.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:27 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:27.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:27 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:27.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:27 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:27.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:27 smithi078 ceph-mon[31997]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:27.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:27 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:27.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:27 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:27.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:27 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:27.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:27 smithi084 ceph-mon[36334]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:27.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:27 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:27.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:27 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:27 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:27 smithi172 ceph-mon[36402]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:30.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.108+0000 7f21ea7fc700 1 -- 172.21.15.78:0/3694444506 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f21f4062a80 con 0x7f21e00a2c20 2024-05-21T15:12:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 9 on host 'smithi078' 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f21e00a2c20 msgr2=0x7f21e00a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f21e00a2c20 0x7f21e00a5130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009fd0 tx=0x7f21f000b7b0 comp rx=0 tx=0).stop 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4109000 msgr2=0x7f21f41a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4109000 0x7f21f41a0d30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005fd0 tx=0x7f21ec00dc30 comp rx=0 tx=0).stop 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 shutdown_connections 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f21e00a2c20 0x7f21e00a5130 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4109000 0x7f21f41a0d30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21f4101520 0x7f21f4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 --2- 172.21.15.78:0/3694444506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21f41066a0 0x7f21f4199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 >> 172.21.15.78:0/3694444506 conn(0x7f21f40fce80 msgr2=0x7f21f4107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 shutdown_connections 2024-05-21T15:12:30.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:12:30.111+0000 7f21fb583700 1 -- 172.21.15.78:0/3694444506 wait complete. 2024-05-21T15:12:30.351 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:30 smithi078 ceph-mon[31997]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:30.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:30.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:30 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:30 smithi084 ceph-mon[36334]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:30 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:30 smithi172 ceph-mon[36402]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:30 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:30.811 DEBUG:teuthology.orchestra.run.smithi078:osd.9> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.9.service 2024-05-21T15:12:30.814 INFO:tasks.cephadm:Deploying osd.10 on smithi084 with /dev/vg_nvme/lv_1... 2024-05-21T15:12:30.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T15:12:31.062 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:12:31.501 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:31 smithi084 ceph-mon[36334]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T15:12:31.501 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:31 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:31.502 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:31 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:31.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:31 smithi078 ceph-mon[31997]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T15:12:31.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:31.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:31 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:31.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:31 smithi172 ceph-mon[36402]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T15:12:31.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:31 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:31.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:31 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:32.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:32 smithi084 ceph-mon[36334]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:32.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:32 smithi084 ceph-mon[36334]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T15:12:32.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:32 smithi084 ceph-mon[36334]: osdmap e65: 10 total, 9 up, 10 in 2024-05-21T15:12:32.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:32 smithi084 ceph-mon[36334]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:12:32.431 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:32 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:32.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:32 smithi078 ceph-mon[31997]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:32 smithi078 ceph-mon[31997]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T15:12:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:32 smithi078 ceph-mon[31997]: osdmap e65: 10 total, 9 up, 10 in 2024-05-21T15:12:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:32 smithi078 ceph-mon[31997]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:12:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:32 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:32 smithi172 ceph-mon[36402]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:32 smithi172 ceph-mon[36402]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T15:12:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:32 smithi172 ceph-mon[36402]: osdmap e65: 10 total, 9 up, 10 in 2024-05-21T15:12:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:32 smithi172 ceph-mon[36402]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-05-21T15:12:32.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:32 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:32.974 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T15:12:32.974 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T15:12:32.974 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-05-21T15:12:32.974 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-05-21T15:12:32.974 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128472 s, 816 MB/s 2024-05-21T15:12:32.974 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-05-21T15:12:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:33 smithi078 ceph-mon[31997]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:12:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:33 smithi078 ceph-mon[31997]: osdmap e66: 10 total, 9 up, 10 in 2024-05-21T15:12:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:33 smithi078 ceph-mon[31997]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' 2024-05-21T15:12:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.349 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:12:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:12:33.137+0000 7ffa336a5700 -1 osd.9 0 waiting for initial osdmap 2024-05-21T15:12:33.349 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:12:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:12:33.154+0000 7ffa2bc53700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:12:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:33 smithi084 ceph-mon[36334]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:12:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:33 smithi084 ceph-mon[36334]: osdmap e66: 10 total, 9 up, 10 in 2024-05-21T15:12:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:33 smithi084 ceph-mon[36334]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' 2024-05-21T15:12:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_1 2024-05-21T15:12:33.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:33 smithi172 ceph-mon[36402]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-05-21T15:12:33.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:33 smithi172 ceph-mon[36402]: osdmap e66: 10 total, 9 up, 10 in 2024-05-21T15:12:33.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:33 smithi172 ceph-mon[36402]: from='osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877]' entity='osd.9' 2024-05-21T15:12:33.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:33.893 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:12:34.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:34 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:12:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:34 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:12:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:34 smithi078 ceph-mon[31997]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:34 smithi078 ceph-mon[31997]: osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] boot 2024-05-21T15:12:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:34 smithi078 ceph-mon[31997]: osdmap e67: 10 total, 10 up, 10 in 2024-05-21T15:12:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:34 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:34 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:12:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:34 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:12:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:34 smithi084 ceph-mon[36334]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:34 smithi084 ceph-mon[36334]: osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] boot 2024-05-21T15:12:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:34 smithi084 ceph-mon[36334]: osdmap e67: 10 total, 10 up, 10 in 2024-05-21T15:12:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:34 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:34 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:12:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:34 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:12:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:34 smithi172 ceph-mon[36402]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-05-21T15:12:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:34 smithi172 ceph-mon[36402]: osd.9 [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] boot 2024-05-21T15:12:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:34 smithi172 ceph-mon[36402]: osdmap e67: 10 total, 10 up, 10 in 2024-05-21T15:12:34.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:34 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:12:35.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 -- 172.21.15.84:0/1573690080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 msgr2=0x7f2d24076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:35.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 --2- 172.21.15.84:0/1573690080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 0x7f2d24076520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009990 tx=0x7f2d18009ca0 comp rx=0 tx=0).stop 2024-05-21T15:12:35.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 -- 172.21.15.84:0/1573690080 shutdown_connections 2024-05-21T15:12:35.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 --2- 172.21.15.84:0/1573690080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d2410b1b0 0x7f2d2410d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:35.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 --2- 172.21.15.84:0/1573690080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d240776f0 0x7f2d24075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:35.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 --2- 172.21.15.84:0/1573690080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 0x7f2d24076520 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:35.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 -- 172.21.15.84:0/1573690080 >> 172.21.15.84:0/1573690080 conn(0x7f2d240fe850 msgr2=0x7f2d24100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:12:35.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 -- 172.21.15.84:0/1573690080 shutdown_connections 2024-05-21T15:12:35.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.505+0000 7f2d29c53700 1 -- 172.21.15.84:0/1573690080 wait complete. 2024-05-21T15:12:35.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d29c53700 1 Processor -- start 2024-05-21T15:12:35.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d29c53700 1 -- start start 2024-05-21T15:12:35.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d29c53700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 0x7f2d2419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d29c53700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d240776f0 0x7f2d2419c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d29c53700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d2410b1b0 0x7f2d241a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d29c53700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241110b0 con 0x7f2d240776f0 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d29c53700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24110f30 con 0x7f2d24076090 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d29c53700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24111230 con 0x7f2d2410b1b0 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d237fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 0x7f2d2419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d237fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 0x7f2d2419bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39934/0 (socket says 172.21.15.84:39934) 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d237fe700 1 -- 172.21.15.84:0/12443062 learned_addr learned my addr 172.21.15.84:0/12443062 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d22ffd700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d240776f0 0x7f2d2419c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d237fe700 1 -- 172.21.15.84:0/12443062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d2410b1b0 msgr2=0x7f2d241a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d237fe700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d2410b1b0 0x7f2d241a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d237fe700 1 -- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d240776f0 msgr2=0x7f2d2419c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:35.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d237fe700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d240776f0 0x7f2d2419c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:35.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.506+0000 7f2d237fe700 1 -- 172.21.15.84:0/12443062 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d241a3690 con 0x7f2d24076090 2024-05-21T15:12:35.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.507+0000 7f2d237fe700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 0x7f2d2419bb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d980 tx=0x7f2d1400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:12:35.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.507+0000 7f2d20ff9700 1 -- 172.21.15.84:0/12443062 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14017070 con 0x7f2d24076090 2024-05-21T15:12:35.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.507+0000 7f2d20ff9700 1 -- 172.21.15.84:0/12443062 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d14009e10 con 0x7f2d24076090 2024-05-21T15:12:35.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.507+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241a3980 con 0x7f2d24076090 2024-05-21T15:12:35.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.507+0000 7f2d20ff9700 1 -- 172.21.15.84:0/12443062 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400f620 con 0x7f2d24076090 2024-05-21T15:12:35.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.507+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a3ec0 con 0x7f2d24076090 2024-05-21T15:12:35.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.508+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d24050170 con 0x7f2d24076090 2024-05-21T15:12:35.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.509+0000 7f2d20ff9700 1 -- 172.21.15.84:0/12443062 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2d14010460 con 0x7f2d24076090 2024-05-21T15:12:35.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.510+0000 7f2d20ff9700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2d0c0a2c20 0x7f2d0c0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:12:35.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.510+0000 7f2d20ff9700 1 -- 172.21.15.84:0/12443062 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f2d140ce3f0 con 0x7f2d24076090 2024-05-21T15:12:35.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.510+0000 7f2d22ffd700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2d0c0a2c20 0x7f2d0c0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:12:35.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.510+0000 7f2d22ffd700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2d0c0a2c20 0x7f2d0c0a5130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800b5c0 tx=0x7f2d1801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:12:35.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.513+0000 7f2d20ff9700 1 -- 172.21.15.84:0/12443062 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2d1409bab0 con 0x7f2d24076090 2024-05-21T15:12:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:35.722+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2d2402e0c0 con 0x7f2d0c0a2c20 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi078 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi078 to 4272M 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:35 smithi084 ceph-mon[36334]: osdmap e68: 10 total, 10 up, 10 in 2024-05-21T15:12:35.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi078 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi078 to 4272M 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:35.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:35.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:35.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:35.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:35.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:35 smithi172 ceph-mon[36402]: osdmap e68: 10 total, 10 up, 10 in 2024-05-21T15:12:36.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi078 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi078 to 4272M 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:36.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:36.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:36.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:36.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:36.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:36.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:36.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:35 smithi078 ceph-mon[31997]: osdmap e68: 10 total, 10 up, 10 in 2024-05-21T15:12:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:36 smithi084 ceph-mon[36334]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:12:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:12:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:12:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:36.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:36 smithi172 ceph-mon[36402]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:12:36.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:12:36.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:12:36.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:37.066 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:36 smithi078 ceph-mon[31997]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:12:37.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:12:37.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:12:37.067 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:37.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:37 smithi084 ceph-mon[36334]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:37.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:37 smithi172 ceph-mon[36402]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:38.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:37 smithi078 ceph-mon[31997]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:38.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:38 smithi084 ceph-mon[36334]: from='client.? 172.21.15.84:0/2541550646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]: dispatch 2024-05-21T15:12:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:38 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]: dispatch 2024-05-21T15:12:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:38 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]': finished 2024-05-21T15:12:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:38 smithi084 ceph-mon[36334]: osdmap e69: 11 total, 10 up, 11 in 2024-05-21T15:12:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:38 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:38.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:38 smithi172 ceph-mon[36402]: from='client.? 172.21.15.84:0/2541550646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]: dispatch 2024-05-21T15:12:38.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:38 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]: dispatch 2024-05-21T15:12:38.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:38 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]': finished 2024-05-21T15:12:38.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:38 smithi172 ceph-mon[36402]: osdmap e69: 11 total, 10 up, 11 in 2024-05-21T15:12:38.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:38 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:39.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:38 smithi078 ceph-mon[31997]: from='client.? 172.21.15.84:0/2541550646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]: dispatch 2024-05-21T15:12:39.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:38 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]: dispatch 2024-05-21T15:12:39.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:38 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e378331e-ef86-4d80-a184-6acc0d9283b2"}]': finished 2024-05-21T15:12:39.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:38 smithi078 ceph-mon[31997]: osdmap e69: 11 total, 10 up, 11 in 2024-05-21T15:12:39.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:38 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:39.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:39 smithi172 ceph-mon[36402]: from='client.? 172.21.15.84:0/309110223' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:12:39.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:39 smithi172 ceph-mon[36402]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:40.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:39 smithi078 ceph-mon[31997]: from='client.? 172.21.15.84:0/309110223' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:12:40.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:39 smithi078 ceph-mon[31997]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:40.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:39 smithi084 ceph-mon[36334]: from='client.? 172.21.15.84:0/309110223' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:12:40.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:39 smithi084 ceph-mon[36334]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:42 smithi078 ceph-mon[31997]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:42.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:42 smithi084 ceph-mon[36334]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:42.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:42 smithi172 ceph-mon[36402]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:44.317 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:44 smithi084 ceph-mon[36334]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:44.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:44 smithi078 ceph-mon[31997]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:44 smithi172 ceph-mon[36402]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:46.566 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:46 smithi078 ceph-mon[31997]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:46.567 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:46 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T15:12:46.567 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:46 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:46.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:46 smithi084 ceph-mon[36334]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:46 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T15:12:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:46 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:46 smithi172 ceph-mon[36402]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:46 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T15:12:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:46 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:47.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:47 smithi084 ceph-mon[36334]: Deploying daemon osd.10 on smithi084 2024-05-21T15:12:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:47 smithi078 ceph-mon[31997]: Deploying daemon osd.10 on smithi084 2024-05-21T15:12:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:47 smithi172 ceph-mon[36402]: Deploying daemon osd.10 on smithi084 2024-05-21T15:12:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:48 smithi084 ceph-mon[36334]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:48 smithi078 ceph-mon[31997]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:48 smithi172 ceph-mon[36402]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:50.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:50 smithi078 ceph-mon[31997]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:50 smithi084 ceph-mon[36334]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:50.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:50 smithi172 ceph-mon[36402]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:52.555 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:52 smithi084 ceph-mon[36334]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:52.555 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:52.555 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:52.555 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:52 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:52 smithi078 ceph-mon[31997]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:52 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:52 smithi172 ceph-mon[36402]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:52.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:52 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:54.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:54 smithi084 ceph-mon[36334]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:54.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:54 smithi078 ceph-mon[31997]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:54.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:54 smithi172 ceph-mon[36402]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:55.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.690+0000 7f2d20ff9700 1 -- 172.21.15.84:0/12443062 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2d1800b7d0 con 0x7f2d0c0a2c20 2024-05-21T15:12:55.694 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 10 on host 'smithi084' 2024-05-21T15:12:55.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2d0c0a2c20 msgr2=0x7f2d0c0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2d0c0a2c20 0x7f2d0c0a5130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800b5c0 tx=0x7f2d1801a040 comp rx=0 tx=0).stop 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 msgr2=0x7f2d2419bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 0x7f2d2419bb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d980 tx=0x7f2d1400dc90 comp rx=0 tx=0).stop 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 shutdown_connections 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2d0c0a2c20 0x7f2d0c0a5130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d2410b1b0 0x7f2d241a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d24076090 0x7f2d2419bb30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 --2- 172.21.15.84:0/12443062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d240776f0 0x7f2d2419c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.692+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 >> 172.21.15.84:0/12443062 conn(0x7f2d240fe850 msgr2=0x7f2d24100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.693+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 shutdown_connections 2024-05-21T15:12:55.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-05-21T15:12:55.693+0000 7f2d29c53700 1 -- 172.21.15.84:0/12443062 wait complete. 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:55.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:55.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:12:55.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:55 smithi084 ceph-mon[36334]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:56.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:12:56.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:55 smithi078 ceph-mon[31997]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:12:56.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:12:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:55 smithi172 ceph-mon[36402]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T15:12:56.327 DEBUG:teuthology.orchestra.run.smithi084:osd.10> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.10.service 2024-05-21T15:12:56.329 INFO:tasks.cephadm:Deploying osd.11 on smithi172 with /dev/vg_nvme/lv_1... 2024-05-21T15:12:56.329 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T15:12:56.575 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:12:57.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:57 smithi084 ceph-mon[36334]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T15:12:57.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:57 smithi084 ceph-mon[36334]: osdmap e70: 11 total, 10 up, 11 in 2024-05-21T15:12:57.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:57 smithi084 ceph-mon[36334]: from='osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:12:57.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:57 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:57.798 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:57 smithi084 ceph-mon[36334]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:12:57.798 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:57 smithi084 ceph-mon[36334]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:57.798 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:12:57 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:12:57.693+0000 7f76d56c9700 -1 osd.10 0 waiting for initial osdmap 2024-05-21T15:12:57.798 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:12:57 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:12:57.709+0000 7f76d1c7f700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:12:58.018 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:57 smithi172 ceph-mon[36402]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T15:12:58.019 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:57 smithi172 ceph-mon[36402]: osdmap e70: 11 total, 10 up, 11 in 2024-05-21T15:12:58.019 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:57 smithi172 ceph-mon[36402]: from='osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:12:58.019 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:57 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:58.019 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:57 smithi172 ceph-mon[36402]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:12:58.019 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:57 smithi172 ceph-mon[36402]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:58.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:57 smithi078 ceph-mon[31997]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T15:12:58.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:57 smithi078 ceph-mon[31997]: osdmap e70: 11 total, 10 up, 11 in 2024-05-21T15:12:58.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:57 smithi078 ceph-mon[31997]: from='osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:12:58.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:57 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:58.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:57 smithi078 ceph-mon[31997]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-05-21T15:12:58.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:57 smithi078 ceph-mon[31997]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-05-21T15:12:58.573 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T15:12:58.573 INFO:teuthology.orchestra.run.smithi172.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T15:12:58.573 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 10+0 records in 2024-05-21T15:12:58.573 INFO:teuthology.orchestra.run.smithi172.stderr:10+0 records out 2024-05-21T15:12:58.573 INFO:teuthology.orchestra.run.smithi172.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126789 s, 827 MB/s 2024-05-21T15:12:58.573 INFO:teuthology.orchestra.run.smithi172.stderr: stderr: 2024-05-21T15:12:58.573 INFO:teuthology.orchestra.run.smithi172.stderr:--> Zapping successful for: 2024-05-21T15:12:59.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:58 smithi084 ceph-mon[36334]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:12:59.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:58 smithi084 ceph-mon[36334]: osdmap e71: 11 total, 10 up, 11 in 2024-05-21T15:12:59.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:58 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:58 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:58 smithi084 ceph-mon[36334]: osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] boot 2024-05-21T15:12:59.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:58 smithi084 ceph-mon[36334]: osdmap e72: 11 total, 11 up, 11 in 2024-05-21T15:12:59.029 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:58 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:58 smithi078 ceph-mon[31997]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:12:59.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:58 smithi078 ceph-mon[31997]: osdmap e71: 11 total, 10 up, 11 in 2024-05-21T15:12:59.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:58 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:58 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:58 smithi078 ceph-mon[31997]: osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] boot 2024-05-21T15:12:59.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:58 smithi078 ceph-mon[31997]: osdmap e72: 11 total, 11 up, 11 in 2024-05-21T15:12:59.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:58 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.173 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:58 smithi172 ceph-mon[36402]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-05-21T15:12:59.173 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:58 smithi172 ceph-mon[36402]: osdmap e71: 11 total, 10 up, 11 in 2024-05-21T15:12:59.173 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:58 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:58 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:58 smithi172 ceph-mon[36402]: osd.10 [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] boot 2024-05-21T15:12:59.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:58 smithi172 ceph-mon[36402]: osdmap e72: 11 total, 11 up, 11 in 2024-05-21T15:12:59.174 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:58 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:12:59.201 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_1 2024-05-21T15:12:59.438 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:12:59.843 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:12:59.843 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:12:59.843 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:12:59.844 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:59.844 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:59.844 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:59.844 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:59.844 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:59.844 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:12:59.844 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:59.844 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:12:59.845 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:12:59.845 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:12:59.845 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:12:59 smithi172 ceph-mon[36402]: osdmap e73: 11 total, 11 up, 11 in 2024-05-21T15:13:00.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:13:00.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:13:00.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:00.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:00.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:00.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:00.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:00.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:00.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:00.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:00.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:00.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:00.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:00.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:12:59 smithi078 ceph-mon[31997]: osdmap e73: 11 total, 11 up, 11 in 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:00.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:00.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:00.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:00.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:12:59 smithi084 ceph-mon[36334]: osdmap e73: 11 total, 11 up, 11 in 2024-05-21T15:13:00.818 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:00 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi084 2024-05-21T15:13:00.818 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:00 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi084 to 4272M 2024-05-21T15:13:00.818 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:00 smithi172 ceph-mon[36402]: osdmap e74: 11 total, 11 up, 11 in 2024-05-21T15:13:01.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 -- 172.21.15.172:0/1463891158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84481020d0 msgr2=0x7f84481024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:01.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 --2- 172.21.15.172:0/1463891158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84481020d0 0x7f84481024e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8438009990 tx=0x7f8438009ca0 comp rx=0 tx=0).stop 2024-05-21T15:13:01.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 -- 172.21.15.172:0/1463891158 shutdown_connections 2024-05-21T15:13:01.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 --2- 172.21.15.172:0/1463891158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8448103ba0 0x7f844810a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:01.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 --2- 172.21.15.172:0/1463891158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84481020d0 0x7f84481024e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:01.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 --2- 172.21.15.172:0/1463891158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8448103200 0x7f8448103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:01.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 -- 172.21.15.172:0/1463891158 >> 172.21.15.172:0/1463891158 conn(0x7f84480fd710 msgr2=0x7f84480ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:01.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 -- 172.21.15.172:0/1463891158 shutdown_connections 2024-05-21T15:13:01.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 -- 172.21.15.172:0/1463891158 wait complete. 2024-05-21T15:13:01.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 Processor -- start 2024-05-21T15:13:01.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.010+0000 7f844e60a700 1 -- start start 2024-05-21T15:13:01.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f844e60a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84481020d0 0x7f84481998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:01.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f844e60a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8448103200 0x7f8448199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f844e60a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8448103ba0 0x7f84481a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f844e60a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f844810dc90 con 0x7f84481020d0 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f844e60a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f844810db10 con 0x7f8448103ba0 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f844e60a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f844810de10 con 0x7f8448103200 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f84477fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8448103200 0x7f8448199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f84477fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8448103200 0x7f8448199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45030/0 (socket says 172.21.15.172:45030) 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f84477fe700 1 -- 172.21.15.172:0/1166077333 learned_addr learned my addr 172.21.15.172:0/1166077333 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f8447fff700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84481020d0 0x7f84481998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f844cba7700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8448103ba0 0x7f84481a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f84477fe700 1 -- 172.21.15.172:0/1166077333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8448103ba0 msgr2=0x7f84481a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:01.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f84477fe700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8448103ba0 0x7f84481a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f84477fe700 1 -- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84481020d0 msgr2=0x7f84481998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f84477fe700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84481020d0 0x7f84481998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.011+0000 7f84477fe700 1 -- 172.21.15.172:0/1166077333 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84481a1440 con 0x7f8448103200 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.012+0000 7f84477fe700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8448103200 0x7f8448199e20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f843c00d950 tx=0x7f843c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.012+0000 7f84457fa700 1 -- 172.21.15.172:0/1166077333 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c016070 con 0x7f8448103200 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.012+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84481a1730 con 0x7f8448103200 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.012+0000 7f84457fa700 1 -- 172.21.15.172:0/1166077333 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c00fd70 con 0x7f8448103200 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.012+0000 7f84457fa700 1 -- 172.21.15.172:0/1166077333 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c00e610 con 0x7f8448103200 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.012+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84481a1cc0 con 0x7f8448103200 2024-05-21T15:13:01.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.013+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8448050170 con 0x7f8448103200 2024-05-21T15:13:01.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.014+0000 7f84457fa700 1 -- 172.21.15.172:0/1166077333 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f843c019070 con 0x7f8448103200 2024-05-21T15:13:01.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.015+0000 7f84457fa700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84300a2c20 0x7f84300a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:01.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.015+0000 7f84457fa700 1 -- 172.21.15.172:0/1166077333 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f843c0ce4a0 con 0x7f8448103200 2024-05-21T15:13:01.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.015+0000 7f8447fff700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84300a2c20 0x7f84300a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:01.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.015+0000 7f8447fff700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84300a2c20 0x7f84300a5130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8438006010 tx=0x7f843801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:01.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.018+0000 7f84457fa700 1 -- 172.21.15.172:0/1166077333 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f843c09ba30 con 0x7f8448103200 2024-05-21T15:13:01.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:00 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi084 2024-05-21T15:13:01.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:00 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi084 to 4272M 2024-05-21T15:13:01.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:00 smithi078 ceph-mon[31997]: osdmap e74: 11 total, 11 up, 11 in 2024-05-21T15:13:01.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:00 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi084 2024-05-21T15:13:01.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:00 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi084 to 4272M 2024-05-21T15:13:01.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:00 smithi084 ceph-mon[36334]: osdmap e74: 11 total, 11 up, 11 in 2024-05-21T15:13:01.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:01.226+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f84480ff710 con 0x7f84300a2c20 2024-05-21T15:13:01.968 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:01 smithi172 ceph-mon[36402]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:01.968 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:01 smithi172 ceph-mon[36402]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:13:01.968 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:01 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:13:01.968 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:01 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:13:01.969 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:01 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:01.969 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:01 smithi172 ceph-mon[36402]: osdmap e75: 11 total, 11 up, 11 in 2024-05-21T15:13:02.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:01 smithi078 ceph-mon[31997]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:02.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:01 smithi078 ceph-mon[31997]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:13:02.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:01 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:13:02.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:01 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:13:02.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:01 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:02.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:01 smithi078 ceph-mon[31997]: osdmap e75: 11 total, 11 up, 11 in 2024-05-21T15:13:02.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:01 smithi084 ceph-mon[36334]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:02.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:01 smithi084 ceph-mon[36334]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:13:02.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:01 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T15:13:02.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:01 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T15:13:02.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:01 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:02.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:01 smithi084 ceph-mon[36334]: osdmap e75: 11 total, 11 up, 11 in 2024-05-21T15:13:03.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:03 smithi078 ceph-mon[31997]: osdmap e76: 11 total, 11 up, 11 in 2024-05-21T15:13:03.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:03 smithi084 ceph-mon[36334]: osdmap e76: 11 total, 11 up, 11 in 2024-05-21T15:13:03.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:03 smithi172 ceph-mon[36402]: osdmap e76: 11 total, 11 up, 11 in 2024-05-21T15:13:04.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:04 smithi078 ceph-mon[31997]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-21T15:13:04.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:04 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/2272434485' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]: dispatch 2024-05-21T15:13:04.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:04 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]: dispatch 2024-05-21T15:13:04.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:04 smithi078 ceph-mon[31997]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]': finished 2024-05-21T15:13:04.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:04 smithi078 ceph-mon[31997]: osdmap e77: 12 total, 11 up, 12 in 2024-05-21T15:13:04.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:04 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:04.506 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:04 smithi172 ceph-mon[36402]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-21T15:13:04.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:04 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/2272434485' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]: dispatch 2024-05-21T15:13:04.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:04 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]: dispatch 2024-05-21T15:13:04.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:04 smithi172 ceph-mon[36402]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]': finished 2024-05-21T15:13:04.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:04 smithi172 ceph-mon[36402]: osdmap e77: 12 total, 11 up, 12 in 2024-05-21T15:13:04.507 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:04 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:04 smithi084 ceph-mon[36334]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-21T15:13:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:04 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/2272434485' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]: dispatch 2024-05-21T15:13:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:04 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]: dispatch 2024-05-21T15:13:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:04 smithi084 ceph-mon[36334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07bc1da2-963e-42e0-9b37-16a9a1be16c5"}]': finished 2024-05-21T15:13:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:04 smithi084 ceph-mon[36334]: osdmap e77: 12 total, 11 up, 12 in 2024-05-21T15:13:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:04 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:05.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:05 smithi078 ceph-mon[31997]: from='client.? 172.21.15.172:0/2852142179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:13:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:05 smithi084 ceph-mon[36334]: from='client.? 172.21.15.172:0/2852142179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:13:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:05 smithi172 ceph-mon[36402]: from='client.? 172.21.15.172:0/2852142179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T15:13:06.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:06 smithi078 ceph-mon[31997]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-05-21T15:13:06.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:06 smithi084 ceph-mon[36334]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-05-21T15:13:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:06 smithi172 ceph-mon[36402]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-05-21T15:13:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:08 smithi172 ceph-mon[36402]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:13:08.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:08 smithi078 ceph-mon[31997]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:13:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:08 smithi084 ceph-mon[36334]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T15:13:10.372 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:10 smithi172 ceph-mon[36402]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:13:10.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:10 smithi078 ceph-mon[31997]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:13:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:10 smithi084 ceph-mon[36334]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T15:13:12.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:12 smithi078 ceph-mon[31997]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T15:13:12.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T15:13:12.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:12.600 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:12 smithi172 ceph-mon[36402]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T15:13:12.600 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T15:13:12.600 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:12 smithi084 ceph-mon[36334]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T15:13:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T15:13:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:13.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:13 smithi078 ceph-mon[31997]: Deploying daemon osd.11 on smithi172 2024-05-21T15:13:13.616 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:13 smithi172 ceph-mon[36402]: Deploying daemon osd.11 on smithi172 2024-05-21T15:13:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:13 smithi084 ceph-mon[36334]: Deploying daemon osd.11 on smithi172 2024-05-21T15:13:14.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:14 smithi172 ceph-mon[36402]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:14 smithi078 ceph-mon[31997]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:14 smithi084 ceph-mon[36334]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:16.557 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:16 smithi172 ceph-mon[36402]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:16.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:16 smithi078 ceph-mon[31997]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:16 smithi084 ceph-mon[36334]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:18 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:18 smithi078 ceph-mon[31997]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:18.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:18 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:18.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:18 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:18.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:18 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:18.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:18 smithi084 ceph-mon[36334]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:18.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:18 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:18.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:18 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:18.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:18 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:18.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:18 smithi172 ceph-mon[36402]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:20 smithi078 ceph-mon[31997]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:20 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:20.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:20 smithi172 ceph-mon[36402]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:20.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:20.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:20.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:20.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:20.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:20 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:20 smithi084 ceph-mon[36334]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:20 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:21.099 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 11 on host 'smithi172' 2024-05-21T15:13:21.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.095+0000 7f84457fa700 1 -- 172.21.15.172:0/1166077333 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f84480ff710 con 0x7f84300a2c20 2024-05-21T15:13:21.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84300a2c20 msgr2=0x7f84300a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:21.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84300a2c20 0x7f84300a5130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8438006010 tx=0x7f843801a040 comp rx=0 tx=0).stop 2024-05-21T15:13:21.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8448103200 msgr2=0x7f8448199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:21.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8448103200 0x7f8448199e20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f843c00d950 tx=0x7f843c00dc60 comp rx=0 tx=0).stop 2024-05-21T15:13:21.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 shutdown_connections 2024-05-21T15:13:21.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84300a2c20 0x7f84300a5130 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:21.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8448103ba0 0x7f84481a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:21.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84481020d0 0x7f84481998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:21.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 --2- 172.21.15.172:0/1166077333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8448103200 0x7f8448199e20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:21.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 >> 172.21.15.172:0/1166077333 conn(0x7f84480fd710 msgr2=0x7f84480ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:21.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 shutdown_connections 2024-05-21T15:13:21.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:21.098+0000 7f844e60a700 1 -- 172.21.15.172:0/1166077333 wait complete. 2024-05-21T15:13:21.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:21 smithi078 ceph-mon[31997]: from='osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:13:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:21 smithi078 ceph-mon[31997]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:13:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:21 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:21 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:21 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:21 smithi084 ceph-mon[36334]: from='osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:13:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:21 smithi084 ceph-mon[36334]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:13:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:21 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:21 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:21 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:21 smithi172 ceph-mon[36402]: from='osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:13:21.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:21 smithi172 ceph-mon[36402]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T15:13:21.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:21 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:21.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:21 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:21.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:21 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:21.801 DEBUG:teuthology.orchestra.run.smithi172:osd.11> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.11.service 2024-05-21T15:13:21.803 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-21T15:13:21.804 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-21T15:13:22.048 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:13:22.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:22 smithi172 ceph-mon[36402]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:22.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:22 smithi172 ceph-mon[36402]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T15:13:22.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:22 smithi172 ceph-mon[36402]: from='osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:13:22.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:22 smithi172 ceph-mon[36402]: osdmap e78: 12 total, 11 up, 12 in 2024-05-21T15:13:22.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:22 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:22.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:22 smithi172 ceph-mon[36402]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:13:22.434 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:13:22 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:13:22.301+0000 7fb194bb7700 -1 osd.11 0 waiting for initial osdmap 2024-05-21T15:13:22.434 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:13:22 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:13:22.318+0000 7fb19116d700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T15:13:22.514 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:22 smithi078 ceph-mon[31997]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:22.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:22 smithi078 ceph-mon[31997]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T15:13:22.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:22 smithi078 ceph-mon[31997]: from='osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:13:22.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:22 smithi078 ceph-mon[31997]: osdmap e78: 12 total, 11 up, 12 in 2024-05-21T15:13:22.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:22 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:22.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:22 smithi078 ceph-mon[31997]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:13:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:22 smithi084 ceph-mon[36334]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:22 smithi084 ceph-mon[36334]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T15:13:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:22 smithi084 ceph-mon[36334]: from='osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:13:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:22 smithi084 ceph-mon[36334]: osdmap e78: 12 total, 11 up, 12 in 2024-05-21T15:13:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:22 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:22 smithi084 ceph-mon[36334]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-05-21T15:13:23.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:23 smithi078 ceph-mon[31997]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:13:23.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:23 smithi078 ceph-mon[31997]: osdmap e79: 12 total, 11 up, 12 in 2024-05-21T15:13:23.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:23 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:23 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:23 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:23 smithi084 ceph-mon[36334]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:13:23.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:23 smithi084 ceph-mon[36334]: osdmap e79: 12 total, 11 up, 12 in 2024-05-21T15:13:23.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:23 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:23 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:23 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:23 smithi172 ceph-mon[36402]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-05-21T15:13:23.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:23 smithi172 ceph-mon[36402]: osdmap e79: 12 total, 11 up, 12 in 2024-05-21T15:13:23.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:23 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:23 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:23 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 -- 172.21.15.78:0/141564893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104cc0 msgr2=0x7f042c10b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 --2- 172.21.15.78:0/141564893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104cc0 0x7f042c10b570 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f042400b0a0 tx=0x7f042400b3b0 comp rx=0 tx=0).stop 2024-05-21T15:13:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 -- 172.21.15.78:0/141564893 shutdown_connections 2024-05-21T15:13:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 --2- 172.21.15.78:0/141564893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104cc0 0x7f042c10b570 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 --2- 172.21.15.78:0/141564893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1031f0 0x7f042c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 --2- 172.21.15.78:0/141564893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f042c104320 0x7f042c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:23.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 -- 172.21.15.78:0/141564893 >> 172.21.15.78:0/141564893 conn(0x7f042c0fe850 msgr2=0x7f042c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 -- 172.21.15.78:0/141564893 shutdown_connections 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.908+0000 7f0433caa700 1 -- 172.21.15.78:0/141564893 wait complete. 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0433caa700 1 Processor -- start 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0433caa700 1 -- start start 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0433caa700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f042c1031f0 0x7f042c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0433caa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104320 0x7f042c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0433caa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c104cc0 0x7f042c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0433caa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f042c10eda0 con 0x7f042c104320 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0433caa700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f042c10ec20 con 0x7f042c1031f0 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0433caa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f042c10ef20 con 0x7f042c104cc0 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431245700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104320 0x7f042c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:23.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431245700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104320 0x7f042c199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59642/0 (socket says 172.21.15.78:59642) 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431245700 1 -- 172.21.15.78:0/3787949031 learned_addr learned my addr 172.21.15.78:0/3787949031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0432247700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c104cc0 0x7f042c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431a46700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f042c1031f0 0x7f042c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431245700 1 -- 172.21.15.78:0/3787949031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c104cc0 msgr2=0x7f042c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431245700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c104cc0 0x7f042c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431245700 1 -- 172.21.15.78:0/3787949031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f042c1031f0 msgr2=0x7f042c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431245700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f042c1031f0 0x7f042c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.909+0000 7f0431245700 1 -- 172.21.15.78:0/3787949031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c1a1340 con 0x7f042c104320 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.910+0000 7f0431245700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104320 0x7f042c199d20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f042800e9a0 tx=0x7f042800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.910+0000 7f0422ffd700 1 -- 172.21.15.78:0/3787949031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042800cbd0 con 0x7f042c104320 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.910+0000 7f0422ffd700 1 -- 172.21.15.78:0/3787949031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f042800cd70 con 0x7f042c104320 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.910+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c1a1630 con 0x7f042c104320 2024-05-21T15:13:23.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.910+0000 7f0422ffd700 1 -- 172.21.15.78:0/3787949031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428018750 con 0x7f042c104320 2024-05-21T15:13:23.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.910+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c1a1b70 con 0x7f042c104320 2024-05-21T15:13:23.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.911+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042c050170 con 0x7f042c104320 2024-05-21T15:13:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.912+0000 7f0422ffd700 1 -- 172.21.15.78:0/3787949031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0428010430 con 0x7f042c104320 2024-05-21T15:13:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.913+0000 7f0422ffd700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f04180a2c20 0x7f04180a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.913+0000 7f0431a46700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f04180a2c20 0x7f04180a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.913+0000 7f0422ffd700 1 -- 172.21.15.78:0/3787949031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f04280cf270 con 0x7f042c104320 2024-05-21T15:13:23.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.913+0000 7f0431a46700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f04180a2c20 0x7f04180a5130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f042c104220 tx=0x7f041c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:23.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:23.916+0000 7f0422ffd700 1 -- 172.21.15.78:0/3787949031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f042809c670 con 0x7f042c104320 2024-05-21T15:13:24.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.113+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f042c02dc60 con 0x7f042c104320 2024-05-21T15:13:24.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.113+0000 7f0422ffd700 1 -- 172.21.15.78:0/3787949031 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f042809bf70 con 0x7f042c104320 2024-05-21T15:13:24.116 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:24.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f04180a2c20 msgr2=0x7f04180a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:24.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f04180a2c20 0x7f04180a5130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f042c104220 tx=0x7f041c00b5b0 comp rx=0 tx=0).stop 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104320 msgr2=0x7f042c199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104320 0x7f042c199d20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f042800e9a0 tx=0x7f042800ecb0 comp rx=0 tx=0).stop 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 shutdown_connections 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f04180a2c20 0x7f04180a5130 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c104cc0 0x7f042c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f042c1031f0 0x7f042c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 --2- 172.21.15.78:0/3787949031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f042c104320 0x7f042c199d20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.115+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 >> 172.21.15.78:0/3787949031 conn(0x7f042c0fe850 msgr2=0x7f042c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.116+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 shutdown_connections 2024-05-21T15:13:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:24.116+0000 7f0433caa700 1 -- 172.21.15.78:0/3787949031 wait complete. 2024-05-21T15:13:24.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:24 smithi078 ceph-mon[31997]: purged_snaps scrub starts 2024-05-21T15:13:24.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:24 smithi078 ceph-mon[31997]: purged_snaps scrub ok 2024-05-21T15:13:24.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:24 smithi078 ceph-mon[31997]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:24.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:24 smithi078 ceph-mon[31997]: osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] boot 2024-05-21T15:13:24.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:24 smithi078 ceph-mon[31997]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T15:13:24.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:24 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:24.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:24 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3787949031' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T15:13:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:24 smithi084 ceph-mon[36334]: purged_snaps scrub starts 2024-05-21T15:13:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:24 smithi084 ceph-mon[36334]: purged_snaps scrub ok 2024-05-21T15:13:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:24 smithi084 ceph-mon[36334]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:24 smithi084 ceph-mon[36334]: osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] boot 2024-05-21T15:13:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:24 smithi084 ceph-mon[36334]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T15:13:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:24 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:24 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3787949031' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T15:13:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:24 smithi172 ceph-mon[36402]: purged_snaps scrub starts 2024-05-21T15:13:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:24 smithi172 ceph-mon[36402]: purged_snaps scrub ok 2024-05-21T15:13:24.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:24 smithi172 ceph-mon[36402]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-21T15:13:24.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:24 smithi172 ceph-mon[36402]: osd.11 [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] boot 2024-05-21T15:13:24.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:24 smithi172 ceph-mon[36402]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T15:13:24.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:24 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:13:24.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:24 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3787949031' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T15:13:24.745 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1716304403,"num_in_osds":12,"osd_in_since":1716304384,"num_remapped_pgs":0} 2024-05-21T15:13:24.746 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T15:13:24.994 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:13:25.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T15:13:25.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.387 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:25.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:25.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:25 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T15:13:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:25.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:25.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:25 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.687 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T15:13:25.687 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:25.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:25 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:26.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:26 smithi078 ceph-mon[31997]: Detected new or changed devices on smithi172 2024-05-21T15:13:26.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:26 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi172 to 4272M 2024-05-21T15:13:26.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:26 smithi078 ceph-mon[31997]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:26 smithi084 ceph-mon[36334]: Detected new or changed devices on smithi172 2024-05-21T15:13:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:26 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi172 to 4272M 2024-05-21T15:13:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:26 smithi084 ceph-mon[36334]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:26.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:26 smithi172 ceph-mon[36402]: Detected new or changed devices on smithi172 2024-05-21T15:13:26.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:26 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi172 to 4272M 2024-05-21T15:13:26.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:26 smithi172 ceph-mon[36402]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:26.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 -- 172.21.15.78:0/2439480706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde01086f0 msgr2=0x7fcde0110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:26.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 --2- 172.21.15.78:0/2439480706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde01086f0 0x7fcde0110050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800b0d0 tx=0x7fcdd800b3e0 comp rx=0 tx=0).stop 2024-05-21T15:13:26.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 -- 172.21.15.78:0/2439480706 shutdown_connections 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 --2- 172.21.15.78:0/2439480706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde01086f0 0x7fcde0110050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 --2- 172.21.15.78:0/2439480706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcde0103470 0x7fcde0103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 --2- 172.21.15.78:0/2439480706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde0107da0 0x7fcde01081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 -- 172.21.15.78:0/2439480706 >> 172.21.15.78:0/2439480706 conn(0x7fcde00fed40 msgr2=0x7fcde0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 -- 172.21.15.78:0/2439480706 shutdown_connections 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.793+0000 7fcde878f700 1 -- 172.21.15.78:0/2439480706 wait complete. 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde878f700 1 Processor -- start 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde878f700 1 -- start start 2024-05-21T15:13:26.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde878f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcde0103470 0x7fcde0075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde878f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0107da0 0x7fcde0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde878f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde01086f0 0x7fcde00764d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde878f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0113470 con 0x7fcde0107da0 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde878f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01132f0 con 0x7fcde0103470 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde878f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01135f0 con 0x7fcde01086f0 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde5d2a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0107da0 0x7fcde0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde5d2a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0107da0 0x7fcde0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59660/0 (socket says 172.21.15.78:59660) 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.794+0000 7fcde5d2a700 1 -- 172.21.15.78:0/1098689779 learned_addr learned my addr 172.21.15.78:0/1098689779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde652b700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcde0103470 0x7fcde0075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde6d2c700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde01086f0 0x7fcde00764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:26.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde5d2a700 1 -- 172.21.15.78:0/1098689779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde01086f0 msgr2=0x7fcde00764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde5d2a700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde01086f0 0x7fcde00764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde5d2a700 1 -- 172.21.15.78:0/1098689779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcde0103470 msgr2=0x7fcde0075a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde5d2a700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcde0103470 0x7fcde0075a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde5d2a700 1 -- 172.21.15.78:0/1098689779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01a80c0 con 0x7fcde0107da0 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde652b700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcde0103470 0x7fcde0075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde5d2a700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0107da0 0x7fcde0075f90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000d980 tx=0x7fcdd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcdd77fe700 1 -- 172.21.15.78:0/1098689779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0017070 con 0x7fcde0107da0 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01a83b0 con 0x7fcde0107da0 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcdd77fe700 1 -- 172.21.15.78:0/1098689779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd0009e10 con 0x7fcde0107da0 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcdd77fe700 1 -- 172.21.15.78:0/1098689779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000f620 con 0x7fcde0107da0 2024-05-21T15:13:26.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.795+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01a8940 con 0x7fcde0107da0 2024-05-21T15:13:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.796+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde0050170 con 0x7fcde0107da0 2024-05-21T15:13:26.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.798+0000 7fcdd77fe700 1 -- 172.21.15.78:0/1098689779 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcdd0010460 con 0x7fcde0107da0 2024-05-21T15:13:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.799+0000 7fcdd77fe700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fcdcc0a2ca0 0x7fcdcc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.799+0000 7fcde652b700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fcdcc0a2ca0 0x7fcdcc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.799+0000 7fcdd77fe700 1 -- 172.21.15.78:0/1098689779 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcdd00ce5e0 con 0x7fcde0107da0 2024-05-21T15:13:26.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.800+0000 7fcde652b700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fcdcc0a2ca0 0x7fcdcc0a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011490 tx=0x7fcddc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:26.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:26.803+0000 7fcdd77fe700 1 -- 172.21.15.78:0/1098689779 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcdd009ba60 con 0x7fcde0107da0 2024-05-21T15:13:27.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.000+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcde002dc60 con 0x7fcde0107da0 2024-05-21T15:13:27.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.001+0000 7fcdd77fe700 1 -- 172.21.15.78:0/1098689779 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18384 (secure 0 0 0) 0x7fcdd009b360 con 0x7fcde0107da0 2024-05-21T15:13:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:27.004 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":81,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:13:24.309719+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fcdcc0a2ca0 msgr2=0x7fcdcc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fcdcc0a2ca0 0x7fcdcc0a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011490 tx=0x7fcddc009260 comp rx=0 tx=0).stop 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0107da0 msgr2=0x7fcde0075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0107da0 0x7fcde0075f90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000d980 tx=0x7fcdd000dc90 comp rx=0 tx=0).stop 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 shutdown_connections 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fcdcc0a2ca0 0x7fcdcc0a51b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde01086f0 0x7fcde00764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcde0103470 0x7fcde0075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:27.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 --2- 172.21.15.78:0/1098689779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0107da0 0x7fcde0075f90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:27.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 >> 172.21.15.78:0/1098689779 conn(0x7fcde00fed40 msgr2=0x7fcde0107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:27.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 shutdown_connections 2024-05-21T15:13:27.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:27.003+0000 7fcde878f700 1 -- 172.21.15.78:0/1098689779 wait complete. 2024-05-21T15:13:27.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:27 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1098689779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:13:27.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:27 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1098689779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:13:27.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:27 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1098689779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:13:27.802 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:13:27.802 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-21T15:13:28.040 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:13:28.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:28 smithi078 ceph-mon[31997]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:28.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:28 smithi084 ceph-mon[36334]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:28 smithi172 ceph-mon[36402]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:29.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 -- 172.21.15.78:0/163091141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101610 msgr2=0x7fbb3c101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:29.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 --2- 172.21.15.78:0/163091141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101610 0x7fbb3c101a20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009990 tx=0x7fbb2c009ca0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 -- 172.21.15.78:0/163091141 shutdown_connections 2024-05-21T15:13:29.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 --2- 172.21.15.78:0/163091141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb3c101f60 0x7fbb3c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 --2- 172.21.15.78:0/163091141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb3c1080d0 0x7fbb3c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 --2- 172.21.15.78:0/163091141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101610 0x7fbb3c101a20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 -- 172.21.15.78:0/163091141 >> 172.21.15.78:0/163091141 conn(0x7fbb3c0fd3a0 msgr2=0x7fbb3c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:29.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 -- 172.21.15.78:0/163091141 shutdown_connections 2024-05-21T15:13:29.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.752+0000 7fbb43be4700 1 -- 172.21.15.78:0/163091141 wait complete. 2024-05-21T15:13:29.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb43be4700 1 Processor -- start 2024-05-21T15:13:29.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb43be4700 1 -- start start 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb43be4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb3c101610 0x7fbb3c1a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb43be4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101f60 0x7fbb3c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb43be4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb3c1080d0 0x7fbb3c1a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb43be4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c119ab0 con 0x7fbb3c101f60 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb43be4700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c119930 con 0x7fbb3c1080d0 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb43be4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c119c30 con 0x7fbb3c101610 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb4117f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101f60 0x7fbb3c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb4117f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101f60 0x7fbb3c1a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57732/0 (socket says 172.21.15.78:57732) 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb4117f700 1 -- 172.21.15.78:0/1211150904 learned_addr learned my addr 172.21.15.78:0/1211150904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb41980700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb3c101610 0x7fbb3c1a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb42181700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb3c1080d0 0x7fbb3c1a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:29.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb4117f700 1 -- 172.21.15.78:0/1211150904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb3c101610 msgr2=0x7fbb3c1a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb4117f700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb3c101610 0x7fbb3c1a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb4117f700 1 -- 172.21.15.78:0/1211150904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb3c1080d0 msgr2=0x7fbb3c1a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb4117f700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb3c1080d0 0x7fbb3c1a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.753+0000 7fbb4117f700 1 -- 172.21.15.78:0/1211150904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c1a5250 con 0x7fbb3c101f60 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.754+0000 7fbb42181700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb3c1080d0 0x7fbb3c1a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.754+0000 7fbb4117f700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101f60 0x7fbb3c1a0920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c000c00 tx=0x7fbb2c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.754+0000 7fbb32ffd700 1 -- 172.21.15.78:0/1211150904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c027070 con 0x7fbb3c101f60 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.754+0000 7fbb32ffd700 1 -- 172.21.15.78:0/1211150904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c004050 con 0x7fbb3c101f60 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.754+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c1a54e0 con 0x7fbb3c101f60 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.754+0000 7fbb32ffd700 1 -- 172.21.15.78:0/1211150904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00f460 con 0x7fbb3c101f60 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.754+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c1ace60 con 0x7fbb3c101f60 2024-05-21T15:13:29.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.755+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c102830 con 0x7fbb3c101f60 2024-05-21T15:13:29.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.756+0000 7fbb32ffd700 1 -- 172.21.15.78:0/1211150904 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbb2c02a070 con 0x7fbb3c101f60 2024-05-21T15:13:29.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.757+0000 7fbb32ffd700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbb280a2c20 0x7fbb280a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:29.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.757+0000 7fbb41980700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbb280a2c20 0x7fbb280a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:29.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.757+0000 7fbb32ffd700 1 -- 172.21.15.78:0/1211150904 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbb2c0cfff0 con 0x7fbb3c101f60 2024-05-21T15:13:29.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.758+0000 7fbb41980700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbb280a2c20 0x7fbb280a5130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb38005950 tx=0x7fbb380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:29.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.760+0000 7fbb32ffd700 1 -- 172.21.15.78:0/1211150904 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbb2c013070 con 0x7fbb3c101f60 2024-05-21T15:13:29.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.954+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb3c1ad030 con 0x7fbb3c101f60 2024-05-21T15:13:29.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.955+0000 7fbb32ffd700 1 -- 172.21.15.78:0/1211150904 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fbb2c09d480 con 0x7fbb3c101f60 2024-05-21T15:13:29.958 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbb280a2c20 msgr2=0x7fbb280a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbb280a2c20 0x7fbb280a5130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb38005950 tx=0x7fbb380058e0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101f60 msgr2=0x7fbb3c1a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101f60 0x7fbb3c1a0920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c000c00 tx=0x7fbb2c0047b0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 shutdown_connections 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbb280a2c20 0x7fbb280a5130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb3c1080d0 0x7fbb3c1a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb3c101610 0x7fbb3c1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 --2- 172.21.15.78:0/1211150904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb3c101f60 0x7fbb3c1a0920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:29.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 >> 172.21.15.78:0/1211150904 conn(0x7fbb3c0fd3a0 msgr2=0x7fbb3c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:29.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 shutdown_connections 2024-05-21T15:13:29.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:29.957+0000 7fbb43be4700 1 -- 172.21.15.78:0/1211150904 wait complete. 2024-05-21T15:13:30.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:30 smithi078 ceph-mon[31997]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:30 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1211150904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:13:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:30 smithi084 ceph-mon[36334]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:30 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1211150904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:13:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:30 smithi172 ceph-mon[36402]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:30 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1211150904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:13:30.729 INFO:tasks.cephadm:Adding mds.a on smithi078 2024-05-21T15:13:30.729 INFO:tasks.cephadm:Adding mds.d on smithi078 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.g on smithi078 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.j on smithi078 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.b on smithi084 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.e on smithi084 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.h on smithi084 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.k on smithi084 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.c on smithi172 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.f on smithi172 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.i on smithi172 2024-05-21T15:13:30.730 INFO:tasks.cephadm:Adding mds.l on smithi172 2024-05-21T15:13:30.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l' 2024-05-21T15:13:30.980 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:13:32.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:32 smithi078 ceph-mon[31997]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:32.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 -- 172.21.15.172:0/1483826804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548108f90 msgr2=0x7fe54810b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:32.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 --2- 172.21.15.172:0/1483826804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548108f90 0x7fe54810b3b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe54000b0a0 tx=0x7fe54000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 -- 172.21.15.172:0/1483826804 shutdown_connections 2024-05-21T15:13:32.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 --2- 172.21.15.172:0/1483826804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548108f90 0x7fe54810b3b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 --2- 172.21.15.172:0/1483826804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe548103cd0 0x7fe5481060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 --2- 172.21.15.172:0/1483826804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe548106630 0x7fe548108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 -- 172.21.15.172:0/1483826804 >> 172.21.15.172:0/1483826804 conn(0x7fe5480fd710 msgr2=0x7fe5480ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:32.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 -- 172.21.15.172:0/1483826804 shutdown_connections 2024-05-21T15:13:32.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.600+0000 7fe54ca71700 1 -- 172.21.15.172:0/1483826804 wait complete. 2024-05-21T15:13:32.606 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:32 smithi172 ceph-mon[36402]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.601+0000 7fe54ca71700 1 Processor -- start 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.601+0000 7fe54ca71700 1 -- start start 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.601+0000 7fe54ca71700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548103cd0 0x7fe548103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.601+0000 7fe54ca71700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe548106630 0x7fe548101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.601+0000 7fe54ca71700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe548108f90 0x7fe548102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54ca71700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe54810ed90 con 0x7fe548106630 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54ca71700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe54810ec10 con 0x7fe548108f90 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54ca71700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe54810ef10 con 0x7fe548103cd0 2024-05-21T15:13:32.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548103cd0 0x7fe548103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548103cd0 0x7fe548103530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48954/0 (socket says 172.21.15.172:48954) 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 -- 172.21.15.172:0/2361531536 learned_addr learned my addr 172.21.15.172:0/2361531536 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe546d9d700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe548108f90 0x7fe548102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe53dd9b700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe548106630 0x7fe548101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 -- 172.21.15.172:0/2361531536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe548108f90 msgr2=0x7fe548102120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe548108f90 0x7fe548102120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 -- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe548106630 msgr2=0x7fe548101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe548106630 0x7fe548101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 -- 172.21.15.172:0/2361531536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5481027f0 con 0x7fe548103cd0 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe54659c700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548103cd0 0x7fe548103530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c8f0 tx=0x7fe53800cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.602+0000 7fe53ffff700 1 -- 172.21.15.172:0/2361531536 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380078d0 con 0x7fe548103cd0 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.603+0000 7fe53ffff700 1 -- 172.21.15.172:0/2361531536 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe53800f450 con 0x7fe548103cd0 2024-05-21T15:13:32.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.603+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5481a3870 con 0x7fe548103cd0 2024-05-21T15:13:32.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.603+0000 7fe53ffff700 1 -- 172.21.15.172:0/2361531536 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538018640 con 0x7fe548103cd0 2024-05-21T15:13:32.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.603+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5481a3d20 con 0x7fe548103cd0 2024-05-21T15:13:32.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.604+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe548050170 con 0x7fe548103cd0 2024-05-21T15:13:32.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.605+0000 7fe53ffff700 1 -- 172.21.15.172:0/2361531536 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe5380187e0 con 0x7fe548103cd0 2024-05-21T15:13:32.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.605+0000 7fe53ffff700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe5340a2ca0 0x7fe5340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:32.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.606+0000 7fe53ffff700 1 -- 172.21.15.172:0/2361531536 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe5380ce950 con 0x7fe548103cd0 2024-05-21T15:13:32.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.606+0000 7fe53dd9b700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe5340a2ca0 0x7fe5340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:32.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.606+0000 7fe53dd9b700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe5340a2ca0 0x7fe5340a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe548102ee0 tx=0x7fe53000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:32.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.611+0000 7fe53ffff700 1 -- 172.21.15.172:0/2361531536 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe538014030 con 0x7fe548103cd0 2024-05-21T15:13:32.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:32 smithi084 ceph-mon[36334]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.819+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}) -- 0x7fe548062b80 con 0x7fe5340a2ca0 2024-05-21T15:13:32.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.828+0000 7fe53ffff700 1 -- 172.21.15.172:0/2361531536 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe548062b80 con 0x7fe5340a2ca0 2024-05-21T15:13:32.831 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mds.all update... 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe5340a2ca0 msgr2=0x7fe5340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe5340a2ca0 0x7fe5340a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe548102ee0 tx=0x7fe53000a380 comp rx=0 tx=0).stop 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548103cd0 msgr2=0x7fe548103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548103cd0 0x7fe548103530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c8f0 tx=0x7fe53800cc00 comp rx=0 tx=0).stop 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 shutdown_connections 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe5340a2ca0 0x7fe5340a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe548108f90 0x7fe548102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe548103cd0 0x7fe548103530 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 --2- 172.21.15.172:0/2361531536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe548106630 0x7fe548101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.830+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 >> 172.21.15.172:0/2361531536 conn(0x7fe5480fd710 msgr2=0x7fe548104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.831+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 shutdown_connections 2024-05-21T15:13:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-05-21T15:13:32.831+0000 7fe54ca71700 1 -- 172.21.15.172:0/2361531536 wait complete. 2024-05-21T15:13:33.594 DEBUG:teuthology.orchestra.run.smithi078:mds.a> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.a.service 2024-05-21T15:13:33.597 DEBUG:teuthology.orchestra.run.smithi078:mds.d> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.d.service 2024-05-21T15:13:33.640 DEBUG:teuthology.orchestra.run.smithi078:mds.g> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.g.service 2024-05-21T15:13:33.683 DEBUG:teuthology.orchestra.run.smithi078:mds.j> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.j.service 2024-05-21T15:13:33.726 DEBUG:teuthology.orchestra.run.smithi084:mds.b> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.b.service 2024-05-21T15:13:33.729 DEBUG:teuthology.orchestra.run.smithi084:mds.e> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.e.service 2024-05-21T15:13:33.774 DEBUG:teuthology.orchestra.run.smithi084:mds.h> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.h.service 2024-05-21T15:13:33.818 DEBUG:teuthology.orchestra.run.smithi084:mds.k> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.k.service 2024-05-21T15:13:33.861 DEBUG:teuthology.orchestra.run.smithi172:mds.c> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.c.service 2024-05-21T15:13:33.864 DEBUG:teuthology.orchestra.run.smithi172:mds.f> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.f.service 2024-05-21T15:13:33.907 DEBUG:teuthology.orchestra.run.smithi172:mds.i> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.i.service 2024-05-21T15:13:33.950 DEBUG:teuthology.orchestra.run.smithi172:mds.l> sudo journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.l.service 2024-05-21T15:13:33.993 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-21T15:13:33.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:13:34.015 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:13:34.015 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:34.016 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:33 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:34.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:13:34.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-05-21T15:13:34.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:34.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:34.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:34.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:34.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:34.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:34.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:34.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:34.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:33 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:34.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.119+0000 7fc291aca640 1 -- 172.21.15.78:0/2732953678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c104030 msgr2=0x7fc28c104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.119+0000 7fc291aca640 1 --2- 172.21.15.78:0/2732953678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c104030 0x7fc28c104440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009900 tx=0x7fc27c02a410 comp rx=0 tx=0).stop 2024-05-21T15:13:34.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 -- 172.21.15.78:0/2732953678 shutdown_connections 2024-05-21T15:13:34.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 --2- 172.21.15.78:0/2732953678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c1006e0 0x7fc28c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 --2- 172.21.15.78:0/2732953678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c1011e0 0x7fc28c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 --2- 172.21.15.78:0/2732953678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c104030 0x7fc28c104440 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 -- 172.21.15.78:0/2732953678 >> 172.21.15.78:0/2732953678 conn(0x7fc28c0fc2e0 msgr2=0x7fc28c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 -- 172.21.15.78:0/2732953678 shutdown_connections 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 -- 172.21.15.78:0/2732953678 wait complete. 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 Processor -- start 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.120+0000 7fc291aca640 1 -- start start 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc291aca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c1006e0 0x7fc28c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc291aca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c1011e0 0x7fc28c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc291aca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c104030 0x7fc28c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc291aca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c10e6d0 con 0x7fc28c1011e0 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc291aca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c10e550 con 0x7fc28c1006e0 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc291aca640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c10e850 con 0x7fc28c104030 2024-05-21T15:13:34.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c1011e0 0x7fc28c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c1011e0 0x7fc28c1997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57764/0 (socket says 172.21.15.78:57764) 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 -- 172.21.15.78:0/2357147949 learned_addr learned my addr 172.21.15.78:0/2357147949 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28bfff640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c104030 0x7fc28c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28b7fe640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c1006e0 0x7fc28c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 -- 172.21.15.78:0/2357147949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c104030 msgr2=0x7fc28c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c104030 0x7fc28c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 -- 172.21.15.78:0/2357147949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c1006e0 msgr2=0x7fc28c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c1006e0 0x7fc28c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 -- 172.21.15.78:0/2357147949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c1a0dc0 con 0x7fc28c1011e0 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc28affd640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c1011e0 0x7fc28c1997c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc28000e9f0 tx=0x7fc28000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:34.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc288ff9640 1 -- 172.21.15.78:0/2357147949 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28000cd80 con 0x7fc28c1011e0 2024-05-21T15:13:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c1a10b0 con 0x7fc28c1011e0 2024-05-21T15:13:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc288ff9640 1 -- 172.21.15.78:0/2357147949 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc280010910 con 0x7fc28c1011e0 2024-05-21T15:13:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc288ff9640 1 -- 172.21.15.78:0/2357147949 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280018990 con 0x7fc28c1011e0 2024-05-21T15:13:34.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.121+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1a1600 con 0x7fc28c1011e0 2024-05-21T15:13:34.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.123+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c1079c0 con 0x7fc28c1011e0 2024-05-21T15:13:34.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.124+0000 7fc288ff9640 1 -- 172.21.15.78:0/2357147949 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc280010ac0 con 0x7fc28c1011e0 2024-05-21T15:13:34.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.125+0000 7fc288ff9640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2740a2b90 0x7fc2740a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.125+0000 7fc28b7fe640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2740a2b90 0x7fc2740a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.125+0000 7fc288ff9640 1 -- 172.21.15.78:0/2357147949 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc2800cf3a0 con 0x7fc28c1011e0 2024-05-21T15:13:34.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.125+0000 7fc28b7fe640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2740a2b90 0x7fc2740a50a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009230 tx=0x7fc27c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:34.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.128+0000 7fc288ff9640 1 -- 172.21.15.78:0/2357147949 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc28009c7a0 con 0x7fc28c1011e0 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:34.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:34.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:33 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:34.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.306+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc28c02d470 con 0x7fc28c1011e0 2024-05-21T15:13:34.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.307+0000 7fc288ff9640 1 -- 172.21.15.78:0/2357147949 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18384 (secure 0 0 0) 0x7fc28009c0a0 con 0x7fc28c1011e0 2024-05-21T15:13:34.310 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:34.310 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":81,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:13:24.309719+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:13:34.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2740a2b90 msgr2=0x7fc2740a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2740a2b90 0x7fc2740a50a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009230 tx=0x7fc27c031400 comp rx=0 tx=0).stop 2024-05-21T15:13:34.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c1011e0 msgr2=0x7fc28c1997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c1011e0 0x7fc28c1997c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc28000e9f0 tx=0x7fc28000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 shutdown_connections 2024-05-21T15:13:34.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2740a2b90 0x7fc2740a50a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c104030 0x7fc28c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c1011e0 0x7fc28c1997c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 --2- 172.21.15.78:0/2357147949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c1006e0 0x7fc28c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 >> 172.21.15.78:0/2357147949 conn(0x7fc28c0fc2e0 msgr2=0x7fc28c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:34.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 shutdown_connections 2024-05-21T15:13:34.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.309+0000 7fc291aca640 1 -- 172.21.15.78:0/2357147949 wait complete. 2024-05-21T15:13:34.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:13:34.324 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.475+0000 7f241c244640 1 -- 172.21.15.78:0/1085436637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414103430 msgr2=0x7f2414109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.475+0000 7f241c244640 1 --2- 172.21.15.78:0/1085436637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414103430 0x7f2414109b70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f240c009ca0 tx=0x7f240c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.475+0000 7f241c244640 1 -- 172.21.15.78:0/1085436637 shutdown_connections 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.475+0000 7f241c244640 1 --2- 172.21.15.78:0/1085436637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2414102950 0x7f2414102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.475+0000 7f241c244640 1 --2- 172.21.15.78:0/1085436637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414103430 0x7f2414109b70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.475+0000 7f241c244640 1 --2- 172.21.15.78:0/1085436637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2414101f70 0x7f2414102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.476+0000 7f241c244640 1 -- 172.21.15.78:0/1085436637 >> 172.21.15.78:0/1085436637 conn(0x7f24140fc330 msgr2=0x7f24140fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.476+0000 7f241c244640 1 -- 172.21.15.78:0/1085436637 shutdown_connections 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.476+0000 7f241c244640 1 -- 172.21.15.78:0/1085436637 wait complete. 2024-05-21T15:13:34.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.476+0000 7f241c244640 1 Processor -- start 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.476+0000 7f241c244640 1 -- start start 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.476+0000 7f241c244640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2414101f70 0x7f2414101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.476+0000 7f241c244640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414102950 0x7f241419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.476+0000 7f241c244640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2414103430 0x7f24141a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f241c244640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f241410d650 con 0x7f2414102950 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f241c244640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f241410d4d0 con 0x7f2414101f70 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f241c244640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f241410d7d0 con 0x7f2414103430 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414102950 0x7f241419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414102950 0x7f241419e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57778/0 (socket says 172.21.15.78:57778) 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 -- 172.21.15.78:0/1049257285 learned_addr learned my addr 172.21.15.78:0/1049257285 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f2419fe0640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2414101f70 0x7f2414101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f241a7e1640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2414103430 0x7f24141a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 -- 172.21.15.78:0/1049257285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2414103430 msgr2=0x7f24141a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2414103430 0x7f24141a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 -- 172.21.15.78:0/1049257285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2414101f70 msgr2=0x7f2414101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2414101f70 0x7f2414101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 -- 172.21.15.78:0/1049257285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24141a0d90 con 0x7f2414102950 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f2419fe0640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2414101f70 0x7f2414101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f24197df640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414102950 0x7f241419e020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f240400e9f0 tx=0x7f240400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f240affd640 1 -- 172.21.15.78:0/1049257285 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240400cd80 con 0x7f2414102950 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24141a1080 con 0x7f2414102950 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f240affd640 1 -- 172.21.15.78:0/1049257285 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2404010910 con 0x7f2414102950 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f240affd640 1 -- 172.21.15.78:0/1049257285 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404018990 con 0x7f2414102950 2024-05-21T15:13:34.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.477+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24141a15d0 con 0x7f2414102950 2024-05-21T15:13:34.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.478+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2414108920 con 0x7f2414102950 2024-05-21T15:13:34.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.480+0000 7f240affd640 1 -- 172.21.15.78:0/1049257285 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2404010ac0 con 0x7f2414102950 2024-05-21T15:13:34.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.480+0000 7f240affd640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f24000a2b90 0x7f24000a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.480+0000 7f2419fe0640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f24000a2b90 0x7f24000a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.480+0000 7f240affd640 1 -- 172.21.15.78:0/1049257285 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f24040cf450 con 0x7f2414102950 2024-05-21T15:13:34.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.481+0000 7f2419fe0640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f24000a2b90 0x7f24000a50a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24100058c0 tx=0x7f2410005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:34.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.484+0000 7f240affd640 1 -- 172.21.15.78:0/1049257285 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f240409c850 con 0x7f2414102950 2024-05-21T15:13:34.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.660+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2414062750 con 0x7f2414102950 2024-05-21T15:13:34.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.661+0000 7f240affd640 1 -- 172.21.15.78:0/1049257285 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f240409c150 con 0x7f2414102950 2024-05-21T15:13:34.664 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:13:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f24000a2b90 msgr2=0x7f24000a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f24000a2b90 0x7f24000a50a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24100058c0 tx=0x7f2410005850 comp rx=0 tx=0).stop 2024-05-21T15:13:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414102950 msgr2=0x7f241419e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414102950 0x7f241419e020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f240400e9f0 tx=0x7f240400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 shutdown_connections 2024-05-21T15:13:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f24000a2b90 0x7f24000a50a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2414103430 0x7f24141a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2414102950 0x7f241419e020 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 --2- 172.21.15.78:0/1049257285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2414101f70 0x7f2414101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 >> 172.21.15.78:0/1049257285 conn(0x7f24140fc330 msgr2=0x7f2414105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:34.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 shutdown_connections 2024-05-21T15:13:34.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.663+0000 7f241c244640 1 -- 172.21.15.78:0/1049257285 wait complete. 2024-05-21T15:13:34.677 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 -- 172.21.15.78:0/2559539811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c1031f0 msgr2=0x7fbf1c105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2559539811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c1031f0 0x7fbf1c105610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009ca0 tx=0x7fbf0c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 -- 172.21.15.78:0/2559539811 shutdown_connections 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2559539811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf1c108710 0x7fbf1c10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2559539811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf1c105be0 0x7fbf1c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2559539811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c1031f0 0x7fbf1c105610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 -- 172.21.15.78:0/2559539811 >> 172.21.15.78:0/2559539811 conn(0x7fbf1c0fcb50 msgr2=0x7fbf1c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 -- 172.21.15.78:0/2559539811 shutdown_connections 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 -- 172.21.15.78:0/2559539811 wait complete. 2024-05-21T15:13:34.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 Processor -- start 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.785+0000 7fbf239cc640 1 -- start start 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf239cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf1c1031f0 0x7fbf1c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf239cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c105be0 0x7fbf1c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf239cc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf1c108710 0x7fbf1c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf239cc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10e6c0 con 0x7fbf1c105be0 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf239cc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10e540 con 0x7fbf1c108710 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf239cc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10e840 con 0x7fbf1c1031f0 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c105be0 0x7fbf1c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c105be0 0x7fbf1c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57804/0 (socket says 172.21.15.78:57804) 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 -- 172.21.15.78:0/2732527344 learned_addr learned my addr 172.21.15.78:0/2732527344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf21f69640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf1c108710 0x7fbf1c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf21768640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf1c1031f0 0x7fbf1c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 -- 172.21.15.78:0/2732527344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf1c1031f0 msgr2=0x7fbf1c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf1c1031f0 0x7fbf1c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 -- 172.21.15.78:0/2732527344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf1c108710 msgr2=0x7fbf1c1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf1c108710 0x7fbf1c1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 -- 172.21.15.78:0/2732527344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c1a0db0 con 0x7fbf1c105be0 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf20f67640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c105be0 0x7fbf1c1997b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000d770 tx=0x7fbf1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf1a7fc640 1 -- 172.21.15.78:0/2732527344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10017070 con 0x7fbf1c105be0 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf1a7fc640 1 -- 172.21.15.78:0/2732527344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf10005500 con 0x7fbf1c105be0 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c1a10a0 con 0x7fbf1c105be0 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf1a7fc640 1 -- 172.21.15.78:0/2732527344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10004e00 con 0x7fbf1c105be0 2024-05-21T15:13:34.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.786+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c1a15a0 con 0x7fbf1c105be0 2024-05-21T15:13:34.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.787+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1c0fe670 con 0x7fbf1c105be0 2024-05-21T15:13:34.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.789+0000 7fbf1a7fc640 1 -- 172.21.15.78:0/2732527344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbf10004fa0 con 0x7fbf1c105be0 2024-05-21T15:13:34.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.789+0000 7fbf1a7fc640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbf040a2b90 0x7fbf040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:34.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.789+0000 7fbf21768640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbf040a2b90 0x7fbf040a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:34.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.789+0000 7fbf1a7fc640 1 -- 172.21.15.78:0/2732527344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbf100ce9a0 con 0x7fbf1c105be0 2024-05-21T15:13:34.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.790+0000 7fbf21768640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbf040a2b90 0x7fbf040a50a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c031c10 tx=0x7fbf0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:34.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:34.793+0000 7fbf1a7fc640 1 -- 172.21.15.78:0/2732527344 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbf1009b750 con 0x7fbf1c105be0 2024-05-21T15:13:35.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.003+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf1c02d2a0 con 0x7fbf1c105be0 2024-05-21T15:13:35.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.004+0000 7fbf1a7fc640 1 -- 172.21.15.78:0/2732527344 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fbf1009fa50 con 0x7fbf1c105be0 2024-05-21T15:13:35.007 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:35.007 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-21T15:13:35.007 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 1 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbf040a2b90 msgr2=0x7fbf040a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbf040a2b90 0x7fbf040a50a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c031c10 tx=0x7fbf0c005fd0 comp rx=0 tx=0).stop 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c105be0 msgr2=0x7fbf1c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c105be0 0x7fbf1c1997b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000d770 tx=0x7fbf1000dc30 comp rx=0 tx=0).stop 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 shutdown_connections 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fbf040a2b90 0x7fbf040a50a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf1c108710 0x7fbf1c1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf1c105be0 0x7fbf1c1997b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 --2- 172.21.15.78:0/2732527344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf1c1031f0 0x7fbf1c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 >> 172.21.15.78:0/2732527344 conn(0x7fbf1c0fcb50 msgr2=0x7fbf1c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 shutdown_connections 2024-05-21T15:13:35.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:35.006+0000 7fbf239cc640 1 -- 172.21.15.78:0/2732527344 wait complete. 2024-05-21T15:13:35.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:34 smithi078 ceph-mon[31997]: Deploying daemon mds.l on smithi172 2024-05-21T15:13:35.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:34 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2357147949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:13:35.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:34 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1049257285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:13:35.127 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:34 smithi172 ceph-mon[36402]: Deploying daemon mds.l on smithi172 2024-05-21T15:13:35.127 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:34 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2357147949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:13:35.127 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:34 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1049257285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:13:35.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:34 smithi084 ceph-mon[36334]: Deploying daemon mds.l on smithi172 2024-05-21T15:13:35.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:34 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2357147949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:13:35.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:34 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1049257285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:13:35.844 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:13:35 smithi172 podman[66093]: 2024-05-21 15:13:35.72740042 +0000 UTC m=+0.129336016 container create e7fb22fe21c2624183265aa930c70d8208c086b8d325d637317badacfcac08f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-21T15:13:36.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:35 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2732527344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:36.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:35 smithi078 ceph-mon[31997]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:36.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:35 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2732527344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:36.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:35 smithi084 ceph-mon[36334]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:36.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:35 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2732527344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:36.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:35 smithi172 ceph-mon[36402]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:36.184 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:13:35 smithi172 podman[66093]: 2024-05-21 15:13:35.946957755 +0000 UTC m=+0.348893361 container init e7fb22fe21c2624183265aa930c70d8208c086b8d325d637317badacfcac08f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-21T15:13:36.184 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:13:35 smithi172 podman[66093]: 2024-05-21 15:13:35.949857938 +0000 UTC m=+0.351793534 container start e7fb22fe21c2624183265aa930c70d8208c086b8d325d637317badacfcac08f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:13:36.184 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:13:35 smithi172 ceph-mds[66112]: -- 172.21.15.172:0/243629571 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56026eedba40 con 0x56026ef53600 2024-05-21T15:13:36.184 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:13:35 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l[66108]: starting mds.l at 2024-05-21T15:13:36.184 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:13:36 smithi172 bash[66093]: e7fb22fe21c2624183265aa930c70d8208c086b8d325d637317badacfcac08f5 2024-05-21T15:13:36.184 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:13:36 smithi172 systemd[1]: Started Ceph mds.l for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:37.015 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] up:boot 2024-05-21T15:13:37.015 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: fsmap 1 up:standby 2024-05-21T15:13:37.015 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:13:37.016 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.016 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.016 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.016 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:37.016 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:37.016 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:37.016 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:36 smithi084 ceph-mon[36334]: Deploying daemon mds.k on smithi084 2024-05-21T15:13:37.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 -- 172.21.15.78:0/2657335563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600101970 msgr2=0x7f0600102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 --2- 172.21.15.78:0/2657335563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600101970 0x7f0600102670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009900 tx=0x7f05f002a410 comp rx=0 tx=0).stop 2024-05-21T15:13:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 -- 172.21.15.78:0/2657335563 shutdown_connections 2024-05-21T15:13:37.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 --2- 172.21.15.78:0/2657335563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06001078b0 0x7f0600109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 --2- 172.21.15.78:0/2657335563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0600102bb0 0x7f0600102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 --2- 172.21.15.78:0/2657335563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0600101970 0x7f0600102670 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 -- 172.21.15.78:0/2657335563 >> 172.21.15.78:0/2657335563 conn(0x7f06000fcb80 msgr2=0x7f06000ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 -- 172.21.15.78:0/2657335563 shutdown_connections 2024-05-21T15:13:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.146+0000 7f0605196640 1 -- 172.21.15.78:0/2657335563 wait complete. 2024-05-21T15:13:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f0605196640 1 Processor -- start 2024-05-21T15:13:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f0605196640 1 -- start start 2024-05-21T15:13:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f0605196640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0600101970 0x7f06001992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f0605196640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0600102bb0 0x7f06001997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f0605196640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06001078b0 0x7f06001a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f0605196640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f060010d5a0 con 0x7f06001078b0 2024-05-21T15:13:37.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f0605196640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f060010d420 con 0x7f0600102bb0 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f0605196640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f060010d720 con 0x7f0600101970 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.147+0000 7f05ff59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06001078b0 0x7f06001a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05ff59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06001078b0 0x7f06001a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57810/0 (socket says 172.21.15.78:57810) 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05ff59e640 1 -- 172.21.15.78:0/2038555712 learned_addr learned my addr 172.21.15.78:0/2038555712 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05fe59c640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0600102bb0 0x7f06001997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05fed9d640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0600101970 0x7f06001992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05ff59e640 1 -- 172.21.15.78:0/2038555712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0600101970 msgr2=0x7f06001992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05ff59e640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0600101970 0x7f06001992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05ff59e640 1 -- 172.21.15.78:0/2038555712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0600102bb0 msgr2=0x7f06001997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05ff59e640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0600102bb0 0x7f06001997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05ff59e640 1 -- 172.21.15.78:0/2038555712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06001a0de0 con 0x7f06001078b0 2024-05-21T15:13:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05fe59c640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0600102bb0 0x7f06001997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:13:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05ff59e640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06001078b0 0x7f06001a0770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05f800eb10 tx=0x7f05f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05f7fff640 1 -- 172.21.15.78:0/2038555712 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004b20 con 0x7f06001078b0 2024-05-21T15:13:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05f7fff640 1 -- 172.21.15.78:0/2038555712 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05f8004cc0 con 0x7f06001078b0 2024-05-21T15:13:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06001a10d0 con 0x7f06001078b0 2024-05-21T15:13:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f05f7fff640 1 -- 172.21.15.78:0/2038555712 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8010760 con 0x7f06001078b0 2024-05-21T15:13:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.148+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06001a15d0 con 0x7f06001078b0 2024-05-21T15:13:37.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.149+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06001a1890 con 0x7f06001078b0 2024-05-21T15:13:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.151+0000 7f05f7fff640 1 -- 172.21.15.78:0/2038555712 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f05f8005020 con 0x7f06001078b0 2024-05-21T15:13:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.151+0000 7f05f7fff640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f05ec0a2b90 0x7f05ec0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.151+0000 7f05fed9d640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f05ec0a2b90 0x7f05ec0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.151+0000 7f05f7fff640 1 -- 172.21.15.78:0/2038555712 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f05f80cf150 con 0x7f06001078b0 2024-05-21T15:13:37.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.152+0000 7f05fed9d640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f05ec0a2b90 0x7f05ec0a50a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05f0005ab0 tx=0x7f05f0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:37.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.156+0000 7f05f7fff640 1 -- 172.21.15.78:0/2038555712 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f05f809c550 con 0x7f06001078b0 2024-05-21T15:13:37.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] up:boot 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: fsmap 1 up:standby 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:37.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:36 smithi078 ceph-mon[31997]: Deploying daemon mds.k on smithi084 2024-05-21T15:13:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.369+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06001a1a60 con 0x7f06001078b0 2024-05-21T15:13:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.369+0000 7f05f7fff640 1 -- 172.21.15.78:0/2038555712 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f05f809be50 con 0x7f06001078b0 2024-05-21T15:13:37.372 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:37.372 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T15:13:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 2 2024-05-21T15:13:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f05ec0a2b90 msgr2=0x7f05ec0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f05ec0a2b90 0x7f05ec0a50a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05f0005ab0 tx=0x7f05f0031400 comp rx=0 tx=0).stop 2024-05-21T15:13:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06001078b0 msgr2=0x7f06001a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06001078b0 0x7f06001a0770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05f800eb10 tx=0x7f05f800c2d0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 shutdown_connections 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f05ec0a2b90 0x7f05ec0a50a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0600102bb0 0x7f06001997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06001078b0 0x7f06001a0770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 --2- 172.21.15.78:0/2038555712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0600101970 0x7f06001992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 >> 172.21.15.78:0/2038555712 conn(0x7f06000fcb80 msgr2=0x7f0600108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 shutdown_connections 2024-05-21T15:13:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:37.371+0000 7f0605196640 1 -- 172.21.15.78:0/2038555712 wait complete. 2024-05-21T15:13:37.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] up:boot 2024-05-21T15:13:37.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: fsmap 1 up:standby 2024-05-21T15:13:37.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:13:37.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:37.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:37.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:37.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:37.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:36 smithi172 ceph-mon[36402]: Deploying daemon mds.k on smithi084 2024-05-21T15:13:38.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:37 smithi084 ceph-mon[36334]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:38.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:37 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2038555712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:38.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:37 smithi078 ceph-mon[31997]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:38.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:37 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2038555712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:38.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:37 smithi172 ceph-mon[36402]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:38.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:37 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2038555712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:38.576 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 systemd[1]: Starting Ceph mds.k for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:13:38.831 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 podman[66214]: 2024-05-21 15:13:38.573212409 +0000 UTC m=+0.099254266 container create bffdc70a4ddcd904f949ea67140fb0bccfbe39d89dfb12d575ff4090955110f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-21T15:13:38.832 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 podman[66214]: 2024-05-21 15:13:38.491232517 +0000 UTC m=+0.017274342 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:13:38.832 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 podman[66214]: 2024-05-21 15:13:38.792195186 +0000 UTC m=+0.318236997 container init bffdc70a4ddcd904f949ea67140fb0bccfbe39d89dfb12d575ff4090955110f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-21T15:13:38.832 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 podman[66214]: 2024-05-21 15:13:38.794722699 +0000 UTC m=+0.320764510 container start bffdc70a4ddcd904f949ea67140fb0bccfbe39d89dfb12d575ff4090955110f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:13:39.146 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 ceph-mds[66233]: -- 172.21.15.84:0/585277681 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56257ea7da40 con 0x56257eaf5a80 2024-05-21T15:13:39.146 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k[66229]: starting mds.k at 2024-05-21T15:13:39.146 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 bash[66214]: bffdc70a4ddcd904f949ea67140fb0bccfbe39d89dfb12d575ff4090955110f1 2024-05-21T15:13:39.146 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:13:38 smithi084 systemd[1]: Started Ceph mds.k for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:39.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.508+0000 7fe77981b640 1 -- 172.21.15.78:0/1091805152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774105bd0 msgr2=0x7fe774108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.508+0000 7fe77981b640 1 --2- 172.21.15.78:0/1091805152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774105bd0 0x7fe774108070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe768009900 tx=0x7fe76802a410 comp rx=0 tx=0).stop 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.508+0000 7fe77981b640 1 -- 172.21.15.78:0/1091805152 shutdown_connections 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.508+0000 7fe77981b640 1 --2- 172.21.15.78:0/1091805152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe774108700 0x7fe77410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.508+0000 7fe77981b640 1 --2- 172.21.15.78:0/1091805152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774105bd0 0x7fe774108070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.508+0000 7fe77981b640 1 --2- 172.21.15.78:0/1091805152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7741031e0 0x7fe774105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.508+0000 7fe77981b640 1 -- 172.21.15.78:0/1091805152 >> 172.21.15.78:0/1091805152 conn(0x7fe7740fcb80 msgr2=0x7fe7740ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.508+0000 7fe77981b640 1 -- 172.21.15.78:0/1091805152 shutdown_connections 2024-05-21T15:13:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.509+0000 7fe77981b640 1 -- 172.21.15.78:0/1091805152 wait complete. 2024-05-21T15:13:39.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.509+0000 7fe77981b640 1 Processor -- start 2024-05-21T15:13:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe77981b640 1 -- start start 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe77981b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7741031e0 0x7fe7741992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe77981b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe774105bd0 0x7fe774199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe77981b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774108700 0x7fe7741a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe77981b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe77410e6c0 con 0x7fe774108700 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe77981b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe77410e540 con 0x7fe774105bd0 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe77981b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe77410e840 con 0x7fe7741031e0 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774108700 0x7fe7741a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774108700 0x7fe7741a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53470/0 (socket says 172.21.15.78:53470) 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7737fe640 1 -- 172.21.15.78:0/729067870 learned_addr learned my addr 172.21.15.78:0/729067870 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7727fc640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe774105bd0 0x7fe774199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe772ffd640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7741031e0 0x7fe7741992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7737fe640 1 -- 172.21.15.78:0/729067870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7741031e0 msgr2=0x7fe7741992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7737fe640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7741031e0 0x7fe7741992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7737fe640 1 -- 172.21.15.78:0/729067870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe774105bd0 msgr2=0x7fe774199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7737fe640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe774105bd0 0x7fe774199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.510+0000 7fe7737fe640 1 -- 172.21.15.78:0/729067870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7741a0d50 con 0x7fe774108700 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.511+0000 7fe7737fe640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774108700 0x7fe7741a0790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00cce0 tx=0x7fe76c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.511+0000 7fe778819640 1 -- 172.21.15.78:0/729067870 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00f450 con 0x7fe774108700 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.511+0000 7fe778819640 1 -- 172.21.15.78:0/729067870 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76c00fad0 con 0x7fe774108700 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.511+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741a0fe0 con 0x7fe774108700 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.511+0000 7fe778819640 1 -- 172.21.15.78:0/729067870 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c018c10 con 0x7fe774108700 2024-05-21T15:13:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.511+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741a1450 con 0x7fe774108700 2024-05-21T15:13:39.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.512+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7740fe6a0 con 0x7fe774108700 2024-05-21T15:13:39.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.514+0000 7fe778819640 1 -- 172.21.15.78:0/729067870 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe76c00f5f0 con 0x7fe774108700 2024-05-21T15:13:39.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.515+0000 7fe778819640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe75c0a2b90 0x7fe75c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:39.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.515+0000 7fe772ffd640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe75c0a2b90 0x7fe75c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:39.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.515+0000 7fe778819640 1 -- 172.21.15.78:0/729067870 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe76c0cf100 con 0x7fe774108700 2024-05-21T15:13:39.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.515+0000 7fe772ffd640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe75c0a2b90 0x7fe75c0a50a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe764005d00 tx=0x7fe76400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:39.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.520+0000 7fe778819640 1 -- 172.21.15.78:0/729067870 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe76c014030 con 0x7fe774108700 2024-05-21T15:13:39.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.745+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe774062750 con 0x7fe774108700 2024-05-21T15:13:39.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.745+0000 7fe778819640 1 -- 172.21.15.78:0/729067870 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fe76c09c500 con 0x7fe774108700 2024-05-21T15:13:39.749 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:39.749 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T15:13:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 3 2024-05-21T15:13:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe75c0a2b90 msgr2=0x7fe75c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe75c0a2b90 0x7fe75c0a50a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe764005d00 tx=0x7fe76400b410 comp rx=0 tx=0).stop 2024-05-21T15:13:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774108700 msgr2=0x7fe7741a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774108700 0x7fe7741a0790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00cce0 tx=0x7fe76c00a620 comp rx=0 tx=0).stop 2024-05-21T15:13:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 shutdown_connections 2024-05-21T15:13:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe75c0a2b90 0x7fe75c0a50a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe774105bd0 0x7fe774199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe774108700 0x7fe7741a0790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 --2- 172.21.15.78:0/729067870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7741031e0 0x7fe7741992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 >> 172.21.15.78:0/729067870 conn(0x7fe7740fcb80 msgr2=0x7fe774103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 shutdown_connections 2024-05-21T15:13:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:39.748+0000 7fe77981b640 1 -- 172.21.15.78:0/729067870 wait complete. 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: Deploying daemon mds.j on smithi078 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] up:boot 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: fsmap 2 up:standby 2024-05-21T15:13:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:13:40.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:40.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:39 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/729067870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:40.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:40.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: Deploying daemon mds.j on smithi078 2024-05-21T15:13:40.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] up:boot 2024-05-21T15:13:40.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: fsmap 2 up:standby 2024-05-21T15:13:40.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:13:40.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:40.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:39 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/729067870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:40.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: Deploying daemon mds.j on smithi078 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] up:boot 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: fsmap 2 up:standby 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:39 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/729067870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:41.347 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 systemd[1]: Starting Ceph mds.j for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:13:41.651 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 podman[72428]: 2024-05-21 15:13:41.299065008 +0000 UTC m=+0.017422606 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:13:41.651 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 podman[72428]: 2024-05-21 15:13:41.427159732 +0000 UTC m=+0.145517339 container create 403f2890c9064f9cfa928fb33f3704d4b0d5fc02b34eb4842ed1cee0fef13e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:13:41.651 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 podman[72428]: 2024-05-21 15:13:41.646087412 +0000 UTC m=+0.364445015 container init 403f2890c9064f9cfa928fb33f3704d4b0d5fc02b34eb4842ed1cee0fef13e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux ) 2024-05-21T15:13:41.651 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 podman[72428]: 2024-05-21 15:13:41.648731523 +0000 UTC m=+0.367089109 container start 403f2890c9064f9cfa928fb33f3704d4b0d5fc02b34eb4842ed1cee0fef13e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-21T15:13:41.763 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:41.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 -- 172.21.15.78:0/2633377922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8334101970 msgr2=0x7f8334102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 --2- 172.21.15.78:0/2633377922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8334101970 0x7f8334102670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8324009900 tx=0x7f832402a410 comp rx=0 tx=0).stop 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 -- 172.21.15.78:0/2633377922 shutdown_connections 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 --2- 172.21.15.78:0/2633377922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83341078b0 0x7f8334109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 --2- 172.21.15.78:0/2633377922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8334102bb0 0x7f8334102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 --2- 172.21.15.78:0/2633377922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8334101970 0x7f8334102670 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 -- 172.21.15.78:0/2633377922 >> 172.21.15.78:0/2633377922 conn(0x7f83340fcb80 msgr2=0x7f83340ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 -- 172.21.15.78:0/2633377922 shutdown_connections 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.890+0000 7f8339273640 1 -- 172.21.15.78:0/2633377922 wait complete. 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8339273640 1 Processor -- start 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8339273640 1 -- start start 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8339273640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8334101970 0x7f83341992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8339273640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8334102bb0 0x7f8334199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:41.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8339273640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83341078b0 0x7f83341a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8339273640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f833410d5c0 con 0x7f83341078b0 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8339273640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f833410d440 con 0x7f8334101970 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8339273640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f833410d740 con 0x7f8334102bb0 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f83337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83341078b0 0x7f83341a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f83337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83341078b0 0x7f83341a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53506/0 (socket says 172.21.15.78:53506) 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f83337fe640 1 -- 172.21.15.78:0/4181754356 learned_addr learned my addr 172.21.15.78:0/4181754356 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8332ffd640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8334101970 0x7f83341992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f832bfff640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8334102bb0 0x7f8334199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f83337fe640 1 -- 172.21.15.78:0/4181754356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8334102bb0 msgr2=0x7f8334199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:41.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f83337fe640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8334102bb0 0x7f8334199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f83337fe640 1 -- 172.21.15.78:0/4181754356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8334101970 msgr2=0x7f83341992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f83337fe640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8334101970 0x7f83341992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f83337fe640 1 -- 172.21.15.78:0/4181754356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83341a0e30 con 0x7f83341078b0 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.891+0000 7f8332ffd640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8334101970 0x7f83341992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.892+0000 7f83337fe640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83341078b0 0x7f83341a07c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f832c00eb10 tx=0x7f832c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.892+0000 7f8330ff9640 1 -- 172.21.15.78:0/4181754356 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c004b20 con 0x7f83341078b0 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.892+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83341a1120 con 0x7f83341078b0 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.892+0000 7f8330ff9640 1 -- 172.21.15.78:0/4181754356 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832c004cc0 con 0x7f83341078b0 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.892+0000 7f8330ff9640 1 -- 172.21.15.78:0/4181754356 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c010760 con 0x7f83341078b0 2024-05-21T15:13:41.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.892+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a1670 con 0x7f83341078b0 2024-05-21T15:13:41.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.893+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83341a1930 con 0x7f83341078b0 2024-05-21T15:13:41.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.894+0000 7f8330ff9640 1 -- 172.21.15.78:0/4181754356 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f832c005000 con 0x7f83341078b0 2024-05-21T15:13:41.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.895+0000 7f8330ff9640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f83200a2b90 0x7f83200a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:41.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.895+0000 7f8332ffd640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f83200a2b90 0x7f83200a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:41.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.895+0000 7f8330ff9640 1 -- 172.21.15.78:0/4181754356 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f832c0cf410 con 0x7f83341078b0 2024-05-21T15:13:41.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.896+0000 7f8332ffd640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f83200a2b90 0x7f83200a50a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8324005ab0 tx=0x7f8324031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:41.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:41.899+0000 7f8330ff9640 1 -- 172.21.15.78:0/4181754356 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f832c053850 con 0x7f83341078b0 2024-05-21T15:13:42.097 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 ceph-mds[72447]: -- 172.21.15.78:0/3196192511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5595aaa03a40 con 0x5595aaa7ba80 2024-05-21T15:13:42.098 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j[72443]: starting mds.j at 2024-05-21T15:13:42.098 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 bash[72428]: 403f2890c9064f9cfa928fb33f3704d4b0d5fc02b34eb4842ed1cee0fef13e5b 2024-05-21T15:13:42.098 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:13:41 smithi078 systemd[1]: Started Ceph mds.j for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:42.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.116+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83341a1b00 con 0x7f83341078b0 2024-05-21T15:13:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.117+0000 7f8330ff9640 1 -- 172.21.15.78:0/4181754356 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f832c09c7f0 con 0x7f83341078b0 2024-05-21T15:13:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:42.120 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T15:13:42.120 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 3 2024-05-21T15:13:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f83200a2b90 msgr2=0x7f83200a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f83200a2b90 0x7f83200a50a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8324005ab0 tx=0x7f8324031400 comp rx=0 tx=0).stop 2024-05-21T15:13:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83341078b0 msgr2=0x7f83341a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83341078b0 0x7f83341a07c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f832c00eb10 tx=0x7f832c00c2d0 comp rx=0 tx=0).stop 2024-05-21T15:13:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 shutdown_connections 2024-05-21T15:13:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f83200a2b90 0x7f83200a50a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83341078b0 0x7f83341a07c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:42.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8334102bb0 0x7f8334199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 --2- 172.21.15.78:0/4181754356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8334101970 0x7f83341992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 >> 172.21.15.78:0/4181754356 conn(0x7f83340fcb80 msgr2=0x7f8334108d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 shutdown_connections 2024-05-21T15:13:42.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:42.119+0000 7f8339273640 1 -- 172.21.15.78:0/4181754356 wait complete. 2024-05-21T15:13:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:42 smithi084 ceph-mon[36334]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:42 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:42 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4181754356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:42.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:42 smithi078 ceph-mon[31997]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:42.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:42.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:42.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:42 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:42.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:42 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4181754356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:42 smithi172 ceph-mon[36402]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:42.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:42.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:42.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:42 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:42.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:42 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4181754356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:43 smithi078 ceph-mon[31997]: Deploying daemon mds.d on smithi078 2024-05-21T15:13:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:43 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.78:6834/4003323899,v1:172.21.15.78:6835/4003323899] up:boot 2024-05-21T15:13:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:43 smithi078 ceph-mon[31997]: fsmap 3 up:standby 2024-05-21T15:13:43.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:43 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:13:43.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:43 smithi084 ceph-mon[36334]: Deploying daemon mds.d on smithi078 2024-05-21T15:13:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:43 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.78:6834/4003323899,v1:172.21.15.78:6835/4003323899] up:boot 2024-05-21T15:13:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:43 smithi084 ceph-mon[36334]: fsmap 3 up:standby 2024-05-21T15:13:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:43 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:13:43.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:43 smithi172 ceph-mon[36402]: Deploying daemon mds.d on smithi078 2024-05-21T15:13:43.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:43 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.78:6834/4003323899,v1:172.21.15.78:6835/4003323899] up:boot 2024-05-21T15:13:43.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:43 smithi172 ceph-mon[36402]: fsmap 3 up:standby 2024-05-21T15:13:43.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:43 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:13:43.999 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:43 smithi078 systemd[1]: Starting Ceph mds.d for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:13:44.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 -- 172.21.15.78:0/2116839883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0102bd0 msgr2=0x7f5be0102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 --2- 172.21.15.78:0/2116839883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0102bd0 0x7f5be0102fe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009900 tx=0x7f5bd402a410 comp rx=0 tx=0).stop 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 -- 172.21.15.78:0/2116839883 shutdown_connections 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 --2- 172.21.15.78:0/2116839883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be01078d0 0x7f5be0109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 --2- 172.21.15.78:0/2116839883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0102bd0 0x7f5be0102fe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 --2- 172.21.15.78:0/2116839883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be0101990 0x7f5be0102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 -- 172.21.15.78:0/2116839883 >> 172.21.15.78:0/2116839883 conn(0x7f5be00fcb80 msgr2=0x7f5be00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 -- 172.21.15.78:0/2116839883 shutdown_connections 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.249+0000 7f5be5965640 1 -- 172.21.15.78:0/2116839883 wait complete. 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5be5965640 1 Processor -- start 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5be5965640 1 -- start start 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5be5965640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be0101990 0x7f5be0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5be5965640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0102bd0 0x7f5be01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5be5965640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be01078d0 0x7f5be019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5be5965640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010d600 con 0x7f5be01078d0 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5be5965640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010d480 con 0x7f5be0102bd0 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5be5965640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010d780 con 0x7f5be0101990 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.250+0000 7f5bdf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be01078d0 0x7f5be019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be01078d0 0x7f5be019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53522/0 (socket says 172.21.15.78:53522) 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdf7fe640 1 -- 172.21.15.78:0/2831029404 learned_addr learned my addr 172.21.15.78:0/2831029404 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bde7fc640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0102bd0 0x7f5be01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdf7fe640 1 -- 172.21.15.78:0/2831029404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be0101990 msgr2=0x7f5be0198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdeffd640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be0101990 0x7f5be0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdf7fe640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be0101990 0x7f5be0198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdf7fe640 1 -- 172.21.15.78:0/2831029404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0102bd0 msgr2=0x7f5be01986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdf7fe640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0102bd0 0x7f5be01986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdf7fe640 1 -- 172.21.15.78:0/2831029404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be019fcb0 con 0x7f5be01078d0 2024-05-21T15:13:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bde7fc640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0102bd0 0x7f5be01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:13:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5bdf7fe640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be01078d0 0x7f5be019f640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800cad0 tx=0x7f5bd800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5be4963640 1 -- 172.21.15.78:0/2831029404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800ae20 con 0x7f5be01078d0 2024-05-21T15:13:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5be4963640 1 -- 172.21.15.78:0/2831029404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd8005020 con 0x7f5be01078d0 2024-05-21T15:13:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be019ffa0 con 0x7f5be01078d0 2024-05-21T15:13:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5be4963640 1 -- 172.21.15.78:0/2831029404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800f6f0 con 0x7f5be01078d0 2024-05-21T15:13:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.251+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be01a0400 con 0x7f5be01078d0 2024-05-21T15:13:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.252+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be01066e0 con 0x7f5be01078d0 2024-05-21T15:13:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.254+0000 7f5be4963640 1 -- 172.21.15.78:0/2831029404 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5bd80051d0 con 0x7f5be01078d0 2024-05-21T15:13:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.254+0000 7f5be4963640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bc80a2b90 0x7f5bc80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.254+0000 7f5bdeffd640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bc80a2b90 0x7f5bc80a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.254+0000 7f5be4963640 1 -- 172.21.15.78:0/2831029404 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5bd80cf030 con 0x7f5be01078d0 2024-05-21T15:13:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.255+0000 7f5bdeffd640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bc80a2b90 0x7f5bc80a50a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0005fd0 tx=0x7f5bd0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:44.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.258+0000 7f5be4963640 1 -- 172.21.15.78:0/2831029404 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5bd8014030 con 0x7f5be01078d0 2024-05-21T15:13:44.302 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:44 smithi078 podman[72768]: 2024-05-21 15:13:44.124012745 +0000 UTC m=+0.018595861 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:13:44.302 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:44 smithi078 podman[72768]: 2024-05-21 15:13:44.299551158 +0000 UTC m=+0.194134250 container create 49275e6098244e035b3c0cbed3b244cadeb22d0f1d785729007f85c35c9972a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T15:13:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.478+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5be002d2a0 con 0x7f5be01078d0 2024-05-21T15:13:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.479+0000 7f5be4963640 1 -- 172.21.15.78:0/2831029404 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f5bd809c430 con 0x7f5be01078d0 2024-05-21T15:13:44.482 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:44.482 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T15:13:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 4 2024-05-21T15:13:44.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bc80a2b90 msgr2=0x7f5bc80a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:44.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bc80a2b90 0x7f5bc80a50a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0005fd0 tx=0x7f5bd0005850 comp rx=0 tx=0).stop 2024-05-21T15:13:44.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be01078d0 msgr2=0x7f5be019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be01078d0 0x7f5be019f640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800cad0 tx=0x7f5bd800cf90 comp rx=0 tx=0).stop 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 shutdown_connections 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bc80a2b90 0x7f5bc80a50a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5be0102bd0 0x7f5be01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be01078d0 0x7f5be019f640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 --2- 172.21.15.78:0/2831029404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5be0101990 0x7f5be0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 >> 172.21.15.78:0/2831029404 conn(0x7f5be00fcb80 msgr2=0x7f5be01038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 shutdown_connections 2024-05-21T15:13:44.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:44.481+0000 7f5be5965640 1 -- 172.21.15.78:0/2831029404 wait complete. 2024-05-21T15:13:44.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:44 smithi078 ceph-mon[31997]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:44 smithi084 ceph-mon[36334]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:44 smithi172 ceph-mon[36402]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:44.847 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:44 smithi078 podman[72768]: 2024-05-21 15:13:44.557973219 +0000 UTC m=+0.452556320 container init 49275e6098244e035b3c0cbed3b244cadeb22d0f1d785729007f85c35c9972a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-21T15:13:44.848 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:44 smithi078 podman[72768]: 2024-05-21 15:13:44.560915775 +0000 UTC m=+0.455498884 container start 49275e6098244e035b3c0cbed3b244cadeb22d0f1d785729007f85c35c9972a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-21T15:13:44.848 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:44 smithi078 ceph-mds[72828]: -- 172.21.15.78:0/409059656 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563caff3fa40 con 0x563caffb7180 2024-05-21T15:13:44.848 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:44 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d[72822]: starting mds.d at 2024-05-21T15:13:44.848 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:44 smithi078 bash[72768]: 49275e6098244e035b3c0cbed3b244cadeb22d0f1d785729007f85c35c9972a5 2024-05-21T15:13:44.848 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:13:44 smithi078 systemd[1]: Started Ceph mds.d for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:45.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:45 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2831029404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:45.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:45 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:45.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:45 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2831029404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:45.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:45.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:45.640 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:45 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:45 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2831029404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:45.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:45 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:46.496 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:46 smithi078 ceph-mon[31997]: Deploying daemon mds.e on smithi084 2024-05-21T15:13:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:46 smithi078 ceph-mon[31997]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:46.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:46 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.78:6836/3299007119,v1:172.21.15.78:6837/3299007119] up:boot 2024-05-21T15:13:46.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:46 smithi078 ceph-mon[31997]: fsmap 4 up:standby 2024-05-21T15:13:46.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:46 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:13:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 -- 172.21.15.78:0/1945215605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c1078d0 msgr2=0x7fe15c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 --2- 172.21.15.78:0/1945215605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c1078d0 0x7fe15c109cf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe154009900 tx=0x7fe15402a410 comp rx=0 tx=0).stop 2024-05-21T15:13:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 -- 172.21.15.78:0/1945215605 shutdown_connections 2024-05-21T15:13:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 --2- 172.21.15.78:0/1945215605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe15c102bd0 0x7fe15c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 --2- 172.21.15.78:0/1945215605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c1078d0 0x7fe15c109cf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 --2- 172.21.15.78:0/1945215605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe15c101990 0x7fe15c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 -- 172.21.15.78:0/1945215605 >> 172.21.15.78:0/1945215605 conn(0x7fe15c0fcb80 msgr2=0x7fe15c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 -- 172.21.15.78:0/1945215605 shutdown_connections 2024-05-21T15:13:46.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.617+0000 7fe160f45640 1 -- 172.21.15.78:0/1945215605 wait complete. 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe160f45640 1 Processor -- start 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe160f45640 1 -- start start 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe160f45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe15c101990 0x7fe15c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe160f45640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c102bd0 0x7fe15c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe160f45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe15c1078d0 0x7fe15c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe160f45640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10e710 con 0x7fe15c102bd0 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe160f45640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10e590 con 0x7fe15c1078d0 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe160f45640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10e890 con 0x7fe15c101990 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe159d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c102bd0 0x7fe15c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe159d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c102bd0 0x7fe15c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53546/0 (socket says 172.21.15.78:53546) 2024-05-21T15:13:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe159d9b640 1 -- 172.21.15.78:0/259667934 learned_addr learned my addr 172.21.15.78:0/259667934 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe15ad9d640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe15c1078d0 0x7fe15c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe15a59c640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe15c101990 0x7fe15c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe159d9b640 1 -- 172.21.15.78:0/259667934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe15c101990 msgr2=0x7fe15c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe159d9b640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe15c101990 0x7fe15c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe159d9b640 1 -- 172.21.15.78:0/259667934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe15c1078d0 msgr2=0x7fe15c1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe159d9b640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe15c1078d0 0x7fe15c1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.618+0000 7fe159d9b640 1 -- 172.21.15.78:0/259667934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c1a0e00 con 0x7fe15c102bd0 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.619+0000 7fe159d9b640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c102bd0 0x7fe15c199800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe15000e9f0 tx=0x7fe15000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.619+0000 7fe14b7fe640 1 -- 172.21.15.78:0/259667934 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15000cd80 con 0x7fe15c102bd0 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.619+0000 7fe14b7fe640 1 -- 172.21.15.78:0/259667934 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe150010910 con 0x7fe15c102bd0 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.619+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c1a10f0 con 0x7fe15c102bd0 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.619+0000 7fe14b7fe640 1 -- 172.21.15.78:0/259667934 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150018990 con 0x7fe15c102bd0 2024-05-21T15:13:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.619+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c1a1640 con 0x7fe15c102bd0 2024-05-21T15:13:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.620+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15c1066e0 con 0x7fe15c102bd0 2024-05-21T15:13:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.621+0000 7fe14b7fe640 1 -- 172.21.15.78:0/259667934 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe150010ac0 con 0x7fe15c102bd0 2024-05-21T15:13:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.622+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe1440a2b90 0x7fe1440a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.622+0000 7fe15a59c640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe1440a2b90 0x7fe1440a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.623+0000 7fe14b7fe640 1 -- 172.21.15.78:0/259667934 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe1500cf3a0 con 0x7fe15c102bd0 2024-05-21T15:13:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.623+0000 7fe15a59c640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe1440a2b90 0x7fe1440a50a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe14c005c40 tx=0x7fe14c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:46.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.626+0000 7fe14b7fe640 1 -- 172.21.15.78:0/259667934 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe15009c7a0 con 0x7fe15c102bd0 2024-05-21T15:13:46.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:46 smithi084 ceph-mon[36334]: Deploying daemon mds.e on smithi084 2024-05-21T15:13:46.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:46 smithi084 ceph-mon[36334]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:46.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:46 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.78:6836/3299007119,v1:172.21.15.78:6837/3299007119] up:boot 2024-05-21T15:13:46.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:46 smithi084 ceph-mon[36334]: fsmap 4 up:standby 2024-05-21T15:13:46.635 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:46 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:13:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:46 smithi172 ceph-mon[36402]: Deploying daemon mds.e on smithi084 2024-05-21T15:13:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:46 smithi172 ceph-mon[36402]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:46 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.78:6836/3299007119,v1:172.21.15.78:6837/3299007119] up:boot 2024-05-21T15:13:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:46 smithi172 ceph-mon[36402]: fsmap 4 up:standby 2024-05-21T15:13:46.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:46 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:13:46.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.840+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe15c02d2a0 con 0x7fe15c102bd0 2024-05-21T15:13:46.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.840+0000 7fe14b7fe640 1 -- 172.21.15.78:0/259667934 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fe15009c0a0 con 0x7fe15c102bd0 2024-05-21T15:13:46.844 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:46.844 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T15:13:46.844 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 5 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.842+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe1440a2b90 msgr2=0x7fe1440a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.842+0000 7fe160f45640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe1440a2b90 0x7fe1440a50a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe14c005c40 tx=0x7fe14c005bd0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c102bd0 msgr2=0x7fe15c199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c102bd0 0x7fe15c199800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe15000e9f0 tx=0x7fe15000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 shutdown_connections 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe1440a2b90 0x7fe1440a50a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe15c1078d0 0x7fe15c1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe15c102bd0 0x7fe15c199800 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 --2- 172.21.15.78:0/259667934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe15c101990 0x7fe15c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 >> 172.21.15.78:0/259667934 conn(0x7fe15c0fcb80 msgr2=0x7fe15c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:46.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 shutdown_connections 2024-05-21T15:13:46.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:46.843+0000 7fe160f45640 1 -- 172.21.15.78:0/259667934 wait complete. 2024-05-21T15:13:46.896 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:46 smithi084 systemd[1]: Starting Ceph mds.e for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:13:47.300 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:47 smithi084 podman[66528]: 2024-05-21 15:13:46.927658518 +0000 UTC m=+0.017151255 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:13:47.300 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:47 smithi084 podman[66528]: 2024-05-21 15:13:47.042080707 +0000 UTC m=+0.131573425 container create a93cd8b3a9103dac29da406609b078118724a604513ccb2b0dcf9241bb6c250b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514) 2024-05-21T15:13:47.300 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:47 smithi084 podman[66528]: 2024-05-21 15:13:47.261525783 +0000 UTC m=+0.351018510 container init a93cd8b3a9103dac29da406609b078118724a604513ccb2b0dcf9241bb6c250b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:13:47.300 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:47 smithi084 podman[66528]: 2024-05-21 15:13:47.264126866 +0000 UTC m=+0.353619589 container start a93cd8b3a9103dac29da406609b078118724a604513ccb2b0dcf9241bb6c250b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-21T15:13:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:47 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/259667934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:47 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/259667934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:47.646 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:47 smithi084 ceph-mds[66548]: -- 172.21.15.84:0/129670678 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634138c9a40 con 0x563413941a80 2024-05-21T15:13:47.646 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:47 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e[66544]: starting mds.e at 2024-05-21T15:13:47.646 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:47 smithi084 bash[66528]: a93cd8b3a9103dac29da406609b078118724a604513ccb2b0dcf9241bb6c250b 2024-05-21T15:13:47.646 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:13:47 smithi084 systemd[1]: Started Ceph mds.e for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:47 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/259667934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:48.410 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:48.410 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6834/4071213416,v1:172.21.15.84:6835/4071213416] up:boot 2024-05-21T15:13:48.410 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: fsmap 5 up:standby 2024-05-21T15:13:48.410 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:13:48.410 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.411 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.411 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.411 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:48.411 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:48.411 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:48 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6834/4071213416,v1:172.21.15.84:6835/4071213416] up:boot 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: fsmap 5 up:standby 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:48 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6834/4071213416,v1:172.21.15.84:6835/4071213416] up:boot 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: fsmap 5 up:standby 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:48 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:48.858 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 -- 172.21.15.78:0/1576112546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc430102ab0 msgr2=0x7fc4301037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 --2- 172.21.15.78:0/1576112546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc430102ab0 0x7fc4301037b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc420009900 tx=0x7fc42002a3f0 comp rx=0 tx=0).stop 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 -- 172.21.15.78:0/1576112546 shutdown_connections 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 --2- 172.21.15.78:0/1576112546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4301089f0 0x7fc43010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 --2- 172.21.15.78:0/1576112546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc430103cf0 0x7fc430104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 --2- 172.21.15.78:0/1576112546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc430102ab0 0x7fc4301037b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 -- 172.21.15.78:0/1576112546 >> 172.21.15.78:0/1576112546 conn(0x7fc4300fdcc0 msgr2=0x7fc430100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 -- 172.21.15.78:0/1576112546 shutdown_connections 2024-05-21T15:13:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 -- 172.21.15.78:0/1576112546 wait complete. 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.977+0000 7fc43481b640 1 Processor -- start 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc43481b640 1 -- start start 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc43481b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc430102ab0 0x7fc430076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc43481b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc430103cf0 0x7fc430074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc43481b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4301089f0 0x7fc4300754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc43481b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc43010e6e0 con 0x7fc4301089f0 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc43481b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc43010e560 con 0x7fc430103cf0 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc43481b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc43010e860 con 0x7fc430102ab0 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4301089f0 0x7fc4300754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc430102ab0 0x7fc430076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc430102ab0 0x7fc430076910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33180/0 (socket says 172.21.15.78:33180) 2024-05-21T15:13:48.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42e59c640 1 -- 172.21.15.78:0/1722033421 learned_addr learned my addr 172.21.15.78:0/1722033421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc425d9b640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc430103cf0 0x7fc430074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42ed9d640 1 -- 172.21.15.78:0/1722033421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc430102ab0 msgr2=0x7fc430076910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42ed9d640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc430102ab0 0x7fc430076910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42ed9d640 1 -- 172.21.15.78:0/1722033421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc430103cf0 msgr2=0x7fc430074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42ed9d640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc430103cf0 0x7fc430074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc42ed9d640 1 -- 172.21.15.78:0/1722033421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc430075b90 con 0x7fc4301089f0 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.978+0000 7fc425d9b640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc430103cf0 0x7fc430074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.979+0000 7fc42ed9d640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4301089f0 0x7fc4300754f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc42800cab0 tx=0x7fc42800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.979+0000 7fc427fff640 1 -- 172.21.15.78:0/1722033421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42800ae20 con 0x7fc4301089f0 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.979+0000 7fc427fff640 1 -- 172.21.15.78:0/1722033421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc42800f930 con 0x7fc4301089f0 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.979+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4301a3340 con 0x7fc4301089f0 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.979+0000 7fc427fff640 1 -- 172.21.15.78:0/1722033421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428018b30 con 0x7fc4301089f0 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.979+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4301a3750 con 0x7fc4301089f0 2024-05-21T15:13:48.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.980+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc430107800 con 0x7fc4301089f0 2024-05-21T15:13:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.981+0000 7fc427fff640 1 -- 172.21.15.78:0/1722033421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc42800fae0 con 0x7fc4301089f0 2024-05-21T15:13:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.982+0000 7fc427fff640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc41c0a2b90 0x7fc41c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.982+0000 7fc42e59c640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc41c0a2b90 0x7fc41c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.982+0000 7fc427fff640 1 -- 172.21.15.78:0/1722033421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4280cf570 con 0x7fc4301089f0 2024-05-21T15:13:48.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.982+0000 7fc42e59c640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc41c0a2b90 0x7fc41c0a50a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc420031610 tx=0x7fc420009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:48.985+0000 7fc427fff640 1 -- 172.21.15.78:0/1722033421 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc428014030 con 0x7fc4301089f0 2024-05-21T15:13:49.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.196+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc43002d2a0 con 0x7fc4301089f0 2024-05-21T15:13:49.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.197+0000 7fc427fff640 1 -- 172.21.15.78:0/1722033421 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fc42809ca00 con 0x7fc4301089f0 2024-05-21T15:13:49.200 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:49.200 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-21T15:13:49.200 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 6 2024-05-21T15:13:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc41c0a2b90 msgr2=0x7fc41c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc41c0a2b90 0x7fc41c0a50a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc420031610 tx=0x7fc420009560 comp rx=0 tx=0).stop 2024-05-21T15:13:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4301089f0 msgr2=0x7fc4300754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4301089f0 0x7fc4300754f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc42800cab0 tx=0x7fc42800cf70 comp rx=0 tx=0).stop 2024-05-21T15:13:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 shutdown_connections 2024-05-21T15:13:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc41c0a2b90 0x7fc41c0a50a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc430103cf0 0x7fc430074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4301089f0 0x7fc4300754f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:49.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 --2- 172.21.15.78:0/1722033421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc430102ab0 0x7fc430076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:49.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 >> 172.21.15.78:0/1722033421 conn(0x7fc4300fdcc0 msgr2=0x7fc4301049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:49.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 shutdown_connections 2024-05-21T15:13:49.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:49.199+0000 7fc43481b640 1 -- 172.21.15.78:0/1722033421 wait complete. 2024-05-21T15:13:49.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:49 smithi078 ceph-mon[31997]: Deploying daemon mds.f on smithi172 2024-05-21T15:13:49.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:49 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1722033421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:49.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:49 smithi084 ceph-mon[36334]: Deploying daemon mds.f on smithi172 2024-05-21T15:13:49.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:49 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1722033421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:49.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:49 smithi172 ceph-mon[36402]: Deploying daemon mds.f on smithi172 2024-05-21T15:13:49.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:49 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1722033421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:49.684 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:49 smithi172 systemd[1]: Starting Ceph mds.f for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:13:49.993 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:49 smithi172 podman[66398]: 2024-05-21 15:13:49.639478493 +0000 UTC m=+0.017493063 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:13:49.994 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:49 smithi172 podman[66398]: 2024-05-21 15:13:49.755400969 +0000 UTC m=+0.133415541 container create 354413a5be18d65fbe74ae4ab7923abd64e6137e781498dae6415556f3079e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:13:50.327 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:49 smithi172 podman[66398]: 2024-05-21 15:13:49.991476258 +0000 UTC m=+0.369490831 container init 354413a5be18d65fbe74ae4ab7923abd64e6137e781498dae6415556f3079e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-21T15:13:50.328 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:49 smithi172 podman[66398]: 2024-05-21 15:13:49.994469658 +0000 UTC m=+0.372484236 container start 354413a5be18d65fbe74ae4ab7923abd64e6137e781498dae6415556f3079e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-21T15:13:50.328 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:50 smithi172 ceph-mds[66416]: -- 172.21.15.172:0/2535816218 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55acf7e5fa40 con 0x55acf7ed7600 2024-05-21T15:13:50.328 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:50 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f[66412]: starting mds.f at 2024-05-21T15:13:50.328 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:50 smithi172 bash[66398]: 354413a5be18d65fbe74ae4ab7923abd64e6137e781498dae6415556f3079e33 2024-05-21T15:13:50.328 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:13:50 smithi172 systemd[1]: Started Ceph mds.f for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:50.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:50 smithi078 ceph-mon[31997]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:50.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:50 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:50 smithi084 ceph-mon[36334]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:50.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:50 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:50.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:50 smithi172 ceph-mon[36402]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:50.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:50.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:50.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:50.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:50 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:51.215 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.335+0000 7f851c738640 1 -- 172.21.15.78:0/3126726273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514105c30 msgr2=0x7f8514108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.335+0000 7f851c738640 1 --2- 172.21.15.78:0/3126726273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514105c30 0x7f8514108110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8504009900 tx=0x7f850402a410 comp rx=0 tx=0).stop 2024-05-21T15:13:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.335+0000 7f851c738640 1 -- 172.21.15.78:0/3126726273 shutdown_connections 2024-05-21T15:13:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.335+0000 7f851c738640 1 --2- 172.21.15.78:0/3126726273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85141087a0 0x7f851410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.335+0000 7f851c738640 1 --2- 172.21.15.78:0/3126726273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514105c30 0x7f8514108110 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.335+0000 7f851c738640 1 --2- 172.21.15.78:0/3126726273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8514100e50 0x7f8514101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.335+0000 7f851c738640 1 -- 172.21.15.78:0/3126726273 >> 172.21.15.78:0/3126726273 conn(0x7f85140fc360 msgr2=0x7f85140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.336+0000 7f851c738640 1 -- 172.21.15.78:0/3126726273 shutdown_connections 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.336+0000 7f851c738640 1 -- 172.21.15.78:0/3126726273 wait complete. 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.336+0000 7f851c738640 1 Processor -- start 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.336+0000 7f851c738640 1 -- start start 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.336+0000 7f851c738640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514100e50 0x7f8514103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.336+0000 7f851c738640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8514105c30 0x7f85141019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851c738640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85141087a0 0x7f8514101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851c738640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f851410e710 con 0x7f8514100e50 2024-05-21T15:13:51.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851c738640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f851410e590 con 0x7f85141087a0 2024-05-21T15:13:51.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851c738640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f851410e890 con 0x7f8514105c30 2024-05-21T15:13:51.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514100e50 0x7f8514103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:51.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514100e50 0x7f8514103320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40222/0 (socket says 172.21.15.78:40222) 2024-05-21T15:13:51.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 -- 172.21.15.78:0/1891928465 learned_addr learned my addr 172.21.15.78:0/1891928465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:51.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851acd5640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85141087a0 0x7f8514101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f8519cd3640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8514105c30 0x7f85141019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 -- 172.21.15.78:0/1891928465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8514105c30 msgr2=0x7f85141019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8514105c30 0x7f85141019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 -- 172.21.15.78:0/1891928465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85141087a0 msgr2=0x7f8514101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85141087a0 0x7f8514101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 -- 172.21.15.78:0/1891928465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8514102630 con 0x7f8514100e50 2024-05-21T15:13:51.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.337+0000 7f851a4d4640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514100e50 0x7f8514103320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f851000e9f0 tx=0x7f851000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:51.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.338+0000 7f850b7fe640 1 -- 172.21.15.78:0/1891928465 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851000ccc0 con 0x7f8514100e50 2024-05-21T15:13:51.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.338+0000 7f850b7fe640 1 -- 172.21.15.78:0/1891928465 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8510005500 con 0x7f8514100e50 2024-05-21T15:13:51.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.338+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85141a3340 con 0x7f8514100e50 2024-05-21T15:13:51.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.338+0000 7f850b7fe640 1 -- 172.21.15.78:0/1891928465 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510010690 con 0x7f8514100e50 2024-05-21T15:13:51.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.338+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85141a3800 con 0x7f8514100e50 2024-05-21T15:13:51.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.340+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85140fde80 con 0x7f8514100e50 2024-05-21T15:13:51.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.341+0000 7f850b7fe640 1 -- 172.21.15.78:0/1891928465 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f85100046c0 con 0x7f8514100e50 2024-05-21T15:13:51.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.341+0000 7f850b7fe640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f85000a2c10 0x7f85000a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:51.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.342+0000 7f8519cd3640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f85000a2c10 0x7f85000a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:51.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.342+0000 7f850b7fe640 1 -- 172.21.15.78:0/1891928465 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f85100cee70 con 0x7f8514100e50 2024-05-21T15:13:51.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.342+0000 7f8519cd3640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f85000a2c10 0x7f85000a5120 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8504009230 tx=0x7f8504031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:51.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.346+0000 7f850b7fe640 1 -- 172.21.15.78:0/1891928465 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f851009c270 con 0x7f8514100e50 2024-05-21T15:13:51.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:51 smithi172 ceph-mon[36402]: Deploying daemon mds.c on smithi172 2024-05-21T15:13:51.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:51 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6834/204442136,v1:172.21.15.172:6835/204442136] up:boot 2024-05-21T15:13:51.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:51 smithi172 ceph-mon[36402]: fsmap 6 up:standby 2024-05-21T15:13:51.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:51 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:13:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.558+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f851402d2a0 con 0x7f8514100e50 2024-05-21T15:13:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.558+0000 7f850b7fe640 1 -- 172.21.15.78:0/1891928465 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f851009bb70 con 0x7f8514100e50 2024-05-21T15:13:51.561 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:51.561 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-21T15:13:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 7 2024-05-21T15:13:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f85000a2c10 msgr2=0x7f85000a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f85000a2c10 0x7f85000a5120 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8504009230 tx=0x7f8504031400 comp rx=0 tx=0).stop 2024-05-21T15:13:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514100e50 msgr2=0x7f8514103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514100e50 0x7f8514103320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f851000e9f0 tx=0x7f851000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 shutdown_connections 2024-05-21T15:13:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f85000a2c10 0x7f85000a5120 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85141087a0 0x7f8514101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8514105c30 0x7f85141019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 --2- 172.21.15.78:0/1891928465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8514100e50 0x7f8514103320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 >> 172.21.15.78:0/1891928465 conn(0x7f85140fc360 msgr2=0x7f8514109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 shutdown_connections 2024-05-21T15:13:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:51.560+0000 7f851c738640 1 -- 172.21.15.78:0/1891928465 wait complete. 2024-05-21T15:13:51.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:51 smithi078 ceph-mon[31997]: Deploying daemon mds.c on smithi172 2024-05-21T15:13:51.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:51 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6834/204442136,v1:172.21.15.172:6835/204442136] up:boot 2024-05-21T15:13:51.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:51 smithi078 ceph-mon[31997]: fsmap 6 up:standby 2024-05-21T15:13:51.574 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:51 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:13:51.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:51 smithi084 ceph-mon[36334]: Deploying daemon mds.c on smithi172 2024-05-21T15:13:51.636 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:51 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6834/204442136,v1:172.21.15.172:6835/204442136] up:boot 2024-05-21T15:13:51.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:51 smithi084 ceph-mon[36334]: fsmap 6 up:standby 2024-05-21T15:13:51.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:51 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:13:52.340 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 systemd[1]: Starting Ceph mds.c for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:13:52.591 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:52 smithi172 ceph-mon[36402]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:52.591 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:52 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1891928465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:52.591 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 podman[66698]: 2024-05-21 15:13:52.364929302 +0000 UTC m=+0.017251142 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:13:52.591 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 podman[66698]: 2024-05-21 15:13:52.467168678 +0000 UTC m=+0.119490541 container create 92ef669fa1973d297791cc5642349843b32e9af729b6f5aa1d5eadbae90e9471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-21T15:13:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:52 smithi078 ceph-mon[31997]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:52 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1891928465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:52 smithi084 ceph-mon[36334]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:52 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1891928465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:53.183 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 podman[66698]: 2024-05-21 15:13:52.703100482 +0000 UTC m=+0.355422324 container init 92ef669fa1973d297791cc5642349843b32e9af729b6f5aa1d5eadbae90e9471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-21T15:13:53.184 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 podman[66698]: 2024-05-21 15:13:52.705691669 +0000 UTC m=+0.358013513 container start 92ef669fa1973d297791cc5642349843b32e9af729b6f5aa1d5eadbae90e9471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-21T15:13:53.184 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 ceph-mds[66717]: -- 172.21.15.172:0/242963945 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f0cb3fa40 con 0x558f0cbb7180 2024-05-21T15:13:53.184 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c[66713]: starting mds.c at 2024-05-21T15:13:53.184 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 bash[66698]: 92ef669fa1973d297791cc5642349843b32e9af729b6f5aa1d5eadbae90e9471 2024-05-21T15:13:53.184 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:13:52 smithi172 systemd[1]: Started Ceph mds.c for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:53.575 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:53.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.694+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/1815454318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18101970 msgr2=0x7f2f18102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:53.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.694+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/1815454318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18101970 0x7f2f18102670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009900 tx=0x7f2f0802a410 comp rx=0 tx=0).stop 2024-05-21T15:13:53.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/1815454318 shutdown_connections 2024-05-21T15:13:53.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/1815454318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f181078b0 0x7f2f18109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/1815454318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f18102bb0 0x7f2f18102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/1815454318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18101970 0x7f2f18102670 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/1815454318 >> 172.21.15.78:0/1815454318 conn(0x7f2f180fcb80 msgr2=0x7f2f180ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/1815454318 shutdown_connections 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/1815454318 wait complete. 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 Processor -- start 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 -- start start 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f18101970 0x7f2f181992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f18102bb0 0x7f2f18199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181078b0 0x7f2f181a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1810e6f0 con 0x7f2f181078b0 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1810e570 con 0x7f2f18101970 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.695+0000 7f2f1d9c3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1810e870 con 0x7f2f18102bb0 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181078b0 0x7f2f181a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181078b0 0x7f2f181a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40228/0 (socket says 172.21.15.78:40228) 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 -- 172.21.15.78:0/701585247 learned_addr learned my addr 172.21.15.78:0/701585247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f0ffff640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f18102bb0 0x7f2f18199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f18101970 0x7f2f181992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 -- 172.21.15.78:0/701585247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f18102bb0 msgr2=0x7f2f18199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f18102bb0 0x7f2f18199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 -- 172.21.15.78:0/701585247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f18101970 msgr2=0x7f2f181992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f18101970 0x7f2f181992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 -- 172.21.15.78:0/701585247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f181a0d60 con 0x7f2f181078b0 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f177fe640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181078b0 0x7f2f181a07a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000cce0 tx=0x7f2f1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f1c9c1640 1 -- 172.21.15.78:0/701585247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000f450 con 0x7f2f181078b0 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f1c9c1640 1 -- 172.21.15.78:0/701585247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1000fad0 con 0x7f2f181078b0 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f181a0ff0 con 0x7f2f181078b0 2024-05-21T15:13:53.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f1c9c1640 1 -- 172.21.15.78:0/701585247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10018c10 con 0x7f2f181078b0 2024-05-21T15:13:53.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.696+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f181a14b0 con 0x7f2f181078b0 2024-05-21T15:13:53.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.697+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f181066c0 con 0x7f2f181078b0 2024-05-21T15:13:53.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.699+0000 7f2f1c9c1640 1 -- 172.21.15.78:0/701585247 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2f1000f5f0 con 0x7f2f181078b0 2024-05-21T15:13:53.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.699+0000 7f2f1c9c1640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2f040a2c10 0x7f2f040a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:53.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.700+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2f040a2c10 0x7f2f040a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:53.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.700+0000 7f2f1c9c1640 1 -- 172.21.15.78:0/701585247 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2f100cf140 con 0x7f2f181078b0 2024-05-21T15:13:53.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.700+0000 7f2f16ffd640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2f040a2c10 0x7f2f040a5120 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f08005ab0 tx=0x7f2f08031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:53.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.703+0000 7f2f1c9c1640 1 -- 172.21.15.78:0/701585247 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f10014030 con 0x7f2f181078b0 2024-05-21T15:13:53.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.919+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f1802d2a0 con 0x7f2f181078b0 2024-05-21T15:13:53.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.920+0000 7f2f1c9c1640 1 -- 172.21.15.78:0/701585247 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f2f1009c650 con 0x7f2f181078b0 2024-05-21T15:13:53.923 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:53.924 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-21T15:13:53.924 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-05-21T15:13:53.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2f040a2c10 msgr2=0x7f2f040a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:53.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2f040a2c10 0x7f2f040a5120 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f08005ab0 tx=0x7f2f08031400 comp rx=0 tx=0).stop 2024-05-21T15:13:53.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181078b0 msgr2=0x7f2f181a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181078b0 0x7f2f181a07a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000cce0 tx=0x7f2f1000a620 comp rx=0 tx=0).stop 2024-05-21T15:13:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 shutdown_connections 2024-05-21T15:13:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2f040a2c10 0x7f2f040a5120 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f181078b0 0x7f2f181a07a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f18102bb0 0x7f2f18199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 --2- 172.21.15.78:0/701585247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f18101970 0x7f2f181992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 >> 172.21.15.78:0/701585247 conn(0x7f2f180fcb80 msgr2=0x7f2f18103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:53.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 shutdown_connections 2024-05-21T15:13:53.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:53.922+0000 7f2f1d9c3640 1 -- 172.21.15.78:0/701585247 wait complete. 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: Deploying daemon mds.h on smithi084 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] up:boot 2024-05-21T15:13:53.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: fsmap 7 up:standby 2024-05-21T15:13:53.937 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:53 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: Deploying daemon mds.h on smithi084 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] up:boot 2024-05-21T15:13:54.086 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: fsmap 7 up:standby 2024-05-21T15:13:54.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:53 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:13:54.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:54.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:54.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:54.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:54.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:54.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:54.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: Deploying daemon mds.h on smithi084 2024-05-21T15:13:54.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:54.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] up:boot 2024-05-21T15:13:54.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: fsmap 7 up:standby 2024-05-21T15:13:54.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:53 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:13:54.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:54 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/701585247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:55.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:54 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/701585247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:55.313 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 systemd[1]: Starting Ceph mds.h for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:13:55.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:54 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/701585247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:55.590 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 podman[66832]: 2024-05-21 15:13:55.213226108 +0000 UTC m=+0.017208905 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:13:55.590 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 podman[66832]: 2024-05-21 15:13:55.32223827 +0000 UTC m=+0.126221054 container create 096aa407a45fb4e975f13e734d7d293d62b00aab5acbfab59a20a26f602728e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-21T15:13:55.590 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 podman[66832]: 2024-05-21 15:13:55.54979113 +0000 UTC m=+0.353773914 container init 096aa407a45fb4e975f13e734d7d293d62b00aab5acbfab59a20a26f602728e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:13:55.590 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 podman[66832]: 2024-05-21 15:13:55.552633933 +0000 UTC m=+0.356616718 container start 096aa407a45fb4e975f13e734d7d293d62b00aab5acbfab59a20a26f602728e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:13:55.590 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 ceph-mds[66851]: -- 172.21.15.84:0/4262591671 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b94987a40 con 0x560b949ffa80 2024-05-21T15:13:55.893 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h[66847]: starting mds.h at 2024-05-21T15:13:55.894 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 bash[66832]: 096aa407a45fb4e975f13e734d7d293d62b00aab5acbfab59a20a26f602728e4 2024-05-21T15:13:55.894 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:13:55 smithi084 systemd[1]: Started Ceph mds.h for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:55.938 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:56.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 -- 172.21.15.78:0/130943091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 msgr2=0x7f64e810ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:56.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 --2- 172.21.15.78:0/130943091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 0x7f64e810ade0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009ca0 tx=0x7f64e002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 -- 172.21.15.78:0/130943091 shutdown_connections 2024-05-21T15:13:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 --2- 172.21.15.78:0/130943091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e8103cc0 0x7f64e81040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 --2- 172.21.15.78:0/130943091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 0x7f64e810ade0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 --2- 172.21.15.78:0/130943091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e8102a80 0x7f64e8103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 -- 172.21.15.78:0/130943091 >> 172.21.15.78:0/130943091 conn(0x7f64e80fdc70 msgr2=0x7f64e8100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 -- 172.21.15.78:0/130943091 shutdown_connections 2024-05-21T15:13:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.059+0000 7f64edb77640 1 -- 172.21.15.78:0/130943091 wait complete. 2024-05-21T15:13:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64edb77640 1 Processor -- start 2024-05-21T15:13:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64edb77640 1 -- start start 2024-05-21T15:13:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64edb77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e8102a80 0x7f64e8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64edb77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e8103cc0 0x7f64e8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64edb77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 0x7f64e81a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64edb77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64e810e700 con 0x7f64e81089c0 2024-05-21T15:13:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64edb77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f64e810e580 con 0x7f64e8103cc0 2024-05-21T15:13:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64edb77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64e810e880 con 0x7f64e8102a80 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 0x7f64e81a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e8102a80 0x7f64e8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 0x7f64e81a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40250/0 (socket says 172.21.15.78:40250) 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64e7fff640 1 -- 172.21.15.78:0/4146010582 learned_addr learned my addr 172.21.15.78:0/4146010582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.060+0000 7f64deffd640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e8103cc0 0x7f64e8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e77fe640 1 -- 172.21.15.78:0/4146010582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e8103cc0 msgr2=0x7f64e8199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e77fe640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e8103cc0 0x7f64e8199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e77fe640 1 -- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 msgr2=0x7f64e81a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e77fe640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 0x7f64e81a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e77fe640 1 -- 172.21.15.78:0/4146010582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e81a0d60 con 0x7f64e8102a80 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e7fff640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 0x7f64e81a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:13:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e77fe640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e8102a80 0x7f64e8199220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f64e8102350 tx=0x7f64d800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e57fa640 1 -- 172.21.15.78:0/4146010582 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d800cc40 con 0x7f64e8102a80 2024-05-21T15:13:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e81a0ff0 con 0x7f64e8102a80 2024-05-21T15:13:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e57fa640 1 -- 172.21.15.78:0/4146010582 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d800cde0 con 0x7f64e8102a80 2024-05-21T15:13:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64e57fa640 1 -- 172.21.15.78:0/4146010582 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8010690 con 0x7f64e8102a80 2024-05-21T15:13:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.061+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e81a14b0 con 0x7f64e8102a80 2024-05-21T15:13:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.062+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e81077d0 con 0x7f64e8102a80 2024-05-21T15:13:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.064+0000 7f64e57fa640 1 -- 172.21.15.78:0/4146010582 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f64d8005030 con 0x7f64e8102a80 2024-05-21T15:13:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.065+0000 7f64e57fa640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f64d40a2b90 0x7f64d40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.065+0000 7f64deffd640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f64d40a2b90 0x7f64d40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.065+0000 7f64e57fa640 1 -- 172.21.15.78:0/4146010582 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f64d80cefb0 con 0x7f64e8102a80 2024-05-21T15:13:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.065+0000 7f64deffd640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f64d40a2b90 0x7f64d40a50a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64e819a2c0 tx=0x7f64d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:56.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.072+0000 7f64e57fa640 1 -- 172.21.15.78:0/4146010582 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f64d809c3b0 con 0x7f64e8102a80 2024-05-21T15:13:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:55 smithi084 ceph-mon[36334]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:55 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:56.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:55 smithi172 ceph-mon[36402]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:56.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:55 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.288+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64e802d2a0 con 0x7f64e8102a80 2024-05-21T15:13:56.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.289+0000 7f64e57fa640 1 -- 172.21.15.78:0/4146010582 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f64d809bcb0 con 0x7f64e8102a80 2024-05-21T15:13:56.292 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:56.292 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-21T15:13:56.292 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 9 2024-05-21T15:13:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f64d40a2b90 msgr2=0x7f64d40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f64d40a2b90 0x7f64d40a50a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64e819a2c0 tx=0x7f64d0005f90 comp rx=0 tx=0).stop 2024-05-21T15:13:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e8102a80 msgr2=0x7f64e8199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e8102a80 0x7f64e8199220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f64e8102350 tx=0x7f64d800ee50 comp rx=0 tx=0).stop 2024-05-21T15:13:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 shutdown_connections 2024-05-21T15:13:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f64d40a2b90 0x7f64d40a50a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64e8103cc0 0x7f64e8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64e81089c0 0x7f64e81a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 --2- 172.21.15.78:0/4146010582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e8102a80 0x7f64e8199220 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 >> 172.21.15.78:0/4146010582 conn(0x7f64e80fdc70 msgr2=0x7f64e81049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 shutdown_connections 2024-05-21T15:13:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:56.291+0000 7f64edb77640 1 -- 172.21.15.78:0/4146010582 wait complete. 2024-05-21T15:13:56.304 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:55 smithi078 ceph-mon[31997]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:56.304 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.305 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.305 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:56.305 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:56.305 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:56.305 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:55 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:57.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:56 smithi084 ceph-mon[36334]: Deploying daemon mds.i on smithi172 2024-05-21T15:13:57.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:56 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6836/3570369413,v1:172.21.15.84:6837/3570369413] up:boot 2024-05-21T15:13:57.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:56 smithi084 ceph-mon[36334]: fsmap 8 up:standby 2024-05-21T15:13:57.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:56 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:13:57.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:56 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4146010582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:57.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:56 smithi172 ceph-mon[36402]: Deploying daemon mds.i on smithi172 2024-05-21T15:13:57.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:56 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6836/3570369413,v1:172.21.15.84:6837/3570369413] up:boot 2024-05-21T15:13:57.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:56 smithi172 ceph-mon[36402]: fsmap 8 up:standby 2024-05-21T15:13:57.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:56 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:13:57.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:56 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4146010582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:56 smithi078 ceph-mon[31997]: Deploying daemon mds.i on smithi172 2024-05-21T15:13:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:56 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6836/3570369413,v1:172.21.15.84:6837/3570369413] up:boot 2024-05-21T15:13:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:56 smithi078 ceph-mon[31997]: fsmap 8 up:standby 2024-05-21T15:13:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:56 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:13:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:56 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4146010582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:57.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:57 smithi172 ceph-mon[36402]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:58.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:13:58.334 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:57 smithi078 ceph-mon[31997]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:58.335 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:13:58 smithi172 podman[67013]: 2024-05-21 15:13:58.300657712 +0000 UTC m=+0.359714119 container start e57ba4470e22ed45cc685e3828e26f563bbae2c28fa20d7e09202b53af840ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-21T15:13:58.341 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:57 smithi084 ceph-mon[36334]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 -- 172.21.15.78:0/461824126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 msgr2=0x7fb8cc102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/461824126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 0x7fb8cc102670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009900 tx=0x7fb8c802a410 comp rx=0 tx=0).stop 2024-05-21T15:13:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 -- 172.21.15.78:0/461824126 shutdown_connections 2024-05-21T15:13:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/461824126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc1078b0 0x7fb8cc109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/461824126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8cc102bb0 0x7fb8cc102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/461824126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 0x7fb8cc102670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 -- 172.21.15.78:0/461824126 >> 172.21.15.78:0/461824126 conn(0x7fb8cc0fcb80 msgr2=0x7fb8cc0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 -- 172.21.15.78:0/461824126 shutdown_connections 2024-05-21T15:13:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.433+0000 7fb8d32a8640 1 -- 172.21.15.78:0/461824126 wait complete. 2024-05-21T15:13:58.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d32a8640 1 Processor -- start 2024-05-21T15:13:58.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d32a8640 1 -- start start 2024-05-21T15:13:58.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d32a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 0x7fb8cc1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d32a8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8cc102bb0 0x7fb8cc199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d32a8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc1078b0 0x7fb8cc1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d32a8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10d5f0 con 0x7fb8cc101970 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d32a8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10d470 con 0x7fb8cc1078b0 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d32a8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc10d770 con 0x7fb8cc102bb0 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d1044640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 0x7fb8cc1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d1044640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 0x7fb8cc1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40262/0 (socket says 172.21.15.78:40262) 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d1044640 1 -- 172.21.15.78:0/4177274699 learned_addr learned my addr 172.21.15.78:0/4177274699 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.434+0000 7fb8d0843640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8cc102bb0 0x7fb8cc199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d1845640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc1078b0 0x7fb8cc1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d1044640 1 -- 172.21.15.78:0/4177274699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8cc102bb0 msgr2=0x7fb8cc199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d1044640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8cc102bb0 0x7fb8cc199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d1044640 1 -- 172.21.15.78:0/4177274699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc1078b0 msgr2=0x7fb8cc1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d1044640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc1078b0 0x7fb8cc1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d1044640 1 -- 172.21.15.78:0/4177274699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc1a0e00 con 0x7fb8cc101970 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d1845640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc1078b0 0x7fb8cc1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d1044640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 0x7fb8cc1992c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8031440 tx=0x7fb8c8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8c27fc640 1 -- 172.21.15.78:0/4177274699 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c803f030 con 0x7fb8cc101970 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8c27fc640 1 -- 172.21.15.78:0/4177274699 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c8003ce0 con 0x7fb8cc101970 2024-05-21T15:13:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc1a1090 con 0x7fb8cc101970 2024-05-21T15:13:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8c27fc640 1 -- 172.21.15.78:0/4177274699 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8035070 con 0x7fb8cc101970 2024-05-21T15:13:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.435+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc1a15d0 con 0x7fb8cc101970 2024-05-21T15:13:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.436+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc1a1830 con 0x7fb8cc101970 2024-05-21T15:13:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.438+0000 7fb8c27fc640 1 -- 172.21.15.78:0/4177274699 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb8c8042030 con 0x7fb8cc101970 2024-05-21T15:13:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.438+0000 7fb8c27fc640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8b80a2c10 0x7fb8b80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:13:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.438+0000 7fb8d0843640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8b80a2c10 0x7fb8b80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:13:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.438+0000 7fb8c27fc640 1 -- 172.21.15.78:0/4177274699 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb8c80ef480 con 0x7fb8cc101970 2024-05-21T15:13:58.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.439+0000 7fb8d0843640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8b80a2c10 0x7fb8b80a5120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc19a360 tx=0x7fb8bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:13:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.444+0000 7fb8c27fc640 1 -- 172.21.15.78:0/4177274699 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8c80bc880 con 0x7fb8cc101970 2024-05-21T15:13:58.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.666+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8cc1a1a00 con 0x7fb8cc101970 2024-05-21T15:13:58.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.667+0000 7fb8c27fc640 1 -- 172.21.15.78:0/4177274699 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7fb8c80bc180 con 0x7fb8cc101970 2024-05-21T15:13:58.671 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:13:58.671 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-21T15:13:58.671 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 9 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8b80a2c10 msgr2=0x7fb8b80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8b80a2c10 0x7fb8b80a5120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc19a360 tx=0x7fb8bc005f90 comp rx=0 tx=0).stop 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 msgr2=0x7fb8cc1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 0x7fb8cc1992c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8031440 tx=0x7fb8c8032ce0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 shutdown_connections 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8b80a2c10 0x7fb8b80a5120 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8cc1078b0 0x7fb8cc1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8cc102bb0 0x7fb8cc199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 --2- 172.21.15.78:0/4177274699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8cc101970 0x7fb8cc1992c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:13:58.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 >> 172.21.15.78:0/4177274699 conn(0x7fb8cc0fcb80 msgr2=0x7fb8cc108d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:13:58.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 shutdown_connections 2024-05-21T15:13:58.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:13:58.669+0000 7fb8d32a8640 1 -- 172.21.15.78:0/4177274699 wait complete. 2024-05-21T15:13:58.684 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:13:58 smithi172 ceph-mds[67032]: -- 172.21.15.172:0/2074540770 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55940af6da40 con 0x55940afe5a80 2024-05-21T15:13:58.684 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:13:58 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i[67028]: starting mds.i at 2024-05-21T15:13:58.684 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:13:58 smithi172 bash[67013]: e57ba4470e22ed45cc685e3828e26f563bbae2c28fa20d7e09202b53af840ad4 2024-05-21T15:13:58.684 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:13:58 smithi172 systemd[1]: Started Ceph mds.i for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:13:59.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:59.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:59.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:59.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: Deploying daemon mds.g on smithi078 2024-05-21T15:13:59.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4177274699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:59.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] up:boot 2024-05-21T15:13:59.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: fsmap 9 up:standby 2024-05-21T15:13:59.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:13:59.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:13:59 smithi078 ceph-mon[31997]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:59.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:59.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:59.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:59.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: Deploying daemon mds.g on smithi078 2024-05-21T15:13:59.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4177274699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:59.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] up:boot 2024-05-21T15:13:59.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: fsmap 9 up:standby 2024-05-21T15:13:59.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:13:59.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:13:59 smithi084 ceph-mon[36334]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:13:59.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:13:59.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:13:59.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:13:59.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:13:59.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: Deploying daemon mds.g on smithi078 2024-05-21T15:13:59.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4177274699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:13:59.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] up:boot 2024-05-21T15:13:59.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: fsmap 9 up:standby 2024-05-21T15:13:59.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:13:59.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:13:59 smithi172 ceph-mon[36402]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:00.686 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:14:00.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 -- 172.21.15.78:0/1373023301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc1078f0 msgr2=0x7f2dbc109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:00.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/1373023301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc1078f0 0x7f2dbc109d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009ca0 tx=0x7f2db402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:14:00.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 -- 172.21.15.78:0/1373023301 shutdown_connections 2024-05-21T15:14:00.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/1373023301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc1078f0 0x7f2dbc109d10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:00.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/1373023301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dbc102bf0 0x7f2dbc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:00.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/1373023301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dbc1019b0 0x7f2dbc1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:00.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 -- 172.21.15.78:0/1373023301 >> 172.21.15.78:0/1373023301 conn(0x7f2dbc0fcba0 msgr2=0x7f2dbc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 -- 172.21.15.78:0/1373023301 shutdown_connections 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 -- 172.21.15.78:0/1373023301 wait complete. 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 Processor -- start 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.804+0000 7f2dc2b16640 1 -- start start 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2dc2b16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dbc1019b0 0x7f2dbc199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2dc2b16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc102bf0 0x7f2dbc199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2dc2b16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dbc1078f0 0x7f2dbc1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2dc2b16640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc10d610 con 0x7f2dbc102bf0 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2dc2b16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc10d490 con 0x7f2dbc1078f0 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2dc2b16640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc10d790 con 0x7f2dbc1019b0 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc102bf0 0x7f2dbc199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc102bf0 0x7f2dbc199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34300/0 (socket says 172.21.15.78:34300) 2024-05-21T15:14:00.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 -- 172.21.15.78:0/477133164 learned_addr learned my addr 172.21.15.78:0/477133164 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2dc08b2640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dbc1019b0 0x7f2dbc199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2dc10b3640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dbc1078f0 0x7f2dbc1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 -- 172.21.15.78:0/477133164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dbc1019b0 msgr2=0x7f2dbc199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dbc1019b0 0x7f2dbc199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 -- 172.21.15.78:0/477133164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dbc1078f0 msgr2=0x7f2dbc1a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dbc1078f0 0x7f2dbc1a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 -- 172.21.15.78:0/477133164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc1a0ce0 con 0x7f2dbc102bf0 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db3fff640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc102bf0 0x7f2dbc199790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c890 tx=0x7f2dac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db1ffb640 1 -- 172.21.15.78:0/477133164 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac004220 con 0x7f2dbc102bf0 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.805+0000 7f2db1ffb640 1 -- 172.21.15.78:0/477133164 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dac0043c0 con 0x7f2dbc102bf0 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.806+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc1a0fd0 con 0x7f2dbc102bf0 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.806+0000 7f2db1ffb640 1 -- 172.21.15.78:0/477133164 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0049e0 con 0x7f2dbc102bf0 2024-05-21T15:14:00.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.806+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc1a1440 con 0x7f2dbc102bf0 2024-05-21T15:14:00.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.807+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dbc106ff0 con 0x7f2dbc102bf0 2024-05-21T15:14:00.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.808+0000 7f2db1ffb640 1 -- 172.21.15.78:0/477133164 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2dac0079b0 con 0x7f2dbc102bf0 2024-05-21T15:14:00.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.808+0000 7f2db1ffb640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2da40a2b90 0x7f2da40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:00.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.809+0000 7f2dc08b2640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2da40a2b90 0x7f2da40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:00.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.809+0000 7f2db1ffb640 1 -- 172.21.15.78:0/477133164 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2dac0cecb0 con 0x7f2dbc102bf0 2024-05-21T15:14:00.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.809+0000 7f2dc08b2640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2da40a2b90 0x7f2da40a50a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2db80058c0 tx=0x7f2db8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:00.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:00.813+0000 7f2db1ffb640 1 -- 172.21.15.78:0/477133164 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2dac014030 con 0x7f2dbc102bf0 2024-05-21T15:14:01.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.034+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dbc02d450 con 0x7f2dbc102bf0 2024-05-21T15:14:01.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.034+0000 7f2db1ffb640 1 -- 172.21.15.78:0/477133164 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f2dac09c0b0 con 0x7f2dbc102bf0 2024-05-21T15:14:01.038 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:01.038 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-21T15:14:01.039 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 10 2024-05-21T15:14:01.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2da40a2b90 msgr2=0x7f2da40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:01.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2da40a2b90 0x7f2da40a50a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2db80058c0 tx=0x7f2db8005850 comp rx=0 tx=0).stop 2024-05-21T15:14:01.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc102bf0 msgr2=0x7f2dbc199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc102bf0 0x7f2dbc199790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c890 tx=0x7f2dac00cd50 comp rx=0 tx=0).stop 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 shutdown_connections 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f2da40a2b90 0x7f2da40a50a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2dbc1078f0 0x7f2dbc1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dbc102bf0 0x7f2dbc199790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 --2- 172.21.15.78:0/477133164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dbc1019b0 0x7f2dbc199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 >> 172.21.15.78:0/477133164 conn(0x7f2dbc0fcba0 msgr2=0x7f2dbc0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.036+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 shutdown_connections 2024-05-21T15:14:01.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:01.037+0000 7f2dc2b16640 1 -- 172.21.15.78:0/477133164 wait complete. 2024-05-21T15:14:01.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:01 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/477133164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:01.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:01 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/477133164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:01.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:01 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/477133164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:01.598 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:14:01 smithi078 systemd[1]: Started Ceph mds.g for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:14:02.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:02 smithi084 ceph-mon[36334]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:02.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:02 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:02 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:02 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:02 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:14:02.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:02 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:14:02.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:02 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:02.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:02 smithi078 ceph-mon[31997]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:02.425 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:02 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:02 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:02 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:02 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:14:02.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:02 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:14:02.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:02 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:02 smithi172 ceph-mon[36402]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:02 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:02 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:02 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:02.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:02 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:14:02.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:02 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:14:02.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:02 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:03.053 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:14:03.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 -- 172.21.15.78:0/185525161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144108720 msgr2=0x7ff14410ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:03.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 --2- 172.21.15.78:0/185525161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144108720 0x7ff14410ac30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009ca0 tx=0x7ff13c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 -- 172.21.15.78:0/185525161 shutdown_connections 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 --2- 172.21.15.78:0/185525161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144108720 0x7ff14410ac30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 --2- 172.21.15.78:0/185525161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff144105bb0 0x7ff144108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 --2- 172.21.15.78:0/185525161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff144100dd0 0x7ff1441011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 -- 172.21.15.78:0/185525161 >> 172.21.15.78:0/185525161 conn(0x7ff1440fc2e0 msgr2=0x7ff1440fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 -- 172.21.15.78:0/185525161 shutdown_connections 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 -- 172.21.15.78:0/185525161 wait complete. 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.167+0000 7ff1497b7640 1 Processor -- start 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1497b7640 1 -- start start 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1497b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff144100dd0 0x7ff1441032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1497b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144105bb0 0x7ff144101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:03.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1497b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff144108720 0x7ff144101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1497b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff14410e750 con 0x7ff144105bb0 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1497b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff14410e5d0 con 0x7ff144108720 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1497b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff14410e8d0 con 0x7ff144100dd0 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144105bb0 0x7ff144101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1427fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144105bb0 0x7ff144101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34342/0 (socket says 172.21.15.78:34342) 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1427fc640 1 -- 172.21.15.78:0/2737319684 learned_addr learned my addr 172.21.15.78:0/2737319684 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1437fe640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff144108720 0x7ff144101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff142ffd640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff144100dd0 0x7ff1441032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1427fc640 1 -- 172.21.15.78:0/2737319684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff144100dd0 msgr2=0x7ff1441032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1427fc640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff144100dd0 0x7ff1441032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1427fc640 1 -- 172.21.15.78:0/2737319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff144108720 msgr2=0x7ff144101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1427fc640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff144108720 0x7ff144101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.168+0000 7ff1427fc640 1 -- 172.21.15.78:0/2737319684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1441025b0 con 0x7ff144105bb0 2024-05-21T15:14:03.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.169+0000 7ff1427fc640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144105bb0 0x7ff144101940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff13800b4a0 tx=0x7ff13800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:03.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.169+0000 7ff12bfff640 1 -- 172.21.15.78:0/2737319684 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138021070 con 0x7ff144105bb0 2024-05-21T15:14:03.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.169+0000 7ff12bfff640 1 -- 172.21.15.78:0/2737319684 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff13801c920 con 0x7ff144105bb0 2024-05-21T15:14:03.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.169+0000 7ff12bfff640 1 -- 172.21.15.78:0/2737319684 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13801a9e0 con 0x7ff144105bb0 2024-05-21T15:14:03.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.169+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff144198eb0 con 0x7ff144105bb0 2024-05-21T15:14:03.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.169+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff144199430 con 0x7ff144105bb0 2024-05-21T15:14:03.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.170+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1440682c0 con 0x7ff144105bb0 2024-05-21T15:14:03.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.172+0000 7ff12bfff640 1 -- 172.21.15.78:0/2737319684 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff138024070 con 0x7ff144105bb0 2024-05-21T15:14:03.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.172+0000 7ff12bfff640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff12c0a2c10 0x7ff12c0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:03.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.173+0000 7ff142ffd640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff12c0a2c10 0x7ff12c0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:03.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.173+0000 7ff12bfff640 1 -- 172.21.15.78:0/2737319684 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff1380d6150 con 0x7ff144105bb0 2024-05-21T15:14:03.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.173+0000 7ff142ffd640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff12c0a2c10 0x7ff12c0a5120 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff134005fd0 tx=0x7ff134005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:03.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.182+0000 7ff12bfff640 1 -- 172.21.15.78:0/2737319684 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1380a2f00 con 0x7ff144105bb0 2024-05-21T15:14:03.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:03 smithi078 ceph-mon[31997]: Deploying daemon mds.a on smithi078 2024-05-21T15:14:03.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:03 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.78:6838/3881579182,v1:172.21.15.78:6839/3881579182] up:boot 2024-05-21T15:14:03.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:03 smithi078 ceph-mon[31997]: fsmap 10 up:standby 2024-05-21T15:14:03.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:03 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:14:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:03 smithi084 ceph-mon[36334]: Deploying daemon mds.a on smithi078 2024-05-21T15:14:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:03 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.78:6838/3881579182,v1:172.21.15.78:6839/3881579182] up:boot 2024-05-21T15:14:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:03 smithi084 ceph-mon[36334]: fsmap 10 up:standby 2024-05-21T15:14:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:03 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:14:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.414+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1440fde00 con 0x7ff144105bb0 2024-05-21T15:14:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.415+0000 7ff12bfff640 1 -- 172.21.15.78:0/2737319684 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7ff1380a7200 con 0x7ff144105bb0 2024-05-21T15:14:03.419 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:03.419 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-21T15:14:03.420 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 11 2024-05-21T15:14:03.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff12c0a2c10 msgr2=0x7ff12c0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff12c0a2c10 0x7ff12c0a5120 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff134005fd0 tx=0x7ff134005850 comp rx=0 tx=0).stop 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144105bb0 msgr2=0x7ff144101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144105bb0 0x7ff144101940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff13800b4a0 tx=0x7ff13800b960 comp rx=0 tx=0).stop 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 shutdown_connections 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff12c0a2c10 0x7ff12c0a5120 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff144108720 0x7ff144101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff144105bb0 0x7ff144101940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 --2- 172.21.15.78:0/2737319684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff144100dd0 0x7ff1441032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 >> 172.21.15.78:0/2737319684 conn(0x7ff1440fc2e0 msgr2=0x7ff144106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 shutdown_connections 2024-05-21T15:14:03.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:03.418+0000 7ff1497b7640 1 -- 172.21.15.78:0/2737319684 wait complete. 2024-05-21T15:14:03.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:03 smithi172 ceph-mon[36402]: Deploying daemon mds.a on smithi078 2024-05-21T15:14:03.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:03 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.78:6838/3881579182,v1:172.21.15.78:6839/3881579182] up:boot 2024-05-21T15:14:03.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:03 smithi172 ceph-mon[36402]: fsmap 10 up:standby 2024-05-21T15:14:03.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:03 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:14:03.848 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:03 smithi078 systemd[1]: Starting Ceph mds.a for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:14:04.145 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:04 smithi078 ceph-mon[31997]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:04.145 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:04 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2737319684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:04.145 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:03 smithi078 podman[73755]: 2024-05-21 15:14:03.793240378 +0000 UTC m=+0.017053281 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:14:04.145 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:03 smithi078 podman[73755]: 2024-05-21 15:14:03.904388418 +0000 UTC m=+0.128201308 container create 4be056043a8e73470c1127e2698787486516f2422801b6d9ab80d4d30007af29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, ceph=True) 2024-05-21T15:14:04.145 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:04 smithi078 podman[73755]: 2024-05-21 15:14:04.140187366 +0000 UTC m=+0.364000274 container init 4be056043a8e73470c1127e2698787486516f2422801b6d9ab80d4d30007af29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:14:04.145 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:04 smithi078 podman[73755]: 2024-05-21 15:14:04.143002308 +0000 UTC m=+0.366815198 container start 4be056043a8e73470c1127e2698787486516f2422801b6d9ab80d4d30007af29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-21T15:14:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:04 smithi084 ceph-mon[36334]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:04 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2737319684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:04.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:04 smithi172 ceph-mon[36402]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:04.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:04 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2737319684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:04.597 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:04 smithi078 ceph-mds[73774]: -- 172.21.15.78:0/2205790069 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648bd4b3a40 con 0x5648bd52b600 2024-05-21T15:14:04.598 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:04 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a[73770]: starting mds.a at 2024-05-21T15:14:04.598 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:04 smithi078 bash[73755]: 4be056043a8e73470c1127e2698787486516f2422801b6d9ab80d4d30007af29 2024-05-21T15:14:04.598 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:14:04 smithi078 systemd[1]: Started Ceph mds.a for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:14:05.435 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:14:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 -- 172.21.15.78:0/3743605123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b01019b0 msgr2=0x7f38b01026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3743605123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b01019b0 0x7f38b01026b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009900 tx=0x7f38a002a410 comp rx=0 tx=0).stop 2024-05-21T15:14:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 -- 172.21.15.78:0/3743605123 shutdown_connections 2024-05-21T15:14:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3743605123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38b01078f0 0x7f38b0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3743605123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b0102bf0 0x7f38b0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3743605123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b01019b0 0x7f38b01026b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 -- 172.21.15.78:0/3743605123 >> 172.21.15.78:0/3743605123 conn(0x7f38b00fcba0 msgr2=0x7f38b00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 -- 172.21.15.78:0/3743605123 shutdown_connections 2024-05-21T15:14:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.568+0000 7f38b5faa640 1 -- 172.21.15.78:0/3743605123 wait complete. 2024-05-21T15:14:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.569+0000 7f38b5faa640 1 Processor -- start 2024-05-21T15:14:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.569+0000 7f38b5faa640 1 -- start start 2024-05-21T15:14:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.569+0000 7f38b5faa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38b01019b0 0x7f38b01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.569+0000 7f38b5faa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b0102bf0 0x7f38b01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.569+0000 7f38b5faa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b01078f0 0x7f38b01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.569+0000 7f38b5faa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38b010e730 con 0x7f38b0102bf0 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.569+0000 7f38b5faa640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38b010e5b0 con 0x7f38b01019b0 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.569+0000 7f38b5faa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38b010e8b0 con 0x7f38b01078f0 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b0102bf0 0x7f38b01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b0102bf0 0x7f38b01997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34384/0 (socket says 172.21.15.78:34384) 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 -- 172.21.15.78:0/3916437776 learned_addr learned my addr 172.21.15.78:0/3916437776 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38af7fe640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38b01019b0 0x7f38b01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38affff640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b01078f0 0x7f38b01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 -- 172.21.15.78:0/3916437776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b01078f0 msgr2=0x7f38b01a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b01078f0 0x7f38b01a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 -- 172.21.15.78:0/3916437776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38b01019b0 msgr2=0x7f38b01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38b01019b0 0x7f38b01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 -- 172.21.15.78:0/3916437776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b01a0df0 con 0x7f38b0102bf0 2024-05-21T15:14:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38aeffd640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b0102bf0 0x7f38b01997f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38a400d770 tx=0x7f38a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38acff9640 1 -- 172.21.15.78:0/3916437776 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4017070 con 0x7f38b0102bf0 2024-05-21T15:14:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38acff9640 1 -- 172.21.15.78:0/3916437776 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a4005500 con 0x7f38b0102bf0 2024-05-21T15:14:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b01a10e0 con 0x7f38b0102bf0 2024-05-21T15:14:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38acff9640 1 -- 172.21.15.78:0/3916437776 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4004e00 con 0x7f38b0102bf0 2024-05-21T15:14:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.570+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b01a15e0 con 0x7f38b0102bf0 2024-05-21T15:14:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.571+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b0106700 con 0x7f38b0102bf0 2024-05-21T15:14:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.573+0000 7f38acff9640 1 -- 172.21.15.78:0/3916437776 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38a4004fa0 con 0x7f38b0102bf0 2024-05-21T15:14:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.573+0000 7f38acff9640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f38980a2b90 0x7f38980a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.574+0000 7f38af7fe640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f38980a2b90 0x7f38980a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.574+0000 7f38acff9640 1 -- 172.21.15.78:0/3916437776 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f38a40ce9a0 con 0x7f38b0102bf0 2024-05-21T15:14:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.574+0000 7f38af7fe640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f38980a2b90 0x7f38980a50a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f38a0005ab0 tx=0x7f38a0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.577+0000 7f38acff9640 1 -- 172.21.15.78:0/3916437776 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38a409b750 con 0x7f38b0102bf0 2024-05-21T15:14:05.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:14:05.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:14:05.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:05.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.78:6840/2631303889,v1:172.21.15.78:6841/2631303889] up:boot 2024-05-21T15:14:05.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: fsmap 11 up:standby 2024-05-21T15:14:05.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:05 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.78:6840/2631303889,v1:172.21.15.78:6841/2631303889] up:boot 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: fsmap 11 up:standby 2024-05-21T15:14:05.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:05 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:14:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T15:14:05.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T15:14:05.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:05.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.78:6840/2631303889,v1:172.21.15.78:6841/2631303889] up:boot 2024-05-21T15:14:05.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: fsmap 11 up:standby 2024-05-21T15:14:05.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:05 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:14:05.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.792+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38b002d2a0 con 0x7f38b0102bf0 2024-05-21T15:14:05.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.792+0000 7f38acff9640 1 -- 172.21.15.78:0/3916437776 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f38a409fa50 con 0x7f38b0102bf0 2024-05-21T15:14:05.796 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:05.797 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-21T15:14:05.797 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 12 2024-05-21T15:14:05.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.794+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f38980a2b90 msgr2=0x7f38980a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.794+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f38980a2b90 0x7f38980a50a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f38a0005ab0 tx=0x7f38a0031400 comp rx=0 tx=0).stop 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.794+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b0102bf0 msgr2=0x7f38b01997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.794+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b0102bf0 0x7f38b01997f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38a400d770 tx=0x7f38a400dc30 comp rx=0 tx=0).stop 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.794+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 shutdown_connections 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.794+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f38980a2b90 0x7f38980a50a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.794+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38b01078f0 0x7f38b01a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.794+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38b0102bf0 0x7f38b01997f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.795+0000 7f38b5faa640 1 --2- 172.21.15.78:0/3916437776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38b01019b0 0x7f38b01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.795+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 >> 172.21.15.78:0/3916437776 conn(0x7f38b00fcba0 msgr2=0x7f38b01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.795+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 shutdown_connections 2024-05-21T15:14:05.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:05.795+0000 7f38b5faa640 1 -- 172.21.15.78:0/3916437776 wait complete. 2024-05-21T15:14:06.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:06 smithi078 ceph-mon[31997]: Deploying daemon mds.b on smithi084 2024-05-21T15:14:06.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:06 smithi078 ceph-mon[31997]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:06.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:06 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3916437776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:06.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:06 smithi084 ceph-mon[36334]: Deploying daemon mds.b on smithi084 2024-05-21T15:14:06.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:06 smithi084 ceph-mon[36334]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:06.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:06 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3916437776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:06.647 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:06 smithi084 systemd[1]: Starting Ceph mds.b for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:14:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:06 smithi172 ceph-mon[36402]: Deploying daemon mds.b on smithi084 2024-05-21T15:14:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:06 smithi172 ceph-mon[36402]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:06 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3916437776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:06.966 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:06 smithi084 podman[67146]: 2024-05-21 15:14:06.602126747 +0000 UTC m=+0.017248297 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:14:06.966 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:06 smithi084 podman[67146]: 2024-05-21 15:14:06.719967256 +0000 UTC m=+0.135088839 container create 57cd46193d8aca42a840a813fcf80923405ce9d13da82e207ad3ea3d053b4d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-21T15:14:07.396 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:06 smithi084 podman[67146]: 2024-05-21 15:14:06.964032656 +0000 UTC m=+0.379154197 container init 57cd46193d8aca42a840a813fcf80923405ce9d13da82e207ad3ea3d053b4d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-21T15:14:07.396 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:06 smithi084 podman[67146]: 2024-05-21 15:14:06.96669686 +0000 UTC m=+0.381818395 container start 57cd46193d8aca42a840a813fcf80923405ce9d13da82e207ad3ea3d053b4d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-21T15:14:07.396 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:06 smithi084 ceph-mds[67165]: -- 172.21.15.84:0/3439049304 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611b355fa40 con 0x5611b35d7180 2024-05-21T15:14:07.396 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:07 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: starting mds.b at 2024-05-21T15:14:07.396 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:07 smithi084 bash[67146]: 57cd46193d8aca42a840a813fcf80923405ce9d13da82e207ad3ea3d053b4d34 2024-05-21T15:14:07.396 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:07 smithi084 systemd[1]: Started Ceph mds.b for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:14:07.811 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:14:07.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.932+0000 7f3ebad1a640 1 -- 172.21.15.78:0/2369934453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4107050 msgr2=0x7f3eb4109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.932+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/2369934453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4107050 0x7f3eb4109470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00adb0 tx=0x7f3eac02b7a0 comp rx=0 tx=0).stop 2024-05-21T15:14:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.932+0000 7f3ebad1a640 1 -- 172.21.15.78:0/2369934453 shutdown_connections 2024-05-21T15:14:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.932+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/2369934453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4107050 0x7f3eb4109470 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.932+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/2369934453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb4102350 0x7f3eb4102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.932+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/2369934453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb4101110 0x7f3eb4101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.932+0000 7f3ebad1a640 1 -- 172.21.15.78:0/2369934453 >> 172.21.15.78:0/2369934453 conn(0x7f3eb40fc2e0 msgr2=0x7f3eb40fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.932+0000 7f3ebad1a640 1 -- 172.21.15.78:0/2369934453 shutdown_connections 2024-05-21T15:14:07.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.933+0000 7f3ebad1a640 1 -- 172.21.15.78:0/2369934453 wait complete. 2024-05-21T15:14:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.933+0000 7f3ebad1a640 1 Processor -- start 2024-05-21T15:14:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.933+0000 7f3ebad1a640 1 -- start start 2024-05-21T15:14:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.933+0000 7f3ebad1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb4101110 0x7f3eb419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:07.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3ebad1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4102350 0x7f3eb419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3ebad1a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb4107050 0x7f3eb41a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3ebad1a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410c540 con 0x7f3eb4102350 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3ebad1a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410c3c0 con 0x7f3eb4101110 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3ebad1a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410c6c0 con 0x7f3eb4107050 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4102350 0x7f3eb419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4102350 0x7f3eb419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34400/0 (socket says 172.21.15.78:34400) 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 -- 172.21.15.78:0/3497212098 learned_addr learned my addr 172.21.15.78:0/3497212098 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb8ab6640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb4101110 0x7f3eb419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb92b7640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb4107050 0x7f3eb41a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 -- 172.21.15.78:0/3497212098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb4107050 msgr2=0x7f3eb41a6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb4107050 0x7f3eb41a6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 -- 172.21.15.78:0/3497212098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb4101110 msgr2=0x7f3eb419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:07.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb4101110 0x7f3eb419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 -- 172.21.15.78:0/3497212098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb41a7380 con 0x7f3eb4102350 2024-05-21T15:14:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb3fff640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4102350 0x7f3eb419fd80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000e9f0 tx=0x7f3ea000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb1ffb640 1 -- 172.21.15.78:0/3497212098 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000cd80 con 0x7f3eb4102350 2024-05-21T15:14:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb1ffb640 1 -- 172.21.15.78:0/3497212098 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea0005500 con 0x7f3eb4102350 2024-05-21T15:14:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb41a7670 con 0x7f3eb4102350 2024-05-21T15:14:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.934+0000 7f3eb1ffb640 1 -- 172.21.15.78:0/3497212098 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0010690 con 0x7f3eb4102350 2024-05-21T15:14:07.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.935+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a7b70 con 0x7f3eb4102350 2024-05-21T15:14:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.936+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb40682c0 con 0x7f3eb4102350 2024-05-21T15:14:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.937+0000 7f3eb1ffb640 1 -- 172.21.15.78:0/3497212098 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3ea00046c0 con 0x7f3eb4102350 2024-05-21T15:14:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.938+0000 7f3eb1ffb640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3ea40a2b90 0x7f3ea40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.938+0000 7f3eb8ab6640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3ea40a2b90 0x7f3ea40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.938+0000 7f3eb1ffb640 1 -- 172.21.15.78:0/3497212098 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3ea00cef80 con 0x7f3eb4102350 2024-05-21T15:14:07.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.938+0000 7f3eb8ab6640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3ea40a2b90 0x7f3ea40a50a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4101d10 tx=0x7f3ea800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:07.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:07.942+0000 7f3eb1ffb640 1 -- 172.21.15.78:0/3497212098 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ea009c380 con 0x7f3eb4102350 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] up:boot 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: fsmap 12 up:standby 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:14:08.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:08 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:14:08.137 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.137 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.137 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.137 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.137 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.137 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:08.137 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] up:boot 2024-05-21T15:14:08.137 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: fsmap 12 up:standby 2024-05-21T15:14:08.138 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:14:08.138 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:08 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:14:08.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.167+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3eb4062750 con 0x7f3eb4102350 2024-05-21T15:14:08.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.168+0000 7f3eb1ffb640 1 -- 172.21.15.78:0/3497212098 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f3ea009bc80 con 0x7f3eb4102350 2024-05-21T15:14:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:08.172 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-21T15:14:08.172 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 13 2024-05-21T15:14:08.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3ea40a2b90 msgr2=0x7f3ea40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3ea40a2b90 0x7f3ea40a50a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4101d10 tx=0x7f3ea800b390 comp rx=0 tx=0).stop 2024-05-21T15:14:08.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4102350 msgr2=0x7f3eb419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4102350 0x7f3eb419fd80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000e9f0 tx=0x7f3ea000edf0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 shutdown_connections 2024-05-21T15:14:08.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3ea40a2b90 0x7f3ea40a50a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3eb4107050 0x7f3eb41a6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3eb4102350 0x7f3eb419fd80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 --2- 172.21.15.78:0/3497212098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eb4101110 0x7f3eb419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 >> 172.21.15.78:0/3497212098 conn(0x7f3eb40fc2e0 msgr2=0x7f3eb4103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:08.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 shutdown_connections 2024-05-21T15:14:08.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.170+0000 7f3ebad1a640 1 -- 172.21.15.78:0/3497212098 wait complete. 2024-05-21T15:14:08.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.300+0000 7fb0848ca640 1 -- 172.21.15.78:0/3459652839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080104030 msgr2=0x7fb080104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.300+0000 7fb0848ca640 1 --2- 172.21.15.78:0/3459652839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080104030 0x7fb080104440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb070009900 tx=0x7fb07002a410 comp rx=0 tx=0).stop 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 -- 172.21.15.78:0/3459652839 shutdown_connections 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 --2- 172.21.15.78:0/3459652839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0801006e0 0x7fb080100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 --2- 172.21.15.78:0/3459652839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0801011e0 0x7fb08010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 --2- 172.21.15.78:0/3459652839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080104030 0x7fb080104440 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 -- 172.21.15.78:0/3459652839 >> 172.21.15.78:0/3459652839 conn(0x7fb0800fc2e0 msgr2=0x7fb0800fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 -- 172.21.15.78:0/3459652839 shutdown_connections 2024-05-21T15:14:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 -- 172.21.15.78:0/3459652839 wait complete. 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 Processor -- start 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.301+0000 7fb0848ca640 1 -- start start 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb0848ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0801006e0 0x7fb0801992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb0848ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0801011e0 0x7fb080199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb0848ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb080104030 0x7fb0801a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb0848ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb08010e6f0 con 0x7fb0801006e0 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb0848ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb08010e570 con 0x7fb080104030 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb0848ca640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb08010e870 con 0x7fb0801011e0 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0801006e0 0x7fb0801992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0801006e0 0x7fb0801992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34412/0 (socket says 172.21.15.78:34412) 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07e59c640 1 -- 172.21.15.78:0/157056492 learned_addr learned my addr 172.21.15.78:0/157056492 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07ed9d640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb080104030 0x7fb0801a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07dd9b640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0801011e0 0x7fb080199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07ed9d640 1 -- 172.21.15.78:0/157056492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0801011e0 msgr2=0x7fb080199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07ed9d640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0801011e0 0x7fb080199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07ed9d640 1 -- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0801006e0 msgr2=0x7fb0801992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07ed9d640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0801006e0 0x7fb0801992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07ed9d640 1 -- 172.21.15.78:0/157056492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0801a0e00 con 0x7fb080104030 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07dd9b640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0801011e0 0x7fb080199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07e59c640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0801006e0 0x7fb0801992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.302+0000 7fb07ed9d640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb080104030 0x7fb0801a0790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e940 tx=0x7fb07800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.303+0000 7fb06f7fe640 1 -- 172.21.15.78:0/157056492 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078004d20 con 0x7fb080104030 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.303+0000 7fb06f7fe640 1 -- 172.21.15.78:0/157056492 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb078005020 con 0x7fb080104030 2024-05-21T15:14:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.303+0000 7fb06f7fe640 1 -- 172.21.15.78:0/157056492 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0780116b0 con 0x7fb080104030 2024-05-21T15:14:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.303+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0801a10f0 con 0x7fb080104030 2024-05-21T15:14:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.303+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0801a15f0 con 0x7fb080104030 2024-05-21T15:14:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.304+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0801079c0 con 0x7fb080104030 2024-05-21T15:14:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.305+0000 7fb06f7fe640 1 -- 172.21.15.78:0/157056492 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb07800cb20 con 0x7fb080104030 2024-05-21T15:14:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.306+0000 7fb06f7fe640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0680a2c10 0x7fb0680a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.306+0000 7fb07e59c640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0680a2c10 0x7fb0680a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.306+0000 7fb06f7fe640 1 -- 172.21.15.78:0/157056492 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb0780cefe0 con 0x7fb080104030 2024-05-21T15:14:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.306+0000 7fb07e59c640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0680a2c10 0x7fb0680a5120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb070031610 tx=0x7fb0700046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:08.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.311+0000 7fb06f7fe640 1 -- 172.21.15.78:0/157056492 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb078016030 con 0x7fb080104030 2024-05-21T15:14:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:08.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:08.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] up:boot 2024-05-21T15:14:08.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: fsmap 12 up:standby 2024-05-21T15:14:08.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:14:08.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:08 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:14:08.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.488+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb080062750 con 0x7fb080104030 2024-05-21T15:14:08.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.490+0000 7fb06f7fe640 1 -- 172.21.15.78:0/157056492 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18384 (secure 0 0 0) 0x7fb07809c380 con 0x7fb080104030 2024-05-21T15:14:08.494 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:08.494 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":81,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:13:24.309719+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0680a2c10 msgr2=0x7fb0680a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0680a2c10 0x7fb0680a5120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb070031610 tx=0x7fb0700046c0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb080104030 msgr2=0x7fb0801a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb080104030 0x7fb0801a0790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e940 tx=0x7fb07800ee00 comp rx=0 tx=0).stop 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 shutdown_connections 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb0680a2c10 0x7fb0680a5120 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb080104030 0x7fb0801a0790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0801011e0 0x7fb080199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 --2- 172.21.15.78:0/157056492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0801006e0 0x7fb0801992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 >> 172.21.15.78:0/157056492 conn(0x7fb0800fc2e0 msgr2=0x7fb080104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 shutdown_connections 2024-05-21T15:14:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.492+0000 7fb0848ca640 1 -- 172.21.15.78:0/157056492 wait complete. 2024-05-21T15:14:08.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:14:08.507 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:14:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 -- 172.21.15.78:0/3209793265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 msgr2=0x7ff6a4102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 --2- 172.21.15.78:0/3209793265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 0x7ff6a4102ff0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009900 tx=0x7ff6a002a410 comp rx=0 tx=0).stop 2024-05-21T15:14:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 -- 172.21.15.78:0/3209793265 shutdown_connections 2024-05-21T15:14:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 --2- 172.21.15.78:0/3209793265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6a41078e0 0x7ff6a4109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 --2- 172.21.15.78:0/3209793265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 0x7ff6a4102ff0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 --2- 172.21.15.78:0/3209793265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a41019a0 0x7ff6a41026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 -- 172.21.15.78:0/3209793265 >> 172.21.15.78:0/3209793265 conn(0x7ff6a40fcb90 msgr2=0x7ff6a40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 -- 172.21.15.78:0/3209793265 shutdown_connections 2024-05-21T15:14:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.621+0000 7ff6ac337640 1 -- 172.21.15.78:0/3209793265 wait complete. 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6ac337640 1 Processor -- start 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6ac337640 1 -- start start 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6ac337640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a41019a0 0x7ff6a406acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6ac337640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 0x7ff6a41a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6ac337640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6a41078e0 0x7ff6a41a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6ac337640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a410d620 con 0x7ff6a4102be0 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6ac337640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a410d4a0 con 0x7ff6a41019a0 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6ac337640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a410d7a0 con 0x7ff6a41078e0 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6a98d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 0x7ff6a41a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6a98d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 0x7ff6a41a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55610/0 (socket says 172.21.15.78:55610) 2024-05-21T15:14:08.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6a98d2640 1 -- 172.21.15.78:0/289760171 learned_addr learned my addr 172.21.15.78:0/289760171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6aa8d4640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6a41078e0 0x7ff6a41a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6aa0d3640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a41019a0 0x7ff6a406acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6a98d2640 1 -- 172.21.15.78:0/289760171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6a41078e0 msgr2=0x7ff6a41a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6a98d2640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6a41078e0 0x7ff6a41a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6a98d2640 1 -- 172.21.15.78:0/289760171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a41019a0 msgr2=0x7ff6a406acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6a98d2640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a41019a0 0x7ff6a406acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.622+0000 7ff6a98d2640 1 -- 172.21.15.78:0/289760171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a41a3010 con 0x7ff6a4102be0 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.623+0000 7ff6a98d2640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 0x7ff6a41a0480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0005940 tx=0x7ff6a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.623+0000 7ff69b7fe640 1 -- 172.21.15.78:0/289760171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a003f030 con 0x7ff6a4102be0 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.623+0000 7ff69b7fe640 1 -- 172.21.15.78:0/289760171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a0031a20 con 0x7ff6a4102be0 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.623+0000 7ff69b7fe640 1 -- 172.21.15.78:0/289760171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0035070 con 0x7ff6a4102be0 2024-05-21T15:14:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.623+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a41a32a0 con 0x7ff6a4102be0 2024-05-21T15:14:08.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.623+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a41a3790 con 0x7ff6a4102be0 2024-05-21T15:14:08.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.624+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a41066f0 con 0x7ff6a4102be0 2024-05-21T15:14:08.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.625+0000 7ff69b7fe640 1 -- 172.21.15.78:0/289760171 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff6a0042030 con 0x7ff6a4102be0 2024-05-21T15:14:08.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.626+0000 7ff69b7fe640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff6900a2b90 0x7ff6900a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.626+0000 7ff6aa0d3640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff6900a2b90 0x7ff6900a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.626+0000 7ff69b7fe640 1 -- 172.21.15.78:0/289760171 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6a00ef140 con 0x7ff6a4102be0 2024-05-21T15:14:08.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.627+0000 7ff6aa0d3640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff6900a2b90 0x7ff6900a50a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6a41025a0 tx=0x7ff694005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:08.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.630+0000 7ff69b7fe640 1 -- 172.21.15.78:0/289760171 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff6a00bbef0 con 0x7ff6a4102be0 2024-05-21T15:14:08.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.843+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6a402d430 con 0x7ff6a4102be0 2024-05-21T15:14:08.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.843+0000 7ff69b7fe640 1 -- 172.21.15.78:0/289760171 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7ff6a00c01f0 con 0x7ff6a4102be0 2024-05-21T15:14:08.847 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:14:08.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.845+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff6900a2b90 msgr2=0x7ff6900a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.845+0000 7ff6ac337640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff6900a2b90 0x7ff6900a50a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6a41025a0 tx=0x7ff694005f90 comp rx=0 tx=0).stop 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.845+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 msgr2=0x7ff6a41a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.845+0000 7ff6ac337640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 0x7ff6a41a0480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0005940 tx=0x7ff6a00042f0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.846+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 shutdown_connections 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.846+0000 7ff6ac337640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff6900a2b90 0x7ff6900a50a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.846+0000 7ff6ac337640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6a41078e0 0x7ff6a41a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.846+0000 7ff6ac337640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a4102be0 0x7ff6a41a0480 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.846+0000 7ff6ac337640 1 --2- 172.21.15.78:0/289760171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6a41019a0 0x7ff6a406acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.846+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 >> 172.21.15.78:0/289760171 conn(0x7ff6a40fcb90 msgr2=0x7ff6a41038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.846+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 shutdown_connections 2024-05-21T15:14:08.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.846+0000 7ff6ac337640 1 -- 172.21.15.78:0/289760171 wait complete. 2024-05-21T15:14:08.862 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-21T15:14:08.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 -- 172.21.15.78:0/220392792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 msgr2=0x7f942c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 --2- 172.21.15.78:0/220392792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 0x7f942c105980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f941c009900 tx=0x7f941c02a410 comp rx=0 tx=0).stop 2024-05-21T15:14:08.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 -- 172.21.15.78:0/220392792 shutdown_connections 2024-05-21T15:14:08.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 --2- 172.21.15.78:0/220392792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f942c108820 0x7f942c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 --2- 172.21.15.78:0/220392792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f942c105ec0 0x7f942c1082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 --2- 172.21.15.78:0/220392792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 0x7f942c105980 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 -- 172.21.15.78:0/220392792 >> 172.21.15.78:0/220392792 conn(0x7f942c0fcb80 msgr2=0x7f942c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:08.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 -- 172.21.15.78:0/220392792 shutdown_connections 2024-05-21T15:14:08.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.986+0000 7f94320e2640 1 -- 172.21.15.78:0/220392792 wait complete. 2024-05-21T15:14:08.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f94320e2640 1 Processor -- start 2024-05-21T15:14:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f94320e2640 1 -- start start 2024-05-21T15:14:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f94320e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 0x7f942c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f94320e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f942c105ec0 0x7f942c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f94320e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f942c108820 0x7f942c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f94320e2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f942c10e710 con 0x7f942c103560 2024-05-21T15:14:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f94320e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f942c10e590 con 0x7f942c108820 2024-05-21T15:14:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f94320e2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f942c10e890 con 0x7f942c105ec0 2024-05-21T15:14:08.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f942b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 0x7f942c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f942b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 0x7f942c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55638/0 (socket says 172.21.15.78:55638) 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f942b7fe640 1 -- 172.21.15.78:0/685875540 learned_addr learned my addr 172.21.15.78:0/685875540 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f942affd640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f942c105ec0 0x7f942c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.987+0000 7f942bfff640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f942c108820 0x7f942c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f942b7fe640 1 -- 172.21.15.78:0/685875540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f942c105ec0 msgr2=0x7f942c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f942b7fe640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f942c105ec0 0x7f942c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f942b7fe640 1 -- 172.21.15.78:0/685875540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f942c108820 msgr2=0x7f942c1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f942b7fe640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f942c108820 0x7f942c1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f942b7fe640 1 -- 172.21.15.78:0/685875540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c1a0dd0 con 0x7f942c103560 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f942b7fe640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 0x7f942c199290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f941c031440 tx=0x7f941c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f9428ff9640 1 -- 172.21.15.78:0/685875540 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c03f030 con 0x7f942c103560 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f9428ff9640 1 -- 172.21.15.78:0/685875540 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f941c003ce0 con 0x7f942c103560 2024-05-21T15:14:08.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c1a1060 con 0x7f942c103560 2024-05-21T15:14:08.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f9428ff9640 1 -- 172.21.15.78:0/685875540 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c035070 con 0x7f942c103560 2024-05-21T15:14:08.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.988+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c1a15a0 con 0x7f942c103560 2024-05-21T15:14:08.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.989+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942c0fe6a0 con 0x7f942c103560 2024-05-21T15:14:08.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.991+0000 7f9428ff9640 1 -- 172.21.15.78:0/685875540 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f941c042030 con 0x7f942c103560 2024-05-21T15:14:08.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.992+0000 7f9428ff9640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f94140a2c10 0x7f94140a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:08.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.992+0000 7f942affd640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f94140a2c10 0x7f94140a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:08.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.992+0000 7f9428ff9640 1 -- 172.21.15.78:0/685875540 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f941c0ef420 con 0x7f942c103560 2024-05-21T15:14:08.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.993+0000 7f942affd640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f94140a2c10 0x7f94140a5120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f942c19a330 tx=0x7f942000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:09.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:08.996+0000 7f9428ff9640 1 -- 172.21.15.78:0/685875540 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f941c0bc1b0 con 0x7f942c103560 2024-05-21T15:14:09.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.202+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f942c062750 con 0x7f942c103560 2024-05-21T15:14:09.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.202+0000 7f9428ff9640 1 -- 172.21.15.78:0/685875540 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f941c0c04b0 con 0x7f942c103560 2024-05-21T15:14:09.206 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:09.207 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-05-21T15:14:09.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:14:09.207 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-05-21T15:14:09.207 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-05-21T15:14:09.207 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:14:09.207 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-05-21T15:14:09.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f94140a2c10 msgr2=0x7f94140a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:09.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f94140a2c10 0x7f94140a5120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f942c19a330 tx=0x7f942000b390 comp rx=0 tx=0).stop 2024-05-21T15:14:09.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 msgr2=0x7f942c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 0x7f942c199290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f941c031440 tx=0x7f941c032ce0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 shutdown_connections 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f94140a2c10 0x7f94140a5120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f942c108820 0x7f942c1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f942c105ec0 0x7f942c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 --2- 172.21.15.78:0/685875540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f942c103560 0x7f942c199290 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.204+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 >> 172.21.15.78:0/685875540 conn(0x7f942c0fcb80 msgr2=0x7f942c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.205+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 shutdown_connections 2024-05-21T15:14:09.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.205+0000 7f94320e2640 1 -- 172.21.15.78:0/685875540 wait complete. 2024-05-21T15:14:09.220 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-21T15:14:09.220 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-21T15:14:09.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 -- 172.21.15.78:0/2077988574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c101370 msgr2=0x7fc28c101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:09.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 --2- 172.21.15.78:0/2077988574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c101370 0x7fc28c101780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc288009930 tx=0x7fc28802a420 comp rx=0 tx=0).stop 2024-05-21T15:14:09.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 -- 172.21.15.78:0/2077988574 shutdown_connections 2024-05-21T15:14:09.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 --2- 172.21.15.78:0/2077988574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c101cc0 0x7fc28c1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 --2- 172.21.15.78:0/2077988574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c102610 0x7fc28c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 --2- 172.21.15.78:0/2077988574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c101370 0x7fc28c101780 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 -- 172.21.15.78:0/2077988574 >> 172.21.15.78:0/2077988574 conn(0x7fc28c0fcb80 msgr2=0x7fc28c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 -- 172.21.15.78:0/2077988574 shutdown_connections 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 -- 172.21.15.78:0/2077988574 wait complete. 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 Processor -- start 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.332+0000 7fc2932b8640 1 -- start start 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2932b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c101370 0x7fc28c19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2932b8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c101cc0 0x7fc28c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2932b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c102610 0x7fc28c1a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2932b8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1108d0 con 0x7fc28c102610 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2932b8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c110750 con 0x7fc28c101370 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2932b8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c110a50 con 0x7fc28c101cc0 2024-05-21T15:14:09.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c102610 0x7fc28c1a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c102610 0x7fc28c1a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55656/0 (socket says 172.21.15.78:55656) 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 -- 172.21.15.78:0/3571714351 learned_addr learned my addr 172.21.15.78:0/3571714351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc290853640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c101cc0 0x7fc28c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291054640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c101370 0x7fc28c19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 -- 172.21.15.78:0/3571714351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c101cc0 msgr2=0x7fc28c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c101cc0 0x7fc28c19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 -- 172.21.15.78:0/3571714351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c101370 msgr2=0x7fc28c19b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c101370 0x7fc28c19b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 -- 172.21.15.78:0/3571714351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c1a30d0 con 0x7fc28c102610 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291054640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c101370 0x7fc28c19b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc291855640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c102610 0x7fc28c1a2a60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc28400e9d0 tx=0x7fc28400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:09.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2827fc640 1 -- 172.21.15.78:0/3571714351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400cd80 con 0x7fc28c102610 2024-05-21T15:14:09.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c1a33c0 con 0x7fc28c102610 2024-05-21T15:14:09.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2827fc640 1 -- 172.21.15.78:0/3571714351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc284005020 con 0x7fc28c102610 2024-05-21T15:14:09.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2827fc640 1 -- 172.21.15.78:0/3571714351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2840116d0 con 0x7fc28c102610 2024-05-21T15:14:09.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.333+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1a3910 con 0x7fc28c102610 2024-05-21T15:14:09.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.334+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c10c3b0 con 0x7fc28c102610 2024-05-21T15:14:09.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.335+0000 7fc2827fc640 1 -- 172.21.15.78:0/3571714351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc2840051d0 con 0x7fc28c102610 2024-05-21T15:14:09.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.336+0000 7fc2827fc640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2780a2b90 0x7fc2780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:09.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.337+0000 7fc291054640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2780a2b90 0x7fc2780a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:09.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.337+0000 7fc2827fc640 1 -- 172.21.15.78:0/3571714351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc2840cf190 con 0x7fc28c102610 2024-05-21T15:14:09.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.337+0000 7fc291054640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2780a2b90 0x7fc2780a50a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc288006010 tx=0x7fc288009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.341+0000 7fc2827fc640 1 -- 172.21.15.78:0/3571714351 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc284016030 con 0x7fc28c102610 2024-05-21T15:14:09.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:09 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3497212098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:09.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:09 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/157056492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:09.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:09 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/289760171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:14:09.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:09 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:09 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:09 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:09 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:09.544+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc28c02d2a0 con 0x7fc28c102610 2024-05-21T15:14:09.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:09 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3497212098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:09.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:09 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/157056492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:09.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:09 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/289760171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:14:09.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:09 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:09 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:09 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:09 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:09 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3497212098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:09 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/157056492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:09 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/289760171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:14:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:09 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:09 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:09 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:09.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:09 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/685875540' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T15:14:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:14:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3571714351' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:14:10.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:10 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:14:10.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.545+0000 7fc2827fc640 1 -- 172.21.15.78:0/3571714351 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fc28409c590 con 0x7fc28c102610 2024-05-21T15:14:10.549 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.547+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2780a2b90 msgr2=0x7fc2780a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.547+0000 7fc2932b8640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2780a2b90 0x7fc2780a50a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc288006010 tx=0x7fc288009560 comp rx=0 tx=0).stop 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c102610 msgr2=0x7fc28c1a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c102610 0x7fc28c1a2a60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc28400e9d0 tx=0x7fc28400ee90 comp rx=0 tx=0).stop 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 shutdown_connections 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc2780a2b90 0x7fc2780a50a0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc288006010 tx=0x7fc288009560 comp rx=0 tx=0).stop 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc28c102610 0x7fc28c1a2a60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c101cc0 0x7fc28c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 --2- 172.21.15.78:0/3571714351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc28c101370 0x7fc28c19b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 >> 172.21.15.78:0/3571714351 conn(0x7fc28c0fcb80 msgr2=0x7fc28c109580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 shutdown_connections 2024-05-21T15:14:10.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.548+0000 7fc2932b8640 1 -- 172.21.15.78:0/3571714351 wait complete. 2024-05-21T15:14:10.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/685875540' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T15:14:10.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:10.566 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:10.566 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:14:10.566 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.566 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3571714351' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:14:10.566 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:10 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:14:10.567 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-21T15:14:10.582 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.582 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/685875540' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T15:14:10.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:10.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:10.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:14:10.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:10.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3571714351' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T15:14:10.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:10 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:14:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 -- 172.21.15.78:0/1975434035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 msgr2=0x7fa4e810ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/1975434035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 0x7fa4e810ae10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009900 tx=0x7fa4d802a410 comp rx=0 tx=0).stop 2024-05-21T15:14:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 -- 172.21.15.78:0/1975434035 shutdown_connections 2024-05-21T15:14:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/1975434035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 0x7fa4e810ae10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/1975434035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e8103cf0 0x7fa4e8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/1975434035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e8102ab0 0x7fa4e81037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 -- 172.21.15.78:0/1975434035 >> 172.21.15.78:0/1975434035 conn(0x7fa4e80fdcc0 msgr2=0x7fa4e8100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 -- 172.21.15.78:0/1975434035 shutdown_connections 2024-05-21T15:14:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 -- 172.21.15.78:0/1975434035 wait complete. 2024-05-21T15:14:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 Processor -- start 2024-05-21T15:14:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.687+0000 7fa4ef60d640 1 -- start start 2024-05-21T15:14:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ef60d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e8102ab0 0x7fa4e819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ef60d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e8103cf0 0x7fa4e819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ef60d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 0x7fa4e81a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ef60d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e810e730 con 0x7fa4e81089f0 2024-05-21T15:14:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ef60d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e810e5b0 con 0x7fa4e8102ab0 2024-05-21T15:14:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ef60d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e810e8b0 con 0x7fa4e8103cf0 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 0x7fa4e81a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 0x7fa4e81a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55674/0 (socket says 172.21.15.78:55674) 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 -- 172.21.15.78:0/4252015271 learned_addr learned my addr 172.21.15.78:0/4252015271 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ed3a9640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e8102ab0 0x7fa4e819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ecba8640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e8103cf0 0x7fa4e819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 -- 172.21.15.78:0/4252015271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e8103cf0 msgr2=0x7fa4e819b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e8103cf0 0x7fa4e819b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 -- 172.21.15.78:0/4252015271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e8102ab0 msgr2=0x7fa4e819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e8102ab0 0x7fa4e819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 -- 172.21.15.78:0/4252015271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e81a2fc0 con 0x7fa4e81089f0 2024-05-21T15:14:10.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4edbaa640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 0x7fa4e81a2950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8031610 tx=0x7fa4d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:10.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4e67fc640 1 -- 172.21.15.78:0/4252015271 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d803f030 con 0x7fa4e81089f0 2024-05-21T15:14:10.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4e67fc640 1 -- 172.21.15.78:0/4252015271 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d8031a20 con 0x7fa4e81089f0 2024-05-21T15:14:10.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e81a3250 con 0x7fa4e81089f0 2024-05-21T15:14:10.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4e67fc640 1 -- 172.21.15.78:0/4252015271 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8035070 con 0x7fa4e81089f0 2024-05-21T15:14:10.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.688+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e81a3740 con 0x7fa4e81089f0 2024-05-21T15:14:10.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.689+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e81a39a0 con 0x7fa4e81089f0 2024-05-21T15:14:10.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.691+0000 7fa4e67fc640 1 -- 172.21.15.78:0/4252015271 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa4d8042030 con 0x7fa4e81089f0 2024-05-21T15:14:10.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.692+0000 7fa4e67fc640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa4d00a2b90 0x7fa4d00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:10.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.692+0000 7fa4ed3a9640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa4d00a2b90 0x7fa4d00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:10.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.692+0000 7fa4e67fc640 1 -- 172.21.15.78:0/4252015271 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fa4d80ef4c0 con 0x7fa4e81089f0 2024-05-21T15:14:10.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.692+0000 7fa4ed3a9640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa4d00a2b90 0x7fa4d00a50a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00095f0 tx=0x7fa4e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:10.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.696+0000 7fa4e67fc640 1 -- 172.21.15.78:0/4252015271 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa4d80bc0e0 con 0x7fa4e81089f0 2024-05-21T15:14:10.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:10.876+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa4e81a4040 con 0x7fa4e81089f0 2024-05-21T15:14:11.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.548+0000 7fa4e67fc640 1 -- 172.21.15.78:0/4252015271 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7fa4d80c03e0 con 0x7fa4e81089f0 2024-05-21T15:14:11.552 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-05-21T15:14:11.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa4d00a2b90 msgr2=0x7fa4d00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:11.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa4d00a2b90 0x7fa4d00a50a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00095f0 tx=0x7fa4e0005f90 comp rx=0 tx=0).stop 2024-05-21T15:14:11.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 msgr2=0x7fa4e81a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:11.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 0x7fa4e81a2950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8031610 tx=0x7fa4d8004270 comp rx=0 tx=0).stop 2024-05-21T15:14:11.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 shutdown_connections 2024-05-21T15:14:11.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa4d00a2b90 0x7fa4d00a50a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4e81089f0 0x7fa4e81a2950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e8103cf0 0x7fa4e819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 --2- 172.21.15.78:0/4252015271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4e8102ab0 0x7fa4e819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.550+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 >> 172.21.15.78:0/4252015271 conn(0x7fa4e80fdcc0 msgr2=0x7fa4e80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:11.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.551+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 shutdown_connections 2024-05-21T15:14:11.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.551+0000 7fa4ef60d640 1 -- 172.21.15.78:0/4252015271 wait complete. 2024-05-21T15:14:11.575 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-21T15:14:11.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:11 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:11.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:11 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3571714351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T15:14:11.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:11 smithi078 ceph-mon[31997]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T15:14:11.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:11 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4252015271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:14:11.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:11 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:11.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:11 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3571714351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T15:14:11.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:11 smithi084 ceph-mon[36334]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T15:14:11.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:11 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4252015271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:14:11.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:11 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:11.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:11 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3571714351' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T15:14:11.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:11 smithi172 ceph-mon[36402]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T15:14:11.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:11 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4252015271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T15:14:11.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 -- 172.21.15.78:0/1058033541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e30102bf0 msgr2=0x7f8e30103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/1058033541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e30102bf0 0x7f8e30103000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009900 tx=0x7f8e2402a410 comp rx=0 tx=0).stop 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 -- 172.21.15.78:0/1058033541 shutdown_connections 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/1058033541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e301078f0 0x7f8e30109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/1058033541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e30102bf0 0x7f8e30103000 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/1058033541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e301019b0 0x7f8e301026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 -- 172.21.15.78:0/1058033541 >> 172.21.15.78:0/1058033541 conn(0x7f8e300fcba0 msgr2=0x7f8e300ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 -- 172.21.15.78:0/1058033541 shutdown_connections 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.689+0000 7f8e35ce2640 1 -- 172.21.15.78:0/1058033541 wait complete. 2024-05-21T15:14:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.690+0000 7f8e35ce2640 1 Processor -- start 2024-05-21T15:14:11.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.690+0000 7f8e35ce2640 1 -- start start 2024-05-21T15:14:11.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.690+0000 7f8e35ce2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e301019b0 0x7f8e30074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:11.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.690+0000 7f8e35ce2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e30102bf0 0x7f8e3019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.690+0000 7f8e35ce2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e301078f0 0x7f8e301a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.690+0000 7f8e35ce2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3010d600 con 0x7f8e301078f0 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.690+0000 7f8e35ce2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3010d480 con 0x7f8e301019b0 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.690+0000 7f8e35ce2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3010d780 con 0x7f8e30102bf0 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e301078f0 0x7f8e301a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e301019b0 0x7f8e30074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e301078f0 0x7f8e301a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55686/0 (socket says 172.21.15.78:55686) 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2ffff640 1 -- 172.21.15.78:0/2328739168 learned_addr learned my addr 172.21.15.78:0/2328739168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2effd640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e30102bf0 0x7f8e3019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2f7fe640 1 -- 172.21.15.78:0/2328739168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e30102bf0 msgr2=0x7f8e3019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2f7fe640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e30102bf0 0x7f8e3019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2f7fe640 1 -- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e301078f0 msgr2=0x7f8e301a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2f7fe640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e301078f0 0x7f8e301a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2f7fe640 1 -- 172.21.15.78:0/2328739168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e301a0dc0 con 0x7f8e301019b0 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2effd640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e30102bf0 0x7f8e3019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2f7fe640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e301019b0 0x7f8e30074810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e30101280 tx=0x7f8e2000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2cff9640 1 -- 172.21.15.78:0/2328739168 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000cc40 con 0x7f8e301019b0 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2cff9640 1 -- 172.21.15.78:0/2328739168 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2000cde0 con 0x7f8e301019b0 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e301a10b0 con 0x7f8e301019b0 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e2cff9640 1 -- 172.21.15.78:0/2328739168 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20010430 con 0x7f8e301019b0 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.691+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e301a1600 con 0x7f8e301019b0 2024-05-21T15:14:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.692+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e30106700 con 0x7f8e301019b0 2024-05-21T15:14:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.693+0000 7f8e2cff9640 1 -- 172.21.15.78:0/2328739168 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8e20043ae0 con 0x7f8e301019b0 2024-05-21T15:14:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.694+0000 7f8e2cff9640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8e180a2b90 0x7f8e180a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.694+0000 7f8e2effd640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8e180a2b90 0x7f8e180a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.694+0000 7f8e2cff9640 1 -- 172.21.15.78:0/2328739168 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f8e200cf410 con 0x7f8e301019b0 2024-05-21T15:14:11.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.694+0000 7f8e2effd640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8e180a2b90 0x7f8e180a50a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e24000c00 tx=0x7f8e240315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:11.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.698+0000 7f8e2cff9640 1 -- 172.21.15.78:0/2328739168 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e2009c480 con 0x7f8e301019b0 2024-05-21T15:14:11.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:11.915+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8e3002d2a0 con 0x7f8e301019b0 2024-05-21T15:14:12.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.554+0000 7f8e2cff9640 1 -- 172.21.15.78:0/2328739168 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8e2009bd80 con 0x7f8e301019b0 2024-05-21T15:14:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 2 and data pool 3 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8e180a2b90 msgr2=0x7f8e180a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8e180a2b90 0x7f8e180a50a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e24000c00 tx=0x7f8e240315b0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e301019b0 msgr2=0x7f8e30074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e301019b0 0x7f8e30074810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e30101280 tx=0x7f8e2000ee50 comp rx=0 tx=0).stop 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 shutdown_connections 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8e180a2b90 0x7f8e180a50a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e301078f0 0x7f8e301a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e30102bf0 0x7f8e3019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 --2- 172.21.15.78:0/2328739168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e301019b0 0x7f8e30074810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 >> 172.21.15.78:0/2328739168 conn(0x7f8e300fcba0 msgr2=0x7f8e301038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 shutdown_connections 2024-05-21T15:14:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.558+0000 7f8e35ce2640 1 -- 172.21.15.78:0/2328739168 wait complete. 2024-05-21T15:14:12.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:12.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:12.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:14:12.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4252015271' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T15:14:12.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T15:14:12.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2328739168' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:14:12.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:14:12.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:12 smithi078 ceph-mon[31997]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T15:14:12.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:14:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4252015271' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2328739168' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:14:12.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:12 smithi084 ceph-mon[36334]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T15:14:12.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:12.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:14:12.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4252015271' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T15:14:12.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T15:14:12.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2328739168' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:14:12.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T15:14:12.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:12 smithi172 ceph-mon[36402]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 -- 172.21.15.78:0/450026271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 msgr2=0x7fc200108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 --2- 172.21.15.78:0/450026271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 0x7fc200108090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009900 tx=0x7fc1e802a410 comp rx=0 tx=0).stop 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 -- 172.21.15.78:0/450026271 shutdown_connections 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 --2- 172.21.15.78:0/450026271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc200108720 0x7fc20010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 --2- 172.21.15.78:0/450026271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 0x7fc200108090 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 --2- 172.21.15.78:0/450026271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc200100dd0 0x7fc2001011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 -- 172.21.15.78:0/450026271 >> 172.21.15.78:0/450026271 conn(0x7fc2000fc2e0 msgr2=0x7fc2000fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 -- 172.21.15.78:0/450026271 shutdown_connections 2024-05-21T15:14:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.686+0000 7fc20487a640 1 -- 172.21.15.78:0/450026271 wait complete. 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc20487a640 1 Processor -- start 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc20487a640 1 -- start start 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc20487a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc200100dd0 0x7fc200199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc20487a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 0x7fc2001997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc20487a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc200108720 0x7fc2001a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc20487a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc20010e750 con 0x7fc200105bb0 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc20487a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc20010e5d0 con 0x7fc200100dd0 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc20487a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc20010e8d0 con 0x7fc200108720 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 0x7fc2001997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 0x7fc2001997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55704/0 (socket says 172.21.15.78:55704) 2024-05-21T15:14:12.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 -- 172.21.15.78:0/4116103016 learned_addr learned my addr 172.21.15.78:0/4116103016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fe59c640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc200100dd0 0x7fc200199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fed9d640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc200108720 0x7fc2001a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 -- 172.21.15.78:0/4116103016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc200108720 msgr2=0x7fc2001a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc200108720 0x7fc2001a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 -- 172.21.15.78:0/4116103016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc200100dd0 msgr2=0x7fc200199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc200100dd0 0x7fc200199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 -- 172.21.15.78:0/4116103016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2001a0d00 con 0x7fc200105bb0 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fe59c640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc200100dd0 0x7fc200199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.687+0000 7fc1fdd9b640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 0x7fc2001997b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005ab0 tx=0x7fc1e8032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.688+0000 7fc1f77fe640 1 -- 172.21.15.78:0/4116103016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e803f030 con 0x7fc200105bb0 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.688+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2001a0f90 con 0x7fc200105bb0 2024-05-21T15:14:12.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.688+0000 7fc1f77fe640 1 -- 172.21.15.78:0/4116103016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1e8003ce0 con 0x7fc200105bb0 2024-05-21T15:14:12.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.688+0000 7fc1f77fe640 1 -- 172.21.15.78:0/4116103016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8035070 con 0x7fc200105bb0 2024-05-21T15:14:12.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.688+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2001a13f0 con 0x7fc200105bb0 2024-05-21T15:14:12.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.689+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2000fde00 con 0x7fc200105bb0 2024-05-21T15:14:12.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.690+0000 7fc1f77fe640 1 -- 172.21.15.78:0/4116103016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc1e8042030 con 0x7fc200105bb0 2024-05-21T15:14:12.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.691+0000 7fc1f77fe640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc1ec0a2b90 0x7fc1ec0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:12.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.691+0000 7fc1fe59c640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc1ec0a2b90 0x7fc1ec0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:12.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.691+0000 7fc1f77fe640 1 -- 172.21.15.78:0/4116103016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fc1e80efa60 con 0x7fc200105bb0 2024-05-21T15:14:12.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.691+0000 7fc1fe59c640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc1ec0a2b90 0x7fc1ec0a50a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc200103aa0 tx=0x7fc1f0007850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:12.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.694+0000 7fc1f77fe640 1 -- 172.21.15.78:0/4116103016 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc1e80bcb10 con 0x7fc200105bb0 2024-05-21T15:14:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.871+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc200062750 con 0x7fc200105bb0 2024-05-21T15:14:12.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.873+0000 7fc1f77fe640 1 -- 172.21.15.78:0/4116103016 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22160 (secure 0 0 0) 0x7fc1e80bc410 con 0x7fc200105bb0 2024-05-21T15:14:12.877 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:12.877 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":84,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:14:11.918556+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:14:12.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc1ec0a2b90 msgr2=0x7fc1ec0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:12.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc1ec0a2b90 0x7fc1ec0a50a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc200103aa0 tx=0x7fc1f0007850 comp rx=0 tx=0).stop 2024-05-21T15:14:12.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 msgr2=0x7fc2001997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:12.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 0x7fc2001997b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005ab0 tx=0x7fc1e8032d00 comp rx=0 tx=0).stop 2024-05-21T15:14:12.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 shutdown_connections 2024-05-21T15:14:12.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc1ec0a2b90 0x7fc1ec0a50a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc200108720 0x7fc2001a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc200105bb0 0x7fc2001997b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 --2- 172.21.15.78:0/4116103016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc200100dd0 0x7fc200199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 >> 172.21.15.78:0/4116103016 conn(0x7fc2000fc2e0 msgr2=0x7fc200109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 shutdown_connections 2024-05-21T15:14:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.875+0000 7fc20487a640 1 -- 172.21.15.78:0/4116103016 wait complete. 2024-05-21T15:14:12.890 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:14:13.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.997+0000 7fb9633fe640 1 -- 172.21.15.78:0/4294846811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c1089f0 msgr2=0x7fb95c10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.997+0000 7fb9633fe640 1 --2- 172.21.15.78:0/4294846811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c1089f0 0x7fb95c10ae10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb954009ca0 tx=0x7fb95402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.997+0000 7fb9633fe640 1 -- 172.21.15.78:0/4294846811 shutdown_connections 2024-05-21T15:14:13.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.997+0000 7fb9633fe640 1 --2- 172.21.15.78:0/4294846811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb95c103cf0 0x7fb95c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 --2- 172.21.15.78:0/4294846811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c1089f0 0x7fb95c10ae10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 --2- 172.21.15.78:0/4294846811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95c102ab0 0x7fb95c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 -- 172.21.15.78:0/4294846811 >> 172.21.15.78:0/4294846811 conn(0x7fb95c0fdcc0 msgr2=0x7fb95c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:13.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 -- 172.21.15.78:0/4294846811 shutdown_connections 2024-05-21T15:14:13.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 -- 172.21.15.78:0/4294846811 wait complete. 2024-05-21T15:14:13.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 Processor -- start 2024-05-21T15:14:13.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 -- start start 2024-05-21T15:14:13.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95c102ab0 0x7fb95c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c103cf0 0x7fb95c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb95c1089f0 0x7fb95c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c10e710 con 0x7fb95c103cf0 2024-05-21T15:14:13.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c10e590 con 0x7fb95c1089f0 2024-05-21T15:14:13.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.998+0000 7fb9633fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c10e890 con 0x7fb95c102ab0 2024-05-21T15:14:13.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c103cf0 0x7fb95c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c103cf0 0x7fb95c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55726/0 (socket says 172.21.15.78:55726) 2024-05-21T15:14:13.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 -- 172.21.15.78:0/1598791402 learned_addr learned my addr 172.21.15.78:0/1598791402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:13.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb96119a640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95c102ab0 0x7fb95c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 -- 172.21.15.78:0/1598791402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95c102ab0 msgr2=0x7fb95c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95c102ab0 0x7fb95c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 -- 172.21.15.78:0/1598791402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb95c1089f0 msgr2=0x7fb95c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb95c1089f0 0x7fb95c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 -- 172.21.15.78:0/1598791402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c1a0c90 con 0x7fb95c103cf0 2024-05-21T15:14:13.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb96119a640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95c102ab0 0x7fb95c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:13.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb960999640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c103cf0 0x7fb95c199740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c920 tx=0x7fb94c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:13.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb9527fc640 1 -- 172.21.15.78:0/1598791402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c007c70 con 0x7fb95c103cf0 2024-05-21T15:14:13.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb9527fc640 1 -- 172.21.15.78:0/1598791402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb94c007e10 con 0x7fb95c103cf0 2024-05-21T15:14:13.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c1a0f80 con 0x7fb95c103cf0 2024-05-21T15:14:13.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb9527fc640 1 -- 172.21.15.78:0/1598791402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c018890 con 0x7fb95c103cf0 2024-05-21T15:14:13.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:12.999+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c1a13f0 con 0x7fb95c103cf0 2024-05-21T15:14:13.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.000+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95c1070f0 con 0x7fb95c103cf0 2024-05-21T15:14:13.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.002+0000 7fb9527fc640 1 -- 172.21.15.78:0/1598791402 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb94c018a50 con 0x7fb95c103cf0 2024-05-21T15:14:13.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.002+0000 7fb9527fc640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9480a2b90 0x7fb9480a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.002+0000 7fb96119a640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9480a2b90 0x7fb9480a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.002+0000 7fb9527fc640 1 -- 172.21.15.78:0/1598791402 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fb94c0cf390 con 0x7fb95c103cf0 2024-05-21T15:14:13.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.003+0000 7fb96119a640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9480a2b90 0x7fb9480a50a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb95c1036b0 tx=0x7fb958005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:13.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.006+0000 7fb9527fc640 1 -- 172.21.15.78:0/1598791402 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb94c014030 con 0x7fb95c103cf0 2024-05-21T15:14:13.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.181+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb95c02d550 con 0x7fb95c103cf0 2024-05-21T15:14:13.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.182+0000 7fb9527fc640 1 -- 172.21.15.78:0/1598791402 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22160 (secure 0 0 0) 0x7fb94c09c4d0 con 0x7fb95c103cf0 2024-05-21T15:14:13.186 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:13.187 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":84,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:14:11.918556+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:14:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9480a2b90 msgr2=0x7fb9480a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9480a2b90 0x7fb9480a50a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb95c1036b0 tx=0x7fb958005db0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c103cf0 msgr2=0x7fb95c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c103cf0 0x7fb95c199740 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c920 tx=0x7fb94c00cde0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 shutdown_connections 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9480a2b90 0x7fb9480a50a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb95c1089f0 0x7fb95c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb95c103cf0 0x7fb95c199740 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 --2- 172.21.15.78:0/1598791402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95c102ab0 0x7fb95c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.184+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 >> 172.21.15.78:0/1598791402 conn(0x7fb95c0fdcc0 msgr2=0x7fb95c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.185+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 shutdown_connections 2024-05-21T15:14:13.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.185+0000 7fb9633fe640 1 -- 172.21.15.78:0/1598791402 wait complete. 2024-05-21T15:14:13.200 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-21T15:14:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 -- 172.21.15.78:0/106137653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01029f0 msgr2=0x7f1be0102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/106137653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01029f0 0x7f1be0102e00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009900 tx=0x7f1bc802a410 comp rx=0 tx=0).stop 2024-05-21T15:14:13.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 -- 172.21.15.78:0/106137653 shutdown_connections 2024-05-21T15:14:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/106137653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be01076f0 0x7f1be0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/106137653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01029f0 0x7f1be0102e00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/106137653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be01017b0 0x7f1be01024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 -- 172.21.15.78:0/106137653 >> 172.21.15.78:0/106137653 conn(0x7f1be00fc9a0 msgr2=0x7f1be00fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 -- 172.21.15.78:0/106137653 shutdown_connections 2024-05-21T15:14:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.312+0000 7f1bdf59e640 1 -- 172.21.15.78:0/106137653 wait complete. 2024-05-21T15:14:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bdf59e640 1 Processor -- start 2024-05-21T15:14:13.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bdf59e640 1 -- start start 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bdf59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be01017b0 0x7f1be01990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bdf59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be01029f0 0x7f1be0199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bdf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01076f0 0x7f1be01a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bdf59e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1be010d420 con 0x7f1be01076f0 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bdf59e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1be010d2a0 con 0x7f1be01029f0 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bdf59e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1be010d5a0 con 0x7f1be01017b0 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bded9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01076f0 0x7f1be01a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bded9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01076f0 0x7f1be01a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55742/0 (socket says 172.21.15.78:55742) 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bded9d640 1 -- 172.21.15.78:0/1533438677 learned_addr learned my addr 172.21.15.78:0/1533438677 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bd7fff640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be01029f0 0x7f1be0199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.313+0000 7f1bde59c640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be01017b0 0x7f1be01990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bded9d640 1 -- 172.21.15.78:0/1533438677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be01017b0 msgr2=0x7f1be01990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bded9d640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be01017b0 0x7f1be01990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bded9d640 1 -- 172.21.15.78:0/1533438677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be01029f0 msgr2=0x7f1be0199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bded9d640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be01029f0 0x7f1be0199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bded9d640 1 -- 172.21.15.78:0/1533438677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be01a0c20 con 0x7f1be01076f0 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bd7fff640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be01029f0 0x7f1be0199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bded9d640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01076f0 0x7f1be01a05b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800eaa0 tx=0x7f1bd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bd77fe640 1 -- 172.21.15.78:0/1533438677 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800cdb0 con 0x7f1be01076f0 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bd77fe640 1 -- 172.21.15.78:0/1533438677 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd8005020 con 0x7f1be01076f0 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be01a0f10 con 0x7f1be01076f0 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bd77fe640 1 -- 172.21.15.78:0/1533438677 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80116b0 con 0x7f1be01076f0 2024-05-21T15:14:13.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.314+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be01a1320 con 0x7f1be01076f0 2024-05-21T15:14:13.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.315+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be01a15e0 con 0x7f1be01076f0 2024-05-21T15:14:13.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.317+0000 7f1bd77fe640 1 -- 172.21.15.78:0/1533438677 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1bd8005560 con 0x7f1be01076f0 2024-05-21T15:14:13.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.318+0000 7f1bd77fe640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bcc0a2b80 0x7f1bcc0a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.318+0000 7f1bde59c640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bcc0a2b80 0x7f1bcc0a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.318+0000 7f1bd77fe640 1 -- 172.21.15.78:0/1533438677 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1bd80cf5e0 con 0x7f1be01076f0 2024-05-21T15:14:13.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.318+0000 7f1bde59c640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bcc0a2b80 0x7f1bcc0a5090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0005fd0 tx=0x7f1bd0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:13.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.321+0000 7f1bd77fe640 1 -- 172.21.15.78:0/1533438677 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1bd8016030 con 0x7f1be01076f0 2024-05-21T15:14:13.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.534+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1be01a17b0 con 0x7f1be01076f0 2024-05-21T15:14:13.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.551+0000 7f1bd77fe640 1 -- 172.21.15.78:0/1533438677 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f1bd809c690 con 0x7f1be01076f0 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bcc0a2b80 msgr2=0x7f1bcc0a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bcc0a2b80 0x7f1bcc0a5090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0005fd0 tx=0x7f1bd0005850 comp rx=0 tx=0).stop 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01076f0 msgr2=0x7f1be01a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01076f0 0x7f1be01a05b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800eaa0 tx=0x7f1bd800ef60 comp rx=0 tx=0).stop 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 shutdown_connections 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bcc0a2b80 0x7f1bcc0a5090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1be01029f0 0x7f1be0199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1be01076f0 0x7f1be01a05b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 --2- 172.21.15.78:0/1533438677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be01017b0 0x7f1be01990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 >> 172.21.15.78:0/1533438677 conn(0x7f1be00fc9a0 msgr2=0x7f1be0068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:13.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 shutdown_connections 2024-05-21T15:14:13.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.553+0000 7f1bdf59e640 1 -- 172.21.15.78:0/1533438677 wait complete. 2024-05-21T15:14:13.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:13 smithi078 ceph-mon[31997]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T15:14:13.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:13 smithi078 ceph-mon[31997]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T15:14:13.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:13 smithi078 ceph-mon[31997]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T15:14:13.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:13 smithi078 ceph-mon[31997]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T15:14:13.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:13 smithi078 ceph-mon[31997]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T15:14:13.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:13 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4116103016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:13.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:13 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1598791402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:13.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:13 smithi078 ceph-mon[31997]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T15:14:13.570 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-21T15:14:13.570 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-21T15:14:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:13 smithi084 ceph-mon[36334]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T15:14:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:13 smithi084 ceph-mon[36334]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T15:14:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:13 smithi084 ceph-mon[36334]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T15:14:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:13 smithi084 ceph-mon[36334]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T15:14:13.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:13 smithi084 ceph-mon[36334]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T15:14:13.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:13 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4116103016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:13.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:13 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1598791402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:13.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:13 smithi084 ceph-mon[36334]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T15:14:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:13 smithi172 ceph-mon[36402]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T15:14:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:13 smithi172 ceph-mon[36402]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T15:14:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:13 smithi172 ceph-mon[36402]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T15:14:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:13 smithi172 ceph-mon[36402]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T15:14:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:13 smithi172 ceph-mon[36402]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T15:14:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:13 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4116103016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:13 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1598791402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:13 smithi172 ceph-mon[36402]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T15:14:13.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 -- 172.21.15.78:0/4103468134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38108840 msgr2=0x7f4c3810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 --2- 172.21.15.78:0/4103468134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38108840 0x7f4c3810ac60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009ca0 tx=0x7f4c3002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 -- 172.21.15.78:0/4103468134 shutdown_connections 2024-05-21T15:14:13.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 --2- 172.21.15.78:0/4103468134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38108840 0x7f4c3810ac60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 --2- 172.21.15.78:0/4103468134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c38105ee0 0x7f4c38108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 --2- 172.21.15.78:0/4103468134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c38103580 0x7f4c381059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 -- 172.21.15.78:0/4103468134 >> 172.21.15.78:0/4103468134 conn(0x7f4c380fcb80 msgr2=0x7f4c380ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 -- 172.21.15.78:0/4103468134 shutdown_connections 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.680+0000 7f4c3f765640 1 -- 172.21.15.78:0/4103468134 wait complete. 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3f765640 1 Processor -- start 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3f765640 1 -- start start 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3f765640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38103580 0x7f4c381995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3f765640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c38105ee0 0x7f4c38199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3f765640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c38108840 0x7f4c381a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3f765640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3810e760 con 0x7f4c38103580 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3f765640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3810e5e0 con 0x7f4c38105ee0 2024-05-21T15:14:13.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3f765640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3810e8e0 con 0x7f4c38108840 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38103580 0x7f4c381995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38103580 0x7f4c381995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55766/0 (socket says 172.21.15.78:55766) 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 -- 172.21.15.78:0/2505774485 learned_addr learned my addr 172.21.15.78:0/2505774485 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3cd00640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c38105ee0 0x7f4c38199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3dd02640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c38108840 0x7f4c381a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 -- 172.21.15.78:0/2505774485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c38108840 msgr2=0x7f4c381a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c38108840 0x7f4c381a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 -- 172.21.15.78:0/2505774485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c38105ee0 msgr2=0x7f4c38199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c38105ee0 0x7f4c38199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 -- 172.21.15.78:0/2505774485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381a1110 con 0x7f4c38103580 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.681+0000 7f4c3d501640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38103580 0x7f4c381995d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400d770 tx=0x7f4c3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.682+0000 7f4c2e7fc640 1 -- 172.21.15.78:0/2505774485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34017070 con 0x7f4c38103580 2024-05-21T15:14:13.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.682+0000 7f4c2e7fc640 1 -- 172.21.15.78:0/2505774485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c34009c30 con 0x7f4c38103580 2024-05-21T15:14:13.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.682+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c381a1400 con 0x7f4c38103580 2024-05-21T15:14:13.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.682+0000 7f4c2e7fc640 1 -- 172.21.15.78:0/2505774485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400f620 con 0x7f4c38103580 2024-05-21T15:14:13.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.682+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381a1a20 con 0x7f4c38103580 2024-05-21T15:14:13.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.683+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3806b250 con 0x7f4c38103580 2024-05-21T15:14:13.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.684+0000 7f4c2e7fc640 1 -- 172.21.15.78:0/2505774485 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4c3401a070 con 0x7f4c38103580 2024-05-21T15:14:13.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.684+0000 7f4c2e7fc640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4c240a2c10 0x7f4c240a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:13.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.685+0000 7f4c3cd00640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4c240a2c10 0x7f4c240a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:13.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.685+0000 7f4c2e7fc640 1 -- 172.21.15.78:0/2505774485 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4c340cea30 con 0x7f4c38103580 2024-05-21T15:14:13.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.685+0000 7f4c3cd00640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4c240a2c10 0x7f4c240a5120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c3819a670 tx=0x7f4c28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:13.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.689+0000 7f4c2e7fc640 1 -- 172.21.15.78:0/2505774485 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c34009de0 con 0x7f4c38103580 2024-05-21T15:14:13.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:13.907+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f4c381a1c80 con 0x7f4c38103580 2024-05-21T15:14:14.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.553+0000 7f4c2e7fc640 1 -- 172.21.15.78:0/2505774485 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f4c3409bae0 con 0x7f4c38103580 2024-05-21T15:14:14.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4c240a2c10 msgr2=0x7f4c240a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:14.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4c240a2c10 0x7f4c240a5120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c3819a670 tx=0x7f4c28008040 comp rx=0 tx=0).stop 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38103580 msgr2=0x7f4c381995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38103580 0x7f4c381995d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400d770 tx=0x7f4c3400dc30 comp rx=0 tx=0).stop 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 shutdown_connections 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4c240a2c10 0x7f4c240a5120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c38105ee0 0x7f4c38199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c38108840 0x7f4c381a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 --2- 172.21.15.78:0/2505774485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c38103580 0x7f4c381995d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.555+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 >> 172.21.15.78:0/2505774485 conn(0x7f4c380fcb80 msgr2=0x7f4c38104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.556+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 shutdown_connections 2024-05-21T15:14:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.556+0000 7f4c3f765640 1 -- 172.21.15.78:0/2505774485 wait complete. 2024-05-21T15:14:14.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:14 smithi078 ceph-mon[31997]: pgmap v284: 97 pgs: 4 creating+peering, 18 active+clean, 75 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:14.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:14 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1533438677' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T15:14:14.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:14 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1533438677' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T15:14:14.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:14 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] up:active 2024-05-21T15:14:14.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:14 smithi078 ceph-mon[31997]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:14:14.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:14 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2505774485' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T15:14:14.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:14 smithi078 ceph-mon[31997]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-21T15:14:14.574 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:14 smithi078 ceph-mon[31997]: Cluster is now healthy 2024-05-21T15:14:14.575 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-21T15:14:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:14 smithi084 ceph-mon[36334]: pgmap v284: 97 pgs: 4 creating+peering, 18 active+clean, 75 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:14 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1533438677' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T15:14:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:14 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1533438677' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T15:14:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:14 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] up:active 2024-05-21T15:14:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:14 smithi084 ceph-mon[36334]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:14:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:14 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2505774485' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T15:14:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:14 smithi084 ceph-mon[36334]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-21T15:14:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:14 smithi084 ceph-mon[36334]: Cluster is now healthy 2024-05-21T15:14:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:14 smithi172 ceph-mon[36402]: pgmap v284: 97 pgs: 4 creating+peering, 18 active+clean, 75 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:14:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:14 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1533438677' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T15:14:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:14 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1533438677' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T15:14:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:14 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] up:active 2024-05-21T15:14:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:14 smithi172 ceph-mon[36402]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:14:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:14 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2505774485' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T15:14:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:14 smithi172 ceph-mon[36402]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-21T15:14:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:14 smithi172 ceph-mon[36402]: Cluster is now healthy 2024-05-21T15:14:14.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 -- 172.21.15.78:0/2695987580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 msgr2=0x7f01c41026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:14.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/2695987580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 0x7f01c41026b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009900 tx=0x7f01b402a410 comp rx=0 tx=0).stop 2024-05-21T15:14:14.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 -- 172.21.15.78:0/2695987580 shutdown_connections 2024-05-21T15:14:14.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/2695987580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01c4102bf0 0x7f01c4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/2695987580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01c41078f0 0x7f01c4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/2695987580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 0x7f01c41026b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 -- 172.21.15.78:0/2695987580 >> 172.21.15.78:0/2695987580 conn(0x7f01c40fcba0 msgr2=0x7f01c40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 -- 172.21.15.78:0/2695987580 shutdown_connections 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.687+0000 7f01cbe3f640 1 -- 172.21.15.78:0/2695987580 wait complete. 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01cbe3f640 1 Processor -- start 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01cbe3f640 1 -- start start 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01cbe3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 0x7f01c4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01cbe3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01c4102bf0 0x7f01c4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01cbe3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01c41078f0 0x7f01c419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01cbe3f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01c410d5e0 con 0x7f01c41019b0 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01cbe3f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01c410d460 con 0x7f01c4102bf0 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01cbe3f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01c410d760 con 0x7f01c41078f0 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01c9bdb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 0x7f01c4198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01c9bdb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 0x7f01c4198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55788/0 (socket says 172.21.15.78:55788) 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.688+0000 7f01c9bdb640 1 -- 172.21.15.78:0/3191323884 learned_addr learned my addr 172.21.15.78:0/3191323884 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c9bdb640 1 -- 172.21.15.78:0/3191323884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01c41078f0 msgr2=0x7f01c419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01ca3dc640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01c41078f0 0x7f01c419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c9bdb640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01c41078f0 0x7f01c419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c93da640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01c4102bf0 0x7f01c4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c9bdb640 1 -- 172.21.15.78:0/3191323884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01c4102bf0 msgr2=0x7f01c4198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c9bdb640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01c4102bf0 0x7f01c4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c9bdb640 1 -- 172.21.15.78:0/3191323884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c419fca0 con 0x7f01c41019b0 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01ca3dc640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01c41078f0 0x7f01c419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c9bdb640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 0x7f01c4198210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01b4006010 tx=0x7f01b40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:14.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c2ffd640 1 -- 172.21.15.78:0/3191323884 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b403f030 con 0x7f01c41019b0 2024-05-21T15:14:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c2ffd640 1 -- 172.21.15.78:0/3191323884 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b4003ce0 con 0x7f01c41019b0 2024-05-21T15:14:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c419ff30 con 0x7f01c41019b0 2024-05-21T15:14:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01c2ffd640 1 -- 172.21.15.78:0/3191323884 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4035070 con 0x7f01c41019b0 2024-05-21T15:14:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.689+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c41a0390 con 0x7f01c41019b0 2024-05-21T15:14:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.690+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c4106700 con 0x7f01c41019b0 2024-05-21T15:14:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.691+0000 7f01c2ffd640 1 -- 172.21.15.78:0/3191323884 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f01b4042030 con 0x7f01c41019b0 2024-05-21T15:14:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.692+0000 7f01c2ffd640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01ac0a2c10 0x7f01ac0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.692+0000 7f01c93da640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01ac0a2c10 0x7f01ac0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.692+0000 7f01c2ffd640 1 -- 172.21.15.78:0/3191323884 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f01b40ef4e0 con 0x7f01c41019b0 2024-05-21T15:14:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.693+0000 7f01c93da640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01ac0a2c10 0x7f01ac0a5120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f01b8005fd0 tx=0x7f01b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:14.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.697+0000 7f01c2ffd640 1 -- 172.21.15.78:0/3191323884 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f01b40bc590 con 0x7f01c41019b0 2024-05-21T15:14:14.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:14.912+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f01c402d2a0 con 0x7f01c41019b0 2024-05-21T15:14:15.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.568+0000 7f01c2ffd640 1 -- 172.21.15.78:0/3191323884 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f01c402d2a0 con 0x7f01c41019b0 2024-05-21T15:14:15.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01ac0a2c10 msgr2=0x7f01ac0a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:15.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01ac0a2c10 0x7f01ac0a5120 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f01b8005fd0 tx=0x7f01b8005850 comp rx=0 tx=0).stop 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 msgr2=0x7f01c4198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 0x7f01c4198210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01b4006010 tx=0x7f01b40043b0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 shutdown_connections 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01ac0a2c10 0x7f01ac0a5120 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01c4102bf0 0x7f01c4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01c41078f0 0x7f01c419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 --2- 172.21.15.78:0/3191323884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01c41019b0 0x7f01c4198210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 >> 172.21.15.78:0/3191323884 conn(0x7f01c40fcba0 msgr2=0x7f01c41038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 shutdown_connections 2024-05-21T15:14:15.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.571+0000 7f01cbe3f640 1 -- 172.21.15.78:0/3191323884 wait complete. 2024-05-21T15:14:15.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:15 smithi078 ceph-mon[31997]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T15:14:15.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:15 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2505774485' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T15:14:15.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:15 smithi078 ceph-mon[31997]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T15:14:15.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:15 smithi078 ceph-mon[31997]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:14:15.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:15 smithi078 ceph-mon[31997]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T15:14:15.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:15 smithi078 ceph-mon[31997]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T15:14:15.590 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:15 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3191323884' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T15:14:15.590 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-21T15:14:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:15 smithi084 ceph-mon[36334]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T15:14:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:15 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2505774485' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T15:14:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:15 smithi084 ceph-mon[36334]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T15:14:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:15 smithi084 ceph-mon[36334]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:14:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:15 smithi084 ceph-mon[36334]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T15:14:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:15 smithi084 ceph-mon[36334]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T15:14:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:15 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3191323884' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T15:14:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:15 smithi172 ceph-mon[36402]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T15:14:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:15 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2505774485' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T15:14:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:15 smithi172 ceph-mon[36402]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T15:14:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:15 smithi172 ceph-mon[36402]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T15:14:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:15 smithi172 ceph-mon[36402]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T15:14:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:15 smithi172 ceph-mon[36402]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T15:14:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:15 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3191323884' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T15:14:15.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 -- 172.21.15.78:0/3893580639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 msgr2=0x7f09981026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:15.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 --2- 172.21.15.78:0/3893580639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 0x7f09981026a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0988009900 tx=0x7f098802a410 comp rx=0 tx=0).stop 2024-05-21T15:14:15.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 -- 172.21.15.78:0/3893580639 shutdown_connections 2024-05-21T15:14:15.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 --2- 172.21.15.78:0/3893580639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0998102be0 0x7f0998102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 --2- 172.21.15.78:0/3893580639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09981078e0 0x7f0998109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 --2- 172.21.15.78:0/3893580639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 0x7f09981026a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 -- 172.21.15.78:0/3893580639 >> 172.21.15.78:0/3893580639 conn(0x7f09980fcb70 msgr2=0x7f09980ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 -- 172.21.15.78:0/3893580639 shutdown_connections 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 -- 172.21.15.78:0/3893580639 wait complete. 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 Processor -- start 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 -- start start 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 0x7f09981992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.702+0000 7f099e731640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0998102be0 0x7f0998199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f099e731640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09981078e0 0x7f09981a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f099e731640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f099810d5d0 con 0x7f09981019a0 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f099e731640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f099810d450 con 0x7f0998102be0 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f099e731640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f099810d750 con 0x7f09981078e0 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 0x7f09981992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:15.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 0x7f09981992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55806/0 (socket says 172.21.15.78:55806) 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 -- 172.21.15.78:0/4202888332 learned_addr learned my addr 172.21.15.78:0/4202888332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f098ffff640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0998102be0 0x7f0998199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f099ccce640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09981078e0 0x7f09981a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 -- 172.21.15.78:0/4202888332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09981078e0 msgr2=0x7f09981a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09981078e0 0x7f09981a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 -- 172.21.15.78:0/4202888332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0998102be0 msgr2=0x7f0998199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0998102be0 0x7f0998199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 -- 172.21.15.78:0/4202888332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09981a0d80 con 0x7f09981019a0 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0997fff640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 0x7f09981992f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0988006010 tx=0x7f0988031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0995ffb640 1 -- 172.21.15.78:0/4202888332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098803f030 con 0x7f09981019a0 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0995ffb640 1 -- 172.21.15.78:0/4202888332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0988003ad0 con 0x7f09981019a0 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09981a1010 con 0x7f09981019a0 2024-05-21T15:14:15.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f0995ffb640 1 -- 172.21.15.78:0/4202888332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988035070 con 0x7f09981019a0 2024-05-21T15:14:15.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.703+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09981a1550 con 0x7f09981019a0 2024-05-21T15:14:15.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.704+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09981a17b0 con 0x7f09981019a0 2024-05-21T15:14:15.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.706+0000 7f0995ffb640 1 -- 172.21.15.78:0/4202888332 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0988042030 con 0x7f09981019a0 2024-05-21T15:14:15.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.707+0000 7f0995ffb640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f09840a2c10 0x7f09840a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:15.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.707+0000 7f098ffff640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f09840a2c10 0x7f09840a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:15.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.707+0000 7f0995ffb640 1 -- 172.21.15.78:0/4202888332 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f09880ef6a0 con 0x7f09981019a0 2024-05-21T15:14:15.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.707+0000 7f098ffff640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f09840a2c10 0x7f09840a5120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f099819a390 tx=0x7f0980005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:15.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.711+0000 7f0995ffb640 1 -- 172.21.15.78:0/4202888332 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f09880bc750 con 0x7f09981019a0 2024-05-21T15:14:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:15.922+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f09981a1980 con 0x7f09981019a0 2024-05-21T15:14:16.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.256+0000 7f0995ffb640 1 -- 172.21.15.78:0/4202888332 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f09880bc050 con 0x7f09981019a0 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f09840a2c10 msgr2=0x7f09840a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f09840a2c10 0x7f09840a5120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f099819a390 tx=0x7f0980005f90 comp rx=0 tx=0).stop 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 msgr2=0x7f09981992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 0x7f09981992f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0988006010 tx=0x7f0988031b60 comp rx=0 tx=0).stop 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 shutdown_connections 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f09840a2c10 0x7f09840a5120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0998102be0 0x7f0998199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09981078e0 0x7f09981a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 --2- 172.21.15.78:0/4202888332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09981019a0 0x7f09981992f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 >> 172.21.15.78:0/4202888332 conn(0x7f09980fcb70 msgr2=0x7f0998108e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:16.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 shutdown_connections 2024-05-21T15:14:16.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.259+0000 7f099e731640 1 -- 172.21.15.78:0/4202888332 wait complete. 2024-05-21T15:14:16.276 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-21T15:14:16.277 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-21T15:14:16.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.387+0000 7f601e079640 1 -- 172.21.15.78:0/3380675351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60181079e0 msgr2=0x7f6018109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.387+0000 7f601e079640 1 --2- 172.21.15.78:0/3380675351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60181079e0 0x7f6018109e00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6010009ca0 tx=0x7f601002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 -- 172.21.15.78:0/3380675351 shutdown_connections 2024-05-21T15:14:16.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 --2- 172.21.15.78:0/3380675351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6018103390 0x7f6018101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 --2- 172.21.15.78:0/3380675351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60181079e0 0x7f6018109e00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 --2- 172.21.15.78:0/3380675351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6018102eb0 0x7f6018101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 -- 172.21.15.78:0/3380675351 >> 172.21.15.78:0/3380675351 conn(0x7f60180fc330 msgr2=0x7f60180fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:16.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 -- 172.21.15.78:0/3380675351 shutdown_connections 2024-05-21T15:14:16.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 -- 172.21.15.78:0/3380675351 wait complete. 2024-05-21T15:14:16.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 Processor -- start 2024-05-21T15:14:16.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.388+0000 7f601e079640 1 -- start start 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f601e079640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6018102eb0 0x7f6018197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f601e079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60181079e0 0x7f60181982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f601e079640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601819d1e0 0x7f601819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f601e079640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f601810d670 con 0x7f601819d1e0 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f601e079640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f601810d4f0 con 0x7f6018102eb0 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f601e079640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f601810d7f0 con 0x7f60181079e0 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601819d1e0 0x7f601819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601819d1e0 0x7f601819f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55830/0 (socket says 172.21.15.78:55830) 2024-05-21T15:14:16.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 -- 172.21.15.78:0/4180031108 learned_addr learned my addr 172.21.15.78:0/4180031108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f60177fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6018102eb0 0x7f6018197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6016ffd640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60181079e0 0x7f60181982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 -- 172.21.15.78:0/4180031108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60181079e0 msgr2=0x7f60181982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60181079e0 0x7f60181982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 -- 172.21.15.78:0/4180031108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6018102eb0 msgr2=0x7f6018197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6018102eb0 0x7f6018197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 -- 172.21.15.78:0/4180031108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601819fc70 con 0x7f601819d1e0 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f60177fe640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6018102eb0 0x7f6018197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6017fff640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601819d1e0 0x7f601819f600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60100319f0 tx=0x7f601002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6014ff9640 1 -- 172.21.15.78:0/4180031108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010003bf0 con 0x7f601819d1e0 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f6014ff9640 1 -- 172.21.15.78:0/4180031108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6010003d90 con 0x7f601819d1e0 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.389+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601819ff00 con 0x7f601819d1e0 2024-05-21T15:14:16.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.390+0000 7f6014ff9640 1 -- 172.21.15.78:0/4180031108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601002fd00 con 0x7f601819d1e0 2024-05-21T15:14:16.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.390+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60181a0370 con 0x7f601819d1e0 2024-05-21T15:14:16.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.390+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60181060e0 con 0x7f601819d1e0 2024-05-21T15:14:16.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.392+0000 7f6014ff9640 1 -- 172.21.15.78:0/4180031108 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6010004040 con 0x7f601819d1e0 2024-05-21T15:14:16.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.392+0000 7f6014ff9640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f60000a2c40 0x7f60000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:16.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.393+0000 7f60177fe640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f60000a2c40 0x7f60000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:16.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.393+0000 7f6014ff9640 1 -- 172.21.15.78:0/4180031108 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f60100f0640 con 0x7f601819d1e0 2024-05-21T15:14:16.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.393+0000 7f60177fe640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f60000a2c40 0x7f60000a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60080058c0 tx=0x7f600800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:16.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.396+0000 7f6014ff9640 1 -- 172.21.15.78:0/4180031108 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f60100bd6e0 con 0x7f601819d1e0 2024-05-21T15:14:16.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:16 smithi078 ceph-mon[31997]: pgmap v285: 97 pgs: 4 creating+peering, 78 active+clean, 15 unknown; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 7 op/s 2024-05-21T15:14:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:16 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3191323884' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T15:14:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:16 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] up:active 2024-05-21T15:14:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:16 smithi078 ceph-mon[31997]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T15:14:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:16 smithi078 ceph-mon[31997]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T15:14:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:16 smithi078 ceph-mon[31997]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-21T15:14:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:16 smithi078 ceph-mon[31997]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-21T15:14:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:16 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4202888332' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T15:14:16.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.597+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6018061f90 con 0x7f60000a2c40 2024-05-21T15:14:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.624+0000 7f6014ff9640 1 -- 172.21.15.78:0/4180031108 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6018061f90 con 0x7f60000a2c40 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f60000a2c40 msgr2=0x7f60000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f60000a2c40 0x7f60000a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60080058c0 tx=0x7f600800a2d0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601819d1e0 msgr2=0x7f601819f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601819d1e0 0x7f601819f600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60100319f0 tx=0x7f601002f380 comp rx=0 tx=0).stop 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 shutdown_connections 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f60000a2c40 0x7f60000a5150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601819d1e0 0x7f601819f600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60181079e0 0x7f60181982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 --2- 172.21.15.78:0/4180031108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6018102eb0 0x7f6018197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 >> 172.21.15.78:0/4180031108 conn(0x7f60180fc330 msgr2=0x7f60180fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 shutdown_connections 2024-05-21T15:14:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.626+0000 7f601e079640 1 -- 172.21.15.78:0/4180031108 wait complete. 2024-05-21T15:14:16.640 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-21T15:14:16.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:16 smithi084 ceph-mon[36334]: pgmap v285: 97 pgs: 4 creating+peering, 78 active+clean, 15 unknown; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 7 op/s 2024-05-21T15:14:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:16 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3191323884' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T15:14:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:16 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] up:active 2024-05-21T15:14:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:16 smithi084 ceph-mon[36334]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T15:14:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:16 smithi084 ceph-mon[36334]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T15:14:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:16 smithi084 ceph-mon[36334]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-21T15:14:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:16 smithi084 ceph-mon[36334]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-21T15:14:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:16 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4202888332' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T15:14:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:16 smithi172 ceph-mon[36402]: pgmap v285: 97 pgs: 4 creating+peering, 78 active+clean, 15 unknown; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 7 op/s 2024-05-21T15:14:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:16 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3191323884' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T15:14:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:16 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] up:active 2024-05-21T15:14:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:16 smithi172 ceph-mon[36402]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T15:14:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:16 smithi172 ceph-mon[36402]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T15:14:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:16 smithi172 ceph-mon[36402]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-21T15:14:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:16 smithi172 ceph-mon[36402]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-21T15:14:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:16 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4202888332' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.753+0000 7ff491745640 1 -- 172.21.15.78:0/1628507447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 msgr2=0x7ff48c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.753+0000 7ff491745640 1 --2- 172.21.15.78:0/1628507447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 0x7ff48c109d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff484009900 tx=0x7ff48402a410 comp rx=0 tx=0).stop 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.753+0000 7ff491745640 1 -- 172.21.15.78:0/1628507447 shutdown_connections 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.753+0000 7ff491745640 1 --2- 172.21.15.78:0/1628507447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 0x7ff48c109d10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.753+0000 7ff491745640 1 --2- 172.21.15.78:0/1628507447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff48c102bf0 0x7ff48c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.753+0000 7ff491745640 1 --2- 172.21.15.78:0/1628507447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48c1019b0 0x7ff48c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.753+0000 7ff491745640 1 -- 172.21.15.78:0/1628507447 >> 172.21.15.78:0/1628507447 conn(0x7ff48c0fcba0 msgr2=0x7ff48c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.753+0000 7ff491745640 1 -- 172.21.15.78:0/1628507447 shutdown_connections 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 -- 172.21.15.78:0/1628507447 wait complete. 2024-05-21T15:14:16.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 Processor -- start 2024-05-21T15:14:16.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 -- start start 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48c1019b0 0x7ff48c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff48c102bf0 0x7ff48c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 0x7ff48c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10d630 con 0x7ff48c1078f0 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10d4b0 con 0x7ff48c1019b0 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.754+0000 7ff491745640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10d7b0 con 0x7ff48c102bf0 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 0x7ff48c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 0x7ff48c1a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55864/0 (socket says 172.21.15.78:55864) 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48b7fe640 1 -- 172.21.15.78:0/4076372238 learned_addr learned my addr 172.21.15.78:0/4076372238 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48affd640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48c1019b0 0x7ff48c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48a7fc640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff48c102bf0 0x7ff48c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:16.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48affd640 1 -- 172.21.15.78:0/4076372238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff48c102bf0 msgr2=0x7ff48c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48affd640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff48c102bf0 0x7ff48c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48affd640 1 -- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 msgr2=0x7ff48c1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48affd640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 0x7ff48c1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48affd640 1 -- 172.21.15.78:0/4076372238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c1a0dd0 con 0x7ff48c1019b0 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48a7fc640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff48c102bf0 0x7ff48c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff48affd640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48c1019b0 0x7ff48c199290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff48c101280 tx=0x7ff47c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff473fff640 1 -- 172.21.15.78:0/4076372238 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c003970 con 0x7ff48c1019b0 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c1a10c0 con 0x7ff48c1019b0 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff473fff640 1 -- 172.21.15.78:0/4076372238 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff47c004030 con 0x7ff48c1019b0 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff473fff640 1 -- 172.21.15.78:0/4076372238 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c010760 con 0x7ff48c1019b0 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.755+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c1a1660 con 0x7ff48c1019b0 2024-05-21T15:14:16.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.756+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48c1a1920 con 0x7ff48c1019b0 2024-05-21T15:14:16.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.758+0000 7ff473fff640 1 -- 172.21.15.78:0/4076372238 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff47c0041e0 con 0x7ff48c1019b0 2024-05-21T15:14:16.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.758+0000 7ff473fff640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff4740a2b90 0x7ff4740a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:16.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.758+0000 7ff48a7fc640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff4740a2b90 0x7ff4740a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:16.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.758+0000 7ff473fff640 1 -- 172.21.15.78:0/4076372238 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff47c0cf5e0 con 0x7ff48c1019b0 2024-05-21T15:14:16.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.759+0000 7ff48a7fc640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff4740a2b90 0x7ff4740a50a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4800058c0 tx=0x7ff480005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:16.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.762+0000 7ff473fff640 1 -- 172.21.15.78:0/4076372238 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff47c09c670 con 0x7ff48c1019b0 2024-05-21T15:14:16.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.967+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff48c074490 con 0x7ff4740a2b90 2024-05-21T15:14:16.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.996+0000 7ff473fff640 1 -- 172.21.15.78:0/4076372238 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff48c074490 con 0x7ff4740a2b90 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff4740a2b90 msgr2=0x7ff4740a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff4740a2b90 0x7ff4740a50a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4800058c0 tx=0x7ff480005d50 comp rx=0 tx=0).stop 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48c1019b0 msgr2=0x7ff48c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48c1019b0 0x7ff48c199290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff48c101280 tx=0x7ff47c00ef90 comp rx=0 tx=0).stop 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 shutdown_connections 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff4740a2b90 0x7ff4740a50a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff48c1078f0 0x7ff48c1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff48c102bf0 0x7ff48c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 --2- 172.21.15.78:0/4076372238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff48c1019b0 0x7ff48c199290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 >> 172.21.15.78:0/4076372238 conn(0x7ff48c0fcba0 msgr2=0x7ff48c073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 shutdown_connections 2024-05-21T15:14:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:16.999+0000 7ff491745640 1 -- 172.21.15.78:0/4076372238 wait complete. 2024-05-21T15:14:17.012 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:14:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:14:16.602+0000 7fc4d730c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:14:17.012 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:14:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:14:16.602+0000 7fc4d730c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:14:17.012 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:14:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:14:16.602+0000 7fc4d730c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:14:17.012 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:14:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:14:16.602+0000 7fc4d730c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:14:17.012 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:14:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:14:16.602+0000 7fc4d730c700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:14:17.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-21T15:14:17.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 -- 172.21.15.78:0/1030898213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8102bd0 msgr2=0x7f3fb8102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 --2- 172.21.15.78:0/1030898213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8102bd0 0x7f3fb8102fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009900 tx=0x7f3fa802a410 comp rx=0 tx=0).stop 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 -- 172.21.15.78:0/1030898213 shutdown_connections 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 --2- 172.21.15.78:0/1030898213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb81078d0 0x7f3fb8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 --2- 172.21.15.78:0/1030898213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8102bd0 0x7f3fb8102fe0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 --2- 172.21.15.78:0/1030898213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb8101990 0x7f3fb8102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 -- 172.21.15.78:0/1030898213 >> 172.21.15.78:0/1030898213 conn(0x7f3fb80fcb80 msgr2=0x7f3fb80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 -- 172.21.15.78:0/1030898213 shutdown_connections 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.127+0000 7f3fc048d640 1 -- 172.21.15.78:0/1030898213 wait complete. 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fc048d640 1 Processor -- start 2024-05-21T15:14:17.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fc048d640 1 -- start start 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fc048d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8101990 0x7f3fb8069910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fc048d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb8102bd0 0x7f3fb819e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fc048d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb81078d0 0x7f3fb81a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fc048d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb810d5e0 con 0x7f3fb8101990 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fc048d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb810d460 con 0x7f3fb81078d0 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fc048d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb810d760 con 0x7f3fb8102bd0 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbe229640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8101990 0x7f3fb8069910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbe229640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8101990 0x7f3fb8069910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55878/0 (socket says 172.21.15.78:55878) 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbe229640 1 -- 172.21.15.78:0/3449121562 learned_addr learned my addr 172.21.15.78:0/3449121562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbea2a640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb81078d0 0x7f3fb81a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbda28640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb8102bd0 0x7f3fb819e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbe229640 1 -- 172.21.15.78:0/3449121562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb8102bd0 msgr2=0x7f3fb819e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbe229640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb8102bd0 0x7f3fb819e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbe229640 1 -- 172.21.15.78:0/3449121562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb81078d0 msgr2=0x7f3fb81a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbe229640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb81078d0 0x7f3fb81a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbe229640 1 -- 172.21.15.78:0/3449121562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb81a10a0 con 0x7f3fb8101990 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.128+0000 7f3fbea2a640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb81078d0 0x7f3fb81a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.129+0000 7f3fbe229640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8101990 0x7f3fb8069910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e980 tx=0x7f3fb400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.129+0000 7f3faf7fe640 1 -- 172.21.15.78:0/3449121562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400cd80 con 0x7f3fb8101990 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.129+0000 7f3faf7fe640 1 -- 172.21.15.78:0/3449121562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb4010910 con 0x7f3fb8101990 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.129+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb81a1390 con 0x7f3fb8101990 2024-05-21T15:14:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.129+0000 7f3faf7fe640 1 -- 172.21.15.78:0/3449121562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40189c0 con 0x7f3fb8101990 2024-05-21T15:14:17.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.129+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb81a18e0 con 0x7f3fb8101990 2024-05-21T15:14:17.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.130+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb81066e0 con 0x7f3fb8101990 2024-05-21T15:14:17.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.131+0000 7f3faf7fe640 1 -- 172.21.15.78:0/3449121562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3fb401f020 con 0x7f3fb8101990 2024-05-21T15:14:17.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.132+0000 7f3faf7fe640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3fa40a2b90 0x7f3fa40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.132+0000 7f3fbda28640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3fa40a2b90 0x7f3fa40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.132+0000 7f3faf7fe640 1 -- 172.21.15.78:0/3449121562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3fb40cf530 con 0x7f3fb8101990 2024-05-21T15:14:17.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.132+0000 7f3fbda28640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3fa40a2b90 0x7f3fa40a50a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8000c00 tx=0x7f3fa8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.135+0000 7f3faf7fe640 1 -- 172.21.15.78:0/3449121562 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3fb409c5e0 con 0x7f3fb8101990 2024-05-21T15:14:17.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:17 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4202888332' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T15:14:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:17 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] up:active 2024-05-21T15:14:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:17 smithi078 ceph-mon[31997]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T15:14:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:17 smithi078 ceph-mon[31997]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-21T15:14:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:17 smithi078 ceph-mon[31997]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-05-21T15:14:17.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:17 smithi078 ceph-mon[31997]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-05-21T15:14:17.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.355+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3fb802d560 con 0x7f3fa40a2b90 2024-05-21T15:14:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.360+0000 7f3faf7fe640 1 -- 172.21.15.78:0/3449121562 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3fb802d560 con 0x7f3fa40a2b90 2024-05-21T15:14:17.363 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_0/56e20639-40e7-449b-888f-925c47c4435a 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3fa40a2b90 msgr2=0x7f3fa40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3fa40a2b90 0x7f3fa40a50a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8000c00 tx=0x7f3fa8031400 comp rx=0 tx=0).stop 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8101990 msgr2=0x7f3fb8069910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8101990 0x7f3fb8069910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e980 tx=0x7f3fb400ee40 comp rx=0 tx=0).stop 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 shutdown_connections 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3fa40a2b90 0x7f3fa40a50a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fb81078d0 0x7f3fb81a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb8102bd0 0x7f3fb819e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 --2- 172.21.15.78:0/3449121562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3fb8101990 0x7f3fb8069910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 >> 172.21.15.78:0/3449121562 conn(0x7f3fb80fcb80 msgr2=0x7f3fb81038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 shutdown_connections 2024-05-21T15:14:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.362+0000 7f3fc048d640 1 -- 172.21.15.78:0/3449121562 wait complete. 2024-05-21T15:14:17.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-21T15:14:17.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 -- 172.21.15.78:0/673316835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c4101990 msgr2=0x7f12c4102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 --2- 172.21.15.78:0/673316835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c4101990 0x7f12c4102690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009900 tx=0x7f12b402a410 comp rx=0 tx=0).stop 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 -- 172.21.15.78:0/673316835 shutdown_connections 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 --2- 172.21.15.78:0/673316835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c41078d0 0x7f12c4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 --2- 172.21.15.78:0/673316835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12c4102bd0 0x7f12c4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 --2- 172.21.15.78:0/673316835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c4101990 0x7f12c4102690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 -- 172.21.15.78:0/673316835 >> 172.21.15.78:0/673316835 conn(0x7f12c40fcb80 msgr2=0x7f12c40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 -- 172.21.15.78:0/673316835 shutdown_connections 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.488+0000 7f12c9977640 1 -- 172.21.15.78:0/673316835 wait complete. 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c9977640 1 Processor -- start 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c9977640 1 -- start start 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c9977640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c4101990 0x7f12c4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c9977640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12c4102bd0 0x7f12c4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c9977640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c41078d0 0x7f12c419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c9977640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410d5e0 con 0x7f12c41078d0 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c9977640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410d460 con 0x7f12c4101990 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c9977640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12c410d760 con 0x7f12c4102bd0 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c41078d0 0x7f12c419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c41078d0 0x7f12c419f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55902/0 (socket says 172.21.15.78:55902) 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c37fe640 1 -- 172.21.15.78:0/1011551817 learned_addr learned my addr 172.21.15.78:0/1011551817 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.489+0000 7f12c2ffd640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c4101990 0x7f12c4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12ba7fc640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12c4102bd0 0x7f12c4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c37fe640 1 -- 172.21.15.78:0/1011551817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12c4102bd0 msgr2=0x7f12c4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c37fe640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12c4102bd0 0x7f12c4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c37fe640 1 -- 172.21.15.78:0/1011551817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c4101990 msgr2=0x7f12c4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c37fe640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c4101990 0x7f12c4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c37fe640 1 -- 172.21.15.78:0/1011551817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c41a0040 con 0x7f12c41078d0 2024-05-21T15:14:17.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c37fe640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c41078d0 0x7f12c419f9d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00eb10 tx=0x7f12bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:17.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c8975640 1 -- 172.21.15.78:0/1011551817 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc004b20 con 0x7f12c41078d0 2024-05-21T15:14:17.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c8975640 1 -- 172.21.15.78:0/1011551817 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12bc004cc0 con 0x7f12c41078d0 2024-05-21T15:14:17.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c41a0330 con 0x7f12c41078d0 2024-05-21T15:14:17.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c8975640 1 -- 172.21.15.78:0/1011551817 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc010760 con 0x7f12c41078d0 2024-05-21T15:14:17.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.490+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c41a0830 con 0x7f12c41078d0 2024-05-21T15:14:17.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.491+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c41066e0 con 0x7f12c41078d0 2024-05-21T15:14:17.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.492+0000 7f12c8975640 1 -- 172.21.15.78:0/1011551817 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f12bc005020 con 0x7f12c41078d0 2024-05-21T15:14:17.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.493+0000 7f12c8975640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f12b00a2b90 0x7f12b00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.493+0000 7f12c2ffd640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f12b00a2b90 0x7f12b00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.493+0000 7f12c8975640 1 -- 172.21.15.78:0/1011551817 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f12bc0cf550 con 0x7f12c41078d0 2024-05-21T15:14:17.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.493+0000 7f12c2ffd640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f12b00a2b90 0x7f12b00a50a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12b4005ab0 tx=0x7f12b4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:17.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.497+0000 7f12c8975640 1 -- 172.21.15.78:0/1011551817 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12bc09c600 con 0x7f12c41078d0 2024-05-21T15:14:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4202888332' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T15:14:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] up:active 2024-05-21T15:14:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-mon[36334]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T15:14:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-mon[36334]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-21T15:14:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-mon[36334]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-05-21T15:14:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-mon[36334]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-05-21T15:14:17.646 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: 2024-05-21T15:14:17.308+0000 7f4ad121b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-21T15:14:17.647 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: 2024-05-21T15:14:17.308+0000 7f4ad121b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-21T15:14:17.647 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: 2024-05-21T15:14:17.308+0000 7f4ad121b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-21T15:14:17.647 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:14:17 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: 2024-05-21T15:14:17.308+0000 7f4ad121b700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-21T15:14:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:17 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4202888332' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T15:14:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:17 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] up:active 2024-05-21T15:14:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:17 smithi172 ceph-mon[36402]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T15:14:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:17 smithi172 ceph-mon[36402]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-21T15:14:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:17 smithi172 ceph-mon[36402]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-05-21T15:14:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:17 smithi172 ceph-mon[36402]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-05-21T15:14:17.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.701+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f12c402d4d0 con 0x7f12b00a2b90 2024-05-21T15:14:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.733+0000 7f12c8975640 1 -- 172.21.15.78:0/1011551817 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12c402d4d0 con 0x7f12b00a2b90 2024-05-21T15:14:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f12b00a2b90 msgr2=0x7f12b00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f12b00a2b90 0x7f12b00a50a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12b4005ab0 tx=0x7f12b4031400 comp rx=0 tx=0).stop 2024-05-21T15:14:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c41078d0 msgr2=0x7f12c419f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c41078d0 0x7f12c419f9d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00eb10 tx=0x7f12bc00c2d0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 shutdown_connections 2024-05-21T15:14:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f12b00a2b90 0x7f12b00a50a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c41078d0 0x7f12c419f9d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12c4102bd0 0x7f12c4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 --2- 172.21.15.78:0/1011551817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c4101990 0x7f12c4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 >> 172.21.15.78:0/1011551817 conn(0x7f12c40fcb80 msgr2=0x7f12c41038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 shutdown_connections 2024-05-21T15:14:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.735+0000 7f12c9977640 1 -- 172.21.15.78:0/1011551817 wait complete. 2024-05-21T15:14:17.749 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-21T15:14:17.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.862+0000 7fb99d557640 1 -- 172.21.15.78:0/2259949362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb998103ce0 msgr2=0x7fb9981040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.862+0000 7fb99d557640 1 --2- 172.21.15.78:0/2259949362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb998103ce0 0x7fb9981040f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009900 tx=0x7fb98c02a410 comp rx=0 tx=0).stop 2024-05-21T15:14:17.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.862+0000 7fb99d557640 1 -- 172.21.15.78:0/2259949362 shutdown_connections 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.862+0000 7fb99d557640 1 --2- 172.21.15.78:0/2259949362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9981089e0 0x7fb99810ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.862+0000 7fb99d557640 1 --2- 172.21.15.78:0/2259949362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb998103ce0 0x7fb9981040f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.862+0000 7fb99d557640 1 --2- 172.21.15.78:0/2259949362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb998102aa0 0x7fb9981037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.862+0000 7fb99d557640 1 -- 172.21.15.78:0/2259949362 >> 172.21.15.78:0/2259949362 conn(0x7fb9980fdc90 msgr2=0x7fb998100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 -- 172.21.15.78:0/2259949362 shutdown_connections 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 -- 172.21.15.78:0/2259949362 wait complete. 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 Processor -- start 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 -- start start 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb998102aa0 0x7fb998199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb998103ce0 0x7fb998199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9981089e0 0x7fb9981a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb99810e770 con 0x7fb9981089e0 2024-05-21T15:14:17.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb99810e5f0 con 0x7fb998102aa0 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.863+0000 7fb99d557640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb99810e8f0 con 0x7fb998103ce0 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb998102aa0 0x7fb998199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb9977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9981089e0 0x7fb9981a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb998102aa0 0x7fb998199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:32780/0 (socket says 172.21.15.78:32780) 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 -- 172.21.15.78:0/1184667557 learned_addr learned my addr 172.21.15.78:0/1184667557 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb9967fc640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb998103ce0 0x7fb998199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 -- 172.21.15.78:0/1184667557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb998103ce0 msgr2=0x7fb998199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb998103ce0 0x7fb998199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 -- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9981089e0 msgr2=0x7fb9981a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9981089e0 0x7fb9981a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 -- 172.21.15.78:0/1184667557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9981a0d90 con 0x7fb998102aa0 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb9977fe640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9981089e0 0x7fb9981a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb996ffd640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb998102aa0 0x7fb998199250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb98800eb90 tx=0x7fb98800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb97ffff640 1 -- 172.21.15.78:0/1184667557 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988003b70 con 0x7fb998102aa0 2024-05-21T15:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb97ffff640 1 -- 172.21.15.78:0/1184667557 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb98800ee90 con 0x7fb998102aa0 2024-05-21T15:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9981a1080 con 0x7fb998102aa0 2024-05-21T15:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb97ffff640 1 -- 172.21.15.78:0/1184667557 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988019820 con 0x7fb998102aa0 2024-05-21T15:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.864+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9981a15d0 con 0x7fb998102aa0 2024-05-21T15:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.865+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9981077f0 con 0x7fb998102aa0 2024-05-21T15:14:17.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.867+0000 7fb97ffff640 1 -- 172.21.15.78:0/1184667557 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb988004b60 con 0x7fb998102aa0 2024-05-21T15:14:17.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.867+0000 7fb97ffff640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9800a2b90 0x7fb9800a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:17.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.867+0000 7fb9967fc640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9800a2b90 0x7fb9800a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:17.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.867+0000 7fb97ffff640 1 -- 172.21.15.78:0/1184667557 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb9880cf7f0 con 0x7fb998102aa0 2024-05-21T15:14:17.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.868+0000 7fb9967fc640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9800a2b90 0x7fb9800a50a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb98c006010 tx=0x7fb98c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:17.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:17.871+0000 7fb97ffff640 1 -- 172.21.15.78:0/1184667557 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb98809c880 con 0x7fb998102aa0 2024-05-21T15:14:18.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.073+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb998061f90 con 0x7fb9800a2b90 2024-05-21T15:14:18.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.078+0000 7fb97ffff640 1 -- 172.21.15.78:0/1184667557 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb998061f90 con 0x7fb9800a2b90 2024-05-21T15:14:18.081 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_1/305cad38-0d45-4543-b894-976a9e5440da 2024-05-21T15:14:18.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9800a2b90 msgr2=0x7fb9800a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9800a2b90 0x7fb9800a50a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb98c006010 tx=0x7fb98c031400 comp rx=0 tx=0).stop 2024-05-21T15:14:18.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb998102aa0 msgr2=0x7fb998199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb998102aa0 0x7fb998199250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb98800eb90 tx=0x7fb98800c6a0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 shutdown_connections 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb9800a2b90 0x7fb9800a50a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9981089e0 0x7fb9981a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb998103ce0 0x7fb998199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 --2- 172.21.15.78:0/1184667557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb998102aa0 0x7fb998199250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.080+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 >> 172.21.15.78:0/1184667557 conn(0x7fb9980fdc90 msgr2=0x7fb9981049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.081+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 shutdown_connections 2024-05-21T15:14:18.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.081+0000 7fb99d557640 1 -- 172.21.15.78:0/1184667557 wait complete. 2024-05-21T15:14:18.095 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.202+0000 7faddb3bc640 1 -- 172.21.15.78:0/3126073152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4102ab0 msgr2=0x7fadd41037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.202+0000 7faddb3bc640 1 --2- 172.21.15.78:0/3126073152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4102ab0 0x7fadd41037b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009ca0 tx=0x7fadc402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 -- 172.21.15.78:0/3126073152 shutdown_connections 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 --2- 172.21.15.78:0/3126073152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd4103cf0 0x7fadd4104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 --2- 172.21.15.78:0/3126073152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd41089f0 0x7fadd410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 --2- 172.21.15.78:0/3126073152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4102ab0 0x7fadd41037b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 -- 172.21.15.78:0/3126073152 >> 172.21.15.78:0/3126073152 conn(0x7fadd40fdcc0 msgr2=0x7fadd4100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 -- 172.21.15.78:0/3126073152 shutdown_connections 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 -- 172.21.15.78:0/3126073152 wait complete. 2024-05-21T15:14:18.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 Processor -- start 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.203+0000 7faddb3bc640 1 -- start start 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7faddb3bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd4102ab0 0x7fadd4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7faddb3bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4103cf0 0x7fadd4199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7faddb3bc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd41089f0 0x7fadd41a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7faddb3bc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fadd410e760 con 0x7fadd4103cf0 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7faddb3bc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fadd410e5e0 con 0x7fadd41089f0 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7faddb3bc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fadd410e8e0 con 0x7fadd4102ab0 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4103cf0 0x7fadd4199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4103cf0 0x7fadd4199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55920/0 (socket says 172.21.15.78:55920) 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 -- 172.21.15.78:0/284580085 learned_addr learned my addr 172.21.15.78:0/284580085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd9959640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd41089f0 0x7fadd41a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd9158640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd4102ab0 0x7fadd4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 -- 172.21.15.78:0/284580085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd4102ab0 msgr2=0x7fadd4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd4102ab0 0x7fadd4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 -- 172.21.15.78:0/284580085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd41089f0 msgr2=0x7fadd41a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd41089f0 0x7fadd41a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 -- 172.21.15.78:0/284580085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd41a0e70 con 0x7fadd4103cf0 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd8957640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4103cf0 0x7fadd4199870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadc800ea60 tx=0x7fadc800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd27fc640 1 -- 172.21.15.78:0/284580085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc800cde0 con 0x7fadd4103cf0 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd27fc640 1 -- 172.21.15.78:0/284580085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc80035f0 con 0x7fadd4103cf0 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd41a1160 con 0x7fadd4103cf0 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7fadd27fc640 1 -- 172.21.15.78:0/284580085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8010670 con 0x7fadd4103cf0 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.204+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd41a1730 con 0x7fadd4103cf0 2024-05-21T15:14:18.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.205+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd4107800 con 0x7fadd4103cf0 2024-05-21T15:14:18.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.207+0000 7fadd27fc640 1 -- 172.21.15.78:0/284580085 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fadc80037a0 con 0x7fadd4103cf0 2024-05-21T15:14:18.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.207+0000 7fadd27fc640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fadbc0a2b90 0x7fadbc0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.207+0000 7fadd9158640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fadbc0a2b90 0x7fadbc0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.207+0000 7fadd27fc640 1 -- 172.21.15.78:0/284580085 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fadc80cf310 con 0x7fadd4103cf0 2024-05-21T15:14:18.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.208+0000 7fadd9158640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fadbc0a2b90 0x7fadbc0a50a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadc4006010 tx=0x7fadc4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:18.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.211+0000 7fadd27fc640 1 -- 172.21.15.78:0/284580085 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fadc809c450 con 0x7fadd4103cf0 2024-05-21T15:14:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.286+0000 7fadd27fc640 1 -- 172.21.15.78:0/284580085 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fadc80983c0 con 0x7fadd4103cf0 2024-05-21T15:14:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 5.0 KiB/s wr, 14 op/s 2024-05-21T15:14:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] up:active 2024-05-21T15:14:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T15:14:18.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T15:14:18.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: Cluster is now healthy 2024-05-21T15:14:18.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-05-21T15:14:18.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-05-21T15:14:18.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:18 smithi078 ceph-mon[31997]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-05-21T15:14:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.433+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadd402d2a0 con 0x7fadd4103cf0 2024-05-21T15:14:18.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.434+0000 7fadd27fc640 1 -- 172.21.15.78:0/284580085 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11925 (secure 0 0 0) 0x7fadc80c7230 con 0x7fadd4103cf0 2024-05-21T15:14:18.437 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:18.437 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:14:18.291660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:14:18.437 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-05-21T15:14:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fadbc0a2b90 msgr2=0x7fadbc0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fadbc0a2b90 0x7fadbc0a50a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadc4006010 tx=0x7fadc4000f40 comp rx=0 tx=0).stop 2024-05-21T15:14:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4103cf0 msgr2=0x7fadd4199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4103cf0 0x7fadd4199870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadc800ea60 tx=0x7fadc800ef20 comp rx=0 tx=0).stop 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 shutdown_connections 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fadbc0a2b90 0x7fadbc0a50a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadd41089f0 0x7fadd41a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fadd4103cf0 0x7fadd4199870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 --2- 172.21.15.78:0/284580085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadd4102ab0 0x7fadd4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 >> 172.21.15.78:0/284580085 conn(0x7fadd40fdcc0 msgr2=0x7fadd41049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 shutdown_connections 2024-05-21T15:14:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.436+0000 7faddb3bc640 1 -- 172.21.15.78:0/284580085 wait complete. 2024-05-21T15:14:18.451 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:14:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 -- 172.21.15.78:0/3305662926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01078f0 msgr2=0x7f42d0109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 --2- 172.21.15.78:0/3305662926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01078f0 0x7f42d0109d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009ca0 tx=0x7f42c802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 -- 172.21.15.78:0/3305662926 shutdown_connections 2024-05-21T15:14:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 --2- 172.21.15.78:0/3305662926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42d0102bf0 0x7f42d0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 --2- 172.21.15.78:0/3305662926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01078f0 0x7f42d0109d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 --2- 172.21.15.78:0/3305662926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d01019b0 0x7f42d01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 -- 172.21.15.78:0/3305662926 >> 172.21.15.78:0/3305662926 conn(0x7f42d00fcba0 msgr2=0x7f42d00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 -- 172.21.15.78:0/3305662926 shutdown_connections 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 -- 172.21.15.78:0/3305662926 wait complete. 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 Processor -- start 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.559+0000 7f42d7032640 1 -- start start 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d7032640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01019b0 0x7f42d0198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d7032640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42d0102bf0 0x7f42d0198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d7032640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d01078f0 0x7f42d019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d7032640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42d010d610 con 0x7f42d01019b0 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d7032640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f42d010d490 con 0x7f42d0102bf0 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d7032640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f42d010d790 con 0x7f42d01078f0 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01019b0 0x7f42d0198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01019b0 0x7f42d0198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50768/0 (socket says 172.21.15.78:50768) 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 -- 172.21.15.78:0/2006450778 learned_addr learned my addr 172.21.15.78:0/2006450778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42cffff640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42d0102bf0 0x7f42d0198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d55cf640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d01078f0 0x7f42d019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 -- 172.21.15.78:0/2006450778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d01078f0 msgr2=0x7f42d019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d01078f0 0x7f42d019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 -- 172.21.15.78:0/2006450778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42d0102bf0 msgr2=0x7f42d0198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42d0102bf0 0x7f42d0198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 -- 172.21.15.78:0/2006450778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d019fc40 con 0x7f42d01019b0 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42cffff640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42d0102bf0 0x7f42d0198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d55cf640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d01078f0 0x7f42d019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d4dce640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01019b0 0x7f42d0198150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42c4005fd0 tx=0x7f42c400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42cdffb640 1 -- 172.21.15.78:0/2006450778 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4017070 con 0x7f42d01019b0 2024-05-21T15:14:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42cdffb640 1 -- 172.21.15.78:0/2006450778 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42c4009c30 con 0x7f42d01019b0 2024-05-21T15:14:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d019ff30 con 0x7f42d01019b0 2024-05-21T15:14:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.560+0000 7f42cdffb640 1 -- 172.21.15.78:0/2006450778 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4003e40 con 0x7f42d01019b0 2024-05-21T15:14:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.561+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d01a03f0 con 0x7f42d01019b0 2024-05-21T15:14:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.562+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d0106700 con 0x7f42d01019b0 2024-05-21T15:14:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.563+0000 7f42cdffb640 1 -- 172.21.15.78:0/2006450778 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f42c4004000 con 0x7f42d01019b0 2024-05-21T15:14:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.563+0000 7f42cdffb640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f42c00a2ca0 0x7f42c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.563+0000 7f42cffff640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f42c00a2ca0 0x7f42c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.563+0000 7f42cdffb640 1 -- 172.21.15.78:0/2006450778 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f42c40cecb0 con 0x7f42d01019b0 2024-05-21T15:14:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.564+0000 7f42cffff640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f42c00a2ca0 0x7f42c00a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0095f0 tx=0x7f42bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.568+0000 7f42cdffb640 1 -- 172.21.15.78:0/2006450778 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f42c4020030 con 0x7f42d01019b0 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 5.0 KiB/s wr, 14 op/s 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] up:active 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: Cluster is now healthy 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-05-21T15:14:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:18 smithi084 ceph-mon[36334]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-05-21T15:14:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 5.0 KiB/s wr, 14 op/s 2024-05-21T15:14:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] up:active 2024-05-21T15:14:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T15:14:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T15:14:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: Cluster is now healthy 2024-05-21T15:14:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-05-21T15:14:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-05-21T15:14:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:18 smithi172 ceph-mon[36402]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-05-21T15:14:18.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.745+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42d002d450 con 0x7f42d01019b0 2024-05-21T15:14:18.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.746+0000 7f42cdffb640 1 -- 172.21.15.78:0/2006450778 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22147 (secure 0 0 0) 0x7f42c409bc60 con 0x7f42d01019b0 2024-05-21T15:14:18.750 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:18.750 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:14:13.233294+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f42c00a2ca0 msgr2=0x7f42c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f42c00a2ca0 0x7f42c00a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0095f0 tx=0x7f42bc005f90 comp rx=0 tx=0).stop 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01019b0 msgr2=0x7f42d0198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01019b0 0x7f42d0198150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42c4005fd0 tx=0x7f42c400dde0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 shutdown_connections 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f42c00a2ca0 0x7f42c00a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42d0102bf0 0x7f42d0198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d01078f0 0x7f42d019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 --2- 172.21.15.78:0/2006450778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42d01019b0 0x7f42d0198150 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 >> 172.21.15.78:0/2006450778 conn(0x7f42d00fcba0 msgr2=0x7f42d01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 shutdown_connections 2024-05-21T15:14:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.749+0000 7f42d7032640 1 -- 172.21.15.78:0/2006450778 wait complete. 2024-05-21T15:14:18.763 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-21T15:14:18.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:14:18.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 -- 172.21.15.78:0/86131894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 msgr2=0x7efc80109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 --2- 172.21.15.78:0/86131894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 0x7efc80109470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efc78009ca0 tx=0x7efc7802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 -- 172.21.15.78:0/86131894 shutdown_connections 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 --2- 172.21.15.78:0/86131894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc80102350 0x7efc80102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 --2- 172.21.15.78:0/86131894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 0x7efc80109470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 --2- 172.21.15.78:0/86131894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc80101110 0x7efc80101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 -- 172.21.15.78:0/86131894 >> 172.21.15.78:0/86131894 conn(0x7efc800fc2e0 msgr2=0x7efc800fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 -- 172.21.15.78:0/86131894 shutdown_connections 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.871+0000 7efc86cd7640 1 -- 172.21.15.78:0/86131894 wait complete. 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc86cd7640 1 Processor -- start 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc86cd7640 1 -- start start 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc86cd7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc80101110 0x7efc80199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc86cd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc80102350 0x7efc80199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc86cd7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 0x7efc801a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc86cd7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010cdd0 con 0x7efc80107050 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc86cd7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010cc50 con 0x7efc80101110 2024-05-21T15:14:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc86cd7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010cf50 con 0x7efc80102350 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc85274640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 0x7efc801a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc85274640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 0x7efc801a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50776/0 (socket says 172.21.15.78:50776) 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc85274640 1 -- 172.21.15.78:0/1635379950 learned_addr learned my addr 172.21.15.78:0/1635379950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc84a73640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc80101110 0x7efc80199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc85274640 1 -- 172.21.15.78:0/1635379950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc80102350 msgr2=0x7efc80199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.872+0000 7efc7ffff640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc80102350 0x7efc80199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc85274640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc80102350 0x7efc80199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc85274640 1 -- 172.21.15.78:0/1635379950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc80101110 msgr2=0x7efc80199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc85274640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc80101110 0x7efc80199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc85274640 1 -- 172.21.15.78:0/1635379950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc801a0d60 con 0x7efc80107050 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc84a73640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc80101110 0x7efc80199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc7ffff640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc80102350 0x7efc80199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:18.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc85274640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 0x7efc801a06f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efc780319f0 tx=0x7efc7802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:18.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc7dffb640 1 -- 172.21.15.78:0/1635379950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78003bf0 con 0x7efc80107050 2024-05-21T15:14:18.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc7dffb640 1 -- 172.21.15.78:0/1635379950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc78003d90 con 0x7efc80107050 2024-05-21T15:14:18.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc801a0ff0 con 0x7efc80107050 2024-05-21T15:14:18.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc7dffb640 1 -- 172.21.15.78:0/1635379950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7802fd00 con 0x7efc80107050 2024-05-21T15:14:18.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.873+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc801a1450 con 0x7efc80107050 2024-05-21T15:14:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.874+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc801a16b0 con 0x7efc80107050 2024-05-21T15:14:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.875+0000 7efc7dffb640 1 -- 172.21.15.78:0/1635379950 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc78004040 con 0x7efc80107050 2024-05-21T15:14:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.876+0000 7efc7dffb640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7efc700a2c20 0x7efc700a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.876+0000 7efc84a73640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7efc700a2c20 0x7efc700a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.876+0000 7efc7dffb640 1 -- 172.21.15.78:0/1635379950 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7efc780f0610 con 0x7efc80107050 2024-05-21T15:14:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.876+0000 7efc84a73640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7efc700a2c20 0x7efc700a5130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc74005fd0 tx=0x7efc7400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:18.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:18.881+0000 7efc7dffb640 1 -- 172.21.15.78:0/1635379950 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efc780bd7f0 con 0x7efc80107050 2024-05-21T15:14:19.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.055+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc801a1880 con 0x7efc700a2c20 2024-05-21T15:14:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.062+0000 7efc7dffb640 1 -- 172.21.15.78:0/1635379950 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258744 (secure 0 0 0) 0x7efc801a1880 con 0x7efc700a2c20 2024-05-21T15:14:19.068 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:19.069 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-05-21T15:14:19.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7efc700a2c20 msgr2=0x7efc700a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:19.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7efc700a2c20 0x7efc700a5130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc74005fd0 tx=0x7efc7400b390 comp rx=0 tx=0).stop 2024-05-21T15:14:19.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 msgr2=0x7efc801a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 0x7efc801a06f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efc780319f0 tx=0x7efc7802f380 comp rx=0 tx=0).stop 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 shutdown_connections 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7efc700a2c20 0x7efc700a5130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc80107050 0x7efc801a06f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc80102350 0x7efc80199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 --2- 172.21.15.78:0/1635379950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc80101110 0x7efc80199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.065+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 >> 172.21.15.78:0/1635379950 conn(0x7efc800fc2e0 msgr2=0x7efc800ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.066+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 shutdown_connections 2024-05-21T15:14:19.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.066+0000 7efc86cd7640 1 -- 172.21.15.78:0/1635379950 wait complete. 2024-05-21T15:14:19.080 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-05-21T15:14:17.241536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593002,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":141,"num_read_kb":124,"num_write":264,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357540,"kb_used_data":12492,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584284,"statfs":{"total":1151940427776,"available":1151574306816,"internally_reserved":0,"allocated":12791808,"data_stored":6989437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2634,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001206"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.254776+0000","last_change":"2024-05-21T15:14:12.254239+0000","last_active":"2024-05-21T15:14:12.254776+0000","last_peered":"2024-05-21T15:14:12.254776+0000","last_clean":"2024-05-21T15:14:12.254776+0000","last_became_active":"2024-05-21T15:14:12.253885+0000","last_became_peered":"2024-05-21T15:14:12.253885+0000","last_unstale":"2024-05-21T15:14:12.254776+0000","last_undegraded":"2024-05-21T15:14:12.254776+0000","last_fullsized":"2024-05-21T15:14:12.254776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:34:07.413828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375498+0000","last_change":"2024-05-21T15:14:12.262984+0000","last_active":"2024-05-21T15:14:14.375498+0000","last_peered":"2024-05-21T15:14:14.375498+0000","last_clean":"2024-05-21T15:14:14.375498+0000","last_became_active":"2024-05-21T15:14:12.262667+0000","last_became_peered":"2024-05-21T15:14:12.262667+0000","last_unstale":"2024-05-21T15:14:14.375498+0000","last_undegraded":"2024-05-21T15:14:14.375498+0000","last_fullsized":"2024-05-21T15:14:14.375498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:47:47.434314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375519+0000","last_change":"2024-05-21T15:14:12.260573+0000","last_active":"2024-05-21T15:14:14.375519+0000","last_peered":"2024-05-21T15:14:14.375519+0000","last_clean":"2024-05-21T15:14:14.375519+0000","last_became_active":"2024-05-21T15:14:12.259073+0000","last_became_peered":"2024-05-21T15:14:12.259073+0000","last_unstale":"2024-05-21T15:14:14.375519+0000","last_undegraded":"2024-05-21T15:14:14.375519+0000","last_fullsized":"2024-05-21T15:14:14.375519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:15:01.919464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375289+0000","last_change":"2024-05-21T15:14:12.256992+0000","last_active":"2024-05-21T15:14:14.375289+0000","last_peered":"2024-05-21T15:14:14.375289+0000","last_clean":"2024-05-21T15:14:14.375289+0000","last_became_active":"2024-05-21T15:14:12.256448+0000","last_became_peered":"2024-05-21T15:14:12.256448+0000","last_unstale":"2024-05-21T15:14:14.375289+0000","last_undegraded":"2024-05-21T15:14:14.375289+0000","last_fullsized":"2024-05-21T15:14:14.375289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:40.569263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484495+0000","last_change":"2024-05-21T15:14:12.251533+0000","last_active":"2024-05-21T15:14:13.484495+0000","last_peered":"2024-05-21T15:14:13.484495+0000","last_clean":"2024-05-21T15:14:13.484495+0000","last_became_active":"2024-05-21T15:14:12.251210+0000","last_became_peered":"2024-05-21T15:14:12.251210+0000","last_unstale":"2024-05-21T15:14:13.484495+0000","last_undegraded":"2024-05-21T15:14:13.484495+0000","last_fullsized":"2024-05-21T15:14:13.484495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:43:34.109179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.259805+0000","last_change":"2024-05-21T15:14:12.251767+0000","last_active":"2024-05-21T15:14:12.259805+0000","last_peered":"2024-05-21T15:14:12.259805+0000","last_clean":"2024-05-21T15:14:12.259805+0000","last_became_active":"2024-05-21T15:14:12.251427+0000","last_became_peered":"2024-05-21T15:14:12.251427+0000","last_unstale":"2024-05-21T15:14:12.259805+0000","last_undegraded":"2024-05-21T15:14:12.259805+0000","last_fullsized":"2024-05-21T15:14:12.259805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:13:48.469030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.255178+0000","last_change":"2024-05-21T15:14:12.253472+0000","last_active":"2024-05-21T15:14:12.255178+0000","last_peered":"2024-05-21T15:14:12.255178+0000","last_clean":"2024-05-21T15:14:12.255178+0000","last_became_active":"2024-05-21T15:14:12.252762+0000","last_became_peered":"2024-05-21T15:14:12.252762+0000","last_unstale":"2024-05-21T15:14:12.255178+0000","last_undegraded":"2024-05-21T15:14:12.255178+0000","last_fullsized":"2024-05-21T15:14:12.255178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:25.384402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.559896+0000","last_change":"2024-05-21T15:14:12.258627+0000","last_active":"2024-05-21T15:14:13.559896+0000","last_peered":"2024-05-21T15:14:13.559896+0000","last_clean":"2024-05-21T15:14:13.559896+0000","last_became_active":"2024-05-21T15:14:12.258118+0000","last_became_peered":"2024-05-21T15:14:12.258118+0000","last_unstale":"2024-05-21T15:14:13.559896+0000","last_undegraded":"2024-05-21T15:14:13.559896+0000","last_fullsized":"2024-05-21T15:14:13.559896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:56.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237924+0000","last_change":"2024-05-21T15:14:12.255998+0000","last_active":"2024-05-21T15:14:13.237924+0000","last_peered":"2024-05-21T15:14:13.237924+0000","last_clean":"2024-05-21T15:14:13.237924+0000","last_became_active":"2024-05-21T15:14:12.255658+0000","last_became_peered":"2024-05-21T15:14:12.255658+0000","last_unstale":"2024-05-21T15:14:13.237924+0000","last_undegraded":"2024-05-21T15:14:13.237924+0000","last_fullsized":"2024-05-21T15:14:13.237924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:44:42.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.260941+0000","last_change":"2024-05-21T15:14:12.257617+0000","last_active":"2024-05-21T15:14:12.260941+0000","last_peered":"2024-05-21T15:14:12.260941+0000","last_clean":"2024-05-21T15:14:12.260941+0000","last_became_active":"2024-05-21T15:14:12.256947+0000","last_became_peered":"2024-05-21T15:14:12.256947+0000","last_unstale":"2024-05-21T15:14:12.260941+0000","last_undegraded":"2024-05-21T15:14:12.260941+0000","last_fullsized":"2024-05-21T15:14:12.260941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:44:30.658968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.260879+0000","last_change":"2024-05-21T15:14:12.260454+0000","last_active":"2024-05-21T15:14:12.260879+0000","last_peered":"2024-05-21T15:14:12.260879+0000","last_clean":"2024-05-21T15:14:12.260879+0000","last_became_active":"2024-05-21T15:14:12.260126+0000","last_became_peered":"2024-05-21T15:14:12.260126+0000","last_unstale":"2024-05-21T15:14:12.260879+0000","last_undegraded":"2024-05-21T15:14:12.260879+0000","last_fullsized":"2024-05-21T15:14:12.260879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:04:55.634333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375092+0000","last_change":"2024-05-21T15:14:12.252071+0000","last_active":"2024-05-21T15:14:14.375092+0000","last_peered":"2024-05-21T15:14:14.375092+0000","last_clean":"2024-05-21T15:14:14.375092+0000","last_became_active":"2024-05-21T15:14:12.251706+0000","last_became_peered":"2024-05-21T15:14:12.251706+0000","last_unstale":"2024-05-21T15:14:14.375092+0000","last_undegraded":"2024-05-21T15:14:14.375092+0000","last_fullsized":"2024-05-21T15:14:14.375092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:02:44.349368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236754+0000","last_change":"2024-05-21T15:14:12.239877+0000","last_active":"2024-05-21T15:14:13.236754+0000","last_peered":"2024-05-21T15:14:13.236754+0000","last_clean":"2024-05-21T15:14:13.236754+0000","last_became_active":"2024-05-21T15:14:12.239615+0000","last_became_peered":"2024-05-21T15:14:12.239615+0000","last_unstale":"2024-05-21T15:14:13.236754+0000","last_undegraded":"2024-05-21T15:14:13.236754+0000","last_fullsized":"2024-05-21T15:14:13.236754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:23.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375262+0000","last_change":"2024-05-21T15:14:12.260349+0000","last_active":"2024-05-21T15:14:14.375262+0000","last_peered":"2024-05-21T15:14:14.375262+0000","last_clean":"2024-05-21T15:14:14.375262+0000","last_became_active":"2024-05-21T15:14:12.259658+0000","last_became_peered":"2024-05-21T15:14:12.259658+0000","last_unstale":"2024-05-21T15:14:14.375262+0000","last_undegraded":"2024-05-21T15:14:14.375262+0000","last_fullsized":"2024-05-21T15:14:14.375262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:59:32.326571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375318+0000","last_change":"2024-05-21T15:14:12.259123+0000","last_active":"2024-05-21T15:14:14.375318+0000","last_peered":"2024-05-21T15:14:14.375318+0000","last_clean":"2024-05-21T15:14:14.375318+0000","last_became_active":"2024-05-21T15:14:12.258791+0000","last_became_peered":"2024-05-21T15:14:12.258791+0000","last_unstale":"2024-05-21T15:14:14.375318+0000","last_undegraded":"2024-05-21T15:14:14.375318+0000","last_fullsized":"2024-05-21T15:14:14.375318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:32:01.046201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237257+0000","last_change":"2024-05-21T15:14:12.264667+0000","last_active":"2024-05-21T15:14:13.237257+0000","last_peered":"2024-05-21T15:14:13.237257+0000","last_clean":"2024-05-21T15:14:13.237257+0000","last_became_active":"2024-05-21T15:14:12.264284+0000","last_became_peered":"2024-05-21T15:14:12.264284+0000","last_unstale":"2024-05-21T15:14:13.237257+0000","last_undegraded":"2024-05-21T15:14:13.237257+0000","last_fullsized":"2024-05-21T15:14:13.237257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:24.861321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.260305+0000","last_change":"2024-05-21T15:14:12.259912+0000","last_active":"2024-05-21T15:14:12.260305+0000","last_peered":"2024-05-21T15:14:12.260305+0000","last_clean":"2024-05-21T15:14:12.260305+0000","last_became_active":"2024-05-21T15:14:12.259578+0000","last_became_peered":"2024-05-21T15:14:12.259578+0000","last_unstale":"2024-05-21T15:14:12.260305+0000","last_undegraded":"2024-05-21T15:14:12.260305+0000","last_fullsized":"2024-05-21T15:14:12.260305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:30.469150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.558317+0000","last_change":"2024-05-21T15:14:12.258355+0000","last_active":"2024-05-21T15:14:13.558317+0000","last_peered":"2024-05-21T15:14:13.558317+0000","last_clean":"2024-05-21T15:14:13.558317+0000","last_became_active":"2024-05-21T15:14:12.257991+0000","last_became_peered":"2024-05-21T15:14:12.257991+0000","last_unstale":"2024-05-21T15:14:13.558317+0000","last_undegraded":"2024-05-21T15:14:13.558317+0000","last_fullsized":"2024-05-21T15:14:13.558317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:54.014553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238585+0000","last_change":"2024-05-21T15:14:12.258873+0000","last_active":"2024-05-21T15:14:13.238585+0000","last_peered":"2024-05-21T15:14:13.238585+0000","last_clean":"2024-05-21T15:14:13.238585+0000","last_became_active":"2024-05-21T15:14:12.258548+0000","last_became_peered":"2024-05-21T15:14:12.258548+0000","last_unstale":"2024-05-21T15:14:13.238585+0000","last_undegraded":"2024-05-21T15:14:13.238585+0000","last_fullsized":"2024-05-21T15:14:13.238585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:00.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.558289+0000","last_change":"2024-05-21T15:14:11.568366+0000","last_active":"2024-05-21T15:14:13.558289+0000","last_peered":"2024-05-21T15:14:13.558289+0000","last_clean":"2024-05-21T15:14:13.558289+0000","last_became_active":"2024-05-21T15:14:11.567885+0000","last_became_peered":"2024-05-21T15:14:11.567885+0000","last_unstale":"2024-05-21T15:14:13.558289+0000","last_undegraded":"2024-05-21T15:14:13.558289+0000","last_fullsized":"2024-05-21T15:14:13.558289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:36:56.553488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560363+0000","last_change":"2024-05-21T15:14:12.261632+0000","last_active":"2024-05-21T15:14:13.560363+0000","last_peered":"2024-05-21T15:14:13.560363+0000","last_clean":"2024-05-21T15:14:13.560363+0000","last_became_active":"2024-05-21T15:14:12.261149+0000","last_became_peered":"2024-05-21T15:14:12.261149+0000","last_unstale":"2024-05-21T15:14:13.560363+0000","last_undegraded":"2024-05-21T15:14:13.560363+0000","last_fullsized":"2024-05-21T15:14:13.560363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:11.349395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560289+0000","last_change":"2024-05-21T15:14:11.567859+0000","last_active":"2024-05-21T15:14:13.560289+0000","last_peered":"2024-05-21T15:14:13.560289+0000","last_clean":"2024-05-21T15:14:13.560289+0000","last_became_active":"2024-05-21T15:14:11.566855+0000","last_became_peered":"2024-05-21T15:14:11.566855+0000","last_unstale":"2024-05-21T15:14:13.560289+0000","last_undegraded":"2024-05-21T15:14:13.560289+0000","last_fullsized":"2024-05-21T15:14:13.560289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:21.568436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375609+0000","last_change":"2024-05-21T15:14:12.262153+0000","last_active":"2024-05-21T15:14:14.375609+0000","last_peered":"2024-05-21T15:14:14.375609+0000","last_clean":"2024-05-21T15:14:14.375609+0000","last_became_active":"2024-05-21T15:14:12.261780+0000","last_became_peered":"2024-05-21T15:14:12.261780+0000","last_unstale":"2024-05-21T15:14:14.375609+0000","last_undegraded":"2024-05-21T15:14:14.375609+0000","last_fullsized":"2024-05-21T15:14:14.375609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:21:25.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.277699+0000","last_change":"2024-05-21T15:14:11.570316+0000","last_active":"2024-05-21T15:14:16.277699+0000","last_peered":"2024-05-21T15:14:16.277699+0000","last_clean":"2024-05-21T15:14:16.277699+0000","last_became_active":"2024-05-21T15:14:11.569967+0000","last_became_peered":"2024-05-21T15:14:11.569967+0000","last_unstale":"2024-05-21T15:14:16.277699+0000","last_undegraded":"2024-05-21T15:14:16.277699+0000","last_fullsized":"2024-05-21T15:14:16.277699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:30:27.364742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237627+0000","last_change":"2024-05-21T15:14:12.261716+0000","last_active":"2024-05-21T15:14:13.237627+0000","last_peered":"2024-05-21T15:14:13.237627+0000","last_clean":"2024-05-21T15:14:13.237627+0000","last_became_active":"2024-05-21T15:14:12.261341+0000","last_became_peered":"2024-05-21T15:14:12.261341+0000","last_unstale":"2024-05-21T15:14:13.237627+0000","last_undegraded":"2024-05-21T15:14:13.237627+0000","last_fullsized":"2024-05-21T15:14:13.237627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:40:10.464974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484649+0000","last_change":"2024-05-21T15:14:11.573305+0000","last_active":"2024-05-21T15:14:13.484649+0000","last_peered":"2024-05-21T15:14:13.484649+0000","last_clean":"2024-05-21T15:14:13.484649+0000","last_became_active":"2024-05-21T15:14:11.572655+0000","last_became_peered":"2024-05-21T15:14:11.572655+0000","last_unstale":"2024-05-21T15:14:13.484649+0000","last_undegraded":"2024-05-21T15:14:13.484649+0000","last_fullsized":"2024-05-21T15:14:13.484649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:37.268440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.255915+0000","last_change":"2024-05-21T15:14:12.253788+0000","last_active":"2024-05-21T15:14:12.255915+0000","last_peered":"2024-05-21T15:14:12.255915+0000","last_clean":"2024-05-21T15:14:12.255915+0000","last_became_active":"2024-05-21T15:14:12.253450+0000","last_became_peered":"2024-05-21T15:14:12.253450+0000","last_unstale":"2024-05-21T15:14:12.255915+0000","last_undegraded":"2024-05-21T15:14:12.255915+0000","last_fullsized":"2024-05-21T15:14:12.255915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:29.115733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.560850+0000","last_change":"2024-05-21T15:14:11.567400+0000","last_active":"2024-05-21T15:14:12.560850+0000","last_peered":"2024-05-21T15:14:12.560850+0000","last_clean":"2024-05-21T15:14:12.560850+0000","last_became_active":"2024-05-21T15:14:11.565995+0000","last_became_peered":"2024-05-21T15:14:11.565995+0000","last_unstale":"2024-05-21T15:14:12.560850+0000","last_undegraded":"2024-05-21T15:14:12.560850+0000","last_fullsized":"2024-05-21T15:14:12.560850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:50.463339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484379+0000","last_change":"2024-05-21T15:14:12.251565+0000","last_active":"2024-05-21T15:14:13.484379+0000","last_peered":"2024-05-21T15:14:13.484379+0000","last_clean":"2024-05-21T15:14:13.484379+0000","last_became_active":"2024-05-21T15:14:12.251289+0000","last_became_peered":"2024-05-21T15:14:12.251289+0000","last_unstale":"2024-05-21T15:14:13.484379+0000","last_undegraded":"2024-05-21T15:14:13.484379+0000","last_fullsized":"2024-05-21T15:14:13.484379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:44.619800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.562875+0000","last_change":"2024-05-21T15:14:11.575193+0000","last_active":"2024-05-21T15:14:12.562875+0000","last_peered":"2024-05-21T15:14:12.562875+0000","last_clean":"2024-05-21T15:14:12.562875+0000","last_became_active":"2024-05-21T15:14:11.574861+0000","last_became_peered":"2024-05-21T15:14:11.574861+0000","last_unstale":"2024-05-21T15:14:12.562875+0000","last_undegraded":"2024-05-21T15:14:12.562875+0000","last_fullsized":"2024-05-21T15:14:12.562875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:54.548408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.256564+0000","last_change":"2024-05-21T15:14:12.254680+0000","last_active":"2024-05-21T15:14:12.256564+0000","last_peered":"2024-05-21T15:14:12.256564+0000","last_clean":"2024-05-21T15:14:12.256564+0000","last_became_active":"2024-05-21T15:14:12.254283+0000","last_became_peered":"2024-05-21T15:14:12.254283+0000","last_unstale":"2024-05-21T15:14:12.256564+0000","last_undegraded":"2024-05-21T15:14:12.256564+0000","last_fullsized":"2024-05-21T15:14:12.256564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:30:38.953642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236377+0000","last_change":"2024-05-21T15:14:11.572087+0000","last_active":"2024-05-21T15:14:13.236377+0000","last_peered":"2024-05-21T15:14:13.236377+0000","last_clean":"2024-05-21T15:14:13.236377+0000","last_became_active":"2024-05-21T15:14:11.571716+0000","last_became_peered":"2024-05-21T15:14:11.571716+0000","last_unstale":"2024-05-21T15:14:13.236377+0000","last_undegraded":"2024-05-21T15:14:13.236377+0000","last_fullsized":"2024-05-21T15:14:13.236377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:07.082766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.559982+0000","last_change":"2024-05-21T15:14:12.252233+0000","last_active":"2024-05-21T15:14:13.559982+0000","last_peered":"2024-05-21T15:14:13.559982+0000","last_clean":"2024-05-21T15:14:13.559982+0000","last_became_active":"2024-05-21T15:14:12.251901+0000","last_became_peered":"2024-05-21T15:14:12.251901+0000","last_unstale":"2024-05-21T15:14:13.559982+0000","last_undegraded":"2024-05-21T15:14:13.559982+0000","last_fullsized":"2024-05-21T15:14:13.559982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:05.107436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.236351+0000","last_change":"2024-05-21T15:14:11.565946+0000","last_active":"2024-05-21T15:14:12.236351+0000","last_peered":"2024-05-21T15:14:12.236351+0000","last_clean":"2024-05-21T15:14:12.236351+0000","last_became_active":"2024-05-21T15:14:11.565601+0000","last_became_peered":"2024-05-21T15:14:11.565601+0000","last_unstale":"2024-05-21T15:14:12.236351+0000","last_undegraded":"2024-05-21T15:14:12.236351+0000","last_fullsized":"2024-05-21T15:14:12.236351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:41:55.243473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.558223+0000","last_change":"2024-05-21T15:14:12.255188+0000","last_active":"2024-05-21T15:14:13.558223+0000","last_peered":"2024-05-21T15:14:13.558223+0000","last_clean":"2024-05-21T15:14:13.558223+0000","last_became_active":"2024-05-21T15:14:12.254503+0000","last_became_peered":"2024-05-21T15:14:12.254503+0000","last_unstale":"2024-05-21T15:14:13.558223+0000","last_undegraded":"2024-05-21T15:14:13.558223+0000","last_fullsized":"2024-05-21T15:14:13.558223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:22:16.336740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.236759+0000","last_change":"2024-05-21T15:14:11.569882+0000","last_active":"2024-05-21T15:14:12.236759+0000","last_peered":"2024-05-21T15:14:12.236759+0000","last_clean":"2024-05-21T15:14:12.236759+0000","last_became_active":"2024-05-21T15:14:11.569532+0000","last_became_peered":"2024-05-21T15:14:11.569532+0000","last_unstale":"2024-05-21T15:14:12.236759+0000","last_undegraded":"2024-05-21T15:14:12.236759+0000","last_fullsized":"2024-05-21T15:14:12.236759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:58:50.081709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237788+0000","last_change":"2024-05-21T15:14:12.262691+0000","last_active":"2024-05-21T15:14:13.237788+0000","last_peered":"2024-05-21T15:14:13.237788+0000","last_clean":"2024-05-21T15:14:13.237788+0000","last_became_active":"2024-05-21T15:14:12.261492+0000","last_became_peered":"2024-05-21T15:14:12.261492+0000","last_unstale":"2024-05-21T15:14:13.237788+0000","last_undegraded":"2024-05-21T15:14:13.237788+0000","last_fullsized":"2024-05-21T15:14:13.237788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:35:00.121256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.243510+0000","last_change":"2024-05-21T15:14:11.569231+0000","last_active":"2024-05-21T15:14:12.243510+0000","last_peered":"2024-05-21T15:14:12.243510+0000","last_clean":"2024-05-21T15:14:12.243510+0000","last_became_active":"2024-05-21T15:14:11.568916+0000","last_became_peered":"2024-05-21T15:14:11.568916+0000","last_unstale":"2024-05-21T15:14:12.243510+0000","last_undegraded":"2024-05-21T15:14:12.243510+0000","last_fullsized":"2024-05-21T15:14:12.243510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:26:13.411854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375549+0000","last_change":"2024-05-21T15:14:12.256307+0000","last_active":"2024-05-21T15:14:14.375549+0000","last_peered":"2024-05-21T15:14:14.375549+0000","last_clean":"2024-05-21T15:14:14.375549+0000","last_became_active":"2024-05-21T15:14:12.255828+0000","last_became_peered":"2024-05-21T15:14:12.255828+0000","last_unstale":"2024-05-21T15:14:14.375549+0000","last_undegraded":"2024-05-21T15:14:14.375549+0000","last_fullsized":"2024-05-21T15:14:14.375549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:11:40.068779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.564053+0000","last_change":"2024-05-21T15:14:11.568239+0000","last_active":"2024-05-21T15:14:12.564053+0000","last_peered":"2024-05-21T15:14:12.564053+0000","last_clean":"2024-05-21T15:14:12.564053+0000","last_became_active":"2024-05-21T15:14:11.567836+0000","last_became_peered":"2024-05-21T15:14:11.567836+0000","last_unstale":"2024-05-21T15:14:12.564053+0000","last_undegraded":"2024-05-21T15:14:12.564053+0000","last_fullsized":"2024-05-21T15:14:12.564053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:10.244172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237859+0000","last_change":"2024-05-21T15:14:12.257762+0000","last_active":"2024-05-21T15:14:13.237859+0000","last_peered":"2024-05-21T15:14:13.237859+0000","last_clean":"2024-05-21T15:14:13.237859+0000","last_became_active":"2024-05-21T15:14:12.256335+0000","last_became_peered":"2024-05-21T15:14:12.256335+0000","last_unstale":"2024-05-21T15:14:13.237859+0000","last_undegraded":"2024-05-21T15:14:13.237859+0000","last_fullsized":"2024-05-21T15:14:13.237859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:35.219673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'6","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.566053+0000","last_change":"2024-05-21T15:14:11.564407+0000","last_active":"2024-05-21T15:14:12.566053+0000","last_peered":"2024-05-21T15:14:12.566053+0000","last_clean":"2024-05-21T15:14:12.566053+0000","last_became_active":"2024-05-21T15:14:11.564012+0000","last_became_peered":"2024-05-21T15:14:11.564012+0000","last_unstale":"2024-05-21T15:14:12.566053+0000","last_undegraded":"2024-05-21T15:14:12.566053+0000","last_fullsized":"2024-05-21T15:14:12.566053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:34:54.177844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484847+0000","last_change":"2024-05-21T15:14:12.253820+0000","last_active":"2024-05-21T15:14:13.484847+0000","last_peered":"2024-05-21T15:14:13.484847+0000","last_clean":"2024-05-21T15:14:13.484847+0000","last_became_active":"2024-05-21T15:14:12.253360+0000","last_became_peered":"2024-05-21T15:14:12.253360+0000","last_unstale":"2024-05-21T15:14:13.484847+0000","last_undegraded":"2024-05-21T15:14:13.484847+0000","last_fullsized":"2024-05-21T15:14:13.484847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:38:19.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.573038+0000","last_change":"2024-05-21T15:14:11.571161+0000","last_active":"2024-05-21T15:14:14.573038+0000","last_peered":"2024-05-21T15:14:14.573038+0000","last_clean":"2024-05-21T15:14:14.573038+0000","last_became_active":"2024-05-21T15:14:11.570837+0000","last_became_peered":"2024-05-21T15:14:11.570837+0000","last_unstale":"2024-05-21T15:14:14.573038+0000","last_undegraded":"2024-05-21T15:14:14.573038+0000","last_fullsized":"2024-05-21T15:14:14.573038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:24:55.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236758+0000","last_change":"2024-05-21T15:14:12.255713+0000","last_active":"2024-05-21T15:14:13.236758+0000","last_peered":"2024-05-21T15:14:13.236758+0000","last_clean":"2024-05-21T15:14:13.236758+0000","last_became_active":"2024-05-21T15:14:12.254458+0000","last_became_peered":"2024-05-21T15:14:12.254458+0000","last_unstale":"2024-05-21T15:14:13.236758+0000","last_undegraded":"2024-05-21T15:14:13.236758+0000","last_fullsized":"2024-05-21T15:14:13.236758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:55:24.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.236195+0000","last_change":"2024-05-21T15:14:11.565916+0000","last_active":"2024-05-21T15:14:12.236195+0000","last_peered":"2024-05-21T15:14:12.236195+0000","last_clean":"2024-05-21T15:14:12.236195+0000","last_became_active":"2024-05-21T15:14:11.565460+0000","last_became_peered":"2024-05-21T15:14:11.565460+0000","last_unstale":"2024-05-21T15:14:12.236195+0000","last_undegraded":"2024-05-21T15:14:12.236195+0000","last_fullsized":"2024-05-21T15:14:12.236195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:09.982208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.253205+0000","last_change":"2024-05-21T15:14:12.251856+0000","last_active":"2024-05-21T15:14:12.253205+0000","last_peered":"2024-05-21T15:14:12.253205+0000","last_clean":"2024-05-21T15:14:12.253205+0000","last_became_active":"2024-05-21T15:14:12.251496+0000","last_became_peered":"2024-05-21T15:14:12.251496+0000","last_unstale":"2024-05-21T15:14:12.253205+0000","last_undegraded":"2024-05-21T15:14:12.253205+0000","last_fullsized":"2024-05-21T15:14:12.253205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:42:19.902467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.374725+0000","last_change":"2024-05-21T15:14:11.565485+0000","last_active":"2024-05-21T15:14:14.374725+0000","last_peered":"2024-05-21T15:14:14.374725+0000","last_clean":"2024-05-21T15:14:14.374725+0000","last_became_active":"2024-05-21T15:14:11.565092+0000","last_became_peered":"2024-05-21T15:14:11.565092+0000","last_unstale":"2024-05-21T15:14:14.374725+0000","last_undegraded":"2024-05-21T15:14:14.374725+0000","last_fullsized":"2024-05-21T15:14:14.374725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:01:16.101793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.263302+0000","last_change":"2024-05-21T15:14:12.262283+0000","last_active":"2024-05-21T15:14:12.263302+0000","last_peered":"2024-05-21T15:14:12.263302+0000","last_clean":"2024-05-21T15:14:12.263302+0000","last_became_active":"2024-05-21T15:14:12.261610+0000","last_became_peered":"2024-05-21T15:14:12.261610+0000","last_unstale":"2024-05-21T15:14:12.263302+0000","last_undegraded":"2024-05-21T15:14:12.263302+0000","last_fullsized":"2024-05-21T15:14:12.263302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:20:53.224276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.262629+0000","last_change":"2024-05-21T15:14:11.569905+0000","last_active":"2024-05-21T15:14:16.262629+0000","last_peered":"2024-05-21T15:14:16.262629+0000","last_clean":"2024-05-21T15:14:16.262629+0000","last_became_active":"2024-05-21T15:14:11.569221+0000","last_became_peered":"2024-05-21T15:14:11.569221+0000","last_unstale":"2024-05-21T15:14:16.262629+0000","last_undegraded":"2024-05-21T15:14:16.262629+0000","last_fullsized":"2024-05-21T15:14:16.262629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:11:04.875890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.256591+0000","last_change":"2024-05-21T15:14:12.255149+0000","last_active":"2024-05-21T15:14:12.256591+0000","last_peered":"2024-05-21T15:14:12.256591+0000","last_clean":"2024-05-21T15:14:12.256591+0000","last_became_active":"2024-05-21T15:14:12.254366+0000","last_became_peered":"2024-05-21T15:14:12.254366+0000","last_unstale":"2024-05-21T15:14:12.256591+0000","last_undegraded":"2024-05-21T15:14:12.256591+0000","last_fullsized":"2024-05-21T15:14:12.256591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:34:30.203231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.264077+0000","last_change":"2024-05-21T15:14:11.569670+0000","last_active":"2024-05-21T15:14:16.264077+0000","last_peered":"2024-05-21T15:14:16.264077+0000","last_clean":"2024-05-21T15:14:16.264077+0000","last_became_active":"2024-05-21T15:14:11.569257+0000","last_became_peered":"2024-05-21T15:14:11.569257+0000","last_unstale":"2024-05-21T15:14:16.264077+0000","last_undegraded":"2024-05-21T15:14:16.264077+0000","last_fullsized":"2024-05-21T15:14:16.264077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:18.584920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.259436+0000","last_change":"2024-05-21T15:14:12.258296+0000","last_active":"2024-05-21T15:14:12.259436+0000","last_peered":"2024-05-21T15:14:12.259436+0000","last_clean":"2024-05-21T15:14:12.259436+0000","last_became_active":"2024-05-21T15:14:12.257919+0000","last_became_peered":"2024-05-21T15:14:12.257919+0000","last_unstale":"2024-05-21T15:14:12.259436+0000","last_undegraded":"2024-05-21T15:14:12.259436+0000","last_fullsized":"2024-05-21T15:14:12.259436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:12:38.852843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.558227+0000","last_change":"2024-05-21T15:14:11.567849+0000","last_active":"2024-05-21T15:14:13.558227+0000","last_peered":"2024-05-21T15:14:13.558227+0000","last_clean":"2024-05-21T15:14:13.558227+0000","last_became_active":"2024-05-21T15:14:11.567476+0000","last_became_peered":"2024-05-21T15:14:11.567476+0000","last_unstale":"2024-05-21T15:14:13.558227+0000","last_undegraded":"2024-05-21T15:14:13.558227+0000","last_fullsized":"2024-05-21T15:14:13.558227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:13.516765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.260181+0000","last_change":"2024-05-21T15:14:12.255165+0000","last_active":"2024-05-21T15:14:12.260181+0000","last_peered":"2024-05-21T15:14:12.260181+0000","last_clean":"2024-05-21T15:14:12.260181+0000","last_became_active":"2024-05-21T15:14:12.254743+0000","last_became_peered":"2024-05-21T15:14:12.254743+0000","last_unstale":"2024-05-21T15:14:12.260181+0000","last_undegraded":"2024-05-21T15:14:12.260181+0000","last_fullsized":"2024-05-21T15:14:12.260181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:21:13.298574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.243298+0000","last_change":"2024-05-21T15:14:11.569745+0000","last_active":"2024-05-21T15:14:12.243298+0000","last_peered":"2024-05-21T15:14:12.243298+0000","last_clean":"2024-05-21T15:14:12.243298+0000","last_became_active":"2024-05-21T15:14:11.569409+0000","last_became_peered":"2024-05-21T15:14:11.569409+0000","last_unstale":"2024-05-21T15:14:12.243298+0000","last_undegraded":"2024-05-21T15:14:12.243298+0000","last_fullsized":"2024-05-21T15:14:12.243298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:00:37.963797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.259847+0000","last_change":"2024-05-21T15:14:12.254631+0000","last_active":"2024-05-21T15:14:12.259847+0000","last_peered":"2024-05-21T15:14:12.259847+0000","last_clean":"2024-05-21T15:14:12.259847+0000","last_became_active":"2024-05-21T15:14:12.254239+0000","last_became_peered":"2024-05-21T15:14:12.254239+0000","last_unstale":"2024-05-21T15:14:12.259847+0000","last_undegraded":"2024-05-21T15:14:12.259847+0000","last_fullsized":"2024-05-21T15:14:12.259847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:35:44.232627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.259064+0000","last_change":"2024-05-21T15:14:12.257568+0000","last_active":"2024-05-21T15:14:12.259064+0000","last_peered":"2024-05-21T15:14:12.259064+0000","last_clean":"2024-05-21T15:14:12.259064+0000","last_became_active":"2024-05-21T15:14:12.256734+0000","last_became_peered":"2024-05-21T15:14:12.256734+0000","last_unstale":"2024-05-21T15:14:12.259064+0000","last_undegraded":"2024-05-21T15:14:12.259064+0000","last_fullsized":"2024-05-21T15:14:12.259064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:44:35.371083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236279+0000","last_change":"2024-05-21T15:14:11.574352+0000","last_active":"2024-05-21T15:14:13.236279+0000","last_peered":"2024-05-21T15:14:13.236279+0000","last_clean":"2024-05-21T15:14:13.236279+0000","last_became_active":"2024-05-21T15:14:11.574041+0000","last_became_peered":"2024-05-21T15:14:11.574041+0000","last_unstale":"2024-05-21T15:14:13.236279+0000","last_undegraded":"2024-05-21T15:14:13.236279+0000","last_fullsized":"2024-05-21T15:14:13.236279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.563218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560464+0000","last_change":"2024-05-21T15:14:12.257886+0000","last_active":"2024-05-21T15:14:13.560464+0000","last_peered":"2024-05-21T15:14:13.560464+0000","last_clean":"2024-05-21T15:14:13.560464+0000","last_became_active":"2024-05-21T15:14:12.257546+0000","last_became_peered":"2024-05-21T15:14:12.257546+0000","last_unstale":"2024-05-21T15:14:13.560464+0000","last_undegraded":"2024-05-21T15:14:13.560464+0000","last_fullsized":"2024-05-21T15:14:13.560464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:20.953421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.573235+0000","last_change":"2024-05-21T15:14:11.568524+0000","last_active":"2024-05-21T15:14:14.573235+0000","last_peered":"2024-05-21T15:14:14.573235+0000","last_clean":"2024-05-21T15:14:14.573235+0000","last_became_active":"2024-05-21T15:14:11.567981+0000","last_became_peered":"2024-05-21T15:14:11.567981+0000","last_unstale":"2024-05-21T15:14:14.573235+0000","last_undegraded":"2024-05-21T15:14:14.573235+0000","last_fullsized":"2024-05-21T15:14:14.573235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.242168+0000","last_change":"2024-05-21T15:13:02.249260+0000","last_active":"2024-05-21T15:14:12.242168+0000","last_peered":"2024-05-21T15:14:12.242168+0000","last_clean":"2024-05-21T15:14:12.242168+0000","last_became_active":"2024-05-21T15:13:02.248637+0000","last_became_peered":"2024-05-21T15:13:02.248637+0000","last_unstale":"2024-05-21T15:14:12.242168+0000","last_undegraded":"2024-05-21T15:14:12.242168+0000","last_fullsized":"2024-05-21T15:14:12.242168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_clean_scrub_stamp":"2024-05-21T15:09:41.506463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:26.250910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236416+0000","last_change":"2024-05-21T15:14:12.259861+0000","last_active":"2024-05-21T15:14:13.236416+0000","last_peered":"2024-05-21T15:14:13.236416+0000","last_clean":"2024-05-21T15:14:13.236416+0000","last_became_active":"2024-05-21T15:14:12.259510+0000","last_became_peered":"2024-05-21T15:14:12.259510+0000","last_unstale":"2024-05-21T15:14:13.236416+0000","last_undegraded":"2024-05-21T15:14:13.236416+0000","last_fullsized":"2024-05-21T15:14:13.236416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:49.548366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.242445+0000","last_change":"2024-05-21T15:14:11.570705+0000","last_active":"2024-05-21T15:14:12.242445+0000","last_peered":"2024-05-21T15:14:12.242445+0000","last_clean":"2024-05-21T15:14:12.242445+0000","last_became_active":"2024-05-21T15:14:11.570403+0000","last_became_peered":"2024-05-21T15:14:11.570403+0000","last_unstale":"2024-05-21T15:14:12.242445+0000","last_undegraded":"2024-05-21T15:14:12.242445+0000","last_fullsized":"2024-05-21T15:14:12.242445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:02:34.156258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.277803+0000","last_change":"2024-05-21T15:14:11.572311+0000","last_active":"2024-05-21T15:14:16.277803+0000","last_peered":"2024-05-21T15:14:16.277803+0000","last_clean":"2024-05-21T15:14:16.277803+0000","last_became_active":"2024-05-21T15:14:11.571949+0000","last_became_peered":"2024-05-21T15:14:11.571949+0000","last_unstale":"2024-05-21T15:14:16.277803+0000","last_undegraded":"2024-05-21T15:14:16.277803+0000","last_fullsized":"2024-05-21T15:14:16.277803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:57:59.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236837+0000","last_change":"2024-05-21T15:14:12.260717+0000","last_active":"2024-05-21T15:14:13.236837+0000","last_peered":"2024-05-21T15:14:13.236837+0000","last_clean":"2024-05-21T15:14:13.236837+0000","last_became_active":"2024-05-21T15:14:12.259911+0000","last_became_peered":"2024-05-21T15:14:12.259911+0000","last_unstale":"2024-05-21T15:14:13.236837+0000","last_undegraded":"2024-05-21T15:14:13.236837+0000","last_fullsized":"2024-05-21T15:14:13.236837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:02.609299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.559573+0000","last_change":"2024-05-21T15:14:11.568061+0000","last_active":"2024-05-21T15:14:13.559573+0000","last_peered":"2024-05-21T15:14:13.559573+0000","last_clean":"2024-05-21T15:14:13.559573+0000","last_became_active":"2024-05-21T15:14:11.567678+0000","last_became_peered":"2024-05-21T15:14:11.567678+0000","last_unstale":"2024-05-21T15:14:13.559573+0000","last_undegraded":"2024-05-21T15:14:13.559573+0000","last_fullsized":"2024-05-21T15:14:13.559573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:07:15.902983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560355+0000","last_change":"2024-05-21T15:14:12.253634+0000","last_active":"2024-05-21T15:14:13.560355+0000","last_peered":"2024-05-21T15:14:13.560355+0000","last_clean":"2024-05-21T15:14:13.560355+0000","last_became_active":"2024-05-21T15:14:12.253284+0000","last_became_peered":"2024-05-21T15:14:12.253284+0000","last_unstale":"2024-05-21T15:14:13.560355+0000","last_undegraded":"2024-05-21T15:14:13.560355+0000","last_fullsized":"2024-05-21T15:14:13.560355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:28:07.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.572074+0000","last_change":"2024-05-21T15:14:11.574147+0000","last_active":"2024-05-21T15:14:14.572074+0000","last_peered":"2024-05-21T15:14:14.572074+0000","last_clean":"2024-05-21T15:14:14.572074+0000","last_became_active":"2024-05-21T15:14:11.573721+0000","last_became_peered":"2024-05-21T15:14:11.573721+0000","last_unstale":"2024-05-21T15:14:14.572074+0000","last_undegraded":"2024-05-21T15:14:14.572074+0000","last_fullsized":"2024-05-21T15:14:14.572074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:10.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375431+0000","last_change":"2024-05-21T15:14:12.258632+0000","last_active":"2024-05-21T15:14:14.375431+0000","last_peered":"2024-05-21T15:14:14.375431+0000","last_clean":"2024-05-21T15:14:14.375431+0000","last_became_active":"2024-05-21T15:14:12.257908+0000","last_became_peered":"2024-05-21T15:14:12.257908+0000","last_unstale":"2024-05-21T15:14:14.375431+0000","last_undegraded":"2024-05-21T15:14:14.375431+0000","last_fullsized":"2024-05-21T15:14:14.375431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:12.062387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.572437+0000","last_change":"2024-05-21T15:14:11.562592+0000","last_active":"2024-05-21T15:14:14.572437+0000","last_peered":"2024-05-21T15:14:14.572437+0000","last_clean":"2024-05-21T15:14:14.572437+0000","last_became_active":"2024-05-21T15:14:11.562265+0000","last_became_peered":"2024-05-21T15:14:11.562265+0000","last_unstale":"2024-05-21T15:14:14.572437+0000","last_undegraded":"2024-05-21T15:14:14.572437+0000","last_fullsized":"2024-05-21T15:14:14.572437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:24.662576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484336+0000","last_change":"2024-05-21T15:14:12.249207+0000","last_active":"2024-05-21T15:14:13.484336+0000","last_peered":"2024-05-21T15:14:13.484336+0000","last_clean":"2024-05-21T15:14:13.484336+0000","last_became_active":"2024-05-21T15:14:12.248819+0000","last_became_peered":"2024-05-21T15:14:12.248819+0000","last_unstale":"2024-05-21T15:14:13.484336+0000","last_undegraded":"2024-05-21T15:14:13.484336+0000","last_fullsized":"2024-05-21T15:14:13.484336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:38.208636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.573818+0000","last_change":"2024-05-21T15:14:11.572044+0000","last_active":"2024-05-21T15:14:14.573818+0000","last_peered":"2024-05-21T15:14:14.573818+0000","last_clean":"2024-05-21T15:14:14.573818+0000","last_became_active":"2024-05-21T15:14:11.570721+0000","last_became_peered":"2024-05-21T15:14:11.570721+0000","last_unstale":"2024-05-21T15:14:14.573818+0000","last_undegraded":"2024-05-21T15:14:14.573818+0000","last_fullsized":"2024-05-21T15:14:14.573818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:34:08.413781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.257417+0000","last_change":"2024-05-21T15:14:12.256327+0000","last_active":"2024-05-21T15:14:12.257417+0000","last_peered":"2024-05-21T15:14:12.257417+0000","last_clean":"2024-05-21T15:14:12.257417+0000","last_became_active":"2024-05-21T15:14:12.255962+0000","last_became_peered":"2024-05-21T15:14:12.255962+0000","last_unstale":"2024-05-21T15:14:12.257417+0000","last_undegraded":"2024-05-21T15:14:12.257417+0000","last_fullsized":"2024-05-21T15:14:12.257417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:55.708072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238226+0000","last_change":"2024-05-21T15:14:11.571062+0000","last_active":"2024-05-21T15:14:13.238226+0000","last_peered":"2024-05-21T15:14:13.238226+0000","last_clean":"2024-05-21T15:14:13.238226+0000","last_became_active":"2024-05-21T15:14:11.570739+0000","last_became_peered":"2024-05-21T15:14:11.570739+0000","last_unstale":"2024-05-21T15:14:13.238226+0000","last_undegraded":"2024-05-21T15:14:13.238226+0000","last_fullsized":"2024-05-21T15:14:13.238226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:00:45.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560264+0000","last_change":"2024-05-21T15:14:12.256993+0000","last_active":"2024-05-21T15:14:13.560264+0000","last_peered":"2024-05-21T15:14:13.560264+0000","last_clean":"2024-05-21T15:14:13.560264+0000","last_became_active":"2024-05-21T15:14:12.256179+0000","last_became_peered":"2024-05-21T15:14:12.256179+0000","last_unstale":"2024-05-21T15:14:13.560264+0000","last_undegraded":"2024-05-21T15:14:13.560264+0000","last_fullsized":"2024-05-21T15:14:13.560264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:01:09.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.573217+0000","last_change":"2024-05-21T15:14:11.569688+0000","last_active":"2024-05-21T15:14:14.573217+0000","last_peered":"2024-05-21T15:14:14.573217+0000","last_clean":"2024-05-21T15:14:14.573217+0000","last_became_active":"2024-05-21T15:14:11.569341+0000","last_became_peered":"2024-05-21T15:14:11.569341+0000","last_unstale":"2024-05-21T15:14:14.573217+0000","last_undegraded":"2024-05-21T15:14:14.573217+0000","last_fullsized":"2024-05-21T15:14:14.573217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:10:26.155956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.262327+0000","last_change":"2024-05-21T15:14:12.260506+0000","last_active":"2024-05-21T15:14:12.262327+0000","last_peered":"2024-05-21T15:14:12.262327+0000","last_clean":"2024-05-21T15:14:12.262327+0000","last_became_active":"2024-05-21T15:14:12.260168+0000","last_became_peered":"2024-05-21T15:14:12.260168+0000","last_unstale":"2024-05-21T15:14:12.262327+0000","last_undegraded":"2024-05-21T15:14:12.262327+0000","last_fullsized":"2024-05-21T15:14:12.262327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:26:41.818781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238460+0000","last_change":"2024-05-21T15:14:11.568630+0000","last_active":"2024-05-21T15:14:13.238460+0000","last_peered":"2024-05-21T15:14:13.238460+0000","last_clean":"2024-05-21T15:14:13.238460+0000","last_became_active":"2024-05-21T15:14:11.568229+0000","last_became_peered":"2024-05-21T15:14:11.568229+0000","last_unstale":"2024-05-21T15:14:13.238460+0000","last_undegraded":"2024-05-21T15:14:13.238460+0000","last_fullsized":"2024-05-21T15:14:13.238460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:21.372395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.260574+0000","last_change":"2024-05-21T15:14:12.259408+0000","last_active":"2024-05-21T15:14:12.260574+0000","last_peered":"2024-05-21T15:14:12.260574+0000","last_clean":"2024-05-21T15:14:12.260574+0000","last_became_active":"2024-05-21T15:14:12.259082+0000","last_became_peered":"2024-05-21T15:14:12.259082+0000","last_unstale":"2024-05-21T15:14:12.260574+0000","last_undegraded":"2024-05-21T15:14:12.260574+0000","last_fullsized":"2024-05-21T15:14:12.260574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:21:31.330653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.563749+0000","last_change":"2024-05-21T15:14:11.571077+0000","last_active":"2024-05-21T15:14:12.563749+0000","last_peered":"2024-05-21T15:14:12.563749+0000","last_clean":"2024-05-21T15:14:12.563749+0000","last_became_active":"2024-05-21T15:14:11.570664+0000","last_became_peered":"2024-05-21T15:14:11.570664+0000","last_unstale":"2024-05-21T15:14:12.563749+0000","last_undegraded":"2024-05-21T15:14:12.563749+0000","last_fullsized":"2024-05-21T15:14:12.563749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:37:16.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.253795+0000","last_change":"2024-05-21T15:14:12.252972+0000","last_active":"2024-05-21T15:14:12.253795+0000","last_peered":"2024-05-21T15:14:12.253795+0000","last_clean":"2024-05-21T15:14:12.253795+0000","last_became_active":"2024-05-21T15:14:12.252591+0000","last_became_peered":"2024-05-21T15:14:12.252591+0000","last_unstale":"2024-05-21T15:14:12.253795+0000","last_undegraded":"2024-05-21T15:14:12.253795+0000","last_fullsized":"2024-05-21T15:14:12.253795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:47:15.351701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236510+0000","last_change":"2024-05-21T15:14:11.577138+0000","last_active":"2024-05-21T15:14:13.236510+0000","last_peered":"2024-05-21T15:14:13.236510+0000","last_clean":"2024-05-21T15:14:13.236510+0000","last_became_active":"2024-05-21T15:14:11.576629+0000","last_became_peered":"2024-05-21T15:14:11.576629+0000","last_unstale":"2024-05-21T15:14:13.236510+0000","last_undegraded":"2024-05-21T15:14:13.236510+0000","last_fullsized":"2024-05-21T15:14:13.236510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:20:20.134764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.258511+0000","last_change":"2024-05-21T15:14:12.256174+0000","last_active":"2024-05-21T15:14:12.258511+0000","last_peered":"2024-05-21T15:14:12.258511+0000","last_clean":"2024-05-21T15:14:12.258511+0000","last_became_active":"2024-05-21T15:14:12.255839+0000","last_became_peered":"2024-05-21T15:14:12.255839+0000","last_unstale":"2024-05-21T15:14:12.258511+0000","last_undegraded":"2024-05-21T15:14:12.258511+0000","last_fullsized":"2024-05-21T15:14:12.258511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:55:11.846577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.257627+0000","last_change":"2024-05-21T15:14:12.257028+0000","last_active":"2024-05-21T15:14:12.257627+0000","last_peered":"2024-05-21T15:14:12.257627+0000","last_clean":"2024-05-21T15:14:12.257627+0000","last_became_active":"2024-05-21T15:14:12.256611+0000","last_became_peered":"2024-05-21T15:14:12.256611+0000","last_unstale":"2024-05-21T15:14:12.257627+0000","last_undegraded":"2024-05-21T15:14:12.257627+0000","last_fullsized":"2024-05-21T15:14:12.257627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.833769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238381+0000","last_change":"2024-05-21T15:14:12.255724+0000","last_active":"2024-05-21T15:14:13.238381+0000","last_peered":"2024-05-21T15:14:13.238381+0000","last_clean":"2024-05-21T15:14:13.238381+0000","last_became_active":"2024-05-21T15:14:12.255272+0000","last_became_peered":"2024-05-21T15:14:12.255272+0000","last_unstale":"2024-05-21T15:14:13.238381+0000","last_undegraded":"2024-05-21T15:14:13.238381+0000","last_fullsized":"2024-05-21T15:14:13.238381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:07:07.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238350+0000","last_change":"2024-05-21T15:14:12.262508+0000","last_active":"2024-05-21T15:14:13.238350+0000","last_peered":"2024-05-21T15:14:13.238350+0000","last_clean":"2024-05-21T15:14:13.238350+0000","last_became_active":"2024-05-21T15:14:12.262155+0000","last_became_peered":"2024-05-21T15:14:12.262155+0000","last_unstale":"2024-05-21T15:14:13.238350+0000","last_undegraded":"2024-05-21T15:14:13.238350+0000","last_fullsized":"2024-05-21T15:14:13.238350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:54:19.982044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375339+0000","last_change":"2024-05-21T15:14:12.251715+0000","last_active":"2024-05-21T15:14:14.375339+0000","last_peered":"2024-05-21T15:14:14.375339+0000","last_clean":"2024-05-21T15:14:14.375339+0000","last_became_active":"2024-05-21T15:14:12.251337+0000","last_became_peered":"2024-05-21T15:14:12.251337+0000","last_unstale":"2024-05-21T15:14:14.375339+0000","last_undegraded":"2024-05-21T15:14:14.375339+0000","last_fullsized":"2024-05-21T15:14:14.375339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:03.562179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236674+0000","last_change":"2024-05-21T15:14:12.253166+0000","last_active":"2024-05-21T15:14:13.236674+0000","last_peered":"2024-05-21T15:14:13.236674+0000","last_clean":"2024-05-21T15:14:13.236674+0000","last_became_active":"2024-05-21T15:14:12.251830+0000","last_became_peered":"2024-05-21T15:14:12.251830+0000","last_unstale":"2024-05-21T15:14:13.236674+0000","last_undegraded":"2024-05-21T15:14:13.236674+0000","last_fullsized":"2024-05-21T15:14:13.236674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:11:58.679568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.257387+0000","last_change":"2024-05-21T15:14:12.256131+0000","last_active":"2024-05-21T15:14:12.257387+0000","last_peered":"2024-05-21T15:14:12.257387+0000","last_clean":"2024-05-21T15:14:12.257387+0000","last_became_active":"2024-05-21T15:14:12.255772+0000","last_became_peered":"2024-05-21T15:14:12.255772+0000","last_unstale":"2024-05-21T15:14:12.257387+0000","last_undegraded":"2024-05-21T15:14:12.257387+0000","last_fullsized":"2024-05-21T15:14:12.257387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:18:59.154936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238543+0000","last_change":"2024-05-21T15:14:12.260829+0000","last_active":"2024-05-21T15:14:13.238543+0000","last_peered":"2024-05-21T15:14:13.238543+0000","last_clean":"2024-05-21T15:14:13.238543+0000","last_became_active":"2024-05-21T15:14:12.259686+0000","last_became_peered":"2024-05-21T15:14:12.259686+0000","last_unstale":"2024-05-21T15:14:13.238543+0000","last_undegraded":"2024-05-21T15:14:13.238543+0000","last_fullsized":"2024-05-21T15:14:13.238543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:48.480814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560225+0000","last_change":"2024-05-21T15:14:12.262892+0000","last_active":"2024-05-21T15:14:13.560225+0000","last_peered":"2024-05-21T15:14:13.560225+0000","last_clean":"2024-05-21T15:14:13.560225+0000","last_became_active":"2024-05-21T15:14:12.262532+0000","last_became_peered":"2024-05-21T15:14:12.262532+0000","last_unstale":"2024-05-21T15:14:13.560225+0000","last_undegraded":"2024-05-21T15:14:13.560225+0000","last_fullsized":"2024-05-21T15:14:13.560225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:22.424783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.254438+0000","last_change":"2024-05-21T15:14:12.250186+0000","last_active":"2024-05-21T15:14:12.254438+0000","last_peered":"2024-05-21T15:14:12.254438+0000","last_clean":"2024-05-21T15:14:12.254438+0000","last_became_active":"2024-05-21T15:14:12.249828+0000","last_became_peered":"2024-05-21T15:14:12.249828+0000","last_unstale":"2024-05-21T15:14:12.254438+0000","last_undegraded":"2024-05-21T15:14:12.254438+0000","last_fullsized":"2024-05-21T15:14:12.254438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:37:42.463095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560105+0000","last_change":"2024-05-21T15:14:12.259845+0000","last_active":"2024-05-21T15:14:13.560105+0000","last_peered":"2024-05-21T15:14:13.560105+0000","last_clean":"2024-05-21T15:14:13.560105+0000","last_became_active":"2024-05-21T15:14:12.259455+0000","last_became_peered":"2024-05-21T15:14:12.259455+0000","last_unstale":"2024-05-21T15:14:13.560105+0000","last_undegraded":"2024-05-21T15:14:13.560105+0000","last_fullsized":"2024-05-21T15:14:13.560105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:24:59.655178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237600+0000","last_change":"2024-05-21T15:14:12.259713+0000","last_active":"2024-05-21T15:14:13.237600+0000","last_peered":"2024-05-21T15:14:13.237600+0000","last_clean":"2024-05-21T15:14:13.237600+0000","last_became_active":"2024-05-21T15:14:12.259057+0000","last_became_peered":"2024-05-21T15:14:12.259057+0000","last_unstale":"2024-05-21T15:14:13.237600+0000","last_undegraded":"2024-05-21T15:14:13.237600+0000","last_fullsized":"2024-05-21T15:14:13.237600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:46:22.377652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236599+0000","last_change":"2024-05-21T15:14:12.254389+0000","last_active":"2024-05-21T15:14:13.236599+0000","last_peered":"2024-05-21T15:14:13.236599+0000","last_clean":"2024-05-21T15:14:13.236599+0000","last_became_active":"2024-05-21T15:14:12.253846+0000","last_became_peered":"2024-05-21T15:14:12.253846+0000","last_unstale":"2024-05-21T15:14:13.236599+0000","last_undegraded":"2024-05-21T15:14:13.236599+0000","last_fullsized":"2024-05-21T15:14:13.236599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:32.932316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.252837+0000","last_change":"2024-05-21T15:14:12.252292+0000","last_active":"2024-05-21T15:14:12.252837+0000","last_peered":"2024-05-21T15:14:12.252837+0000","last_clean":"2024-05-21T15:14:12.252837+0000","last_became_active":"2024-05-21T15:14:12.251816+0000","last_became_peered":"2024-05-21T15:14:12.251816+0000","last_unstale":"2024-05-21T15:14:12.252837+0000","last_undegraded":"2024-05-21T15:14:12.252837+0000","last_fullsized":"2024-05-21T15:14:12.252837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:58:51.769621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2634,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":8882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645329,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1519616,"data_stored":1022125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332128,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712520,"statfs":{"total":95995035648,"available":95961620480,"internally_reserved":0,"allocated":1482752,"data_stored":1011242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":440789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":438625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920815,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1507328,"data_stored":1020501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:14:19.086 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:14:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 -- 172.21.15.78:0/2511632249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc102ab0 msgr2=0x7f43bc1037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:19.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 --2- 172.21.15.78:0/2511632249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc102ab0 0x7f43bc1037b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009900 tx=0x7f43b002a410 comp rx=0 tx=0).stop 2024-05-21T15:14:19.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 -- 172.21.15.78:0/2511632249 shutdown_connections 2024-05-21T15:14:19.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 --2- 172.21.15.78:0/2511632249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43bc1089f0 0x7f43bc10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 --2- 172.21.15.78:0/2511632249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43bc103cf0 0x7f43bc104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 --2- 172.21.15.78:0/2511632249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc102ab0 0x7f43bc1037b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 -- 172.21.15.78:0/2511632249 >> 172.21.15.78:0/2511632249 conn(0x7f43bc0fdcc0 msgr2=0x7f43bc100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:19.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 -- 172.21.15.78:0/2511632249 shutdown_connections 2024-05-21T15:14:19.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.228+0000 7f43c3136640 1 -- 172.21.15.78:0/2511632249 wait complete. 2024-05-21T15:14:19.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c3136640 1 Processor -- start 2024-05-21T15:14:19.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c3136640 1 -- start start 2024-05-21T15:14:19.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c3136640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43bc102ab0 0x7f43bc199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:19.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c3136640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43bc103cf0 0x7f43bc199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:19.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c3136640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc1089f0 0x7f43bc1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:19.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c3136640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc10e730 con 0x7f43bc103cf0 2024-05-21T15:14:19.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c3136640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc10e5b0 con 0x7f43bc1089f0 2024-05-21T15:14:19.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c3136640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc10e8b0 con 0x7f43bc102ab0 2024-05-21T15:14:19.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43bc103cf0 0x7f43bc199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:19.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43bc103cf0 0x7f43bc199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50804/0 (socket says 172.21.15.78:50804) 2024-05-21T15:14:19.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 -- 172.21.15.78:0/349194620 learned_addr learned my addr 172.21.15.78:0/349194620 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:19.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c16d3640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc1089f0 0x7f43bc1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:19.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c0ed2640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43bc102ab0 0x7f43bc199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:19.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 -- 172.21.15.78:0/349194620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43bc102ab0 msgr2=0x7f43bc199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:19.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43bc102ab0 0x7f43bc199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 -- 172.21.15.78:0/349194620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc1089f0 msgr2=0x7f43bc1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:19.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc1089f0 0x7f43bc1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 -- 172.21.15.78:0/349194620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc1a0e40 con 0x7f43bc103cf0 2024-05-21T15:14:19.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43c16d3640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc1089f0 0x7f43bc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:19.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.229+0000 7f43bbfff640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43bc103cf0 0x7f43bc199840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43a800d950 tx=0x7f43a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:19.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.230+0000 7f43b9ffb640 1 -- 172.21.15.78:0/349194620 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8017030 con 0x7f43bc103cf0 2024-05-21T15:14:19.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.230+0000 7f43b9ffb640 1 -- 172.21.15.78:0/349194620 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a8004020 con 0x7f43bc103cf0 2024-05-21T15:14:19.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.230+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc1a1130 con 0x7f43bc103cf0 2024-05-21T15:14:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.230+0000 7f43b9ffb640 1 -- 172.21.15.78:0/349194620 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80047a0 con 0x7f43bc103cf0 2024-05-21T15:14:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.230+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc1a1680 con 0x7f43bc103cf0 2024-05-21T15:14:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.231+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43bc107800 con 0x7f43bc103cf0 2024-05-21T15:14:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.232+0000 7f43b9ffb640 1 -- 172.21.15.78:0/349194620 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f43a80041d0 con 0x7f43bc103cf0 2024-05-21T15:14:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.233+0000 7f43b9ffb640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f43ac0a2c20 0x7f43ac0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.233+0000 7f43c0ed2640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f43ac0a2c20 0x7f43ac0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.233+0000 7f43b9ffb640 1 -- 172.21.15.78:0/349194620 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f43a80cea30 con 0x7f43bc103cf0 2024-05-21T15:14:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.233+0000 7f43c0ed2640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f43ac0a2c20 0x7f43ac0a5130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43b002aa00 tx=0x7f43b0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.238+0000 7f43b9ffb640 1 -- 172.21.15.78:0/349194620 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f43a809b9e0 con 0x7f43bc103cf0 2024-05-21T15:14:19.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T15:14:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] up:active 2024-05-21T15:14:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-05-21T15:14:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:14:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/284580085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:19 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2006450778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:19.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.412+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43bc061f90 con 0x7f43ac0a2c20 2024-05-21T15:14:19.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.419+0000 7f43b9ffb640 1 -- 172.21.15.78:0/349194620 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258816 (secure 0 0 0) 0x7f43bc061f90 con 0x7f43ac0a2c20 2024-05-21T15:14:19.426 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-05-21T15:14:19.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f43ac0a2c20 msgr2=0x7f43ac0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:19.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f43ac0a2c20 0x7f43ac0a5130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43b002aa00 tx=0x7f43b0005a20 comp rx=0 tx=0).stop 2024-05-21T15:14:19.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43bc103cf0 msgr2=0x7f43bc199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:19.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43bc103cf0 0x7f43bc199840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43a800d950 tx=0x7f43a800de10 comp rx=0 tx=0).stop 2024-05-21T15:14:19.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 shutdown_connections 2024-05-21T15:14:19.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f43ac0a2c20 0x7f43ac0a5130 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43bc1089f0 0x7f43bc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43bc103cf0 0x7f43bc199840 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 --2- 172.21.15.78:0/349194620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43bc102ab0 0x7f43bc199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:19.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 >> 172.21.15.78:0/349194620 conn(0x7f43bc0fdcc0 msgr2=0x7f43bc1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:19.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 shutdown_connections 2024-05-21T15:14:19.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:19.422+0000 7f43c3136640 1 -- 172.21.15.78:0/349194620 wait complete. 2024-05-21T15:14:19.437 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-05-21T15:14:19.241959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632259,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":170,"num_read_kb":182,"num_write":328,"num_write_kb":3030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":216,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357728,"kb_used_data":12704,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584096,"statfs":{"total":1151940427776,"available":1151574114304,"internally_reserved":0,"allocated":13008896,"data_stored":7077874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41891,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":34,"num_read_kb":63,"num_write":147,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.691277"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.254776+0000","last_change":"2024-05-21T15:14:12.254239+0000","last_active":"2024-05-21T15:14:12.254776+0000","last_peered":"2024-05-21T15:14:12.254776+0000","last_clean":"2024-05-21T15:14:12.254776+0000","last_became_active":"2024-05-21T15:14:12.253885+0000","last_became_peered":"2024-05-21T15:14:12.253885+0000","last_unstale":"2024-05-21T15:14:12.254776+0000","last_undegraded":"2024-05-21T15:14:12.254776+0000","last_fullsized":"2024-05-21T15:14:12.254776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:34:07.413828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375498+0000","last_change":"2024-05-21T15:14:12.262984+0000","last_active":"2024-05-21T15:14:14.375498+0000","last_peered":"2024-05-21T15:14:14.375498+0000","last_clean":"2024-05-21T15:14:14.375498+0000","last_became_active":"2024-05-21T15:14:12.262667+0000","last_became_peered":"2024-05-21T15:14:12.262667+0000","last_unstale":"2024-05-21T15:14:14.375498+0000","last_undegraded":"2024-05-21T15:14:14.375498+0000","last_fullsized":"2024-05-21T15:14:14.375498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:47:47.434314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375519+0000","last_change":"2024-05-21T15:14:12.260573+0000","last_active":"2024-05-21T15:14:14.375519+0000","last_peered":"2024-05-21T15:14:14.375519+0000","last_clean":"2024-05-21T15:14:14.375519+0000","last_became_active":"2024-05-21T15:14:12.259073+0000","last_became_peered":"2024-05-21T15:14:12.259073+0000","last_unstale":"2024-05-21T15:14:14.375519+0000","last_undegraded":"2024-05-21T15:14:14.375519+0000","last_fullsized":"2024-05-21T15:14:14.375519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:15:01.919464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375289+0000","last_change":"2024-05-21T15:14:12.256992+0000","last_active":"2024-05-21T15:14:14.375289+0000","last_peered":"2024-05-21T15:14:14.375289+0000","last_clean":"2024-05-21T15:14:14.375289+0000","last_became_active":"2024-05-21T15:14:12.256448+0000","last_became_peered":"2024-05-21T15:14:12.256448+0000","last_unstale":"2024-05-21T15:14:14.375289+0000","last_undegraded":"2024-05-21T15:14:14.375289+0000","last_fullsized":"2024-05-21T15:14:14.375289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:40.569263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484495+0000","last_change":"2024-05-21T15:14:12.251533+0000","last_active":"2024-05-21T15:14:13.484495+0000","last_peered":"2024-05-21T15:14:13.484495+0000","last_clean":"2024-05-21T15:14:13.484495+0000","last_became_active":"2024-05-21T15:14:12.251210+0000","last_became_peered":"2024-05-21T15:14:12.251210+0000","last_unstale":"2024-05-21T15:14:13.484495+0000","last_undegraded":"2024-05-21T15:14:13.484495+0000","last_fullsized":"2024-05-21T15:14:13.484495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:43:34.109179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375671+0000","last_change":"2024-05-21T15:14:12.251767+0000","last_active":"2024-05-21T15:14:14.375671+0000","last_peered":"2024-05-21T15:14:14.375671+0000","last_clean":"2024-05-21T15:14:14.375671+0000","last_became_active":"2024-05-21T15:14:12.251427+0000","last_became_peered":"2024-05-21T15:14:12.251427+0000","last_unstale":"2024-05-21T15:14:14.375671+0000","last_undegraded":"2024-05-21T15:14:14.375671+0000","last_fullsized":"2024-05-21T15:14:14.375671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:48:41.804074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376131+0000","last_change":"2024-05-21T15:14:12.253472+0000","last_active":"2024-05-21T15:14:14.376131+0000","last_peered":"2024-05-21T15:14:14.376131+0000","last_clean":"2024-05-21T15:14:14.376131+0000","last_became_active":"2024-05-21T15:14:12.252762+0000","last_became_peered":"2024-05-21T15:14:12.252762+0000","last_unstale":"2024-05-21T15:14:14.376131+0000","last_undegraded":"2024-05-21T15:14:14.376131+0000","last_fullsized":"2024-05-21T15:14:14.376131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:11:44.497404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.559896+0000","last_change":"2024-05-21T15:14:12.258627+0000","last_active":"2024-05-21T15:14:13.559896+0000","last_peered":"2024-05-21T15:14:13.559896+0000","last_clean":"2024-05-21T15:14:13.559896+0000","last_became_active":"2024-05-21T15:14:12.258118+0000","last_became_peered":"2024-05-21T15:14:12.258118+0000","last_unstale":"2024-05-21T15:14:13.559896+0000","last_undegraded":"2024-05-21T15:14:13.559896+0000","last_fullsized":"2024-05-21T15:14:13.559896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:56.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237924+0000","last_change":"2024-05-21T15:14:12.255998+0000","last_active":"2024-05-21T15:14:13.237924+0000","last_peered":"2024-05-21T15:14:13.237924+0000","last_clean":"2024-05-21T15:14:13.237924+0000","last_became_active":"2024-05-21T15:14:12.255658+0000","last_became_peered":"2024-05-21T15:14:12.255658+0000","last_unstale":"2024-05-21T15:14:13.237924+0000","last_undegraded":"2024-05-21T15:14:13.237924+0000","last_fullsized":"2024-05-21T15:14:13.237924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:44:42.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376321+0000","last_change":"2024-05-21T15:14:12.257617+0000","last_active":"2024-05-21T15:14:14.376321+0000","last_peered":"2024-05-21T15:14:14.376321+0000","last_clean":"2024-05-21T15:14:14.376321+0000","last_became_active":"2024-05-21T15:14:12.256947+0000","last_became_peered":"2024-05-21T15:14:12.256947+0000","last_unstale":"2024-05-21T15:14:14.376321+0000","last_undegraded":"2024-05-21T15:14:14.376321+0000","last_fullsized":"2024-05-21T15:14:14.376321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:45:58.174954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376622+0000","last_change":"2024-05-21T15:14:12.260454+0000","last_active":"2024-05-21T15:14:14.376622+0000","last_peered":"2024-05-21T15:14:14.376622+0000","last_clean":"2024-05-21T15:14:14.376622+0000","last_became_active":"2024-05-21T15:14:12.260126+0000","last_became_peered":"2024-05-21T15:14:12.260126+0000","last_unstale":"2024-05-21T15:14:14.376622+0000","last_undegraded":"2024-05-21T15:14:14.376622+0000","last_fullsized":"2024-05-21T15:14:14.376622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:49:45.212050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375092+0000","last_change":"2024-05-21T15:14:12.252071+0000","last_active":"2024-05-21T15:14:14.375092+0000","last_peered":"2024-05-21T15:14:14.375092+0000","last_clean":"2024-05-21T15:14:14.375092+0000","last_became_active":"2024-05-21T15:14:12.251706+0000","last_became_peered":"2024-05-21T15:14:12.251706+0000","last_unstale":"2024-05-21T15:14:14.375092+0000","last_undegraded":"2024-05-21T15:14:14.375092+0000","last_fullsized":"2024-05-21T15:14:14.375092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:02:44.349368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236754+0000","last_change":"2024-05-21T15:14:12.239877+0000","last_active":"2024-05-21T15:14:13.236754+0000","last_peered":"2024-05-21T15:14:13.236754+0000","last_clean":"2024-05-21T15:14:13.236754+0000","last_became_active":"2024-05-21T15:14:12.239615+0000","last_became_peered":"2024-05-21T15:14:12.239615+0000","last_unstale":"2024-05-21T15:14:13.236754+0000","last_undegraded":"2024-05-21T15:14:13.236754+0000","last_fullsized":"2024-05-21T15:14:13.236754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:23.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375262+0000","last_change":"2024-05-21T15:14:12.260349+0000","last_active":"2024-05-21T15:14:14.375262+0000","last_peered":"2024-05-21T15:14:14.375262+0000","last_clean":"2024-05-21T15:14:14.375262+0000","last_became_active":"2024-05-21T15:14:12.259658+0000","last_became_peered":"2024-05-21T15:14:12.259658+0000","last_unstale":"2024-05-21T15:14:14.375262+0000","last_undegraded":"2024-05-21T15:14:14.375262+0000","last_fullsized":"2024-05-21T15:14:14.375262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:59:32.326571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375318+0000","last_change":"2024-05-21T15:14:12.259123+0000","last_active":"2024-05-21T15:14:14.375318+0000","last_peered":"2024-05-21T15:14:14.375318+0000","last_clean":"2024-05-21T15:14:14.375318+0000","last_became_active":"2024-05-21T15:14:12.258791+0000","last_became_peered":"2024-05-21T15:14:12.258791+0000","last_unstale":"2024-05-21T15:14:14.375318+0000","last_undegraded":"2024-05-21T15:14:14.375318+0000","last_fullsized":"2024-05-21T15:14:14.375318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:32:01.046201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237257+0000","last_change":"2024-05-21T15:14:12.264667+0000","last_active":"2024-05-21T15:14:13.237257+0000","last_peered":"2024-05-21T15:14:13.237257+0000","last_clean":"2024-05-21T15:14:13.237257+0000","last_became_active":"2024-05-21T15:14:12.264284+0000","last_became_peered":"2024-05-21T15:14:12.264284+0000","last_unstale":"2024-05-21T15:14:13.237257+0000","last_undegraded":"2024-05-21T15:14:13.237257+0000","last_fullsized":"2024-05-21T15:14:13.237257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:24.861321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.260305+0000","last_change":"2024-05-21T15:14:12.259912+0000","last_active":"2024-05-21T15:14:12.260305+0000","last_peered":"2024-05-21T15:14:12.260305+0000","last_clean":"2024-05-21T15:14:12.260305+0000","last_became_active":"2024-05-21T15:14:12.259578+0000","last_became_peered":"2024-05-21T15:14:12.259578+0000","last_unstale":"2024-05-21T15:14:12.260305+0000","last_undegraded":"2024-05-21T15:14:12.260305+0000","last_fullsized":"2024-05-21T15:14:12.260305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:52:30.469150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.558317+0000","last_change":"2024-05-21T15:14:12.258355+0000","last_active":"2024-05-21T15:14:13.558317+0000","last_peered":"2024-05-21T15:14:13.558317+0000","last_clean":"2024-05-21T15:14:13.558317+0000","last_became_active":"2024-05-21T15:14:12.257991+0000","last_became_peered":"2024-05-21T15:14:12.257991+0000","last_unstale":"2024-05-21T15:14:13.558317+0000","last_undegraded":"2024-05-21T15:14:13.558317+0000","last_fullsized":"2024-05-21T15:14:13.558317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:54.014553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238585+0000","last_change":"2024-05-21T15:14:12.258873+0000","last_active":"2024-05-21T15:14:13.238585+0000","last_peered":"2024-05-21T15:14:13.238585+0000","last_clean":"2024-05-21T15:14:13.238585+0000","last_became_active":"2024-05-21T15:14:12.258548+0000","last_became_peered":"2024-05-21T15:14:12.258548+0000","last_unstale":"2024-05-21T15:14:13.238585+0000","last_undegraded":"2024-05-21T15:14:13.238585+0000","last_fullsized":"2024-05-21T15:14:13.238585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:00.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.558289+0000","last_change":"2024-05-21T15:14:11.568366+0000","last_active":"2024-05-21T15:14:13.558289+0000","last_peered":"2024-05-21T15:14:13.558289+0000","last_clean":"2024-05-21T15:14:13.558289+0000","last_became_active":"2024-05-21T15:14:11.567885+0000","last_became_peered":"2024-05-21T15:14:11.567885+0000","last_unstale":"2024-05-21T15:14:13.558289+0000","last_undegraded":"2024-05-21T15:14:13.558289+0000","last_fullsized":"2024-05-21T15:14:13.558289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:36:56.553488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560363+0000","last_change":"2024-05-21T15:14:12.261632+0000","last_active":"2024-05-21T15:14:13.560363+0000","last_peered":"2024-05-21T15:14:13.560363+0000","last_clean":"2024-05-21T15:14:13.560363+0000","last_became_active":"2024-05-21T15:14:12.261149+0000","last_became_peered":"2024-05-21T15:14:12.261149+0000","last_unstale":"2024-05-21T15:14:13.560363+0000","last_undegraded":"2024-05-21T15:14:13.560363+0000","last_fullsized":"2024-05-21T15:14:13.560363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:11.349395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560289+0000","last_change":"2024-05-21T15:14:11.567859+0000","last_active":"2024-05-21T15:14:13.560289+0000","last_peered":"2024-05-21T15:14:13.560289+0000","last_clean":"2024-05-21T15:14:13.560289+0000","last_became_active":"2024-05-21T15:14:11.566855+0000","last_became_peered":"2024-05-21T15:14:11.566855+0000","last_unstale":"2024-05-21T15:14:13.560289+0000","last_undegraded":"2024-05-21T15:14:13.560289+0000","last_fullsized":"2024-05-21T15:14:13.560289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:21.568436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375609+0000","last_change":"2024-05-21T15:14:12.262153+0000","last_active":"2024-05-21T15:14:14.375609+0000","last_peered":"2024-05-21T15:14:14.375609+0000","last_clean":"2024-05-21T15:14:14.375609+0000","last_became_active":"2024-05-21T15:14:12.261780+0000","last_became_peered":"2024-05-21T15:14:12.261780+0000","last_unstale":"2024-05-21T15:14:14.375609+0000","last_undegraded":"2024-05-21T15:14:14.375609+0000","last_fullsized":"2024-05-21T15:14:14.375609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:21:25.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.277699+0000","last_change":"2024-05-21T15:14:11.570316+0000","last_active":"2024-05-21T15:14:16.277699+0000","last_peered":"2024-05-21T15:14:16.277699+0000","last_clean":"2024-05-21T15:14:16.277699+0000","last_became_active":"2024-05-21T15:14:11.569967+0000","last_became_peered":"2024-05-21T15:14:11.569967+0000","last_unstale":"2024-05-21T15:14:16.277699+0000","last_undegraded":"2024-05-21T15:14:16.277699+0000","last_fullsized":"2024-05-21T15:14:16.277699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:30:27.364742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237627+0000","last_change":"2024-05-21T15:14:12.261716+0000","last_active":"2024-05-21T15:14:13.237627+0000","last_peered":"2024-05-21T15:14:13.237627+0000","last_clean":"2024-05-21T15:14:13.237627+0000","last_became_active":"2024-05-21T15:14:12.261341+0000","last_became_peered":"2024-05-21T15:14:12.261341+0000","last_unstale":"2024-05-21T15:14:13.237627+0000","last_undegraded":"2024-05-21T15:14:13.237627+0000","last_fullsized":"2024-05-21T15:14:13.237627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:40:10.464974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484649+0000","last_change":"2024-05-21T15:14:11.573305+0000","last_active":"2024-05-21T15:14:13.484649+0000","last_peered":"2024-05-21T15:14:13.484649+0000","last_clean":"2024-05-21T15:14:13.484649+0000","last_became_active":"2024-05-21T15:14:11.572655+0000","last_became_peered":"2024-05-21T15:14:11.572655+0000","last_unstale":"2024-05-21T15:14:13.484649+0000","last_undegraded":"2024-05-21T15:14:13.484649+0000","last_fullsized":"2024-05-21T15:14:13.484649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:37.268440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375483+0000","last_change":"2024-05-21T15:14:12.253788+0000","last_active":"2024-05-21T15:14:14.375483+0000","last_peered":"2024-05-21T15:14:14.375483+0000","last_clean":"2024-05-21T15:14:14.375483+0000","last_became_active":"2024-05-21T15:14:12.253450+0000","last_became_peered":"2024-05-21T15:14:12.253450+0000","last_unstale":"2024-05-21T15:14:14.375483+0000","last_undegraded":"2024-05-21T15:14:14.375483+0000","last_fullsized":"2024-05-21T15:14:14.375483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:57.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:17.269062+0000","last_change":"2024-05-21T15:14:11.567400+0000","last_active":"2024-05-21T15:14:17.269062+0000","last_peered":"2024-05-21T15:14:17.269062+0000","last_clean":"2024-05-21T15:14:17.269062+0000","last_became_active":"2024-05-21T15:14:11.565995+0000","last_became_peered":"2024-05-21T15:14:11.565995+0000","last_unstale":"2024-05-21T15:14:17.269062+0000","last_undegraded":"2024-05-21T15:14:17.269062+0000","last_fullsized":"2024-05-21T15:14:17.269062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:50.463339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484379+0000","last_change":"2024-05-21T15:14:12.251565+0000","last_active":"2024-05-21T15:14:13.484379+0000","last_peered":"2024-05-21T15:14:13.484379+0000","last_clean":"2024-05-21T15:14:13.484379+0000","last_became_active":"2024-05-21T15:14:12.251289+0000","last_became_peered":"2024-05-21T15:14:12.251289+0000","last_unstale":"2024-05-21T15:14:13.484379+0000","last_undegraded":"2024-05-21T15:14:13.484379+0000","last_fullsized":"2024-05-21T15:14:13.484379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:44.619800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.276522+0000","last_change":"2024-05-21T15:14:11.575193+0000","last_active":"2024-05-21T15:14:16.276522+0000","last_peered":"2024-05-21T15:14:16.276522+0000","last_clean":"2024-05-21T15:14:16.276522+0000","last_became_active":"2024-05-21T15:14:11.574861+0000","last_became_peered":"2024-05-21T15:14:11.574861+0000","last_unstale":"2024-05-21T15:14:16.276522+0000","last_undegraded":"2024-05-21T15:14:16.276522+0000","last_fullsized":"2024-05-21T15:14:16.276522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:54.548408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375654+0000","last_change":"2024-05-21T15:14:12.254680+0000","last_active":"2024-05-21T15:14:14.375654+0000","last_peered":"2024-05-21T15:14:14.375654+0000","last_clean":"2024-05-21T15:14:14.375654+0000","last_became_active":"2024-05-21T15:14:12.254283+0000","last_became_peered":"2024-05-21T15:14:12.254283+0000","last_unstale":"2024-05-21T15:14:14.375654+0000","last_undegraded":"2024-05-21T15:14:14.375654+0000","last_fullsized":"2024-05-21T15:14:14.375654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:59:32.760390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:15.586568+0000","last_change":"2024-05-21T15:14:11.572087+0000","last_active":"2024-05-21T15:14:15.586568+0000","last_peered":"2024-05-21T15:14:15.586568+0000","last_clean":"2024-05-21T15:14:15.586568+0000","last_became_active":"2024-05-21T15:14:11.571716+0000","last_became_peered":"2024-05-21T15:14:11.571716+0000","last_unstale":"2024-05-21T15:14:15.586568+0000","last_undegraded":"2024-05-21T15:14:15.586568+0000","last_fullsized":"2024-05-21T15:14:15.586568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:07.082766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.559982+0000","last_change":"2024-05-21T15:14:12.252233+0000","last_active":"2024-05-21T15:14:13.559982+0000","last_peered":"2024-05-21T15:14:13.559982+0000","last_clean":"2024-05-21T15:14:13.559982+0000","last_became_active":"2024-05-21T15:14:12.251901+0000","last_became_peered":"2024-05-21T15:14:12.251901+0000","last_unstale":"2024-05-21T15:14:13.559982+0000","last_undegraded":"2024-05-21T15:14:13.559982+0000","last_fullsized":"2024-05-21T15:14:13.559982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:05.107436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:17.283797+0000","last_change":"2024-05-21T15:14:11.565946+0000","last_active":"2024-05-21T15:14:17.283797+0000","last_peered":"2024-05-21T15:14:17.283797+0000","last_clean":"2024-05-21T15:14:17.283797+0000","last_became_active":"2024-05-21T15:14:11.565601+0000","last_became_peered":"2024-05-21T15:14:11.565601+0000","last_unstale":"2024-05-21T15:14:17.283797+0000","last_undegraded":"2024-05-21T15:14:17.283797+0000","last_fullsized":"2024-05-21T15:14:17.283797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:41:55.243473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.558223+0000","last_change":"2024-05-21T15:14:12.255188+0000","last_active":"2024-05-21T15:14:13.558223+0000","last_peered":"2024-05-21T15:14:13.558223+0000","last_clean":"2024-05-21T15:14:13.558223+0000","last_became_active":"2024-05-21T15:14:12.254503+0000","last_became_peered":"2024-05-21T15:14:12.254503+0000","last_unstale":"2024-05-21T15:14:13.558223+0000","last_undegraded":"2024-05-21T15:14:13.558223+0000","last_fullsized":"2024-05-21T15:14:13.558223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:22:16.336740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:18.269598+0000","last_change":"2024-05-21T15:14:11.569882+0000","last_active":"2024-05-21T15:14:18.269598+0000","last_peered":"2024-05-21T15:14:18.269598+0000","last_clean":"2024-05-21T15:14:18.269598+0000","last_became_active":"2024-05-21T15:14:11.569532+0000","last_became_peered":"2024-05-21T15:14:11.569532+0000","last_unstale":"2024-05-21T15:14:18.269598+0000","last_undegraded":"2024-05-21T15:14:18.269598+0000","last_fullsized":"2024-05-21T15:14:18.269598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:58:50.081709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237788+0000","last_change":"2024-05-21T15:14:12.262691+0000","last_active":"2024-05-21T15:14:13.237788+0000","last_peered":"2024-05-21T15:14:13.237788+0000","last_clean":"2024-05-21T15:14:13.237788+0000","last_became_active":"2024-05-21T15:14:12.261492+0000","last_became_peered":"2024-05-21T15:14:12.261492+0000","last_unstale":"2024-05-21T15:14:13.237788+0000","last_undegraded":"2024-05-21T15:14:13.237788+0000","last_fullsized":"2024-05-21T15:14:13.237788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:35:00.121256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.572926+0000","last_change":"2024-05-21T15:14:11.569231+0000","last_active":"2024-05-21T15:14:14.572926+0000","last_peered":"2024-05-21T15:14:14.572926+0000","last_clean":"2024-05-21T15:14:14.572926+0000","last_became_active":"2024-05-21T15:14:11.568916+0000","last_became_peered":"2024-05-21T15:14:11.568916+0000","last_unstale":"2024-05-21T15:14:14.572926+0000","last_undegraded":"2024-05-21T15:14:14.572926+0000","last_fullsized":"2024-05-21T15:14:14.572926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:26:13.411854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375549+0000","last_change":"2024-05-21T15:14:12.256307+0000","last_active":"2024-05-21T15:14:14.375549+0000","last_peered":"2024-05-21T15:14:14.375549+0000","last_clean":"2024-05-21T15:14:14.375549+0000","last_became_active":"2024-05-21T15:14:12.255828+0000","last_became_peered":"2024-05-21T15:14:12.255828+0000","last_unstale":"2024-05-21T15:14:14.375549+0000","last_undegraded":"2024-05-21T15:14:14.375549+0000","last_fullsized":"2024-05-21T15:14:14.375549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:11:40.068779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.266173+0000","last_change":"2024-05-21T15:14:11.568239+0000","last_active":"2024-05-21T15:14:16.266173+0000","last_peered":"2024-05-21T15:14:16.266173+0000","last_clean":"2024-05-21T15:14:16.266173+0000","last_became_active":"2024-05-21T15:14:11.567836+0000","last_became_peered":"2024-05-21T15:14:11.567836+0000","last_unstale":"2024-05-21T15:14:16.266173+0000","last_undegraded":"2024-05-21T15:14:16.266173+0000","last_fullsized":"2024-05-21T15:14:16.266173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:10.244172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237859+0000","last_change":"2024-05-21T15:14:12.257762+0000","last_active":"2024-05-21T15:14:13.237859+0000","last_peered":"2024-05-21T15:14:13.237859+0000","last_clean":"2024-05-21T15:14:13.237859+0000","last_became_active":"2024-05-21T15:14:12.256335+0000","last_became_peered":"2024-05-21T15:14:12.256335+0000","last_unstale":"2024-05-21T15:14:13.237859+0000","last_undegraded":"2024-05-21T15:14:13.237859+0000","last_fullsized":"2024-05-21T15:14:13.237859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:35.219673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'22","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:18.274509+0000","last_change":"2024-05-21T15:14:11.564407+0000","last_active":"2024-05-21T15:14:18.274509+0000","last_peered":"2024-05-21T15:14:18.274509+0000","last_clean":"2024-05-21T15:14:18.274509+0000","last_became_active":"2024-05-21T15:14:11.564012+0000","last_became_peered":"2024-05-21T15:14:11.564012+0000","last_unstale":"2024-05-21T15:14:18.274509+0000","last_undegraded":"2024-05-21T15:14:18.274509+0000","last_fullsized":"2024-05-21T15:14:18.274509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:34:54.177844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":42,"num_write":24,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484847+0000","last_change":"2024-05-21T15:14:12.253820+0000","last_active":"2024-05-21T15:14:13.484847+0000","last_peered":"2024-05-21T15:14:13.484847+0000","last_clean":"2024-05-21T15:14:13.484847+0000","last_became_active":"2024-05-21T15:14:12.253360+0000","last_became_peered":"2024-05-21T15:14:12.253360+0000","last_unstale":"2024-05-21T15:14:13.484847+0000","last_undegraded":"2024-05-21T15:14:13.484847+0000","last_fullsized":"2024-05-21T15:14:13.484847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:38:19.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.573038+0000","last_change":"2024-05-21T15:14:11.571161+0000","last_active":"2024-05-21T15:14:14.573038+0000","last_peered":"2024-05-21T15:14:14.573038+0000","last_clean":"2024-05-21T15:14:14.573038+0000","last_became_active":"2024-05-21T15:14:11.570837+0000","last_became_peered":"2024-05-21T15:14:11.570837+0000","last_unstale":"2024-05-21T15:14:14.573038+0000","last_undegraded":"2024-05-21T15:14:14.573038+0000","last_fullsized":"2024-05-21T15:14:14.573038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:24:55.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236758+0000","last_change":"2024-05-21T15:14:12.255713+0000","last_active":"2024-05-21T15:14:13.236758+0000","last_peered":"2024-05-21T15:14:13.236758+0000","last_clean":"2024-05-21T15:14:13.236758+0000","last_became_active":"2024-05-21T15:14:12.254458+0000","last_became_peered":"2024-05-21T15:14:12.254458+0000","last_unstale":"2024-05-21T15:14:13.236758+0000","last_undegraded":"2024-05-21T15:14:13.236758+0000","last_fullsized":"2024-05-21T15:14:13.236758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:55:24.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:17.283912+0000","last_change":"2024-05-21T15:14:11.565916+0000","last_active":"2024-05-21T15:14:17.283912+0000","last_peered":"2024-05-21T15:14:17.283912+0000","last_clean":"2024-05-21T15:14:17.283912+0000","last_became_active":"2024-05-21T15:14:11.565460+0000","last_became_peered":"2024-05-21T15:14:11.565460+0000","last_unstale":"2024-05-21T15:14:17.283912+0000","last_undegraded":"2024-05-21T15:14:17.283912+0000","last_fullsized":"2024-05-21T15:14:17.283912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:09.982208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376405+0000","last_change":"2024-05-21T15:14:12.251856+0000","last_active":"2024-05-21T15:14:14.376405+0000","last_peered":"2024-05-21T15:14:14.376405+0000","last_clean":"2024-05-21T15:14:14.376405+0000","last_became_active":"2024-05-21T15:14:12.251496+0000","last_became_peered":"2024-05-21T15:14:12.251496+0000","last_unstale":"2024-05-21T15:14:14.376405+0000","last_undegraded":"2024-05-21T15:14:14.376405+0000","last_fullsized":"2024-05-21T15:14:14.376405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:17.566276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.374725+0000","last_change":"2024-05-21T15:14:11.565485+0000","last_active":"2024-05-21T15:14:14.374725+0000","last_peered":"2024-05-21T15:14:14.374725+0000","last_clean":"2024-05-21T15:14:14.374725+0000","last_became_active":"2024-05-21T15:14:11.565092+0000","last_became_peered":"2024-05-21T15:14:11.565092+0000","last_unstale":"2024-05-21T15:14:14.374725+0000","last_undegraded":"2024-05-21T15:14:14.374725+0000","last_fullsized":"2024-05-21T15:14:14.374725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:01:16.101793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375596+0000","last_change":"2024-05-21T15:14:12.262283+0000","last_active":"2024-05-21T15:14:14.375596+0000","last_peered":"2024-05-21T15:14:14.375596+0000","last_clean":"2024-05-21T15:14:14.375596+0000","last_became_active":"2024-05-21T15:14:12.261610+0000","last_became_peered":"2024-05-21T15:14:12.261610+0000","last_unstale":"2024-05-21T15:14:14.375596+0000","last_undegraded":"2024-05-21T15:14:14.375596+0000","last_fullsized":"2024-05-21T15:14:14.375596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:44:33.087115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.262629+0000","last_change":"2024-05-21T15:14:11.569905+0000","last_active":"2024-05-21T15:14:16.262629+0000","last_peered":"2024-05-21T15:14:16.262629+0000","last_clean":"2024-05-21T15:14:16.262629+0000","last_became_active":"2024-05-21T15:14:11.569221+0000","last_became_peered":"2024-05-21T15:14:11.569221+0000","last_unstale":"2024-05-21T15:14:16.262629+0000","last_undegraded":"2024-05-21T15:14:16.262629+0000","last_fullsized":"2024-05-21T15:14:16.262629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:11:04.875890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375777+0000","last_change":"2024-05-21T15:14:12.255149+0000","last_active":"2024-05-21T15:14:14.375777+0000","last_peered":"2024-05-21T15:14:14.375777+0000","last_clean":"2024-05-21T15:14:14.375777+0000","last_became_active":"2024-05-21T15:14:12.254366+0000","last_became_peered":"2024-05-21T15:14:12.254366+0000","last_unstale":"2024-05-21T15:14:14.375777+0000","last_undegraded":"2024-05-21T15:14:14.375777+0000","last_fullsized":"2024-05-21T15:14:14.375777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:49:05.806218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.264077+0000","last_change":"2024-05-21T15:14:11.569670+0000","last_active":"2024-05-21T15:14:16.264077+0000","last_peered":"2024-05-21T15:14:16.264077+0000","last_clean":"2024-05-21T15:14:16.264077+0000","last_became_active":"2024-05-21T15:14:11.569257+0000","last_became_peered":"2024-05-21T15:14:11.569257+0000","last_unstale":"2024-05-21T15:14:16.264077+0000","last_undegraded":"2024-05-21T15:14:16.264077+0000","last_fullsized":"2024-05-21T15:14:16.264077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:18.584920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.259436+0000","last_change":"2024-05-21T15:14:12.258296+0000","last_active":"2024-05-21T15:14:12.259436+0000","last_peered":"2024-05-21T15:14:12.259436+0000","last_clean":"2024-05-21T15:14:12.259436+0000","last_became_active":"2024-05-21T15:14:12.257919+0000","last_became_peered":"2024-05-21T15:14:12.257919+0000","last_unstale":"2024-05-21T15:14:12.259436+0000","last_undegraded":"2024-05-21T15:14:12.259436+0000","last_fullsized":"2024-05-21T15:14:12.259436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:12:38.852843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.558227+0000","last_change":"2024-05-21T15:14:11.567849+0000","last_active":"2024-05-21T15:14:13.558227+0000","last_peered":"2024-05-21T15:14:13.558227+0000","last_clean":"2024-05-21T15:14:13.558227+0000","last_became_active":"2024-05-21T15:14:11.567476+0000","last_became_peered":"2024-05-21T15:14:11.567476+0000","last_unstale":"2024-05-21T15:14:13.558227+0000","last_undegraded":"2024-05-21T15:14:13.558227+0000","last_fullsized":"2024-05-21T15:14:13.558227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:13.516765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376010+0000","last_change":"2024-05-21T15:14:12.255165+0000","last_active":"2024-05-21T15:14:14.376010+0000","last_peered":"2024-05-21T15:14:14.376010+0000","last_clean":"2024-05-21T15:14:14.376010+0000","last_became_active":"2024-05-21T15:14:12.254743+0000","last_became_peered":"2024-05-21T15:14:12.254743+0000","last_unstale":"2024-05-21T15:14:14.376010+0000","last_undegraded":"2024-05-21T15:14:14.376010+0000","last_fullsized":"2024-05-21T15:14:14.376010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:53:11.059597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376293+0000","last_change":"2024-05-21T15:14:11.569745+0000","last_active":"2024-05-21T15:14:14.376293+0000","last_peered":"2024-05-21T15:14:14.376293+0000","last_clean":"2024-05-21T15:14:14.376293+0000","last_became_active":"2024-05-21T15:14:11.569409+0000","last_became_peered":"2024-05-21T15:14:11.569409+0000","last_unstale":"2024-05-21T15:14:14.376293+0000","last_undegraded":"2024-05-21T15:14:14.376293+0000","last_fullsized":"2024-05-21T15:14:14.376293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:00:37.963797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376915+0000","last_change":"2024-05-21T15:14:12.254631+0000","last_active":"2024-05-21T15:14:14.376915+0000","last_peered":"2024-05-21T15:14:14.376915+0000","last_clean":"2024-05-21T15:14:14.376915+0000","last_became_active":"2024-05-21T15:14:12.254239+0000","last_became_peered":"2024-05-21T15:14:12.254239+0000","last_unstale":"2024-05-21T15:14:14.376915+0000","last_undegraded":"2024-05-21T15:14:14.376915+0000","last_fullsized":"2024-05-21T15:14:14.376915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:55:17.135550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375657+0000","last_change":"2024-05-21T15:14:12.257568+0000","last_active":"2024-05-21T15:14:14.375657+0000","last_peered":"2024-05-21T15:14:14.375657+0000","last_clean":"2024-05-21T15:14:14.375657+0000","last_became_active":"2024-05-21T15:14:12.256734+0000","last_became_peered":"2024-05-21T15:14:12.256734+0000","last_unstale":"2024-05-21T15:14:14.375657+0000","last_undegraded":"2024-05-21T15:14:14.375657+0000","last_fullsized":"2024-05-21T15:14:14.375657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:51:15.271444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:18.269047+0000","last_change":"2024-05-21T15:14:11.574352+0000","last_active":"2024-05-21T15:14:18.269047+0000","last_peered":"2024-05-21T15:14:18.269047+0000","last_clean":"2024-05-21T15:14:18.269047+0000","last_became_active":"2024-05-21T15:14:11.574041+0000","last_became_peered":"2024-05-21T15:14:11.574041+0000","last_unstale":"2024-05-21T15:14:18.269047+0000","last_undegraded":"2024-05-21T15:14:18.269047+0000","last_fullsized":"2024-05-21T15:14:18.269047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.563218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560464+0000","last_change":"2024-05-21T15:14:12.257886+0000","last_active":"2024-05-21T15:14:13.560464+0000","last_peered":"2024-05-21T15:14:13.560464+0000","last_clean":"2024-05-21T15:14:13.560464+0000","last_became_active":"2024-05-21T15:14:12.257546+0000","last_became_peered":"2024-05-21T15:14:12.257546+0000","last_unstale":"2024-05-21T15:14:13.560464+0000","last_undegraded":"2024-05-21T15:14:13.560464+0000","last_fullsized":"2024-05-21T15:14:13.560464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:20.953421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.573235+0000","last_change":"2024-05-21T15:14:11.568524+0000","last_active":"2024-05-21T15:14:14.573235+0000","last_peered":"2024-05-21T15:14:14.573235+0000","last_clean":"2024-05-21T15:14:14.573235+0000","last_became_active":"2024-05-21T15:14:11.567981+0000","last_became_peered":"2024-05-21T15:14:11.567981+0000","last_unstale":"2024-05-21T15:14:14.573235+0000","last_undegraded":"2024-05-21T15:14:14.573235+0000","last_fullsized":"2024-05-21T15:14:14.573235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375730+0000","last_change":"2024-05-21T15:13:02.249260+0000","last_active":"2024-05-21T15:14:14.375730+0000","last_peered":"2024-05-21T15:14:14.375730+0000","last_clean":"2024-05-21T15:14:14.375730+0000","last_became_active":"2024-05-21T15:13:02.248637+0000","last_became_peered":"2024-05-21T15:13:02.248637+0000","last_unstale":"2024-05-21T15:14:14.375730+0000","last_undegraded":"2024-05-21T15:14:14.375730+0000","last_fullsized":"2024-05-21T15:14:14.375730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_clean_scrub_stamp":"2024-05-21T15:09:41.506463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:26.250910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236416+0000","last_change":"2024-05-21T15:14:12.259861+0000","last_active":"2024-05-21T15:14:13.236416+0000","last_peered":"2024-05-21T15:14:13.236416+0000","last_clean":"2024-05-21T15:14:13.236416+0000","last_became_active":"2024-05-21T15:14:12.259510+0000","last_became_peered":"2024-05-21T15:14:12.259510+0000","last_unstale":"2024-05-21T15:14:13.236416+0000","last_undegraded":"2024-05-21T15:14:13.236416+0000","last_fullsized":"2024-05-21T15:14:13.236416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:49.548366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:15.586462+0000","last_change":"2024-05-21T15:14:11.570705+0000","last_active":"2024-05-21T15:14:15.586462+0000","last_peered":"2024-05-21T15:14:15.586462+0000","last_clean":"2024-05-21T15:14:15.586462+0000","last_became_active":"2024-05-21T15:14:11.570403+0000","last_became_peered":"2024-05-21T15:14:11.570403+0000","last_unstale":"2024-05-21T15:14:15.586462+0000","last_undegraded":"2024-05-21T15:14:15.586462+0000","last_fullsized":"2024-05-21T15:14:15.586462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:02:34.156258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.277803+0000","last_change":"2024-05-21T15:14:11.572311+0000","last_active":"2024-05-21T15:14:16.277803+0000","last_peered":"2024-05-21T15:14:16.277803+0000","last_clean":"2024-05-21T15:14:16.277803+0000","last_became_active":"2024-05-21T15:14:11.571949+0000","last_became_peered":"2024-05-21T15:14:11.571949+0000","last_unstale":"2024-05-21T15:14:16.277803+0000","last_undegraded":"2024-05-21T15:14:16.277803+0000","last_fullsized":"2024-05-21T15:14:16.277803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:57:59.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236837+0000","last_change":"2024-05-21T15:14:12.260717+0000","last_active":"2024-05-21T15:14:13.236837+0000","last_peered":"2024-05-21T15:14:13.236837+0000","last_clean":"2024-05-21T15:14:13.236837+0000","last_became_active":"2024-05-21T15:14:12.259911+0000","last_became_peered":"2024-05-21T15:14:12.259911+0000","last_unstale":"2024-05-21T15:14:13.236837+0000","last_undegraded":"2024-05-21T15:14:13.236837+0000","last_fullsized":"2024-05-21T15:14:13.236837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:02.609299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.559573+0000","last_change":"2024-05-21T15:14:11.568061+0000","last_active":"2024-05-21T15:14:13.559573+0000","last_peered":"2024-05-21T15:14:13.559573+0000","last_clean":"2024-05-21T15:14:13.559573+0000","last_became_active":"2024-05-21T15:14:11.567678+0000","last_became_peered":"2024-05-21T15:14:11.567678+0000","last_unstale":"2024-05-21T15:14:13.559573+0000","last_undegraded":"2024-05-21T15:14:13.559573+0000","last_fullsized":"2024-05-21T15:14:13.559573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:07:15.902983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560355+0000","last_change":"2024-05-21T15:14:12.253634+0000","last_active":"2024-05-21T15:14:13.560355+0000","last_peered":"2024-05-21T15:14:13.560355+0000","last_clean":"2024-05-21T15:14:13.560355+0000","last_became_active":"2024-05-21T15:14:12.253284+0000","last_became_peered":"2024-05-21T15:14:12.253284+0000","last_unstale":"2024-05-21T15:14:13.560355+0000","last_undegraded":"2024-05-21T15:14:13.560355+0000","last_fullsized":"2024-05-21T15:14:13.560355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:28:07.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.572074+0000","last_change":"2024-05-21T15:14:11.574147+0000","last_active":"2024-05-21T15:14:14.572074+0000","last_peered":"2024-05-21T15:14:14.572074+0000","last_clean":"2024-05-21T15:14:14.572074+0000","last_became_active":"2024-05-21T15:14:11.573721+0000","last_became_peered":"2024-05-21T15:14:11.573721+0000","last_unstale":"2024-05-21T15:14:14.572074+0000","last_undegraded":"2024-05-21T15:14:14.572074+0000","last_fullsized":"2024-05-21T15:14:14.572074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:10.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375431+0000","last_change":"2024-05-21T15:14:12.258632+0000","last_active":"2024-05-21T15:14:14.375431+0000","last_peered":"2024-05-21T15:14:14.375431+0000","last_clean":"2024-05-21T15:14:14.375431+0000","last_became_active":"2024-05-21T15:14:12.257908+0000","last_became_peered":"2024-05-21T15:14:12.257908+0000","last_unstale":"2024-05-21T15:14:14.375431+0000","last_undegraded":"2024-05-21T15:14:14.375431+0000","last_fullsized":"2024-05-21T15:14:14.375431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:12.062387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.572437+0000","last_change":"2024-05-21T15:14:11.562592+0000","last_active":"2024-05-21T15:14:14.572437+0000","last_peered":"2024-05-21T15:14:14.572437+0000","last_clean":"2024-05-21T15:14:14.572437+0000","last_became_active":"2024-05-21T15:14:11.562265+0000","last_became_peered":"2024-05-21T15:14:11.562265+0000","last_unstale":"2024-05-21T15:14:14.572437+0000","last_undegraded":"2024-05-21T15:14:14.572437+0000","last_fullsized":"2024-05-21T15:14:14.572437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:24.662576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.484336+0000","last_change":"2024-05-21T15:14:12.249207+0000","last_active":"2024-05-21T15:14:13.484336+0000","last_peered":"2024-05-21T15:14:13.484336+0000","last_clean":"2024-05-21T15:14:13.484336+0000","last_became_active":"2024-05-21T15:14:12.248819+0000","last_became_peered":"2024-05-21T15:14:12.248819+0000","last_unstale":"2024-05-21T15:14:13.484336+0000","last_undegraded":"2024-05-21T15:14:13.484336+0000","last_fullsized":"2024-05-21T15:14:13.484336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:38.208636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.573818+0000","last_change":"2024-05-21T15:14:11.572044+0000","last_active":"2024-05-21T15:14:14.573818+0000","last_peered":"2024-05-21T15:14:14.573818+0000","last_clean":"2024-05-21T15:14:14.573818+0000","last_became_active":"2024-05-21T15:14:11.570721+0000","last_became_peered":"2024-05-21T15:14:11.570721+0000","last_unstale":"2024-05-21T15:14:14.573818+0000","last_undegraded":"2024-05-21T15:14:14.573818+0000","last_fullsized":"2024-05-21T15:14:14.573818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:34:08.413781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.257417+0000","last_change":"2024-05-21T15:14:12.256327+0000","last_active":"2024-05-21T15:14:12.257417+0000","last_peered":"2024-05-21T15:14:12.257417+0000","last_clean":"2024-05-21T15:14:12.257417+0000","last_became_active":"2024-05-21T15:14:12.255962+0000","last_became_peered":"2024-05-21T15:14:12.255962+0000","last_unstale":"2024-05-21T15:14:12.257417+0000","last_undegraded":"2024-05-21T15:14:12.257417+0000","last_fullsized":"2024-05-21T15:14:12.257417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:55.708072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238226+0000","last_change":"2024-05-21T15:14:11.571062+0000","last_active":"2024-05-21T15:14:13.238226+0000","last_peered":"2024-05-21T15:14:13.238226+0000","last_clean":"2024-05-21T15:14:13.238226+0000","last_became_active":"2024-05-21T15:14:11.570739+0000","last_became_peered":"2024-05-21T15:14:11.570739+0000","last_unstale":"2024-05-21T15:14:13.238226+0000","last_undegraded":"2024-05-21T15:14:13.238226+0000","last_fullsized":"2024-05-21T15:14:13.238226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:00:45.895992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560264+0000","last_change":"2024-05-21T15:14:12.256993+0000","last_active":"2024-05-21T15:14:13.560264+0000","last_peered":"2024-05-21T15:14:13.560264+0000","last_clean":"2024-05-21T15:14:13.560264+0000","last_became_active":"2024-05-21T15:14:12.256179+0000","last_became_peered":"2024-05-21T15:14:12.256179+0000","last_unstale":"2024-05-21T15:14:13.560264+0000","last_undegraded":"2024-05-21T15:14:13.560264+0000","last_fullsized":"2024-05-21T15:14:13.560264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:01:09.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.573217+0000","last_change":"2024-05-21T15:14:11.569688+0000","last_active":"2024-05-21T15:14:14.573217+0000","last_peered":"2024-05-21T15:14:14.573217+0000","last_clean":"2024-05-21T15:14:14.573217+0000","last_became_active":"2024-05-21T15:14:11.569341+0000","last_became_peered":"2024-05-21T15:14:11.569341+0000","last_unstale":"2024-05-21T15:14:14.573217+0000","last_undegraded":"2024-05-21T15:14:14.573217+0000","last_fullsized":"2024-05-21T15:14:14.573217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:10:26.155956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376444+0000","last_change":"2024-05-21T15:14:12.260506+0000","last_active":"2024-05-21T15:14:14.376444+0000","last_peered":"2024-05-21T15:14:14.376444+0000","last_clean":"2024-05-21T15:14:14.376444+0000","last_became_active":"2024-05-21T15:14:12.260168+0000","last_became_peered":"2024-05-21T15:14:12.260168+0000","last_unstale":"2024-05-21T15:14:14.376444+0000","last_undegraded":"2024-05-21T15:14:14.376444+0000","last_fullsized":"2024-05-21T15:14:14.376444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:11.321622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238460+0000","last_change":"2024-05-21T15:14:11.568630+0000","last_active":"2024-05-21T15:14:13.238460+0000","last_peered":"2024-05-21T15:14:13.238460+0000","last_clean":"2024-05-21T15:14:13.238460+0000","last_became_active":"2024-05-21T15:14:11.568229+0000","last_became_peered":"2024-05-21T15:14:11.568229+0000","last_unstale":"2024-05-21T15:14:13.238460+0000","last_undegraded":"2024-05-21T15:14:13.238460+0000","last_fullsized":"2024-05-21T15:14:13.238460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:21.372395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376711+0000","last_change":"2024-05-21T15:14:12.259408+0000","last_active":"2024-05-21T15:14:14.376711+0000","last_peered":"2024-05-21T15:14:14.376711+0000","last_clean":"2024-05-21T15:14:14.376711+0000","last_became_active":"2024-05-21T15:14:12.259082+0000","last_became_peered":"2024-05-21T15:14:12.259082+0000","last_unstale":"2024-05-21T15:14:14.376711+0000","last_undegraded":"2024-05-21T15:14:14.376711+0000","last_fullsized":"2024-05-21T15:14:14.376711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:32:09.059307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:16.277273+0000","last_change":"2024-05-21T15:14:11.571077+0000","last_active":"2024-05-21T15:14:16.277273+0000","last_peered":"2024-05-21T15:14:16.277273+0000","last_clean":"2024-05-21T15:14:16.277273+0000","last_became_active":"2024-05-21T15:14:11.570664+0000","last_became_peered":"2024-05-21T15:14:11.570664+0000","last_unstale":"2024-05-21T15:14:16.277273+0000","last_undegraded":"2024-05-21T15:14:16.277273+0000","last_fullsized":"2024-05-21T15:14:16.277273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:37:16.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376494+0000","last_change":"2024-05-21T15:14:12.252972+0000","last_active":"2024-05-21T15:14:14.376494+0000","last_peered":"2024-05-21T15:14:14.376494+0000","last_clean":"2024-05-21T15:14:14.376494+0000","last_became_active":"2024-05-21T15:14:12.252591+0000","last_became_peered":"2024-05-21T15:14:12.252591+0000","last_unstale":"2024-05-21T15:14:14.376494+0000","last_undegraded":"2024-05-21T15:14:14.376494+0000","last_fullsized":"2024-05-21T15:14:14.376494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:14:16.430371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:17.281878+0000","last_change":"2024-05-21T15:14:11.577138+0000","last_active":"2024-05-21T15:14:17.281878+0000","last_peered":"2024-05-21T15:14:17.281878+0000","last_clean":"2024-05-21T15:14:17.281878+0000","last_became_active":"2024-05-21T15:14:11.576629+0000","last_became_peered":"2024-05-21T15:14:11.576629+0000","last_unstale":"2024-05-21T15:14:17.281878+0000","last_undegraded":"2024-05-21T15:14:17.281878+0000","last_fullsized":"2024-05-21T15:14:17.281878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:20:20.134764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.376716+0000","last_change":"2024-05-21T15:14:12.256174+0000","last_active":"2024-05-21T15:14:14.376716+0000","last_peered":"2024-05-21T15:14:14.376716+0000","last_clean":"2024-05-21T15:14:14.376716+0000","last_became_active":"2024-05-21T15:14:12.255839+0000","last_became_peered":"2024-05-21T15:14:12.255839+0000","last_unstale":"2024-05-21T15:14:14.376716+0000","last_undegraded":"2024-05-21T15:14:14.376716+0000","last_fullsized":"2024-05-21T15:14:14.376716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:12:40.743118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.257627+0000","last_change":"2024-05-21T15:14:12.257028+0000","last_active":"2024-05-21T15:14:12.257627+0000","last_peered":"2024-05-21T15:14:12.257627+0000","last_clean":"2024-05-21T15:14:12.257627+0000","last_became_active":"2024-05-21T15:14:12.256611+0000","last_became_peered":"2024-05-21T15:14:12.256611+0000","last_unstale":"2024-05-21T15:14:12.257627+0000","last_undegraded":"2024-05-21T15:14:12.257627+0000","last_fullsized":"2024-05-21T15:14:12.257627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.833769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238381+0000","last_change":"2024-05-21T15:14:12.255724+0000","last_active":"2024-05-21T15:14:13.238381+0000","last_peered":"2024-05-21T15:14:13.238381+0000","last_clean":"2024-05-21T15:14:13.238381+0000","last_became_active":"2024-05-21T15:14:12.255272+0000","last_became_peered":"2024-05-21T15:14:12.255272+0000","last_unstale":"2024-05-21T15:14:13.238381+0000","last_undegraded":"2024-05-21T15:14:13.238381+0000","last_fullsized":"2024-05-21T15:14:13.238381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:07:07.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238350+0000","last_change":"2024-05-21T15:14:12.262508+0000","last_active":"2024-05-21T15:14:13.238350+0000","last_peered":"2024-05-21T15:14:13.238350+0000","last_clean":"2024-05-21T15:14:13.238350+0000","last_became_active":"2024-05-21T15:14:12.262155+0000","last_became_peered":"2024-05-21T15:14:12.262155+0000","last_unstale":"2024-05-21T15:14:13.238350+0000","last_undegraded":"2024-05-21T15:14:13.238350+0000","last_fullsized":"2024-05-21T15:14:13.238350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:54:19.982044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375339+0000","last_change":"2024-05-21T15:14:12.251715+0000","last_active":"2024-05-21T15:14:14.375339+0000","last_peered":"2024-05-21T15:14:14.375339+0000","last_clean":"2024-05-21T15:14:14.375339+0000","last_became_active":"2024-05-21T15:14:12.251337+0000","last_became_peered":"2024-05-21T15:14:12.251337+0000","last_unstale":"2024-05-21T15:14:14.375339+0000","last_undegraded":"2024-05-21T15:14:14.375339+0000","last_fullsized":"2024-05-21T15:14:14.375339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:03.562179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236674+0000","last_change":"2024-05-21T15:14:12.253166+0000","last_active":"2024-05-21T15:14:13.236674+0000","last_peered":"2024-05-21T15:14:13.236674+0000","last_clean":"2024-05-21T15:14:13.236674+0000","last_became_active":"2024-05-21T15:14:12.251830+0000","last_became_peered":"2024-05-21T15:14:12.251830+0000","last_unstale":"2024-05-21T15:14:13.236674+0000","last_undegraded":"2024-05-21T15:14:13.236674+0000","last_fullsized":"2024-05-21T15:14:13.236674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:11:58.679568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.257387+0000","last_change":"2024-05-21T15:14:12.256131+0000","last_active":"2024-05-21T15:14:12.257387+0000","last_peered":"2024-05-21T15:14:12.257387+0000","last_clean":"2024-05-21T15:14:12.257387+0000","last_became_active":"2024-05-21T15:14:12.255772+0000","last_became_peered":"2024-05-21T15:14:12.255772+0000","last_unstale":"2024-05-21T15:14:12.257387+0000","last_undegraded":"2024-05-21T15:14:12.257387+0000","last_fullsized":"2024-05-21T15:14:12.257387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:18:59.154936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.238543+0000","last_change":"2024-05-21T15:14:12.260829+0000","last_active":"2024-05-21T15:14:13.238543+0000","last_peered":"2024-05-21T15:14:13.238543+0000","last_clean":"2024-05-21T15:14:13.238543+0000","last_became_active":"2024-05-21T15:14:12.259686+0000","last_became_peered":"2024-05-21T15:14:12.259686+0000","last_unstale":"2024-05-21T15:14:13.238543+0000","last_undegraded":"2024-05-21T15:14:13.238543+0000","last_fullsized":"2024-05-21T15:14:13.238543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:48.480814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560225+0000","last_change":"2024-05-21T15:14:12.262892+0000","last_active":"2024-05-21T15:14:13.560225+0000","last_peered":"2024-05-21T15:14:13.560225+0000","last_clean":"2024-05-21T15:14:13.560225+0000","last_became_active":"2024-05-21T15:14:12.262532+0000","last_became_peered":"2024-05-21T15:14:12.262532+0000","last_unstale":"2024-05-21T15:14:13.560225+0000","last_undegraded":"2024-05-21T15:14:13.560225+0000","last_fullsized":"2024-05-21T15:14:13.560225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:22.424783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:14.375404+0000","last_change":"2024-05-21T15:14:12.250186+0000","last_active":"2024-05-21T15:14:14.375404+0000","last_peered":"2024-05-21T15:14:14.375404+0000","last_clean":"2024-05-21T15:14:14.375404+0000","last_became_active":"2024-05-21T15:14:12.249828+0000","last_became_peered":"2024-05-21T15:14:12.249828+0000","last_unstale":"2024-05-21T15:14:14.375404+0000","last_undegraded":"2024-05-21T15:14:14.375404+0000","last_fullsized":"2024-05-21T15:14:14.375404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:58:11.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.560105+0000","last_change":"2024-05-21T15:14:12.259845+0000","last_active":"2024-05-21T15:14:13.560105+0000","last_peered":"2024-05-21T15:14:13.560105+0000","last_clean":"2024-05-21T15:14:13.560105+0000","last_became_active":"2024-05-21T15:14:12.259455+0000","last_became_peered":"2024-05-21T15:14:12.259455+0000","last_unstale":"2024-05-21T15:14:13.560105+0000","last_undegraded":"2024-05-21T15:14:13.560105+0000","last_fullsized":"2024-05-21T15:14:13.560105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:24:59.655178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.237600+0000","last_change":"2024-05-21T15:14:12.259713+0000","last_active":"2024-05-21T15:14:13.237600+0000","last_peered":"2024-05-21T15:14:13.237600+0000","last_clean":"2024-05-21T15:14:13.237600+0000","last_became_active":"2024-05-21T15:14:12.259057+0000","last_became_peered":"2024-05-21T15:14:12.259057+0000","last_unstale":"2024-05-21T15:14:13.237600+0000","last_undegraded":"2024-05-21T15:14:13.237600+0000","last_fullsized":"2024-05-21T15:14:13.237600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:46:22.377652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-21T15:14:13.236599+0000","last_change":"2024-05-21T15:14:12.254389+0000","last_active":"2024-05-21T15:14:13.236599+0000","last_peered":"2024-05-21T15:14:13.236599+0000","last_clean":"2024-05-21T15:14:13.236599+0000","last_became_active":"2024-05-21T15:14:12.253846+0000","last_became_peered":"2024-05-21T15:14:12.253846+0000","last_unstale":"2024-05-21T15:14:13.236599+0000","last_undegraded":"2024-05-21T15:14:13.236599+0000","last_fullsized":"2024-05-21T15:14:13.236599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:32.932316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-21T15:14:12.252837+0000","last_change":"2024-05-21T15:14:12.252292+0000","last_active":"2024-05-21T15:14:12.252837+0000","last_peered":"2024-05-21T15:14:12.252837+0000","last_clean":"2024-05-21T15:14:12.252837+0000","last_became_active":"2024-05-21T15:14:12.251816+0000","last_became_peered":"2024-05-21T15:14:12.251816+0000","last_unstale":"2024-05-21T15:14:12.252837+0000","last_undegraded":"2024-05-21T15:14:12.252837+0000","last_fullsized":"2024-05-21T15:14:12.252837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:58:51.769621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41774,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":34,"num_read_kb":63,"num_write":146,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":53611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1068935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1031744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":440789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":438625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920815,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1556480,"data_stored":1032217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":39929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:14:19.446 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-21T15:14:19.447 INFO:tasks.cephadm:Setting up client nodes... 2024-05-21T15:14:19.447 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-21T15:14:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T15:14:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] up:active 2024-05-21T15:14:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-05-21T15:14:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:14:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/284580085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:19.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:19 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2006450778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:19.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:14:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-21T15:14:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] up:active 2024-05-21T15:14:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-05-21T15:14:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:14:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/284580085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:14:19.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:19 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2006450778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:19.714 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:20.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:20 smithi078 ceph-mon[31997]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:14:20.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:20 smithi078 ceph-mon[31997]: pgmap v287: 97 pgs: 97 active+clean; 617 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-05-21T15:14:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:20 smithi084 ceph-mon[36334]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:14:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:20 smithi084 ceph-mon[36334]: pgmap v287: 97 pgs: 97 active+clean; 617 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-05-21T15:14:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:20 smithi172 ceph-mon[36402]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:14:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:20 smithi172 ceph-mon[36402]: pgmap v287: 97 pgs: 97 active+clean; 617 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 12 KiB/s wr, 23 op/s 2024-05-21T15:14:21.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.271+0000 7f79dd9b8700 1 -- 172.21.15.78:0/4186452639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d8103b90 msgr2=0x7f79d810a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.271+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/4186452639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d8103b90 0x7f79d810a440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b0a0 tx=0x7f79d000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.271+0000 7f79dd9b8700 1 -- 172.21.15.78:0/4186452639 shutdown_connections 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.271+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/4186452639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d8103b90 0x7f79d810a440 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.271+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/4186452639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d81020c0 0x7f79d81024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.271+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/4186452639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79d81031f0 0x7f79d8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.271+0000 7f79dd9b8700 1 -- 172.21.15.78:0/4186452639 >> 172.21.15.78:0/4186452639 conn(0x7f79d80fd700 msgr2=0x7f79d80ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.272+0000 7f79dd9b8700 1 -- 172.21.15.78:0/4186452639 shutdown_connections 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.272+0000 7f79dd9b8700 1 -- 172.21.15.78:0/4186452639 wait complete. 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.272+0000 7f79dd9b8700 1 Processor -- start 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.272+0000 7f79dd9b8700 1 -- start start 2024-05-21T15:14:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.272+0000 7f79dd9b8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d81020c0 0x7f79d81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.272+0000 7f79dd9b8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d81031f0 0x7f79d8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79dd9b8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79d8103b90 0x7f79d819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79dd9b8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810dc90 con 0x7f79d81031f0 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79dd9b8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810db10 con 0x7f79d81020c0 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79dd9b8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810de10 con 0x7f79d8103b90 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d81031f0 0x7f79d8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d81031f0 0x7f79d8198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50816/0 (socket says 172.21.15.78:50816) 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 -- 172.21.15.78:0/3987028616 learned_addr learned my addr 172.21.15.78:0/3987028616 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79d6ffd700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d81020c0 0x7f79d81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79d77fe700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79d8103b90 0x7f79d819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 -- 172.21.15.78:0/3987028616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79d8103b90 msgr2=0x7f79d819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79d8103b90 0x7f79d819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 -- 172.21.15.78:0/3987028616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d81020c0 msgr2=0x7f79d81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d81020c0 0x7f79d81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 -- 172.21.15.78:0/3987028616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d81a0250 con 0x7f79d81031f0 2024-05-21T15:14:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79cffff700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d81031f0 0x7f79d8198c30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f79c0007b50 tx=0x7f79c0007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79dc9b6700 1 -- 172.21.15.78:0/3987028616 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0019020 con 0x7f79d81031f0 2024-05-21T15:14:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d806a4f0 con 0x7f79d81031f0 2024-05-21T15:14:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79dc9b6700 1 -- 172.21.15.78:0/3987028616 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c000ce50 con 0x7f79d81031f0 2024-05-21T15:14:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.273+0000 7f79dc9b6700 1 -- 172.21.15.78:0/3987028616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c000e580 con 0x7f79d81031f0 2024-05-21T15:14:21.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.274+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d806aa80 con 0x7f79d81031f0 2024-05-21T15:14:21.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.275+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d8050170 con 0x7f79d81031f0 2024-05-21T15:14:21.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.276+0000 7f79dc9b6700 1 -- 172.21.15.78:0/3987028616 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f79c0005000 con 0x7f79d81031f0 2024-05-21T15:14:21.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.276+0000 7f79dc9b6700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79c40a2cb0 0x7f79c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:21.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.277+0000 7f79d6ffd700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79c40a2cb0 0x7f79c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:21.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.277+0000 7f79dc9b6700 1 -- 172.21.15.78:0/3987028616 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f79c00ce240 con 0x7f79d81031f0 2024-05-21T15:14:21.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.277+0000 7f79d6ffd700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79c40a2cb0 0x7f79c40a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f79d81030f0 tx=0x7f79c800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:21.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.281+0000 7f79dc9b6700 1 -- 172.21.15.78:0/3987028616 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f79c0023020 con 0x7f79d81031f0 2024-05-21T15:14:21.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.566+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f79d8108f20 con 0x7f79d81031f0 2024-05-21T15:14:21.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:21 smithi078 ceph-mon[31997]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:14:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.571+0000 7f79dc9b6700 1 -- 172.21.15.78:0/3987028616 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f79c009b280 con 0x7f79d81031f0 2024-05-21T15:14:21.574 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-05-21T15:14:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBNukxm24LTIRAADiQKmwz8PZyJYQT2LZUJNA== 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79c40a2cb0 msgr2=0x7f79c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79c40a2cb0 0x7f79c40a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f79d81030f0 tx=0x7f79c800a380 comp rx=0 tx=0).stop 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d81031f0 msgr2=0x7f79d8198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d81031f0 0x7f79d8198c30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f79c0007b50 tx=0x7f79c0007e60 comp rx=0 tx=0).stop 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 shutdown_connections 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79c40a2cb0 0x7f79c40a51c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79d8103b90 0x7f79d819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79d81020c0 0x7f79d81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 --2- 172.21.15.78:0/3987028616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79d81031f0 0x7f79d8198c30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 >> 172.21.15.78:0/3987028616 conn(0x7f79d80fd700 msgr2=0x7f79d8106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 shutdown_connections 2024-05-21T15:14:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:21.573+0000 7f79dd9b8700 1 -- 172.21.15.78:0/3987028616 wait complete. 2024-05-21T15:14:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:21 smithi084 ceph-mon[36334]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:14:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:21 smithi172 ceph-mon[36402]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:14:22.267 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:14:22.268 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-21T15:14:22.268 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-21T15:14:22.307 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-21T15:14:22.307 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-21T15:14:22.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-21T15:14:22.586 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:22.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:22 smithi078 ceph-mon[31997]: pgmap v288: 97 pgs: 97 active+clean; 619 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-05-21T15:14:22.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:22 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3987028616' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T15:14:22.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:22 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3987028616' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T15:14:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:22 smithi084 ceph-mon[36334]: pgmap v288: 97 pgs: 97 active+clean; 619 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-05-21T15:14:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:22 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3987028616' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T15:14:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:22 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3987028616' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T15:14:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:22 smithi172 ceph-mon[36402]: pgmap v288: 97 pgs: 97 active+clean; 619 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-05-21T15:14:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:22 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3987028616' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T15:14:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:22 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3987028616' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T15:14:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 -- 172.21.15.78:0/2952216091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101fb0 msgr2=0x7f953410ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 --2- 172.21.15.78:0/2952216091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101fb0 0x7f953410ffa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9524009990 tx=0x7f9524009ca0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 -- 172.21.15.78:0/2952216091 shutdown_connections 2024-05-21T15:14:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 --2- 172.21.15.78:0/2952216091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101fb0 0x7f953410ffa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 --2- 172.21.15.78:0/2952216091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9534109260 0x7f9534109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 --2- 172.21.15.78:0/2952216091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9534101660 0x7f9534101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 -- 172.21.15.78:0/2952216091 >> 172.21.15.78:0/2952216091 conn(0x7f95340fd570 msgr2=0x7f95340ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 -- 172.21.15.78:0/2952216091 shutdown_connections 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.108+0000 7f9539bd9700 1 -- 172.21.15.78:0/2952216091 wait complete. 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9539bd9700 1 Processor -- start 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9539bd9700 1 -- start start 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9539bd9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101660 0x7f953419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9539bd9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9534101fb0 0x7f953419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9539bd9700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9534109260 0x7f95341a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9539bd9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95341134c0 con 0x7f9534101660 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9539bd9700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9534113340 con 0x7f9534101fb0 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9539bd9700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9534113640 con 0x7f9534109260 2024-05-21T15:14:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f95337fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101660 0x7f953419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f95337fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101660 0x7f953419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50836/0 (socket says 172.21.15.78:50836) 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f95337fe700 1 -- 172.21.15.78:0/157546521 learned_addr learned my addr 172.21.15.78:0/157546521 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9532ffd700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9534101fb0 0x7f953419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.109+0000 7f9533fff700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9534109260 0x7f95341a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f95337fe700 1 -- 172.21.15.78:0/157546521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9534109260 msgr2=0x7f95341a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f95337fe700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9534109260 0x7f95341a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f95337fe700 1 -- 172.21.15.78:0/157546521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9534101fb0 msgr2=0x7f953419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f95337fe700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9534101fb0 0x7f953419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f95337fe700 1 -- 172.21.15.78:0/157546521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95341a0e80 con 0x7f9534101660 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f9533fff700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9534109260 0x7f95341a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f9532ffd700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9534101fb0 0x7f953419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f95337fe700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101660 0x7f953419bfb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f952c0060b0 tx=0x7f952c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f9530ff9700 1 -- 172.21.15.78:0/157546521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c01a070 con 0x7f9534101660 2024-05-21T15:14:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f9530ff9700 1 -- 172.21.15.78:0/157546521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f952c00de20 con 0x7f9534101660 2024-05-21T15:14:24.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95341a1170 con 0x7f9534101660 2024-05-21T15:14:24.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f9530ff9700 1 -- 172.21.15.78:0/157546521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c014790 con 0x7f9534101660 2024-05-21T15:14:24.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.110+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341a8a90 con 0x7f9534101660 2024-05-21T15:14:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.111+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9534050170 con 0x7f9534101660 2024-05-21T15:14:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.112+0000 7f9530ff9700 1 -- 172.21.15.78:0/157546521 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f952c01d070 con 0x7f9534101660 2024-05-21T15:14:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.113+0000 7f9530ff9700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f95200a2cb0 0x7f95200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.113+0000 7f9532ffd700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f95200a2cb0 0x7f95200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.113+0000 7f9530ff9700 1 -- 172.21.15.78:0/157546521 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f952c0cee10 con 0x7f9534101660 2024-05-21T15:14:24.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.113+0000 7f9532ffd700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f95200a2cb0 0x7f95200a51c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9534100400 tx=0x7f951c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:24.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.116+0000 7f9530ff9700 1 -- 172.21.15.78:0/157546521 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f952c010070 con 0x7f9534101660 2024-05-21T15:14:24.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.371+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f95341a8c60 con 0x7f9534101660 2024-05-21T15:14:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.376+0000 7f9530ff9700 1 -- 172.21.15.78:0/157546521 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7f952c09bdc0 con 0x7f9534101660 2024-05-21T15:14:24.382 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:24.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f95200a2cb0 msgr2=0x7f95200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f95200a2cb0 0x7f95200a51c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9534100400 tx=0x7f951c008040 comp rx=0 tx=0).stop 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101660 msgr2=0x7f953419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101660 0x7f953419bfb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f952c0060b0 tx=0x7f952c007f90 comp rx=0 tx=0).stop 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 shutdown_connections 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f95200a2cb0 0x7f95200a51c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9534109260 0x7f95341a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9534101660 0x7f953419bfb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 --2- 172.21.15.78:0/157546521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9534101fb0 0x7f953419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 >> 172.21.15.78:0/157546521 conn(0x7f95340fd570 msgr2=0x7f953410f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 shutdown_connections 2024-05-21T15:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:24.379+0000 7f9539bd9700 1 -- 172.21.15.78:0/157546521 wait complete. 2024-05-21T15:14:24.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:24 smithi078 ceph-mon[31997]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-05-21T15:14:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:24 smithi084 ceph-mon[36334]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-05-21T15:14:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:24 smithi172 ceph-mon[36402]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-05-21T15:14:25.040 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2228258510},{"type":"v1","addr":"172.21.15.78:6801","nonce":2228258510}]},"active_addr":"172.21.15.78:6801/2228258510","active_change":"2024-05-21T15:07:15.158407+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.78:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2012910515}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2604499013}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2194549734}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":3526992601}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":3746961366}]}]} 2024-05-21T15:14:25.050 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-21T15:14:25.051 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-21T15:14:25.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T15:14:25.292 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:25.321 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:25 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/157546521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T15:14:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:25 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/157546521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T15:14:25.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:25 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/157546521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T15:14:26.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:26 smithi078 ceph-mon[31997]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.9 KiB/s wr, 27 op/s 2024-05-21T15:14:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:26 smithi084 ceph-mon[36334]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.9 KiB/s wr, 27 op/s 2024-05-21T15:14:26.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:26 smithi172 ceph-mon[36402]: pgmap v290: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.9 KiB/s wr, 27 op/s 2024-05-21T15:14:26.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 -- 172.21.15.78:0/1113971972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc104c30 msgr2=0x7faadc10d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:26.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 --2- 172.21.15.78:0/1113971972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc104c30 0x7faadc10d180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faac4009990 tx=0x7faac4009ca0 comp rx=0 tx=0).stop 2024-05-21T15:14:26.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 -- 172.21.15.78:0/1113971972 shutdown_connections 2024-05-21T15:14:26.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 --2- 172.21.15.78:0/1113971972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc10d750 0x7faadc10fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:26.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 --2- 172.21.15.78:0/1113971972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faadc1042e0 0x7faadc1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:26.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 --2- 172.21.15.78:0/1113971972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc104c30 0x7faadc10d180 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:26.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 -- 172.21.15.78:0/1113971972 >> 172.21.15.78:0/1113971972 conn(0x7faadc0fdc40 msgr2=0x7faadc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:26.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 -- 172.21.15.78:0/1113971972 shutdown_connections 2024-05-21T15:14:26.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.839+0000 7faae1ece700 1 -- 172.21.15.78:0/1113971972 wait complete. 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.840+0000 7faae1ece700 1 Processor -- start 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faae1ece700 1 -- start start 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faae1ece700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc1042e0 0x7faadc199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faae1ece700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faadc104c30 0x7faadc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faae1ece700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc10d750 0x7faadc19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faae1ece700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faadc113480 con 0x7faadc10d750 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faae1ece700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faadc113300 con 0x7faadc104c30 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faae1ece700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faadc113600 con 0x7faadc1042e0 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc10d750 0x7faadc19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faadc104c30 0x7faadc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc10d750 0x7faadc19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50850/0 (socket says 172.21.15.78:50850) 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 -- 172.21.15.78:0/1609225362 learned_addr learned my addr 172.21.15.78:0/1609225362 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadb7fe700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc1042e0 0x7faadc199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 -- 172.21.15.78:0/1609225362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc1042e0 msgr2=0x7faadc199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc1042e0 0x7faadc199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 -- 172.21.15.78:0/1609225362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faadc104c30 msgr2=0x7faadc19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faadc104c30 0x7faadc19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 -- 172.21.15.78:0/1609225362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc19ec20 con 0x7faadc10d750 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.841+0000 7faadbfff700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc10d750 0x7faadc19e580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faacc00b550 tx=0x7faacc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.842+0000 7faad8ff9700 1 -- 172.21.15.78:0/1609225362 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc015020 con 0x7faadc10d750 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.842+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc19ef10 con 0x7faadc10d750 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.842+0000 7faad8ff9700 1 -- 172.21.15.78:0/1609225362 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faacc011d10 con 0x7faadc10d750 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.842+0000 7faad8ff9700 1 -- 172.21.15.78:0/1609225362 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc010e00 con 0x7faadc10d750 2024-05-21T15:14:26.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.842+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc1a6830 con 0x7faadc10d750 2024-05-21T15:14:26.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.843+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faadc050170 con 0x7faadc10d750 2024-05-21T15:14:26.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.844+0000 7faad8ff9700 1 -- 172.21.15.78:0/1609225362 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faacc018020 con 0x7faadc10d750 2024-05-21T15:14:26.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.845+0000 7faad8ff9700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7faac80a2cb0 0x7faac80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:26.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.845+0000 7faad8ff9700 1 -- 172.21.15.78:0/1609225362 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7faacc0ceb10 con 0x7faadc10d750 2024-05-21T15:14:26.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.845+0000 7faadb7fe700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7faac80a2cb0 0x7faac80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:26.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.845+0000 7faadb7fe700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7faac80a2cb0 0x7faac80a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faad4009550 tx=0x7faad40091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:26.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:26.848+0000 7faad8ff9700 1 -- 172.21.15.78:0/1609225362 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faacc01f020 con 0x7faadc10d750 2024-05-21T15:14:27.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.044+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faadc19b0b0 con 0x7faadc10d750 2024-05-21T15:14:27.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.046+0000 7faad8ff9700 1 -- 172.21.15.78:0/1609225362 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22147 (secure 0 0 0) 0x7faadc19b0b0 con 0x7faadc10d750 2024-05-21T15:14:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:27.049 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:14:13.233294+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:14:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7faac80a2cb0 msgr2=0x7faac80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7faac80a2cb0 0x7faac80a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faad4009550 tx=0x7faad40091f0 comp rx=0 tx=0).stop 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc10d750 msgr2=0x7faadc19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc10d750 0x7faadc19e580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faacc00b550 tx=0x7faacc00b860 comp rx=0 tx=0).stop 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 shutdown_connections 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7faac80a2cb0 0x7faac80a51c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faadc10d750 0x7faadc19e580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faadc1042e0 0x7faadc199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 --2- 172.21.15.78:0/1609225362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faadc104c30 0x7faadc19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.048+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 >> 172.21.15.78:0/1609225362 conn(0x7faadc0fdc40 msgr2=0x7faadc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.049+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 shutdown_connections 2024-05-21T15:14:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:27.049+0000 7faae1ece700 1 -- 172.21.15.78:0/1609225362 wait complete. 2024-05-21T15:14:27.592 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:27 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1609225362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:27.620 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-21T15:14:27.620 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T15:14:27.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:27 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1609225362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:27.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:27 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1609225362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:27.863 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:28.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:28 smithi078 ceph-mon[31997]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-05-21T15:14:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:28 smithi084 ceph-mon[36334]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-05-21T15:14:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:28 smithi172 ceph-mon[36402]: pgmap v291: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-05-21T15:14:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/1264765856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c28107d80 msgr2=0x7f3c28108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/1264765856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c28107d80 0x7f3c28108190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c140099c0 tx=0x7f3c14009cd0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/1264765856 shutdown_connections 2024-05-21T15:14:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/1264765856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c281086d0 0x7f3c28110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/1264765856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c28103450 0x7f3c28103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/1264765856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c28107d80 0x7f3c28108190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/1264765856 >> 172.21.15.78:0/1264765856 conn(0x7f3c280fed20 msgr2=0x7f3c28101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:29.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/1264765856 shutdown_connections 2024-05-21T15:14:29.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/1264765856 wait complete. 2024-05-21T15:14:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.443+0000 7f3c2ec3d700 1 Processor -- start 2024-05-21T15:14:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2ec3d700 1 -- start start 2024-05-21T15:14:29.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2ec3d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c28103450 0x7f3c28075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2ec3d700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c28107d80 0x7f3c28075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2ec3d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c281086d0 0x7f3c280764d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2ec3d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281134a0 con 0x7f3c281086d0 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2ec3d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28113320 con 0x7f3c28103450 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2ec3d700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28113620 con 0x7f3c28107d80 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2d1da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c281086d0 0x7f3c280764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2c9d9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c28103450 0x7f3c28075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2d1da700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c281086d0 0x7f3c280764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33354/0 (socket says 172.21.15.78:33354) 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2c9d9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c28103450 0x7f3c28075a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:52174/0 (socket says 172.21.15.78:52174) 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2d1da700 1 -- 172.21.15.78:0/3844161455 learned_addr learned my addr 172.21.15.78:0/3844161455 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c27fff700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c28107d80 0x7f3c28075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:29.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2d1da700 1 -- 172.21.15.78:0/3844161455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c28107d80 msgr2=0x7f3c28075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2d1da700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c28107d80 0x7f3c28075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2d1da700 1 -- 172.21.15.78:0/3844161455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c28103450 msgr2=0x7f3c28075a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2d1da700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c28103450 0x7f3c28075a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2d1da700 1 -- 172.21.15.78:0/3844161455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c281a80c0 con 0x7f3c281086d0 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.444+0000 7f3c2c9d9700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c28103450 0x7f3c28075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.445+0000 7f3c2d1da700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c281086d0 0x7f3c280764d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000b970 tx=0x7f3c2000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.445+0000 7f3c25ffb700 1 -- 172.21.15.78:0/3844161455 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000d860 con 0x7f3c281086d0 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.445+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c281a83b0 con 0x7f3c281086d0 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.445+0000 7f3c25ffb700 1 -- 172.21.15.78:0/3844161455 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2000e430 con 0x7f3c281086d0 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.445+0000 7f3c25ffb700 1 -- 172.21.15.78:0/3844161455 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20017660 con 0x7f3c281086d0 2024-05-21T15:14:29.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.445+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c281a88c0 con 0x7f3c281086d0 2024-05-21T15:14:29.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.446+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c28050170 con 0x7f3c281086d0 2024-05-21T15:14:29.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.447+0000 7f3c25ffb700 1 -- 172.21.15.78:0/3844161455 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3c2000da00 con 0x7f3c281086d0 2024-05-21T15:14:29.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.448+0000 7f3c25ffb700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3c180a2d30 0x7f3c180a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:29.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.448+0000 7f3c2c9d9700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3c180a2d30 0x7f3c180a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:29.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.448+0000 7f3c25ffb700 1 -- 172.21.15.78:0/3844161455 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3c200ceb40 con 0x7f3c281086d0 2024-05-21T15:14:29.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.449+0000 7f3c2c9d9700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3c180a2d30 0x7f3c180a5240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00afd0 tx=0x7f3c1c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:29.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.453+0000 7f3c25ffb700 1 -- 172.21.15.78:0/3844161455 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c20013030 con 0x7f3c281086d0 2024-05-21T15:14:29.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.651+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c28076f80 con 0x7f3c281086d0 2024-05-21T15:14:29.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.652+0000 7f3c25ffb700 1 -- 172.21.15.78:0/3844161455 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22147 (secure 0 0 0) 0x7f3c2001e2f0 con 0x7f3c281086d0 2024-05-21T15:14:29.658 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:14:29.661 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":85,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:14:13.233294+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:14:29.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3c180a2d30 msgr2=0x7f3c180a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3c180a2d30 0x7f3c180a5240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00afd0 tx=0x7f3c1c00a380 comp rx=0 tx=0).stop 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c281086d0 msgr2=0x7f3c280764d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c281086d0 0x7f3c280764d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000b970 tx=0x7f3c2000bc80 comp rx=0 tx=0).stop 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 shutdown_connections 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f3c180a2d30 0x7f3c180a5240 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c281086d0 0x7f3c280764d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c28103450 0x7f3c28075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 --2- 172.21.15.78:0/3844161455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c28107d80 0x7f3c28075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:29.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 >> 172.21.15.78:0/3844161455 conn(0x7f3c280fed20 msgr2=0x7f3c281074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:29.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 shutdown_connections 2024-05-21T15:14:29.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:29.655+0000 7f3c2ec3d700 1 -- 172.21.15.78:0/3844161455 wait complete. 2024-05-21T15:14:30.318 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-21T15:14:30.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-21T15:14:30.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-21T15:14:30.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-21T15:14:30.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-21T15:14:30.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-21T15:14:30.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-21T15:14:30.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-21T15:14:30.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-21T15:14:30.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-21T15:14:30.323 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-21T15:14:30.323 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-21T15:14:30.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:30 smithi078 ceph-mon[31997]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-05-21T15:14:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:30 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3844161455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:30 smithi084 ceph-mon[36334]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-05-21T15:14:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:30 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3844161455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:30 smithi172 ceph-mon[36402]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-05-21T15:14:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:30 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3844161455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:14:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:31.369 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:31.371 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:31.372 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:31.374 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:31.377 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:31.378 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:31.781 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:32.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:32 smithi078 ceph-mon[31997]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-05-21T15:14:32.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:32 smithi084 ceph-mon[36334]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-05-21T15:14:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:32 smithi172 ceph-mon[36402]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-05-21T15:14:32.733 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:34.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:34 smithi078 ceph-mon[31997]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-21T15:14:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:34 smithi084 ceph-mon[36334]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-21T15:14:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:34 smithi172 ceph-mon[36402]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-21T15:14:36.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:36 smithi078 ceph-mon[31997]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-21T15:14:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:36 smithi084 ceph-mon[36334]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-21T15:14:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:36 smithi172 ceph-mon[36402]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-05-21T15:14:38.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:38 smithi078 ceph-mon[31997]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-21T15:14:38.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 -- 172.21.15.78:0/1871869308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 msgr2=0x7f47a0103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:38.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 --2- 172.21.15.78:0/1871869308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 0x7f47a0103860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47900099c0 tx=0x7f4790009cd0 comp rx=0 tx=0).stop 2024-05-21T15:14:38.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 -- 172.21.15.78:0/1871869308 shutdown_connections 2024-05-21T15:14:38.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 --2- 172.21.15.78:0/1871869308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a01086d0 0x7f47a0110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:38.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 --2- 172.21.15.78:0/1871869308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 0x7f47a0103860 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:38.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 --2- 172.21.15.78:0/1871869308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a0107d80 0x7f47a0108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:38.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 -- 172.21.15.78:0/1871869308 >> 172.21.15.78:0/1871869308 conn(0x7f47a00fed20 msgr2=0x7f47a0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:38.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 -- 172.21.15.78:0/1871869308 shutdown_connections 2024-05-21T15:14:38.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.625+0000 7f47a51e8700 1 -- 172.21.15.78:0/1871869308 wait complete. 2024-05-21T15:14:38.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f47a51e8700 1 Processor -- start 2024-05-21T15:14:38.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f47a51e8700 1 -- start start 2024-05-21T15:14:38.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f47a51e8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 0x7f47a019f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:38.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f47a51e8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a0107d80 0x7f47a019fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:38.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f47a51e8700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a01086d0 0x7f47a0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:38.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f47a51e8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0113450 con 0x7f47a0103450 2024-05-21T15:14:38.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f47a51e8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47a01132d0 con 0x7f47a0107d80 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f47a51e8700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47a01135d0 con 0x7f47a01086d0 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f479ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 0x7f47a019f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f479ed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 0x7f47a019f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48198/0 (socket says 172.21.15.78:48198) 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.626+0000 7f479ed9d700 1 -- 172.21.15.78:0/4294665316 learned_addr learned my addr 172.21.15.78:0/4294665316 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479f59e700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a01086d0 0x7f47a0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479e59c700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a0107d80 0x7f47a019fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479ed9d700 1 -- 172.21.15.78:0/4294665316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a01086d0 msgr2=0x7f47a0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479ed9d700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a01086d0 0x7f47a0199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479ed9d700 1 -- 172.21.15.78:0/4294665316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a0107d80 msgr2=0x7f47a019fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479ed9d700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a0107d80 0x7f47a019fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479ed9d700 1 -- 172.21.15.78:0/4294665316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a019a2c0 con 0x7f47a0103450 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479e59c700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a0107d80 0x7f47a019fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:38.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f479ed9d700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 0x7f47a019f9b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4790006010 tx=0x7f4790009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:38.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790027070 con 0x7f47a0103450 2024-05-21T15:14:38.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4790003780 con 0x7f47a0103450 2024-05-21T15:14:38.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a019a550 con 0x7f47a0103450 2024-05-21T15:14:38.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790022da0 con 0x7f47a0103450 2024-05-21T15:14:38.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.627+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01a6760 con 0x7f47a0103450 2024-05-21T15:14:38.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.628+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47a0062dd0 con 0x7f47a0103450 2024-05-21T15:14:38.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.629+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f479002a070 con 0x7f47a0103450 2024-05-21T15:14:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.630+0000 7f4787fff700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f47880a2d30 0x7f47880a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.630+0000 7f479e59c700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f47880a2d30 0x7f47880a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.630+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f47900d0720 con 0x7f47a0103450 2024-05-21T15:14:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.630+0000 7f4787fff700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] conn(0x7f47880ac560 0x7f47880ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.630+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 --> [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47900043d0 con 0x7f47880ac560 2024-05-21T15:14:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.630+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f47900d0c00 con 0x7f47a0103450 2024-05-21T15:14:38.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.631+0000 7f479e59c700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f47880a2d30 0x7f47880a5240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47a019b270 tx=0x7f4794005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.631+0000 7f479f59e700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] conn(0x7f47880ac560 0x7f47880ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:38.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.631+0000 7f479f59e700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] conn(0x7f47880ac560 0x7f47880ae9b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:38.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.632+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 <== osd.11 v2:172.21.15.172:6824/1663785983 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47900043d0 con 0x7f47880ac560 2024-05-21T15:14:38.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:38 smithi084 ceph-mon[36334]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-21T15:14:38.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.648+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 --> [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47a00631d0 con 0x7f47880ac560 2024-05-21T15:14:38.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f4787fff700 1 -- 172.21.15.78:0/4294665316 <== osd.11 v2:172.21.15.172:6824/1663785983 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f47a00631d0 con 0x7f47880ac560 2024-05-21T15:14:38.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 >> [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] conn(0x7f47880ac560 msgr2=0x7f47880ae9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:38.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] conn(0x7f47880ac560 0x7f47880ae9b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:38.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f47880a2d30 msgr2=0x7f47880a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:38.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f47880a2d30 0x7f47880a5240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f47a019b270 tx=0x7f4794005f50 comp rx=0 tx=0).stop 2024-05-21T15:14:38.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 msgr2=0x7f47a019f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:38.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 --2- 172.21.15.78:0/4294665316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0103450 0x7f47a019f9b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4790006010 tx=0x7f4790009f80 comp rx=0 tx=0).stop 2024-05-21T15:14:38.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f479f59e700 1 -- 172.21.15.78:0/4294665316 reap_dead start 2024-05-21T15:14:38.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 shutdown_connections 2024-05-21T15:14:38.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 >> 172.21.15.78:0/4294665316 conn(0x7f47a00fed20 msgr2=0x7f47a0107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:38.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 shutdown_connections 2024-05-21T15:14:38.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:38.649+0000 7f47a51e8700 1 -- 172.21.15.78:0/4294665316 wait complete. 2024-05-21T15:14:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:38 smithi172 ceph-mon[36402]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-21T15:14:40.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:40 smithi078 ceph-mon[31997]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-21T15:14:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:40 smithi084 ceph-mon[36334]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-21T15:14:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:40 smithi172 ceph-mon[36402]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-21T15:14:41.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.368+0000 7f8dd6427700 1 -- 172.21.15.78:0/3204337139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 msgr2=0x7f8dd01070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.368+0000 7f8dd6427700 1 --2- 172.21.15.78:0/3204337139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 0x7f8dd01070a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40099c0 tx=0x7f8dc4009cd0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.369+0000 7f8dd6427700 1 -- 172.21.15.78:0/3204337139 shutdown_connections 2024-05-21T15:14:41.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.369+0000 7f8dd6427700 1 --2- 172.21.15.78:0/3204337139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd01075e0 0x7f8dd010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.369+0000 7f8dd6427700 1 --2- 172.21.15.78:0/3204337139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dd0102360 0x7f8dd0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.369+0000 7f8dd6427700 1 --2- 172.21.15.78:0/3204337139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 0x7f8dd01070a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.369+0000 7f8dd6427700 1 -- 172.21.15.78:0/3204337139 >> 172.21.15.78:0/3204337139 conn(0x7f8dd00fddb0 msgr2=0x7f8dd01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:41.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.369+0000 7f8dd6427700 1 -- 172.21.15.78:0/3204337139 shutdown_connections 2024-05-21T15:14:41.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.369+0000 7f8dd6427700 1 -- 172.21.15.78:0/3204337139 wait complete. 2024-05-21T15:14:41.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dd6427700 1 Processor -- start 2024-05-21T15:14:41.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dd6427700 1 -- start start 2024-05-21T15:14:41.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dd6427700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dd0102360 0x7f8dd0108fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dd6427700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 0x7f8dd0109500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dd6427700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd01075e0 0x7f8dd0109a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dd6427700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0112360 con 0x7f8dd0106c90 2024-05-21T15:14:41.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dd6427700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01121e0 con 0x7f8dd0102360 2024-05-21T15:14:41.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dd6427700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01124e0 con 0x7f8dd01075e0 2024-05-21T15:14:41.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dcf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 0x7f8dd0109500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dcf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 0x7f8dd0109500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48228/0 (socket says 172.21.15.78:48228) 2024-05-21T15:14:41.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.370+0000 7f8dcffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dd0102360 0x7f8dd0108fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dd0102360 0x7f8dd0108fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:46594/0 (socket says 172.21.15.78:46594) 2024-05-21T15:14:41.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcffff700 1 -- 172.21.15.78:0/147725424 learned_addr learned my addr 172.21.15.78:0/147725424 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:41.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dd49c4700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd01075e0 0x7f8dd0109a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcffff700 1 -- 172.21.15.78:0/147725424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd01075e0 msgr2=0x7f8dd0109a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcffff700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd01075e0 0x7f8dd0109a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcffff700 1 -- 172.21.15.78:0/147725424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 msgr2=0x7f8dd0109500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcffff700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 0x7f8dd0109500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcffff700 1 -- 172.21.15.78:0/147725424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd01ac610 con 0x7f8dd0102360 2024-05-21T15:14:41.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcf7fe700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8dd0106c90 0x7f8dd0109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:14:41.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.371+0000 7f8dcffff700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dd0102360 0x7f8dd0108fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000c9b0 tx=0x7f8dc000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:41.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.372+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0007b00 con 0x7f8dd0102360 2024-05-21T15:14:41.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.372+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd01ac840 con 0x7f8dd0102360 2024-05-21T15:14:41.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.372+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc000ce50 con 0x7f8dd0102360 2024-05-21T15:14:41.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.372+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd01accf0 con 0x7f8dd0102360 2024-05-21T15:14:41.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.372+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0005650 con 0x7f8dd0102360 2024-05-21T15:14:41.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.374+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8dd010edf0 con 0x7f8dd0102360 2024-05-21T15:14:41.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.375+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8dc000f450 con 0x7f8dd0102360 2024-05-21T15:14:41.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.376+0000 7f8dcd7fa700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8db80a2d30 0x7f8db80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.376+0000 7f8dcf7fe700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8db80a2d30 0x7f8db80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.376+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8dc00cf0f0 con 0x7f8dd0102360 2024-05-21T15:14:41.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.377+0000 7f8dcd7fa700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] conn(0x7f8db80ac560 0x7f8db80ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.377+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 --> [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8db80af080 con 0x7f8db80ac560 2024-05-21T15:14:41.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.377+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8dc00d6050 con 0x7f8dd0102360 2024-05-21T15:14:41.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.377+0000 7f8dcf7fe700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8db80a2d30 0x7f8db80a5240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009990 tx=0x7f8dc401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:41.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.377+0000 7f8dd49c4700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] conn(0x7f8db80ac560 0x7f8db80ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.378+0000 7f8dd49c4700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] conn(0x7f8db80ac560 0x7f8db80ae9b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:41.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.378+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 <== osd.10 v2:172.21.15.84:6824/1416740807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8db80af080 con 0x7f8db80ac560 2024-05-21T15:14:41.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.395+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 --> [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8dd0103810 con 0x7f8db80ac560 2024-05-21T15:14:41.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.396+0000 7f8dcd7fa700 1 -- 172.21.15.78:0/147725424 <== osd.10 v2:172.21.15.84:6824/1416740807 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8dd0103810 con 0x7f8db80ac560 2024-05-21T15:14:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.396+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 >> [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] conn(0x7f8db80ac560 msgr2=0x7f8db80ae9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.396+0000 7f8dd6427700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] conn(0x7f8db80ac560 0x7f8db80ae9b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8db80a2d30 msgr2=0x7f8db80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd6427700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8db80a2d30 0x7f8db80a5240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009990 tx=0x7f8dc401a040 comp rx=0 tx=0).stop 2024-05-21T15:14:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dd0102360 msgr2=0x7f8dd0108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd6427700 1 --2- 172.21.15.78:0/147725424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8dd0102360 0x7f8dd0108fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000c9b0 tx=0x7f8dc000ccc0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd49c4700 1 -- 172.21.15.78:0/147725424 reap_dead start 2024-05-21T15:14:41.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 shutdown_connections 2024-05-21T15:14:41.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 >> 172.21.15.78:0/147725424 conn(0x7f8dd00fddb0 msgr2=0x7f8dd01085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:41.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 shutdown_connections 2024-05-21T15:14:41.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.397+0000 7f8dd6427700 1 -- 172.21.15.78:0/147725424 wait complete. 2024-05-21T15:14:41.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.988+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1563739530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2481023d0 msgr2=0x7fd2481027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.988+0000 7fd24fa7e700 1 --2- 172.21.15.78:0/1563739530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2481023d0 0x7fd2481027e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd2440099c0 tx=0x7fd244009cd0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.989+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1563739530 shutdown_connections 2024-05-21T15:14:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.989+0000 7fd24fa7e700 1 --2- 172.21.15.78:0/1563739530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd248107600 0x7fd24810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.989+0000 7fd24fa7e700 1 --2- 172.21.15.78:0/1563739530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2481023d0 0x7fd2481027e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.989+0000 7fd24fa7e700 1 --2- 172.21.15.78:0/1563739530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd248106cb0 0x7fd2481070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.989+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1563739530 >> 172.21.15.78:0/1563739530 conn(0x7fd248077c80 msgr2=0x7fd2480780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.989+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1563739530 shutdown_connections 2024-05-21T15:14:41.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.989+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1563739530 wait complete. 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24fa7e700 1 Processor -- start 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24fa7e700 1 -- start start 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24fa7e700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2481023d0 0x7fd24819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24fa7e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd248106cb0 0x7fd24819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24fa7e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd248107600 0x7fd24819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24fa7e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd248112380 con 0x7fd248106cb0 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24fa7e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd248112200 con 0x7fd248107600 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24fa7e700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd248112500 con 0x7fd2481023d0 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd248106cb0 0x7fd24819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd248106cb0 0x7fd24819b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48242/0 (socket says 172.21.15.78:48242) 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 -- 172.21.15.78:0/1849258085 learned_addr learned my addr 172.21.15.78:0/1849258085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:41.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24e01b700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd248107600 0x7fd24819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d81a700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2481023d0 0x7fd24819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 -- 172.21.15.78:0/1849258085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2481023d0 msgr2=0x7fd24819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2481023d0 0x7fd24819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 -- 172.21.15.78:0/1849258085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd248107600 msgr2=0x7fd24819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd248107600 0x7fd24819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 -- 172.21.15.78:0/1849258085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2481a5ed0 con 0x7fd248106cb0 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d81a700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2481023d0 0x7fd24819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24e01b700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd248107600 0x7fd24819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.990+0000 7fd24d019700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd248106cb0 0x7fd24819b0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c8f0 tx=0x7fd23800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.991+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238007b00 con 0x7fd248106cb0 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.991+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd238007ca0 con 0x7fd248106cb0 2024-05-21T15:14:41.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.991+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2481a61c0 con 0x7fd248106cb0 2024-05-21T15:14:41.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.991+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238018640 con 0x7fd248106cb0 2024-05-21T15:14:41.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.991+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2481a6750 con 0x7fd248106cb0 2024-05-21T15:14:41.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.992+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd248062d50 con 0x7fd248106cb0 2024-05-21T15:14:41.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.993+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd23800f450 con 0x7fd248106cb0 2024-05-21T15:14:41.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd23effd700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2340a2d30 0x7fd2340a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd24d81a700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2340a2d30 0x7fd2340a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd2380cf250 con 0x7fd248106cb0 2024-05-21T15:14:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd23effd700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] conn(0x7fd2340ac560 0x7fd2340ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 --> [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd238018e50 con 0x7fd2340ac560 2024-05-21T15:14:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd2380cf730 con 0x7fd248106cb0 2024-05-21T15:14:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd24e01b700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] conn(0x7fd2340ac560 0x7fd2340ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd24d81a700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2340a2d30 0x7fd2340a5240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd244009750 tx=0x7fd244005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.994+0000 7fd24e01b700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] conn(0x7fd2340ac560 0x7fd2340ae9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:41.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:41.995+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 <== osd.3 v2:172.21.15.78:6810/586567339 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd238018e50 con 0x7fd2340ac560 2024-05-21T15:14:42.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.011+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 --> [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd248063180 con 0x7fd2340ac560 2024-05-21T15:14:42.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.011+0000 7fd23effd700 1 -- 172.21.15.78:0/1849258085 <== osd.3 v2:172.21.15.78:6810/586567339 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd248063180 con 0x7fd2340ac560 2024-05-21T15:14:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] conn(0x7fd2340ac560 msgr2=0x7fd2340ae9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] conn(0x7fd2340ac560 0x7fd2340ae9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2340a2d30 msgr2=0x7fd2340a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2340a2d30 0x7fd2340a5240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd244009750 tx=0x7fd244005e80 comp rx=0 tx=0).stop 2024-05-21T15:14:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd248106cb0 msgr2=0x7fd24819b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 --2- 172.21.15.78:0/1849258085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd248106cb0 0x7fd24819b0c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c8f0 tx=0x7fd23800ccb0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24e01b700 1 -- 172.21.15.78:0/1849258085 reap_dead start 2024-05-21T15:14:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 shutdown_connections 2024-05-21T15:14:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 >> 172.21.15.78:0/1849258085 conn(0x7fd248077c80 msgr2=0x7fd248100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 shutdown_connections 2024-05-21T15:14:42.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.012+0000 7fd24fa7e700 1 -- 172.21.15.78:0/1849258085 wait complete. 2024-05-21T15:14:42.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 -- 172.21.15.78:0/3201804578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8107d90 msgr2=0x7ffbb81081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 --2- 172.21.15.78:0/3201804578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8107d90 0x7ffbb81081a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0099c0 tx=0x7ffbac009cd0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 -- 172.21.15.78:0/3201804578 shutdown_connections 2024-05-21T15:14:42.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 --2- 172.21.15.78:0/3201804578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb81086e0 0x7ffbb8110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 --2- 172.21.15.78:0/3201804578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbb8103460 0x7ffbb8103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 --2- 172.21.15.78:0/3201804578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8107d90 0x7ffbb81081a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 -- 172.21.15.78:0/3201804578 >> 172.21.15.78:0/3201804578 conn(0x7ffbb80fed50 msgr2=0x7ffbb81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 -- 172.21.15.78:0/3201804578 shutdown_connections 2024-05-21T15:14:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.559+0000 7ffbbd989700 1 -- 172.21.15.78:0/3201804578 wait complete. 2024-05-21T15:14:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.560+0000 7ffbbd989700 1 Processor -- start 2024-05-21T15:14:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.560+0000 7ffbbd989700 1 -- start start 2024-05-21T15:14:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.560+0000 7ffbbd989700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8103460 0x7ffbb8199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.560+0000 7ffbbd989700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb8107d90 0x7ffbb819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.560+0000 7ffbbd989700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbb81086e0 0x7ffbb819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.560+0000 7ffbbd989700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113440 con 0x7ffbb8103460 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.560+0000 7ffbbd989700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81132c0 con 0x7ffbb81086e0 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.560+0000 7ffbbd989700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81135c0 con 0x7ffbb8107d90 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8103460 0x7ffbb8199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8103460 0x7ffbb8199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48260/0 (socket says 172.21.15.78:48260) 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 -- 172.21.15.78:0/3694487536 learned_addr learned my addr 172.21.15.78:0/3694487536 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb67fc700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb8107d90 0x7ffbb819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb77fe700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbb81086e0 0x7ffbb819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 -- 172.21.15.78:0/3694487536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb8107d90 msgr2=0x7ffbb819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb8107d90 0x7ffbb819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 -- 172.21.15.78:0/3694487536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbb81086e0 msgr2=0x7ffbb819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbb81086e0 0x7ffbb819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 -- 172.21.15.78:0/3694487536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb819ebb0 con 0x7ffbb8103460 2024-05-21T15:14:42.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb67fc700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb8107d90 0x7ffbb819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:42.566 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:42 smithi078 ceph-mon[31997]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:14:42.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbb6ffd700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8103460 0x7ffbb8199d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffba800d980 tx=0x7ffba800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:42.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8016070 con 0x7ffbb8103460 2024-05-21T15:14:42.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb819eea0 con 0x7ffbb8103460 2024-05-21T15:14:42.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb819f400 con 0x7ffbb8103460 2024-05-21T15:14:42.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffba8004500 con 0x7ffbb8103460 2024-05-21T15:14:42.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.561+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8004d10 con 0x7ffbb8103460 2024-05-21T15:14:42.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.562+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffba4001030 con 0x7ffbb8103460 2024-05-21T15:14:42.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.565+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffba800bb60 con 0x7ffbb8103460 2024-05-21T15:14:42.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.565+0000 7ffbbc987700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ffba00a2cb0 0x7ffba00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.566+0000 7ffbb67fc700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ffba00a2cb0 0x7ffba00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.566+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ffba80cf3d0 con 0x7ffbb8103460 2024-05-21T15:14:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.566+0000 7ffbbc987700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] conn(0x7ffba00ac4e0 0x7ffba00ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.566+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 --> [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffba8015ec0 con 0x7ffba00ac4e0 2024-05-21T15:14:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.566+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ffba80cf8b0 con 0x7ffbb8103460 2024-05-21T15:14:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.566+0000 7ffbb77fe700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] conn(0x7ffba00ac4e0 0x7ffba00ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.566+0000 7ffbb67fc700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ffba00a2cb0 0x7ffba00a51c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffbac000c00 tx=0x7ffbac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.566+0000 7ffbb77fe700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] conn(0x7ffba00ac4e0 0x7ffba00ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:42.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.567+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 <== osd.1 v2:172.21.15.84:6800/3339409975 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffba8015ec0 con 0x7ffba00ac4e0 2024-05-21T15:14:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.583+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 --> [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffba4002e60 con 0x7ffba00ac4e0 2024-05-21T15:14:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbc987700 1 -- 172.21.15.78:0/3694487536 <== osd.1 v2:172.21.15.84:6800/3339409975 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffba4002e60 con 0x7ffba00ac4e0 2024-05-21T15:14:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 >> [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] conn(0x7ffba00ac4e0 msgr2=0x7ffba00ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbd989700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] conn(0x7ffba00ac4e0 0x7ffba00ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ffba00a2cb0 msgr2=0x7ffba00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbd989700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ffba00a2cb0 0x7ffba00a51c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffbac000c00 tx=0x7ffbac01a040 comp rx=0 tx=0).stop 2024-05-21T15:14:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8103460 msgr2=0x7ffbb8199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbd989700 1 --2- 172.21.15.78:0/3694487536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffbb8103460 0x7ffbb8199d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffba800d980 tx=0x7ffba800dd40 comp rx=0 tx=0).stop 2024-05-21T15:14:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbb77fe700 1 -- 172.21.15.78:0/3694487536 reap_dead start 2024-05-21T15:14:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 shutdown_connections 2024-05-21T15:14:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.584+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 >> 172.21.15.78:0/3694487536 conn(0x7ffbb80fed50 msgr2=0x7ffbb8107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.585+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 shutdown_connections 2024-05-21T15:14:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:42.585+0000 7ffbbd989700 1 -- 172.21.15.78:0/3694487536 wait complete. 2024-05-21T15:14:42.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:42 smithi084 ceph-mon[36334]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:14:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:42 smithi172 ceph-mon[36402]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:14:43.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 -- 172.21.15.78:0/4038864177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 msgr2=0x7ff068107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:43.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 --2- 172.21.15.78:0/4038864177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 0x7ff068107490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff058009990 tx=0x7ff058009ca0 comp rx=0 tx=0).stop 2024-05-21T15:14:43.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 -- 172.21.15.78:0/4038864177 shutdown_connections 2024-05-21T15:14:43.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 --2- 172.21.15.78:0/4038864177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0681030c0 0x7ff06810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:43.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 --2- 172.21.15.78:0/4038864177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 0x7ff068107490 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:43.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 --2- 172.21.15.78:0/4038864177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff068102770 0x7ff068102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:43.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 -- 172.21.15.78:0/4038864177 >> 172.21.15.78:0/4038864177 conn(0x7ff0680fd570 msgr2=0x7ff0680ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:43.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 -- 172.21.15.78:0/4038864177 shutdown_connections 2024-05-21T15:14:43.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.410+0000 7ff070311700 1 -- 172.21.15.78:0/4038864177 wait complete. 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff070311700 1 Processor -- start 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff070311700 1 -- start start 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff070311700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff068102770 0x7ff06819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff070311700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0681030c0 0x7ff06819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff070311700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 0x7ff068198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff070311700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff068112340 con 0x7ff068107080 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff070311700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0681121c0 con 0x7ff0681030c0 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff070311700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0681124c0 con 0x7ff068102770 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff06e8ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 0x7ff068198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff06e8ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 0x7ff068198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48286/0 (socket says 172.21.15.78:48286) 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff06e8ae700 1 -- 172.21.15.78:0/3119014199 learned_addr learned my addr 172.21.15.78:0/3119014199 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff06d8ac700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0681030c0 0x7ff06819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.411+0000 7ff06e0ad700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff068102770 0x7ff06819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff06e8ae700 1 -- 172.21.15.78:0/3119014199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff068102770 msgr2=0x7ff06819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff06e8ae700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff068102770 0x7ff06819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff06e8ae700 1 -- 172.21.15.78:0/3119014199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0681030c0 msgr2=0x7ff06819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff06e8ae700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0681030c0 0x7ff06819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff06e8ae700 1 -- 172.21.15.78:0/3119014199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0681990e0 con 0x7ff068107080 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff06e8ae700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 0x7ff068198aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff060007d70 tx=0x7ff06000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060016020 con 0x7ff068107080 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06000ad40 con 0x7ff068107080 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0681993d0 con 0x7ff068107080 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0600116f0 con 0x7ff068107080 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.412+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0681998c0 con 0x7ff068107080 2024-05-21T15:14:43.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.413+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff068062dd0 con 0x7ff068107080 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.414+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff060019020 con 0x7ff068107080 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.415+0000 7ff05f7fe700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0540a2cb0 0x7ff0540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.415+0000 7ff06e0ad700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0540a2cb0 0x7ff0540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.415+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff0600cf560 con 0x7ff068107080 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.415+0000 7ff05f7fe700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] conn(0x7ff0540ac4e0 0x7ff0540ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.415+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 --> [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0540af000 con 0x7ff0540ac4e0 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.415+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff0600cfa40 con 0x7ff068107080 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.415+0000 7ff06e0ad700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0540a2cb0 0x7ff0540a51c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff05800b5c0 tx=0x7ff058005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.415+0000 7ff06d8ac700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] conn(0x7ff0540ac4e0 0x7ff0540ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:43.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.416+0000 7ff06d8ac700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] conn(0x7ff0540ac4e0 0x7ff0540ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:43.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.417+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 <== osd.8 v2:172.21.15.172:6816/3263777193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0540af000 con 0x7ff0540ac4e0 2024-05-21T15:14:43.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.433+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 --> [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff068063250 con 0x7ff0540ac4e0 2024-05-21T15:14:43.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.433+0000 7ff05f7fe700 1 -- 172.21.15.78:0/3119014199 <== osd.8 v2:172.21.15.172:6816/3263777193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff068063250 con 0x7ff0540ac4e0 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 >> [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] conn(0x7ff0540ac4e0 msgr2=0x7ff0540ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] conn(0x7ff0540ac4e0 0x7ff0540ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0540a2cb0 msgr2=0x7ff0540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0540a2cb0 0x7ff0540a51c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff05800b5c0 tx=0x7ff058005fb0 comp rx=0 tx=0).stop 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 msgr2=0x7ff068198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 --2- 172.21.15.78:0/3119014199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff068107080 0x7ff068198aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff060007d70 tx=0x7ff06000a340 comp rx=0 tx=0).stop 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff06e8ae700 1 -- 172.21.15.78:0/3119014199 reap_dead start 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 shutdown_connections 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 >> 172.21.15.78:0/3119014199 conn(0x7ff0680fd570 msgr2=0x7ff068106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 shutdown_connections 2024-05-21T15:14:43.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:43.434+0000 7ff070311700 1 -- 172.21.15.78:0/3119014199 wait complete. 2024-05-21T15:14:44.206 INFO:teuthology.orchestra.run.smithi078.stdout:343597383698 2024-05-21T15:14:44.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-21T15:14:44.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:44 smithi078 ceph-mon[31997]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:14:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:44 smithi084 ceph-mon[36334]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:14:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:44 smithi172 ceph-mon[36402]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:14:45.265 INFO:teuthology.orchestra.run.smithi078.stdout:309237645335 2024-05-21T15:14:45.266 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-21T15:14:46.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.000+0000 7f62e7d73700 1 -- 172.21.15.78:0/3511077234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e01075e0 msgr2=0x7f62e010ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:46.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.000+0000 7f62e7d73700 1 --2- 172.21.15.78:0/3511077234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e01075e0 0x7f62e010ef40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62d800b0d0 tx=0x7f62d800b3e0 comp rx=0 tx=0).stop 2024-05-21T15:14:46.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.000+0000 7f62e7d73700 1 -- 172.21.15.78:0/3511077234 shutdown_connections 2024-05-21T15:14:46.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.000+0000 7f62e7d73700 1 --2- 172.21.15.78:0/3511077234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e01075e0 0x7f62e010ef40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:46.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.000+0000 7f62e7d73700 1 --2- 172.21.15.78:0/3511077234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e0102360 0x7f62e0102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:46.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.000+0000 7f62e7d73700 1 --2- 172.21.15.78:0/3511077234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e0106c90 0x7f62e01070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:46.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.000+0000 7f62e7d73700 1 -- 172.21.15.78:0/3511077234 >> 172.21.15.78:0/3511077234 conn(0x7f62e00fddb0 msgr2=0x7f62e01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:46.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 -- 172.21.15.78:0/3511077234 shutdown_connections 2024-05-21T15:14:46.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 -- 172.21.15.78:0/3511077234 wait complete. 2024-05-21T15:14:46.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 Processor -- start 2024-05-21T15:14:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 -- start start 2024-05-21T15:14:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e0102360 0x7f62e0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e0106c90 0x7f62e019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e01075e0 0x7f62e019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01134b0 con 0x7f62e0102360 2024-05-21T15:14:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.001+0000 7f62e7d73700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0113330 con 0x7f62e0106c90 2024-05-21T15:14:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e7d73700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0113630 con 0x7f62e01075e0 2024-05-21T15:14:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e0102360 0x7f62e0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e0102360 0x7f62e0199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48318/0 (socket says 172.21.15.78:48318) 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 -- 172.21.15.78:0/3443218370 learned_addr learned my addr 172.21.15.78:0/3443218370 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e530e700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e0106c90 0x7f62e019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e6310700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e01075e0 0x7f62e019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 -- 172.21.15.78:0/3443218370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e01075e0 msgr2=0x7f62e019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62e01075e0 0x7f62e019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 -- 172.21.15.78:0/3443218370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e0106c90 msgr2=0x7f62e019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e0106c90 0x7f62e019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:46.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 -- 172.21.15.78:0/3443218370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e019ec10 con 0x7f62e0102360 2024-05-21T15:14:46.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e530e700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62e0106c90 0x7f62e019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:14:46.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e5b0f700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e0102360 0x7f62e0199d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00d950 tx=0x7f62dc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:46.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc016070 con 0x7f62e0102360 2024-05-21T15:14:46.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62dc009b40 con 0x7f62e0102360 2024-05-21T15:14:46.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e019ef00 con 0x7f62e0102360 2024-05-21T15:14:46.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00f620 con 0x7f62e0102360 2024-05-21T15:14:46.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.002+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e01a6820 con 0x7f62e0102360 2024-05-21T15:14:46.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.003+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62e0062dd0 con 0x7f62e0102360 2024-05-21T15:14:46.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.005+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f62dc019070 con 0x7f62e0102360 2024-05-21T15:14:46.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.005+0000 7f62d6ffd700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f62cc0a2cb0 0x7f62cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:46.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.005+0000 7f62e530e700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f62cc0a2cb0 0x7f62cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:46.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.005+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f62dc0cf010 con 0x7f62e0102360 2024-05-21T15:14:46.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.006+0000 7f62d6ffd700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] conn(0x7f62cc0ac4e0 0x7f62cc0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:46.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.006+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 --> [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62dc014e80 con 0x7f62cc0ac4e0 2024-05-21T15:14:46.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.006+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f62dc0cf4f0 con 0x7f62e0102360 2024-05-21T15:14:46.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.006+0000 7f62e530e700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f62cc0a2cb0 0x7f62cc0a51c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005e10 tx=0x7f62d0005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:46.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.006+0000 7f62e6310700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] conn(0x7f62cc0ac4e0 0x7f62cc0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:46.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.006+0000 7f62e6310700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] conn(0x7f62cc0ac4e0 0x7f62cc0ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:46.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.007+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 <== osd.2 v2:172.21.15.172:6800/55763200 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62dc014e80 con 0x7f62cc0ac4e0 2024-05-21T15:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.023+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 --> [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62e0063250 con 0x7f62cc0ac4e0 2024-05-21T15:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62d6ffd700 1 -- 172.21.15.78:0/3443218370 <== osd.2 v2:172.21.15.172:6800/55763200 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f62e0063250 con 0x7f62cc0ac4e0 2024-05-21T15:14:46.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 >> [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] conn(0x7f62cc0ac4e0 msgr2=0x7f62cc0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:46.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] conn(0x7f62cc0ac4e0 0x7f62cc0ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:46.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f62cc0a2cb0 msgr2=0x7f62cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:46.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f62cc0a2cb0 0x7f62cc0a51c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005e10 tx=0x7f62d0005d60 comp rx=0 tx=0).stop 2024-05-21T15:14:46.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e0102360 msgr2=0x7f62e0199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:46.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 --2- 172.21.15.78:0/3443218370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62e0102360 0x7f62e0199d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00d950 tx=0x7f62dc00dd10 comp rx=0 tx=0).stop 2024-05-21T15:14:46.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e6310700 1 -- 172.21.15.78:0/3443218370 reap_dead start 2024-05-21T15:14:46.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 shutdown_connections 2024-05-21T15:14:46.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 >> 172.21.15.78:0/3443218370 conn(0x7f62e00fddb0 msgr2=0x7f62e0106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:46.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 shutdown_connections 2024-05-21T15:14:46.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:46.024+0000 7f62e7d73700 1 -- 172.21.15.78:0/3443218370 wait complete. 2024-05-21T15:14:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:46 smithi078 ceph-mon[31997]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-21T15:14:46.640 INFO:teuthology.orchestra.run.smithi078.stdout:77309411392 2024-05-21T15:14:46.641 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-21T15:14:46.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:46 smithi084 ceph-mon[36334]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-21T15:14:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:46 smithi172 ceph-mon[36402]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-21T15:14:46.981 INFO:teuthology.orchestra.run.smithi078.stdout:55834574915 2024-05-21T15:14:46.981 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-21T15:14:47.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:47 smithi078 ceph-mon[31997]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T15:14:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:47 smithi172 ceph-mon[36402]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T15:14:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:47 smithi084 ceph-mon[36334]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T15:14:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/3917728978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010f4e0 msgr2=0x7f9fe010f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 --2- 172.21.15.78:0/3917728978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010f4e0 0x7f9fe010f8f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009990 tx=0x7f9fd4009ca0 comp rx=0 tx=0).stop 2024-05-21T15:14:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/3917728978 shutdown_connections 2024-05-21T15:14:48.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 --2- 172.21.15.78:0/3917728978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe010fe30 0x7f9fe0117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 --2- 172.21.15.78:0/3917728978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe010b6c0 0x7f9fe010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 --2- 172.21.15.78:0/3917728978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010f4e0 0x7f9fe010f8f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/3917728978 >> 172.21.15.78:0/3917728978 conn(0x7f9fe006e350 msgr2=0x7f9fe006e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/3917728978 shutdown_connections 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.679+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/3917728978 wait complete. 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fe4ea2700 1 Processor -- start 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fe4ea2700 1 -- start start 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fe4ea2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010b6c0 0x7f9fe00716e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:48.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fe4ea2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe010f4e0 0x7f9fe0071c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fe4ea2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe010fe30 0x7f9fe0072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fe4ea2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe011abc0 con 0x7f9fe010b6c0 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fe4ea2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe011aa40 con 0x7f9fe010fe30 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fe4ea2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe011ad40 con 0x7f9fe010f4e0 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fde59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010b6c0 0x7f9fe00716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fde59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010b6c0 0x7f9fe00716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50200/0 (socket says 172.21.15.78:50200) 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.680+0000 7f9fde59c700 1 -- 172.21.15.78:0/774014124 learned_addr learned my addr 172.21.15.78:0/774014124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.681+0000 7f9fddd9b700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe010f4e0 0x7f9fe0071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:48.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.681+0000 7f9fde59c700 1 -- 172.21.15.78:0/774014124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe010f4e0 msgr2=0x7f9fe0071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.681+0000 7f9fde59c700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe010f4e0 0x7f9fe0071c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.681+0000 7f9fde59c700 1 -- 172.21.15.78:0/774014124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe010fe30 msgr2=0x7f9fe0072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.681+0000 7f9fde59c700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fe010fe30 0x7f9fe0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.681+0000 7f9fde59c700 1 -- 172.21.15.78:0/774014124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe01b08d0 con 0x7f9fe010b6c0 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.681+0000 7f9fde59c700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010b6c0 0x7f9fe00716e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000c9e0 tx=0x7f9fd000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.681+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0007aa0 con 0x7f9fe010b6c0 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.682+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe01b0bc0 con 0x7f9fe010b6c0 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.682+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd0007c40 con 0x7f9fe010b6c0 2024-05-21T15:14:48.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.682+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0018780 con 0x7f9fe010b6c0 2024-05-21T15:14:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.682+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe01b1150 con 0x7f9fe010b6c0 2024-05-21T15:14:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.683+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9fc0001030 con 0x7f9fe010b6c0 2024-05-21T15:14:48.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.685+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9fd0018920 con 0x7f9fe010b6c0 2024-05-21T15:14:48.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.686+0000 7f9fcf7fe700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9fc80a2cb0 0x7f9fc80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:48.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.686+0000 7f9fddd9b700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9fc80a2cb0 0x7f9fc80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:48.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.686+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9fd00cf550 con 0x7f9fe010b6c0 2024-05-21T15:14:48.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.686+0000 7f9fcf7fe700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] conn(0x7f9fc80ac4e0 0x7f9fc80ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:48.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.686+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 --> [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fd0026e50 con 0x7f9fc80ac4e0 2024-05-21T15:14:48.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.686+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f9fd00cfa30 con 0x7f9fe010b6c0 2024-05-21T15:14:48.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.686+0000 7f9fded9d700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] conn(0x7f9fc80ac4e0 0x7f9fc80ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:48.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.687+0000 7f9fded9d700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] conn(0x7f9fc80ac4e0 0x7f9fc80ae930 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:48.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.688+0000 7f9fddd9b700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9fc80a2cb0 0x7f9fc80a51c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9fe006e980 tx=0x7f9fd401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:48.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.688+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 <== osd.6 v2:172.21.15.78:6818/4029644708 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fd0026e50 con 0x7f9fc80ac4e0 2024-05-21T15:14:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 --> [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9fc0002e60 con 0x7f9fc80ac4e0 2024-05-21T15:14:48.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fcf7fe700 1 -- 172.21.15.78:0/774014124 <== osd.6 v2:172.21.15.78:6818/4029644708 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9fc0002e60 con 0x7f9fc80ac4e0 2024-05-21T15:14:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] conn(0x7f9fc80ac4e0 msgr2=0x7f9fc80ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fe4ea2700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] conn(0x7f9fc80ac4e0 0x7f9fc80ae930 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9fc80a2cb0 msgr2=0x7f9fc80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fe4ea2700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9fc80a2cb0 0x7f9fc80a51c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9fe006e980 tx=0x7f9fd401a040 comp rx=0 tx=0).stop 2024-05-21T15:14:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010b6c0 msgr2=0x7f9fe00716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:48.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fe4ea2700 1 --2- 172.21.15.78:0/774014124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fe010b6c0 0x7f9fe00716e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000c9e0 tx=0x7f9fd000cda0 comp rx=0 tx=0).stop 2024-05-21T15:14:48.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.715+0000 7f9fded9d700 1 -- 172.21.15.78:0/774014124 reap_dead start 2024-05-21T15:14:48.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.716+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 shutdown_connections 2024-05-21T15:14:48.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.716+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 >> 172.21.15.78:0/774014124 conn(0x7f9fe006e350 msgr2=0x7f9fe010a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:48.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.716+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 shutdown_connections 2024-05-21T15:14:48.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:48.716+0000 7f9fe4ea2700 1 -- 172.21.15.78:0/774014124 wait complete. 2024-05-21T15:14:49.515 INFO:teuthology.orchestra.run.smithi078.stdout:253403070498 2024-05-21T15:14:49.516 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-21T15:14:50.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:50 smithi078 ceph-mon[31997]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-21T15:14:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:50 smithi084 ceph-mon[36334]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-21T15:14:50.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:50 smithi172 ceph-mon[36402]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-05-21T15:14:50.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.772+0000 7fabb7050700 1 -- 172.21.15.78:0/2494268410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0102340 msgr2=0x7fabb0102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:50.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.772+0000 7fabb7050700 1 --2- 172.21.15.78:0/2494268410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0102340 0x7fabb0102750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fabac00b0d0 tx=0x7fabac00b3e0 comp rx=0 tx=0).stop 2024-05-21T15:14:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.772+0000 7fabb7050700 1 -- 172.21.15.78:0/2494268410 shutdown_connections 2024-05-21T15:14:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.772+0000 7fabb7050700 1 --2- 172.21.15.78:0/2494268410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb01075c0 0x7fabb010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.772+0000 7fabb7050700 1 --2- 172.21.15.78:0/2494268410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0102340 0x7fabb0102750 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.772+0000 7fabb7050700 1 --2- 172.21.15.78:0/2494268410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabb0106c70 0x7fabb0107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.772+0000 7fabb7050700 1 -- 172.21.15.78:0/2494268410 >> 172.21.15.78:0/2494268410 conn(0x7fabb00fdc40 msgr2=0x7fabb0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 -- 172.21.15.78:0/2494268410 shutdown_connections 2024-05-21T15:14:50.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 -- 172.21.15.78:0/2494268410 wait complete. 2024-05-21T15:14:50.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 Processor -- start 2024-05-21T15:14:50.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 -- start start 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabb0102340 0x7fabb019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0106c70 0x7fabb019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb01075c0 0x7fabb019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01123b0 con 0x7fabb0106c70 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0112230 con 0x7fabb01075c0 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.773+0000 7fabb7050700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0112530 con 0x7fabb0102340 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0106c70 0x7fabb019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0106c70 0x7fabb019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50214/0 (socket says 172.21.15.78:50214) 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 -- 172.21.15.78:0/2769622508 learned_addr learned my addr 172.21.15.78:0/2769622508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:50.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7fabb4dec700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabb0102340 0x7fabb019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7fabb55ed700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb01075c0 0x7fabb019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 -- 172.21.15.78:0/2769622508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabb0102340 msgr2=0x7fabb019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabb0102340 0x7fabb019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 -- 172.21.15.78:0/2769622508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb01075c0 msgr2=0x7fabb019b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabb01075c0 0x7fabb019b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 -- 172.21.15.78:0/2769622508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb01a5f00 con 0x7fabb0106c70 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba7fff700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0106c70 0x7fabb019b0f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faba000b500 tx=0x7faba000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0015020 con 0x7fabb0106c70 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb01a61f0 con 0x7fabb0106c70 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0011d10 con 0x7fabb0106c70 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.774+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0010de0 con 0x7fabb0106c70 2024-05-21T15:14:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.775+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb01a6780 con 0x7fabb0106c70 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.775+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabb0062d50 con 0x7fabb0106c70 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.777+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faba0018020 con 0x7fabb0106c70 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.777+0000 7faba5ffb700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fab980a2cb0 0x7fab980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.777+0000 7fabb4dec700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fab980a2cb0 0x7fab980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.777+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7faba00cebf0 con 0x7fabb0106c70 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.778+0000 7faba5ffb700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] conn(0x7fab980ac4e0 0x7fab980ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.778+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 --> [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faba0013ec0 con 0x7fab980ac4e0 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.778+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7faba00cf0d0 con 0x7fabb0106c70 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.778+0000 7fabb4dec700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fab980a2cb0 0x7fab980a51c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fabac009900 tx=0x7fabac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.778+0000 7fabb55ed700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] conn(0x7fab980ac4e0 0x7fab980ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.778+0000 7fabb55ed700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] conn(0x7fab980ac4e0 0x7fab980ae930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:50.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.779+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 <== osd.0 v2:172.21.15.78:6802/780655399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faba0013ec0 con 0x7fab980ac4e0 2024-05-21T15:14:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 --> [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fabb0063150 con 0x7fab980ac4e0 2024-05-21T15:14:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7faba5ffb700 1 -- 172.21.15.78:0/2769622508 <== osd.0 v2:172.21.15.78:6802/780655399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fabb0063150 con 0x7fab980ac4e0 2024-05-21T15:14:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] conn(0x7fab980ac4e0 msgr2=0x7fab980ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7fabb7050700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] conn(0x7fab980ac4e0 0x7fab980ae930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fab980a2cb0 msgr2=0x7fab980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7fabb7050700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fab980a2cb0 0x7fab980a51c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fabac009900 tx=0x7fabac00b040 comp rx=0 tx=0).stop 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0106c70 msgr2=0x7fabb019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7fabb7050700 1 --2- 172.21.15.78:0/2769622508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabb0106c70 0x7fabb019b0f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faba000b500 tx=0x7faba000b810 comp rx=0 tx=0).stop 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.795+0000 7fabb55ed700 1 -- 172.21.15.78:0/2769622508 reap_dead start 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.796+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 shutdown_connections 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.796+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 >> 172.21.15.78:0/2769622508 conn(0x7fabb00fdc40 msgr2=0x7fabb0105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.796+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 shutdown_connections 2024-05-21T15:14:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:50.796+0000 7fabb7050700 1 -- 172.21.15.78:0/2769622508 wait complete. 2024-05-21T15:14:51.391 INFO:teuthology.orchestra.run.smithi078.stdout:107374182459 2024-05-21T15:14:51.392 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-21T15:14:52.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:52 smithi078 ceph-mon[31997]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-21T15:14:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/83331183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0075330 msgr2=0x7fc3e0113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 --2- 172.21.15.78:0/83331183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0075330 0x7fc3e0113ac0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009990 tx=0x7fc3cc009ca0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/83331183 shutdown_connections 2024-05-21T15:14:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 --2- 172.21.15.78:0/83331183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3e0114000 0x7fc3e0116420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 --2- 172.21.15.78:0/83331183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3e00749e0 0x7fc3e0074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 --2- 172.21.15.78:0/83331183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0075330 0x7fc3e0113ac0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/83331183 >> 172.21.15.78:0/83331183 conn(0x7fc3e0078b90 msgr2=0x7fc3e0078fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:52.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/83331183 shutdown_connections 2024-05-21T15:14:52.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.463+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/83331183 wait complete. 2024-05-21T15:14:52.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e5ba2700 1 Processor -- start 2024-05-21T15:14:52.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e5ba2700 1 -- start start 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e5ba2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3e00749e0 0x7fc3e01a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e5ba2700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3e0075330 0x7fc3e01a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e5ba2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0114000 0x7fc3e01a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e5ba2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01198b0 con 0x7fc3e0114000 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e5ba2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0119730 con 0x7fc3e00749e0 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e5ba2700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0119a30 con 0x7fc3e0075330 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e53a1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0114000 0x7fc3e01a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e53a1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0114000 0x7fc3e01a49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50222/0 (socket says 172.21.15.78:50222) 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e53a1700 1 -- 172.21.15.78:0/1340036148 learned_addr learned my addr 172.21.15.78:0/1340036148 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3e4ba0700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3e00749e0 0x7fc3e01a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.464+0000 7fc3dffff700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3e0075330 0x7fc3e01a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3e53a1700 1 -- 172.21.15.78:0/1340036148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3e0075330 msgr2=0x7fc3e01a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3e53a1700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3e0075330 0x7fc3e01a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3e53a1700 1 -- 172.21.15.78:0/1340036148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3e00749e0 msgr2=0x7fc3e01a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3e53a1700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3e00749e0 0x7fc3e01a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3e53a1700 1 -- 172.21.15.78:0/1340036148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e01a5050 con 0x7fc3e0114000 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3dffff700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3e0075330 0x7fc3e01a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3e53a1700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0114000 0x7fc3e01a49b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800ada0 tx=0x7fc3d800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8017020 con 0x7fc3e0114000 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e01a5340 con 0x7fc3e0114000 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d8004bb0 con 0x7fc3e0114000 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8005520 con 0x7fc3e0114000 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.465+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e01acc60 con 0x7fc3e0114000 2024-05-21T15:14:52.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.466+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3e0100730 con 0x7fc3e0114000 2024-05-21T15:14:52.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.467+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc3d80056c0 con 0x7fc3e0114000 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.468+0000 7fc3ddffb700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc3d00a2cb0 0x7fc3d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.468+0000 7fc3e4ba0700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc3d00a2cb0 0x7fc3d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.468+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc3d80cf100 con 0x7fc3e0114000 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.468+0000 7fc3ddffb700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] conn(0x7fc3d00ac4e0 0x7fc3d00ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.468+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 --> [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3d8019e50 con 0x7fc3d00ac4e0 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.468+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fc3d80cf5e0 con 0x7fc3e0114000 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.469+0000 7fc3e4ba0700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc3d00a2cb0 0x7fc3d00a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4005fd0 tx=0x7fc3d400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.469+0000 7fc3dffff700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] conn(0x7fc3d00ac4e0 0x7fc3d00ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.469+0000 7fc3dffff700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] conn(0x7fc3d00ac4e0 0x7fc3d00ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:52.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.470+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 <== osd.7 v2:172.21.15.84:6816/429797253 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3d8019e50 con 0x7fc3d00ac4e0 2024-05-21T15:14:52.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.487+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 --> [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc3e0076120 con 0x7fc3d00ac4e0 2024-05-21T15:14:52.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.487+0000 7fc3ddffb700 1 -- 172.21.15.78:0/1340036148 <== osd.7 v2:172.21.15.84:6816/429797253 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc3e0076120 con 0x7fc3d00ac4e0 2024-05-21T15:14:52.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 >> [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] conn(0x7fc3d00ac4e0 msgr2=0x7fc3d00ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] conn(0x7fc3d00ac4e0 0x7fc3d00ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc3d00a2cb0 msgr2=0x7fc3d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc3d00a2cb0 0x7fc3d00a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4005fd0 tx=0x7fc3d400a560 comp rx=0 tx=0).stop 2024-05-21T15:14:52.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0114000 msgr2=0x7fc3e01a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 --2- 172.21.15.78:0/1340036148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0114000 0x7fc3e01a49b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800ada0 tx=0x7fc3d800c330 comp rx=0 tx=0).stop 2024-05-21T15:14:52.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e53a1700 1 -- 172.21.15.78:0/1340036148 reap_dead start 2024-05-21T15:14:52.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 shutdown_connections 2024-05-21T15:14:52.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 >> 172.21.15.78:0/1340036148 conn(0x7fc3e0078b90 msgr2=0x7fc3e00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:52.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 shutdown_connections 2024-05-21T15:14:52.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.488+0000 7fc3e5ba2700 1 -- 172.21.15.78:0/1340036148 wait complete. 2024-05-21T15:14:52.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:52 smithi084 ceph-mon[36334]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-21T15:14:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:52 smithi172 ceph-mon[36402]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 -- 172.21.15.78:0/3110282394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c103470 msgr2=0x7fb81c103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 --2- 172.21.15.78:0/3110282394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c103470 0x7fb81c103880 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b0d0 tx=0x7fb81800b3e0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 -- 172.21.15.78:0/3110282394 shutdown_connections 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 --2- 172.21.15.78:0/3110282394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb81c1086f0 0x7fb81c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 --2- 172.21.15.78:0/3110282394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c103470 0x7fb81c103880 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 --2- 172.21.15.78:0/3110282394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81c107da0 0x7fb81c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 -- 172.21.15.78:0/3110282394 >> 172.21.15.78:0/3110282394 conn(0x7fb81c0feec0 msgr2=0x7fb81c1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 -- 172.21.15.78:0/3110282394 shutdown_connections 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.869+0000 7fb8241c7700 1 -- 172.21.15.78:0/3110282394 wait complete. 2024-05-21T15:14:52.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb8241c7700 1 Processor -- start 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb8241c7700 1 -- start start 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb8241c7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81c103470 0x7fb81c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb8241c7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb81c107da0 0x7fb81c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb8241c7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c1086f0 0x7fb81c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb8241c7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c1134e0 con 0x7fb81c1086f0 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb8241c7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c113360 con 0x7fb81c107da0 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb8241c7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c113660 con 0x7fb81c103470 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.870+0000 7fb822764700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c1086f0 0x7fb81c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb822764700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c1086f0 0x7fb81c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50234/0 (socket says 172.21.15.78:50234) 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb822764700 1 -- 172.21.15.78:0/363125128 learned_addr learned my addr 172.21.15.78:0/363125128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb821f63700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81c103470 0x7fb81c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb821762700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb81c107da0 0x7fb81c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb822764700 1 -- 172.21.15.78:0/363125128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81c103470 msgr2=0x7fb81c199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb822764700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81c103470 0x7fb81c199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb822764700 1 -- 172.21.15.78:0/363125128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb81c107da0 msgr2=0x7fb81c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb822764700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb81c107da0 0x7fb81c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb822764700 1 -- 172.21.15.78:0/363125128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c19ec50 con 0x7fb81c1086f0 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb821f63700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81c103470 0x7fb81c199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb821762700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb81c107da0 0x7fb81c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb822764700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c1086f0 0x7fb81c19e660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c900 tx=0x7fb81400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814007b00 con 0x7fb81c1086f0 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb814007ca0 con 0x7fb81c1086f0 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c19ef40 con 0x7fb81c1086f0 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814018660 con 0x7fb81c1086f0 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.871+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c19f430 con 0x7fb81c1086f0 2024-05-21T15:14:52.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.872+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb81c062d90 con 0x7fb81c1086f0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.873+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb81400f450 con 0x7fb81c1086f0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.874+0000 7fb812ffd700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8080a2cb0 0x7fb8080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.874+0000 7fb821f63700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8080a2cb0 0x7fb8080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.874+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb8140cf100 con 0x7fb81c1086f0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.874+0000 7fb812ffd700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] conn(0x7fb8080ac4e0 0x7fb8080ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.874+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 --> [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb814015e70 con 0x7fb8080ac4e0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.874+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb8140cf5e0 con 0x7fb81c1086f0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.874+0000 7fb821762700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] conn(0x7fb8080ac4e0 0x7fb8080ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.875+0000 7fb821f63700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8080a2cb0 0x7fb8080a51c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb818009fd0 tx=0x7fb81800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.875+0000 7fb821762700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] conn(0x7fb8080ac4e0 0x7fb8080ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:52.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.875+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 <== osd.9 v2:172.21.15.78:6826/2889072877 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb814015e70 con 0x7fb8080ac4e0 2024-05-21T15:14:52.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 --> [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb81c063210 con 0x7fb8080ac4e0 2024-05-21T15:14:52.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb812ffd700 1 -- 172.21.15.78:0/363125128 <== osd.9 v2:172.21.15.78:6826/2889072877 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb81c063210 con 0x7fb8080ac4e0 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] conn(0x7fb8080ac4e0 msgr2=0x7fb8080ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] conn(0x7fb8080ac4e0 0x7fb8080ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8080a2cb0 msgr2=0x7fb8080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8080a2cb0 0x7fb8080a51c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb818009fd0 tx=0x7fb81800b040 comp rx=0 tx=0).stop 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c1086f0 msgr2=0x7fb81c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 --2- 172.21.15.78:0/363125128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb81c1086f0 0x7fb81c19e660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c900 tx=0x7fb81400ccc0 comp rx=0 tx=0).stop 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb822764700 1 -- 172.21.15.78:0/363125128 reap_dead start 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 shutdown_connections 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.893+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 >> 172.21.15.78:0/363125128 conn(0x7fb81c0feec0 msgr2=0x7fb81c106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.894+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 shutdown_connections 2024-05-21T15:14:52.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:52.894+0000 7fb8241c7700 1 -- 172.21.15.78:0/363125128 wait complete. 2024-05-21T15:14:53.416 INFO:teuthology.orchestra.run.smithi078.stdout:197568495662 2024-05-21T15:14:53.417 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 -- 172.21.15.78:0/954497924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc102a90 msgr2=0x7f5bcc102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 --2- 172.21.15.78:0/954497924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc102a90 0x7f5bcc102ea0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00099c0 tx=0x7f5bc0009cd0 comp rx=0 tx=0).stop 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 -- 172.21.15.78:0/954497924 shutdown_connections 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 --2- 172.21.15.78:0/954497924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bcc1077f0 0x7f5bcc10f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 --2- 172.21.15.78:0/954497924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc102a90 0x7f5bcc102ea0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 --2- 172.21.15.78:0/954497924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bcc106ea0 0x7f5bcc1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 -- 172.21.15.78:0/954497924 >> 172.21.15.78:0/954497924 conn(0x7f5bcc0fd3a0 msgr2=0x7f5bcc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 -- 172.21.15.78:0/954497924 shutdown_connections 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.652+0000 7f5bd301a700 1 -- 172.21.15.78:0/954497924 wait complete. 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.653+0000 7f5bd301a700 1 Processor -- start 2024-05-21T15:14:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.653+0000 7f5bd301a700 1 -- start start 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.653+0000 7f5bd301a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bcc102a90 0x7f5bcc198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.653+0000 7f5bd301a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bcc106ea0 0x7f5bcc199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.653+0000 7f5bd301a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc1077f0 0x7f5bcc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.653+0000 7f5bd301a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1125e0 con 0x7f5bcc1077f0 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.653+0000 7f5bd301a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc112460 con 0x7f5bcc106ea0 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.653+0000 7f5bd301a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc112760 con 0x7f5bcc102a90 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc1077f0 0x7f5bcc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc1077f0 0x7f5bcc1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50264/0 (socket says 172.21.15.78:50264) 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 -- 172.21.15.78:0/3492644497 learned_addr learned my addr 172.21.15.78:0/3492644497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bcbfff700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bcc106ea0 0x7f5bcc199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd0db6700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bcc102a90 0x7f5bcc198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 -- 172.21.15.78:0/3492644497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bcc102a90 msgr2=0x7f5bcc198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bcc102a90 0x7f5bcc198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 -- 172.21.15.78:0/3492644497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bcc106ea0 msgr2=0x7f5bcc199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5bcc106ea0 0x7f5bcc199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 -- 172.21.15.78:0/3492644497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc1a82d0 con 0x7f5bcc1077f0 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd15b7700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc1077f0 0x7f5bcc1996d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4007d40 tx=0x7f5bc400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4016020 con 0x7f5bcc1077f0 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc1a85c0 con 0x7f5bcc1077f0 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc400acb0 con 0x7f5bcc1077f0 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.654+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40115e0 con 0x7f5bcc1077f0 2024-05-21T15:14:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.655+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc1a8b00 con 0x7f5bcc1077f0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.655+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5bcc062dd0 con 0x7f5bcc1077f0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.657+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5bc4019020 con 0x7f5bcc1077f0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.657+0000 7f5bc9ffb700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bbc0a2cb0 0x7f5bbc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.658+0000 7f5bd0db6700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bbc0a2cb0 0x7f5bbc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.658+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5bc40cf350 con 0x7f5bcc1077f0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.658+0000 7f5bc9ffb700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] conn(0x7f5bbc0ac4e0 0x7f5bbc0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.658+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 --> [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bc4027ea0 con 0x7f5bbc0ac4e0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.658+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f5bc40cf830 con 0x7f5bcc1077f0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.658+0000 7f5bd0db6700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bbc0a2cb0 0x7f5bbc0a51c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0000c00 tx=0x7f5bc001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.658+0000 7f5bcbfff700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] conn(0x7f5bbc0ac4e0 0x7f5bbc0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.659+0000 7f5bcbfff700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] conn(0x7f5bbc0ac4e0 0x7f5bbc0ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.659+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 <== osd.4 v2:172.21.15.84:6808/2716060044 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bc4027ea0 con 0x7f5bbc0ac4e0 2024-05-21T15:14:53.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.675+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 --> [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5bcc063250 con 0x7f5bbc0ac4e0 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.676+0000 7f5bc9ffb700 1 -- 172.21.15.78:0/3492644497 <== osd.4 v2:172.21.15.84:6808/2716060044 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5bcc063250 con 0x7f5bbc0ac4e0 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.676+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 >> [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] conn(0x7f5bbc0ac4e0 msgr2=0x7f5bbc0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.676+0000 7f5bd301a700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] conn(0x7f5bbc0ac4e0 0x7f5bbc0ae930 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bbc0a2cb0 msgr2=0x7f5bbc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd301a700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5bbc0a2cb0 0x7f5bbc0a51c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0000c00 tx=0x7f5bc001a040 comp rx=0 tx=0).stop 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc1077f0 msgr2=0x7f5bcc1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd301a700 1 --2- 172.21.15.78:0/3492644497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bcc1077f0 0x7f5bcc1996d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4007d40 tx=0x7f5bc400a340 comp rx=0 tx=0).stop 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd15b7700 1 -- 172.21.15.78:0/3492644497 reap_dead start 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 shutdown_connections 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 >> 172.21.15.78:0/3492644497 conn(0x7f5bcc0fd3a0 msgr2=0x7f5bcc1060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 shutdown_connections 2024-05-21T15:14:53.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:53.677+0000 7f5bd301a700 1 -- 172.21.15.78:0/3492644497 wait complete. 2024-05-21T15:14:54.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:54 smithi078 ceph-mon[31997]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-21T15:14:54.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:54 smithi084 ceph-mon[36334]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-21T15:14:54.673 INFO:teuthology.orchestra.run.smithi078.stdout:34359738443 2024-05-21T15:14:54.674 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-21T15:14:54.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:54 smithi172 ceph-mon[36402]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-21T15:14:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.870+0000 7fc5c5bb1700 1 -- 172.21.15.78:0/703387068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 msgr2=0x7fc5c0103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.870+0000 7fc5c5bb1700 1 --2- 172.21.15.78:0/703387068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 0x7fc5c0103870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00099c0 tx=0x7fc5b0009cd0 comp rx=0 tx=0).stop 2024-05-21T15:14:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.870+0000 7fc5c5bb1700 1 -- 172.21.15.78:0/703387068 shutdown_connections 2024-05-21T15:14:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.870+0000 7fc5c5bb1700 1 --2- 172.21.15.78:0/703387068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c01086e0 0x7fc5c0110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.870+0000 7fc5c5bb1700 1 --2- 172.21.15.78:0/703387068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 0x7fc5c0103870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.870+0000 7fc5c5bb1700 1 --2- 172.21.15.78:0/703387068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c0107d90 0x7fc5c01081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.870+0000 7fc5c5bb1700 1 -- 172.21.15.78:0/703387068 >> 172.21.15.78:0/703387068 conn(0x7fc5c00fecf0 msgr2=0x7fc5c0101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.871+0000 7fc5c5bb1700 1 -- 172.21.15.78:0/703387068 shutdown_connections 2024-05-21T15:14:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.871+0000 7fc5c5bb1700 1 -- 172.21.15.78:0/703387068 wait complete. 2024-05-21T15:14:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.871+0000 7fc5c5bb1700 1 Processor -- start 2024-05-21T15:14:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.871+0000 7fc5c5bb1700 1 -- start start 2024-05-21T15:14:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5c5bb1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 0x7fc5c0199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5c5bb1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c0107d90 0x7fc5c019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5c5bb1700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c01086e0 0x7fc5c019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5c5bb1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c0113460 con 0x7fc5c0103460 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5c5bb1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01132e0 con 0x7fc5c0107d90 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5c5bb1700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01135e0 con 0x7fc5c01086e0 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 0x7fc5c0199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 0x7fc5c0199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50288/0 (socket says 172.21.15.78:50288) 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 -- 172.21.15.78:0/2260683196 learned_addr learned my addr 172.21.15.78:0/2260683196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bffff700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c01086e0 0x7fc5c019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5b7fff700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c0107d90 0x7fc5c019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 -- 172.21.15.78:0/2260683196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c01086e0 msgr2=0x7fc5c019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c01086e0 0x7fc5c019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 -- 172.21.15.78:0/2260683196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c0107d90 msgr2=0x7fc5c019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c0107d90 0x7fc5c019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 -- 172.21.15.78:0/2260683196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c019eb50 con 0x7fc5c0103460 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5b7fff700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c0107d90 0x7fc5c019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bf7fe700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 0x7fc5c0199d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0006010 tx=0x7fc5b0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0027070 con 0x7fc5c0103460 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b0004050 con 0x7fc5c0103460 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0022e10 con 0x7fc5c0103460 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5c5bb1700 1 -- 172.21.15.78:0/2260683196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c019ede0 con 0x7fc5c0103460 2024-05-21T15:14:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.872+0000 7fc5c5bb1700 1 -- 172.21.15.78:0/2260683196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c019f2c0 con 0x7fc5c0103460 2024-05-21T15:14:54.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.873+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5c0062dd0 con 0x7fc5c0103460 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.875+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc5b002a070 con 0x7fc5c0103460 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.875+0000 7fc5bd7fa700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5ac0a2cb0 0x7fc5ac0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.875+0000 7fc5b7fff700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5ac0a2cb0 0x7fc5ac0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.875+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc5b00d0520 con 0x7fc5c0103460 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.876+0000 7fc5bd7fa700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] conn(0x7fc5ac0ac4e0 0x7fc5ac0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.876+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 --> [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5b0001d30 con 0x7fc5ac0ac4e0 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.876+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fc5b00d0a00 con 0x7fc5c0103460 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.876+0000 7fc5b7fff700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5ac0a2cb0 0x7fc5ac0a51c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8005fd0 tx=0x7fc5a8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.876+0000 7fc5bffff700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] conn(0x7fc5ac0ac4e0 0x7fc5ac0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:14:54.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.876+0000 7fc5bffff700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] conn(0x7fc5ac0ac4e0 0x7fc5ac0ae930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:14:54.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.877+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 <== osd.5 v2:172.21.15.172:6808/3395704434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5b0001d30 con 0x7fc5ac0ac4e0 2024-05-21T15:14:54.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.893+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 --> [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5c00631d0 con 0x7fc5ac0ac4e0 2024-05-21T15:14:54.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5bd7fa700 1 -- 172.21.15.78:0/2260683196 <== osd.5 v2:172.21.15.172:6808/3395704434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc5c00631d0 con 0x7fc5ac0ac4e0 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 >> [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] conn(0x7fc5ac0ac4e0 msgr2=0x7fc5ac0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5b67fc700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] conn(0x7fc5ac0ac4e0 0x7fc5ac0ae930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5ac0a2cb0 msgr2=0x7fc5ac0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5b67fc700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5ac0a2cb0 0x7fc5ac0a51c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8005fd0 tx=0x7fc5a8005e20 comp rx=0 tx=0).stop 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 msgr2=0x7fc5c0199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5b67fc700 1 --2- 172.21.15.78:0/2260683196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c0103460 0x7fc5c0199d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0006010 tx=0x7fc5b0009f80 comp rx=0 tx=0).stop 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5bffff700 1 -- 172.21.15.78:0/2260683196 reap_dead start 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 shutdown_connections 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.894+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 >> 172.21.15.78:0/2260683196 conn(0x7fc5c00fecf0 msgr2=0x7fc5c0107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.895+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 shutdown_connections 2024-05-21T15:14:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:14:54.895+0000 7fc5b67fc700 1 -- 172.21.15.78:0/2260683196 wait complete. 2024-05-21T15:14:55.297 INFO:teuthology.orchestra.run.smithi078.stdout:141733920823 2024-05-21T15:14:55.297 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-21T15:14:56.017 INFO:teuthology.orchestra.run.smithi078.stdout:163208757300 2024-05-21T15:14:56.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-21T15:14:56.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:56 smithi078 ceph-mon[31997]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-21T15:14:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:56 smithi084 ceph-mon[36334]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-21T15:14:56.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:56 smithi172 ceph-mon[36402]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-21T15:14:56.714 INFO:teuthology.orchestra.run.smithi078.stdout:219043332137 2024-05-21T15:14:56.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-21T15:14:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:56.958 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:56.959 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:57.325 INFO:teuthology.orchestra.run.smithi078.stdout:287762808863 2024-05-21T15:14:57.325 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-21T15:14:57.347 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:57.352 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:57.360 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:14:58.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:14:58 smithi078 ceph-mon[31997]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:14:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:14:58 smithi084 ceph-mon[36334]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:14:58.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:14:58 smithi172 ceph-mon[36402]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:00.586 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:00 smithi078 ceph-mon[31997]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:00.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:00 smithi084 ceph-mon[36334]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:00.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:00 smithi172 ceph-mon[36402]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:01.997 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:01.998 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:02.566 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:02 smithi078 ceph-mon[31997]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:02.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:02 smithi084 ceph-mon[36334]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:02.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:02 smithi172 ceph-mon[36402]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 -- 172.21.15.78:0/2836597719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008101660 msgr2=0x7f2008101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 --2- 172.21.15.78:0/2836597719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008101660 0x7f2008101a70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009990 tx=0x7f1ff8009ca0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 -- 172.21.15.78:0/2836597719 shutdown_connections 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 --2- 172.21.15.78:0/2836597719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2008101fb0 0x7f200810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 --2- 172.21.15.78:0/2836597719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2008109260 0x7f2008109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 --2- 172.21.15.78:0/2836597719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008101660 0x7f2008101a70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 -- 172.21.15.78:0/2836597719 >> 172.21.15.78:0/2836597719 conn(0x7f20080fd570 msgr2=0x7f20080ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 -- 172.21.15.78:0/2836597719 shutdown_connections 2024-05-21T15:15:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 -- 172.21.15.78:0/2836597719 wait complete. 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 Processor -- start 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.750+0000 7f200f64c700 1 -- start start 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200f64c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2008101660 0x7f200819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200f64c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2008101fb0 0x7f200819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200f64c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008109260 0x7f2008197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200f64c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20081112b0 con 0x7f2008109260 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200f64c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2008111130 con 0x7f2008101660 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200f64c700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2008111430 con 0x7f2008101fb0 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200dbe9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008109260 0x7f2008197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200dbe9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008109260 0x7f2008197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33640/0 (socket says 172.21.15.78:33640) 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200dbe9700 1 -- 172.21.15.78:0/2439273761 learned_addr learned my addr 172.21.15.78:0/2439273761 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200cbe7700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2008101fb0 0x7f200819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200d3e8700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2008101660 0x7f200819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200dbe9700 1 -- 172.21.15.78:0/2439273761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2008101fb0 msgr2=0x7f200819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200dbe9700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2008101fb0 0x7f200819dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200dbe9700 1 -- 172.21.15.78:0/2439273761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2008101660 msgr2=0x7f200819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200dbe9700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2008101660 0x7f200819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.751+0000 7f200dbe9700 1 -- 172.21.15.78:0/2439273761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20081980e0 con 0x7f2008109260 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.752+0000 7f200dbe9700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008109260 0x7f2008197a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2000007ee0 tx=0x7f200000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.752+0000 7f1ffe7fc700 1 -- 172.21.15.78:0/2439273761 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000dd20 con 0x7f2008109260 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.752+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20081983d0 con 0x7f2008109260 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.752+0000 7f1ffe7fc700 1 -- 172.21.15.78:0/2439273761 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000004020 con 0x7f2008109260 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.752+0000 7f1ffe7fc700 1 -- 172.21.15.78:0/2439273761 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000015660 con 0x7f2008109260 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.752+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081a4650 con 0x7f2008109260 2024-05-21T15:15:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.753+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200810b6d0 con 0x7f2008109260 2024-05-21T15:15:04.146 INFO:teuthology.orchestra.run.smithi078.stdout:77309411395 2024-05-21T15:15:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.755+0000 7f1ffe7fc700 1 -- 172.21.15.78:0/2439273761 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f20000041d0 con 0x7f2008109260 2024-05-21T15:15:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.756+0000 7f1ffe7fc700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1ff40a2cb0 0x7f1ff40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.756+0000 7f200d3e8700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1ff40a2cb0 0x7f1ff40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.756+0000 7f1ffe7fc700 1 -- 172.21.15.78:0/2439273761 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f20000cf290 con 0x7f2008109260 2024-05-21T15:15:04.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.757+0000 7f200d3e8700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1ff40a2cb0 0x7f1ff40a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f200806b850 tx=0x7f200400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.759+0000 7f1ffe7fc700 1 -- 172.21.15.78:0/2439273761 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2000010070 con 0x7f2008109260 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.960+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2008198f10 con 0x7f2008109260 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.960+0000 7f1ffe7fc700 1 -- 172.21.15.78:0/2439273761 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f200009c240 con 0x7f2008109260 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1ff40a2cb0 msgr2=0x7f1ff40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1ff40a2cb0 0x7f1ff40a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f200806b850 tx=0x7f200400b700 comp rx=0 tx=0).stop 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008109260 msgr2=0x7f2008197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008109260 0x7f2008197a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2000007ee0 tx=0x7f200000d3b0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 shutdown_connections 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1ff40a2cb0 0x7f1ff40a51c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2008109260 0x7f2008197a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2008101660 0x7f200819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 --2- 172.21.15.78:0/2439273761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2008101fb0 0x7f200819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:04.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 >> 172.21.15.78:0/2439273761 conn(0x7f20080fd570 msgr2=0x7f20080ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:04.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 shutdown_connections 2024-05-21T15:15:04.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:03.962+0000 7f200f64c700 1 -- 172.21.15.78:0/2439273761 wait complete. 2024-05-21T15:15:04.333 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:04 smithi078 ceph-mon[31997]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:04.333 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:04 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2439273761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T15:15:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:04 smithi084 ceph-mon[36334]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:04 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2439273761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T15:15:04.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:04 smithi172 ceph-mon[36402]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:04.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:04 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2439273761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T15:15:04.833 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411392 got 77309411395 for osd.2 2024-05-21T15:15:04.834 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:05.724 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:05.725 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:05.725 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:05.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.741+0000 7fb6b71c0700 1 -- 172.21.15.78:0/4116992172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b01075c0 msgr2=0x7fb6b010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:05.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.742+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/4116992172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b01075c0 0x7fb6b010ef20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800b0d0 tx=0x7fb6a800b3e0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.742+0000 7fb6b71c0700 1 -- 172.21.15.78:0/4116992172 shutdown_connections 2024-05-21T15:15:05.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.742+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/4116992172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b01075c0 0x7fb6b010ef20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.742+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/4116992172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6b0102340 0x7fb6b0102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.742+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/4116992172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6b0106c70 0x7fb6b0107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.742+0000 7fb6b71c0700 1 -- 172.21.15.78:0/4116992172 >> 172.21.15.78:0/4116992172 conn(0x7fb6b00fdc40 msgr2=0x7fb6b0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:05.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.742+0000 7fb6b71c0700 1 -- 172.21.15.78:0/4116992172 shutdown_connections 2024-05-21T15:15:05.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.742+0000 7fb6b71c0700 1 -- 172.21.15.78:0/4116992172 wait complete. 2024-05-21T15:15:05.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.743+0000 7fb6b71c0700 1 Processor -- start 2024-05-21T15:15:05.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.743+0000 7fb6b71c0700 1 -- start start 2024-05-21T15:15:05.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b71c0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6b0102340 0x7fb6b010b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b71c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0106c70 0x7fb6b010b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b71c0700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6b01075c0 0x7fb6b010bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b4f5c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6b0102340 0x7fb6b010b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b4f5c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6b0102340 0x7fb6b010b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38858/0 (socket says 172.21.15.78:38858) 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b4f5c700 1 -- 172.21.15.78:0/567815780 learned_addr learned my addr 172.21.15.78:0/567815780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6affff700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0106c70 0x7fb6b010b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0112340 con 0x7fb6b0106c70 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b575d700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6b01075c0 0x7fb6b010bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.744+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01121c0 con 0x7fb6b0102340 2024-05-21T15:15:05.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.745+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01124c0 con 0x7fb6b01075c0 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.745+0000 7fb6affff700 1 -- 172.21.15.78:0/567815780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6b01075c0 msgr2=0x7fb6b010bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.745+0000 7fb6affff700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6b01075c0 0x7fb6b010bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.745+0000 7fb6affff700 1 -- 172.21.15.78:0/567815780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6b0102340 msgr2=0x7fb6b010b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.745+0000 7fb6affff700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6b0102340 0x7fb6b010b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.745+0000 7fb6affff700 1 -- 172.21.15.78:0/567815780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b01ac4b0 con 0x7fb6b0106c70 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.745+0000 7fb6b4f5c700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6b0102340 0x7fb6b010b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.745+0000 7fb6affff700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0106c70 0x7fb6b010b6f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0075930 tx=0x7fb69c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.746+0000 7fb6adffb700 1 -- 172.21.15.78:0/567815780 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c017070 con 0x7fb6b0106c70 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.746+0000 7fb6adffb700 1 -- 172.21.15.78:0/567815780 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb69c009e10 con 0x7fb6b0106c70 2024-05-21T15:15:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.746+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b01ac7a0 con 0x7fb6b0106c70 2024-05-21T15:15:05.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.746+0000 7fb6adffb700 1 -- 172.21.15.78:0/567815780 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00f3c0 con 0x7fb6b0106c70 2024-05-21T15:15:05.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.746+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b01acd30 con 0x7fb6b0106c70 2024-05-21T15:15:05.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.747+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b01037f0 con 0x7fb6b0106c70 2024-05-21T15:15:05.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.749+0000 7fb6adffb700 1 -- 172.21.15.78:0/567815780 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb69c01a070 con 0x7fb6b0106c70 2024-05-21T15:15:05.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.749+0000 7fb6adffb700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb6a00a2d30 0x7fb6a00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:05.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.750+0000 7fb6adffb700 1 -- 172.21.15.78:0/567815780 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb69c0ce900 con 0x7fb6b0106c70 2024-05-21T15:15:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.750+0000 7fb6b4f5c700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb6a00a2d30 0x7fb6a00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.750+0000 7fb6b4f5c700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb6a00a2d30 0x7fb6a00a5240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005d90 tx=0x7fb6a4005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.754+0000 7fb6adffb700 1 -- 172.21.15.78:0/567815780 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb69c09b930 con 0x7fb6b0106c70 2024-05-21T15:15:05.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.966+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb6b01acf90 con 0x7fb6b0106c70 2024-05-21T15:15:05.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.966+0000 7fb6adffb700 1 -- 172.21.15.78:0/567815780 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb69c09b230 con 0x7fb6b0106c70 2024-05-21T15:15:05.969 INFO:teuthology.orchestra.run.smithi078.stdout:34359738446 2024-05-21T15:15:05.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb6a00a2d30 msgr2=0x7fb6a00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:05.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb6a00a2d30 0x7fb6a00a5240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005d90 tx=0x7fb6a4005d20 comp rx=0 tx=0).stop 2024-05-21T15:15:05.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0106c70 msgr2=0x7fb6b010b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0106c70 0x7fb6b010b6f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0075930 tx=0x7fb69c00dc30 comp rx=0 tx=0).stop 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 shutdown_connections 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb6a00a2d30 0x7fb6a00a5240 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6b01075c0 0x7fb6b010bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6b0102340 0x7fb6b010b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 --2- 172.21.15.78:0/567815780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b0106c70 0x7fb6b010b6f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 >> 172.21.15.78:0/567815780 conn(0x7fb6b00fdc40 msgr2=0x7fb6b0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 shutdown_connections 2024-05-21T15:15:05.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:05.968+0000 7fb6b71c0700 1 -- 172.21.15.78:0/567815780 wait complete. 2024-05-21T15:15:06.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 -- 172.21.15.78:0/2228679217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540101610 msgr2=0x7fa540101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:06.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 --2- 172.21.15.78:0/2228679217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540101610 0x7fa540101a20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa534009990 tx=0x7fa534009ca0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.542 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:06 smithi078 ceph-mon[31997]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-05-21T15:15:06.542 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:06 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/567815780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T15:15:06.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 -- 172.21.15.78:0/2228679217 shutdown_connections 2024-05-21T15:15:06.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 --2- 172.21.15.78:0/2228679217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540101f60 0x7fa54010fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 --2- 172.21.15.78:0/2228679217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540108180 0x7fa540108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 --2- 172.21.15.78:0/2228679217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540101610 0x7fa540101a20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 -- 172.21.15.78:0/2228679217 >> 172.21.15.78:0/2228679217 conn(0x7fa5400fd3a0 msgr2=0x7fa5400ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 -- 172.21.15.78:0/2228679217 shutdown_connections 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.538+0000 7fa546957700 1 -- 172.21.15.78:0/2228679217 wait complete. 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.539+0000 7fa546957700 1 Processor -- start 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.539+0000 7fa546957700 1 -- start start 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.539+0000 7fa546957700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540101610 0x7fa540199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.539+0000 7fa546957700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540101f60 0x7fa54019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.539+0000 7fa546957700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540108180 0x7fa54019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.539+0000 7fa546957700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa540113570 con 0x7fa540108180 2024-05-21T15:15:06.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.539+0000 7fa546957700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401133f0 con 0x7fa540101610 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.539+0000 7fa546957700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5401136f0 con 0x7fa540101f60 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540108180 0x7fa54019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540108180 0x7fa54019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33658/0 (socket says 172.21.15.78:33658) 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 -- 172.21.15.78:0/2785345233 learned_addr learned my addr 172.21.15.78:0/2785345233 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa53ffff700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540101610 0x7fa540199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa53f7fe700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540101f60 0x7fa54019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 -- 172.21.15.78:0/2785345233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540101f60 msgr2=0x7fa54019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540101f60 0x7fa54019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 -- 172.21.15.78:0/2785345233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540101610 msgr2=0x7fa540199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540101610 0x7fa540199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 -- 172.21.15.78:0/2785345233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54019ec30 con 0x7fa540108180 2024-05-21T15:15:06.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa544ef4700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540108180 0x7fa54019e640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa538007d70 tx=0x7fa53800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:06.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa53d7fa700 1 -- 172.21.15.78:0/2785345233 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538016020 con 0x7fa540108180 2024-05-21T15:15:06.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa53d7fa700 1 -- 172.21.15.78:0/2785345233 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53800ac90 con 0x7fa540108180 2024-05-21T15:15:06.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa53d7fa700 1 -- 172.21.15.78:0/2785345233 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5380115c0 con 0x7fa540108180 2024-05-21T15:15:06.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54019ef20 con 0x7fa540108180 2024-05-21T15:15:06.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.540+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54019f3e0 con 0x7fa540108180 2024-05-21T15:15:06.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.541+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa540050170 con 0x7fa540108180 2024-05-21T15:15:06.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.544+0000 7fa53d7fa700 1 -- 172.21.15.78:0/2785345233 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa538019020 con 0x7fa540108180 2024-05-21T15:15:06.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.544+0000 7fa53d7fa700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5280a2d30 0x7fa5280a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:06.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.545+0000 7fa53ffff700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5280a2d30 0x7fa5280a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:06.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.545+0000 7fa53d7fa700 1 -- 172.21.15.78:0/2785345233 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa5380cf230 con 0x7fa540108180 2024-05-21T15:15:06.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.545+0000 7fa53ffff700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5280a2d30 0x7fa5280a5240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa54006b8f0 tx=0x7fa53000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:06.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.548+0000 7fa53d7fa700 1 -- 172.21.15.78:0/2785345233 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa538020020 con 0x7fa540108180 2024-05-21T15:15:06.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:06 smithi084 ceph-mon[36334]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-05-21T15:15:06.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:06 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/567815780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T15:15:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:06 smithi172 ceph-mon[36402]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-05-21T15:15:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:06 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/567815780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T15:15:06.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.792+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fa54019b170 con 0x7fa540108180 2024-05-21T15:15:06.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.793+0000 7fa53d7fa700 1 -- 172.21.15.78:0/2785345233 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa53809c2f0 con 0x7fa540108180 2024-05-21T15:15:06.796 INFO:teuthology.orchestra.run.smithi078.stdout:343597383704 2024-05-21T15:15:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5280a2d30 msgr2=0x7fa5280a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5280a2d30 0x7fa5280a5240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa54006b8f0 tx=0x7fa53000a380 comp rx=0 tx=0).stop 2024-05-21T15:15:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540108180 msgr2=0x7fa54019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540108180 0x7fa54019e640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa538007d70 tx=0x7fa53800a340 comp rx=0 tx=0).stop 2024-05-21T15:15:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 shutdown_connections 2024-05-21T15:15:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5280a2d30 0x7fa5280a5240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa540108180 0x7fa54019e640 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa540101610 0x7fa540199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 --2- 172.21.15.78:0/2785345233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa540101f60 0x7fa54019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 >> 172.21.15.78:0/2785345233 conn(0x7fa5400fd3a0 msgr2=0x7fa540105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 shutdown_connections 2024-05-21T15:15:06.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:06.795+0000 7fa546957700 1 -- 172.21.15.78:0/2785345233 wait complete. 2024-05-21T15:15:07.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:07 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2785345233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T15:15:07.500 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383698 got 343597383704 for osd.11 2024-05-21T15:15:07.500 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:07.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:07 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2785345233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T15:15:07.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:07 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2785345233' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T15:15:08.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:08 smithi078 ceph-mon[31997]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:08.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:08 smithi084 ceph-mon[36334]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:08.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:08 smithi172 ceph-mon[36402]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:10.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:10 smithi078 ceph-mon[31997]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:10 smithi084 ceph-mon[36334]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:10.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:10 smithi172 ceph-mon[36402]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:10.808 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738446 for osd.0 2024-05-21T15:15:10.808 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.654+0000 7f1613ec7700 1 -- 172.21.15.78:0/2680083872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 msgr2=0x7f160c102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.654+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2680083872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 0x7f160c102770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f16080099c0 tx=0x7f1608009cd0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 -- 172.21.15.78:0/2680083872 shutdown_connections 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2680083872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f160c1075e0 0x7f160c10ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2680083872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 0x7f160c102770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2680083872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f160c106c90 0x7f160c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 -- 172.21.15.78:0/2680083872 >> 172.21.15.78:0/2680083872 conn(0x7f160c0fddb0 msgr2=0x7f160c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 -- 172.21.15.78:0/2680083872 shutdown_connections 2024-05-21T15:15:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 -- 172.21.15.78:0/2680083872 wait complete. 2024-05-21T15:15:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 Processor -- start 2024-05-21T15:15:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.655+0000 7f1613ec7700 1 -- start start 2024-05-21T15:15:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1613ec7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 0x7f160c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1613ec7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f160c106c90 0x7f160c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1613ec7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f160c1075e0 0x7f160c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1613ec7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f160c112360 con 0x7f160c102360 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1613ec7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1121e0 con 0x7f160c106c90 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1613ec7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1124e0 con 0x7f160c1075e0 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 0x7f160c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 0x7f160c199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44756/0 (socket says 172.21.15.78:44756) 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 -- 172.21.15.78:0/2268212970 learned_addr learned my addr 172.21.15.78:0/2268212970 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1612464700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f160c1075e0 0x7f160c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611462700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f160c106c90 0x7f160c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 -- 172.21.15.78:0/2268212970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f160c1075e0 msgr2=0x7f160c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f160c1075e0 0x7f160c19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 -- 172.21.15.78:0/2268212970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f160c106c90 msgr2=0x7f160c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f160c106c90 0x7f160c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 -- 172.21.15.78:0/2268212970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c19ec70 con 0x7f160c102360 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1611c63700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 0x7f160c199da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f16080059d0 tx=0x7f16080038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.656+0000 7f1602ffd700 1 -- 172.21.15.78:0/2268212970 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608027070 con 0x7f160c102360 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.657+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c19ef00 con 0x7f160c102360 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.657+0000 7f1602ffd700 1 -- 172.21.15.78:0/2268212970 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1608004050 con 0x7f160c102360 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.657+0000 7f1602ffd700 1 -- 172.21.15.78:0/2268212970 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608022db0 con 0x7f160c102360 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.657+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c0fe1c0 con 0x7f160c102360 2024-05-21T15:15:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.658+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160c103810 con 0x7f160c102360 2024-05-21T15:15:11.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.659+0000 7f1602ffd700 1 -- 172.21.15.78:0/2268212970 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f160802a070 con 0x7f160c102360 2024-05-21T15:15:11.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.660+0000 7f1602ffd700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15f80a2d30 0x7f15f80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:11.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.660+0000 7f1611462700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15f80a2d30 0x7f15f80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:11.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.661+0000 7f1602ffd700 1 -- 172.21.15.78:0/2268212970 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f16080d0480 con 0x7f160c102360 2024-05-21T15:15:11.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.661+0000 7f1611462700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15f80a2d30 0x7f15f80a5240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005fd0 tx=0x7f15fc005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.664+0000 7f1602ffd700 1 -- 172.21.15.78:0/2268212970 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1608031080 con 0x7f160c102360 2024-05-21T15:15:11.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.887+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f160c19b100 con 0x7f160c102360 2024-05-21T15:15:11.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.888+0000 7f1602ffd700 1 -- 172.21.15.78:0/2268212970 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1608013070 con 0x7f160c102360 2024-05-21T15:15:11.892 INFO:teuthology.orchestra.run.smithi078.stdout:141733920827 2024-05-21T15:15:11.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15f80a2d30 msgr2=0x7f15f80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15f80a2d30 0x7f15f80a5240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005fd0 tx=0x7f15fc005f60 comp rx=0 tx=0).stop 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 msgr2=0x7f160c199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 0x7f160c199da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f16080059d0 tx=0x7f16080038b0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 shutdown_connections 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15f80a2d30 0x7f15f80a5240 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f160c1075e0 0x7f160c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f160c102360 0x7f160c199da0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 --2- 172.21.15.78:0/2268212970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f160c106c90 0x7f160c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 >> 172.21.15.78:0/2268212970 conn(0x7f160c0fddb0 msgr2=0x7f160c06b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 shutdown_connections 2024-05-21T15:15:11.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:11.890+0000 7f1613ec7700 1 -- 172.21.15.78:0/2268212970 wait complete. 2024-05-21T15:15:12.608 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:12 smithi172 ceph-mon[36402]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:12.608 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:12 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:15:12.608 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:12 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2268212970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T15:15:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:12 smithi084 ceph-mon[36334]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:12 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:15:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:12 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2268212970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T15:15:12.667 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:12 smithi078 ceph-mon[31997]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:12.667 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:12 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:15:12.667 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:12 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2268212970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 -- 172.21.15.78:0/574511763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 msgr2=0x7fc92010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 --2- 172.21.15.78:0/574511763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 0x7fc92010ef60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc91800b0d0 tx=0x7fc91800b3e0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 -- 172.21.15.78:0/574511763 shutdown_connections 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 --2- 172.21.15.78:0/574511763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 0x7fc92010ef60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 --2- 172.21.15.78:0/574511763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9201023d0 0x7fc9201027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 --2- 172.21.15.78:0/574511763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc920106cb0 0x7fc9201070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 -- 172.21.15.78:0/574511763 >> 172.21.15.78:0/574511763 conn(0x7fc920077c80 msgr2=0x7fc9200780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 -- 172.21.15.78:0/574511763 shutdown_connections 2024-05-21T15:15:12.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.683+0000 7fc927744700 1 -- 172.21.15.78:0/574511763 wait complete. 2024-05-21T15:15:12.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc927744700 1 Processor -- start 2024-05-21T15:15:12.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc927744700 1 -- start start 2024-05-21T15:15:12.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc927744700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9201023d0 0x7fc92019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc927744700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc920106cb0 0x7fc92019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc927744700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 0x7fc92019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc927744700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc920112380 con 0x7fc920107600 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc927744700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc920112200 con 0x7fc9201023d0 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc927744700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc920112500 con 0x7fc920106cb0 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc925ce1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 0x7fc92019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc925ce1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 0x7fc92019b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44768/0 (socket says 172.21.15.78:44768) 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc925ce1700 1 -- 172.21.15.78:0/542926035 learned_addr learned my addr 172.21.15.78:0/542926035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc9254e0700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9201023d0 0x7fc92019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc924cdf700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc920106cb0 0x7fc92019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:12.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc925ce1700 1 -- 172.21.15.78:0/542926035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc920106cb0 msgr2=0x7fc92019b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc925ce1700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc920106cb0 0x7fc92019b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc925ce1700 1 -- 172.21.15.78:0/542926035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9201023d0 msgr2=0x7fc92019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.684+0000 7fc925ce1700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9201023d0 0x7fc92019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.685+0000 7fc925ce1700 1 -- 172.21.15.78:0/542926035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9201a5e80 con 0x7fc920107600 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.685+0000 7fc925ce1700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 0x7fc92019b660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc918006070 tx=0x7fc918004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.685+0000 7fc9127fc700 1 -- 172.21.15.78:0/542926035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918021070 con 0x7fc920107600 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.685+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9201a6110 con 0x7fc920107600 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.685+0000 7fc9127fc700 1 -- 172.21.15.78:0/542926035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc918007b80 con 0x7fc920107600 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.685+0000 7fc9127fc700 1 -- 172.21.15.78:0/542926035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918005330 con 0x7fc920107600 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.685+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9201a6610 con 0x7fc920107600 2024-05-21T15:15:12.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.686+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc920050170 con 0x7fc920107600 2024-05-21T15:15:12.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.687+0000 7fc9127fc700 1 -- 172.21.15.78:0/542926035 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc918024070 con 0x7fc920107600 2024-05-21T15:15:12.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.688+0000 7fc9127fc700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc90c0a2cb0 0x7fc90c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:12.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.688+0000 7fc9254e0700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc90c0a2cb0 0x7fc90c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:12.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.688+0000 7fc9127fc700 1 -- 172.21.15.78:0/542926035 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc91800f020 con 0x7fc920107600 2024-05-21T15:15:12.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.688+0000 7fc9254e0700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc90c0a2cb0 0x7fc90c0a51c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc91c005950 tx=0x7fc91c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:12.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.691+0000 7fc9127fc700 1 -- 172.21.15.78:0/542926035 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc91802b020 con 0x7fc920107600 2024-05-21T15:15:12.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.889+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc9201a6930 con 0x7fc920107600 2024-05-21T15:15:12.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.889+0000 7fc9127fc700 1 -- 172.21.15.78:0/542926035 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc91809dae0 con 0x7fc920107600 2024-05-21T15:15:12.893 INFO:teuthology.orchestra.run.smithi078.stdout:309237645341 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc90c0a2cb0 msgr2=0x7fc90c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc90c0a2cb0 0x7fc90c0a51c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc91c005950 tx=0x7fc91c00b410 comp rx=0 tx=0).stop 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 msgr2=0x7fc92019b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 0x7fc92019b660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc918006070 tx=0x7fc918004d00 comp rx=0 tx=0).stop 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 shutdown_connections 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc90c0a2cb0 0x7fc90c0a51c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc920107600 0x7fc92019b660 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9201023d0 0x7fc92019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 --2- 172.21.15.78:0/542926035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc920106cb0 0x7fc92019b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 >> 172.21.15.78:0/542926035 conn(0x7fc920077c80 msgr2=0x7fc9200ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 shutdown_connections 2024-05-21T15:15:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:12.892+0000 7fc927744700 1 -- 172.21.15.78:0/542926035 wait complete. 2024-05-21T15:15:13.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:13 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/542926035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T15:15:13.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:13 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:13 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/542926035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T15:15:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:13 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:13 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:13 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:13 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:13 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/542926035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T15:15:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:13 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 -- 172.21.15.78:0/3867307175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 msgr2=0x7f7cb81081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/3867307175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 0x7f7cb81081b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0099c0 tx=0x7f7cac009cd0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 -- 172.21.15.78:0/3867307175 shutdown_connections 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/3867307175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb81086f0 0x7f7cb8110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/3867307175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb8103470 0x7f7cb8103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/3867307175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 0x7f7cb81081b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 -- 172.21.15.78:0/3867307175 >> 172.21.15.78:0/3867307175 conn(0x7f7cb80fed40 msgr2=0x7f7cb8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 -- 172.21.15.78:0/3867307175 shutdown_connections 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.721+0000 7f7cbcc51700 1 -- 172.21.15.78:0/3867307175 wait complete. 2024-05-21T15:15:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cbcc51700 1 Processor -- start 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cbcc51700 1 -- start start 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cbcc51700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb8103470 0x7f7cb8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cbcc51700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 0x7f7cb819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cbcc51700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb81086f0 0x7f7cb819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cbcc51700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb8113450 con 0x7f7cb8107da0 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cbcc51700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb81132d0 con 0x7f7cb81086f0 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cbcc51700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb81135d0 con 0x7f7cb8103470 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cb5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 0x7f7cb819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cb5d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 0x7f7cb819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44778/0 (socket says 172.21.15.78:44778) 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cb5d9b700 1 -- 172.21.15.78:0/2122939736 learned_addr learned my addr 172.21.15.78:0/2122939736 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cb659c700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb8103470 0x7f7cb8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.722+0000 7f7cb6d9d700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb81086f0 0x7f7cb819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7cb5d9b700 1 -- 172.21.15.78:0/2122939736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb8103470 msgr2=0x7f7cb8199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7cb5d9b700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb8103470 0x7f7cb8199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7cb5d9b700 1 -- 172.21.15.78:0/2122939736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb81086f0 msgr2=0x7f7cb819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7cb5d9b700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb81086f0 0x7f7cb819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7cb5d9b700 1 -- 172.21.15.78:0/2122939736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb819ec70 con 0x7f7cb8107da0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7cb5d9b700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 0x7f7cb819a340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c010 tx=0x7f7cac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7ca77fe700 1 -- 172.21.15.78:0/2122939736 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac004830 con 0x7f7cb8107da0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb819ef00 con 0x7f7cb8107da0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7ca77fe700 1 -- 172.21.15.78:0/2122939736 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac005020 con 0x7f7cb8107da0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7ca77fe700 1 -- 172.21.15.78:0/2122939736 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac003cb0 con 0x7f7cb8107da0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.723+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb819f3d0 con 0x7f7cb8107da0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.724+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb8050170 con 0x7f7cb8107da0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.725+0000 7f7ca77fe700 1 -- 172.21.15.78:0/2122939736 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7cac0049d0 con 0x7f7cb8107da0 2024-05-21T15:15:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.726+0000 7f7ca77fe700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7ca00a2cb0 0x7f7ca00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.726+0000 7f7cb659c700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7ca00a2cb0 0x7f7ca00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.726+0000 7f7ca77fe700 1 -- 172.21.15.78:0/2122939736 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7cac0cff10 con 0x7f7cb8107da0 2024-05-21T15:15:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.726+0000 7f7cb659c700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7ca00a2cb0 0x7f7ca00a51c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8005950 tx=0x7f7ca800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.731+0000 7f7ca77fe700 1 -- 172.21.15.78:0/2122939736 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7cac013070 con 0x7f7cb8107da0 2024-05-21T15:15:13.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.939+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7cb802dc60 con 0x7f7cb8107da0 2024-05-21T15:15:13.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.939+0000 7f7ca77fe700 1 -- 172.21.15.78:0/2122939736 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7cac0095a0 con 0x7f7cb8107da0 2024-05-21T15:15:13.943 INFO:teuthology.orchestra.run.smithi078.stdout:55834574921 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7ca00a2cb0 msgr2=0x7f7ca00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7ca00a2cb0 0x7f7ca00a51c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8005950 tx=0x7f7ca800b410 comp rx=0 tx=0).stop 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 msgr2=0x7f7cb819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 0x7f7cb819a340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c010 tx=0x7f7cac0047b0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 shutdown_connections 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7ca00a2cb0 0x7f7ca00a51c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb81086f0 0x7f7cb819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7cb8103470 0x7f7cb8199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 --2- 172.21.15.78:0/2122939736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cb8107da0 0x7f7cb819a340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 >> 172.21.15.78:0/2122939736 conn(0x7f7cb80fed40 msgr2=0x7f7cb8109650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:13.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 shutdown_connections 2024-05-21T15:15:13.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:13.942+0000 7f7cbcc51700 1 -- 172.21.15.78:0/2122939736 wait complete. 2024-05-21T15:15:14.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:14 smithi078 ceph-mon[31997]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:14.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:14 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2122939736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T15:15:14.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:14 smithi084 ceph-mon[36334]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:14 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2122939736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T15:15:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:14 smithi172 ceph-mon[36402]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:14 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2122939736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T15:15:15.372 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574915 got 55834574921 for osd.1 2024-05-21T15:15:15.373 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:15.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-05-21T15:15:15.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:15 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-21T15:15:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-21T15:15:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-21T15:15:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-05-21T15:15:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-05-21T15:15:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-21T15:15:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-21T15:15:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-05-21T15:15:15.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2024-05-21T15:15:15.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-05-21T15:15:15.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:15 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-21T15:15:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-21T15:15:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-21T15:15:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-05-21T15:15:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-05-21T15:15:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-21T15:15:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-21T15:15:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-05-21T15:15:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]: dispatch 2024-05-21T15:15:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-05-21T15:15:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:15 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-05-21T15:15:16.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:16.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-21T15:15:16.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-21T15:15:16.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-05-21T15:15:16.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-05-21T15:15:16.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-21T15:15:16.577 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-21T15:15:16.577 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-05-21T15:15:16.577 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2024-05-21T15:15:16.577 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-05-21T15:15:16.577 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-21T15:15:16.577 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:16 smithi078 ceph-mon[31997]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T15:15:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 -- 172.21.15.78:0/2773262834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f411010d750 msgr2=0x7f411010fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 --2- 172.21.15.78:0/2773262834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f411010d750 0x7f411010fc30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f410800b0a0 tx=0x7f410800b3b0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 -- 172.21.15.78:0/2773262834 shutdown_connections 2024-05-21T15:15:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 --2- 172.21.15.78:0/2773262834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f411010d750 0x7f411010fc30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 --2- 172.21.15.78:0/2773262834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41101042e0 0x7f41101046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 --2- 172.21.15.78:0/2773262834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4110104c30 0x7f411010d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 -- 172.21.15.78:0/2773262834 >> 172.21.15.78:0/2773262834 conn(0x7f41100fdc40 msgr2=0x7f4110100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 -- 172.21.15.78:0/2773262834 shutdown_connections 2024-05-21T15:15:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.603+0000 7f4115e63700 1 -- 172.21.15.78:0/2773262834 wait complete. 2024-05-21T15:15:16.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.604+0000 7f4115e63700 1 Processor -- start 2024-05-21T15:15:16.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.604+0000 7f4115e63700 1 -- start start 2024-05-21T15:15:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.604+0000 7f4115e63700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101042e0 0x7f4110199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.604+0000 7f4115e63700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4110104c30 0x7f411019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f4115e63700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411010d750 0x7f411019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f4115e63700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41101134f0 con 0x7f41101042e0 2024-05-21T15:15:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f4115e63700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4110113370 con 0x7f4110104c30 2024-05-21T15:15:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f4115e63700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4110113670 con 0x7f411010d750 2024-05-21T15:15:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101042e0 0x7f4110199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101042e0 0x7f4110199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44804/0 (socket says 172.21.15.78:44804) 2024-05-21T15:15:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 -- 172.21.15.78:0/3320641723 learned_addr learned my addr 172.21.15.78:0/3320641723 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410ffff700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411010d750 0x7f411019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410effd700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4110104c30 0x7f411019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 -- 172.21.15.78:0/3320641723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411010d750 msgr2=0x7f411019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411010d750 0x7f411019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 -- 172.21.15.78:0/3320641723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4110104c30 msgr2=0x7f411019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4110104c30 0x7f411019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 -- 172.21.15.78:0/3320641723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411019ec70 con 0x7f41101042e0 2024-05-21T15:15:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410effd700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4110104c30 0x7f411019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410f7fe700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101042e0 0x7f4110199da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f410400b530 tx=0x7f410400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410cff9700 1 -- 172.21.15.78:0/3320641723 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104015020 con 0x7f41101042e0 2024-05-21T15:15:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411019ef60 con 0x7f41101042e0 2024-05-21T15:15:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410cff9700 1 -- 172.21.15.78:0/3320641723 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4104011d10 con 0x7f41101042e0 2024-05-21T15:15:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f410cff9700 1 -- 172.21.15.78:0/3320641723 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104010e50 con 0x7f41101042e0 2024-05-21T15:15:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.605+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41101a6880 con 0x7f41101042e0 2024-05-21T15:15:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.607+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4110050170 con 0x7f41101042e0 2024-05-21T15:15:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.608+0000 7f410cff9700 1 -- 172.21.15.78:0/3320641723 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4104018020 con 0x7f41101042e0 2024-05-21T15:15:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.608+0000 7f410cff9700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f40f80a2e50 0x7f40f80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.608+0000 7f410effd700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f40f80a2e50 0x7f40f80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.608+0000 7f410cff9700 1 -- 172.21.15.78:0/3320641723 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f41040ce980 con 0x7f41101042e0 2024-05-21T15:15:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.609+0000 7f410effd700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f40f80a2e50 0x7f40f80a5360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41101010e0 tx=0x7f4100008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.613+0000 7f410cff9700 1 -- 172.21.15.78:0/3320641723 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f410401f020 con 0x7f41101042e0 2024-05-21T15:15:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-21T15:15:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-21T15:15:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-05-21T15:15:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-05-21T15:15:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-21T15:15:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-21T15:15:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-05-21T15:15:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2024-05-21T15:15:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-05-21T15:15:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-21T15:15:16.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:16 smithi084 ceph-mon[36334]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T15:15:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 6]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-05-21T15:15:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:16 smithi172 ceph-mon[36402]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T15:15:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 -- 172.21.15.78:0/1211094629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101660 msgr2=0x7f2228101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1211094629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101660 0x7f2228101a70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2224009990 tx=0x7f2224009ca0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 -- 172.21.15.78:0/1211094629 shutdown_connections 2024-05-21T15:15:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1211094629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2228101fb0 0x7f222810ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1211094629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2228109260 0x7f2228109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1211094629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101660 0x7f2228101a70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 -- 172.21.15.78:0/1211094629 >> 172.21.15.78:0/1211094629 conn(0x7f22280fd570 msgr2=0x7f22280ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 -- 172.21.15.78:0/1211094629 shutdown_connections 2024-05-21T15:15:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.794+0000 7f222f4fe700 1 -- 172.21.15.78:0/1211094629 wait complete. 2024-05-21T15:15:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222f4fe700 1 Processor -- start 2024-05-21T15:15:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222f4fe700 1 -- start start 2024-05-21T15:15:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222f4fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2228101660 0x7f2228102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222f4fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101fb0 0x7f2228102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222f4fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2228109260 0x7f22281032c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222f4fe700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2228113430 con 0x7f2228101fb0 2024-05-21T15:15:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222f4fe700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22281132b0 con 0x7f2228101660 2024-05-21T15:15:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222f4fe700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22281135b0 con 0x7f2228109260 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222ca99700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101fb0 0x7f2228102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.795+0000 7f222ca99700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101fb0 0x7f2228102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44812/0 (socket says 172.21.15.78:44812) 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222ca99700 1 -- 172.21.15.78:0/1237364240 learned_addr learned my addr 172.21.15.78:0/1237364240 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222da9b700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2228109260 0x7f22281032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222d29a700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2228101660 0x7f2228102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222ca99700 1 -- 172.21.15.78:0/1237364240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2228109260 msgr2=0x7f22281032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222ca99700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2228109260 0x7f22281032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222ca99700 1 -- 172.21.15.78:0/1237364240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2228101660 msgr2=0x7f2228102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222ca99700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2228101660 0x7f2228102840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222ca99700 1 -- 172.21.15.78:0/1237364240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22281a81a0 con 0x7f2228101fb0 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222da9b700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2228109260 0x7f22281032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222d29a700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2228101660 0x7f2228102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222ca99700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101fb0 0x7f2228102d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2224000c00 tx=0x7f2224003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f221e7fc700 1 -- 172.21.15.78:0/1237364240 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2224027070 con 0x7f2228101fb0 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f221e7fc700 1 -- 172.21.15.78:0/1237364240 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2224004190 con 0x7f2228101fb0 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22281a8430 con 0x7f2228101fb0 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f221e7fc700 1 -- 172.21.15.78:0/1237364240 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222400f460 con 0x7f2228101fb0 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.796+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22281a8880 con 0x7f2228101fb0 2024-05-21T15:15:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.797+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2228050170 con 0x7f2228101fb0 2024-05-21T15:15:16.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.799+0000 7f221e7fc700 1 -- 172.21.15.78:0/1237364240 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f222402a070 con 0x7f2228101fb0 2024-05-21T15:15:16.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.800+0000 7f221e7fc700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f22140a2cb0 0x7f22140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:16.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.800+0000 7f222d29a700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f22140a2cb0 0x7f22140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:16.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.800+0000 7f221e7fc700 1 -- 172.21.15.78:0/1237364240 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f22240d0550 con 0x7f2228101fb0 2024-05-21T15:15:16.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.800+0000 7f222d29a700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f22140a2cb0 0x7f22140a51c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2218005fd0 tx=0x7f2218005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:16.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.803+0000 7f221e7fc700 1 -- 172.21.15.78:0/1237364240 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2224013070 con 0x7f2228101fb0 2024-05-21T15:15:16.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.822+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f411019b100 con 0x7f41101042e0 2024-05-21T15:15:16.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.822+0000 7f410cff9700 1 -- 172.21.15.78:0/3320641723 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f410409b870 con 0x7f41101042e0 2024-05-21T15:15:16.827 INFO:teuthology.orchestra.run.smithi078.stdout:219043332141 2024-05-21T15:15:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f40f80a2e50 msgr2=0x7f40f80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f40f80a2e50 0x7f40f80a5360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41101010e0 tx=0x7f4100008040 comp rx=0 tx=0).stop 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101042e0 msgr2=0x7f4110199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101042e0 0x7f4110199da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f410400b530 tx=0x7f410400b8f0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 shutdown_connections 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f40f80a2e50 0x7f40f80a5360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f411010d750 0x7f411019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41101042e0 0x7f4110199da0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 --2- 172.21.15.78:0/3320641723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4110104c30 0x7f411019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.824+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 >> 172.21.15.78:0/3320641723 conn(0x7f41100fdc40 msgr2=0x7f41100fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.825+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 shutdown_connections 2024-05-21T15:15:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:16.825+0000 7f4115e63700 1 -- 172.21.15.78:0/3320641723 wait complete. 2024-05-21T15:15:17.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.025+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f222802dc60 con 0x7f2228101fb0 2024-05-21T15:15:17.030 INFO:teuthology.orchestra.run.smithi078.stdout:287762808868 2024-05-21T15:15:17.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.026+0000 7f221e7fc700 1 -- 172.21.15.78:0/1237364240 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f222409d3c0 con 0x7f2228101fb0 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f22140a2cb0 msgr2=0x7f22140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f22140a2cb0 0x7f22140a51c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2218005fd0 tx=0x7f2218005db0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101fb0 msgr2=0x7f2228102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101fb0 0x7f2228102d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2224000c00 tx=0x7f2224003730 comp rx=0 tx=0).stop 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 shutdown_connections 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f22140a2cb0 0x7f22140a51c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2228109260 0x7f22281032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2228101660 0x7f2228102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 --2- 172.21.15.78:0/1237364240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2228101fb0 0x7f2228102d80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 >> 172.21.15.78:0/1237364240 conn(0x7f22280fd570 msgr2=0x7f22280ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:17.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 shutdown_connections 2024-05-21T15:15:17.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.028+0000 7f222f4fe700 1 -- 172.21.15.78:0/1237364240 wait complete. 2024-05-21T15:15:17.426 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920827 for osd.4 2024-05-21T15:15:17.427 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:17.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.626+0000 7f7a5838b700 1 -- 172.21.15.78:0/4195591125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a50107080 msgr2=0x7f7a50107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.626+0000 7f7a5838b700 1 --2- 172.21.15.78:0/4195591125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a50107080 0x7f7a50107490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009990 tx=0x7f7a4c009ca0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.626+0000 7f7a5838b700 1 -- 172.21.15.78:0/4195591125 shutdown_connections 2024-05-21T15:15:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.626+0000 7f7a5838b700 1 --2- 172.21.15.78:0/4195591125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a501030c0 0x7f7a5010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.626+0000 7f7a5838b700 1 --2- 172.21.15.78:0/4195591125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a50107080 0x7f7a50107490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.626+0000 7f7a5838b700 1 --2- 172.21.15.78:0/4195591125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a50102770 0x7f7a50102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.626+0000 7f7a5838b700 1 -- 172.21.15.78:0/4195591125 >> 172.21.15.78:0/4195591125 conn(0x7f7a500fd570 msgr2=0x7f7a500ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:17.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:17 smithi078 ceph-mon[31997]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T15:15:17.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:17 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3320641723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T15:15:17.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:17 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1237364240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T15:15:17.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:17 smithi078 ceph-mon[31997]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:15:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.627+0000 7f7a5838b700 1 -- 172.21.15.78:0/4195591125 shutdown_connections 2024-05-21T15:15:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.627+0000 7f7a5838b700 1 -- 172.21.15.78:0/4195591125 wait complete. 2024-05-21T15:15:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.627+0000 7f7a5838b700 1 Processor -- start 2024-05-21T15:15:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.627+0000 7f7a5838b700 1 -- start start 2024-05-21T15:15:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a5838b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a50102770 0x7f7a50104be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:17.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a5838b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501030c0 0x7f7a50105120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a5838b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a50107080 0x7f7a50105660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a5838b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50112360 con 0x7f7a501030c0 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a5838b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501121e0 con 0x7f7a50107080 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a5838b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501124e0 con 0x7f7a50102770 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501030c0 0x7f7a50105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501030c0 0x7f7a50105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44838/0 (socket says 172.21.15.78:44838) 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 -- 172.21.15.78:0/1678957783 learned_addr learned my addr 172.21.15.78:0/1678957783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a56127700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a50102770 0x7f7a50104be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 -- 172.21.15.78:0/1678957783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a50102770 msgr2=0x7f7a50104be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a50102770 0x7f7a50104be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a56928700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a50107080 0x7f7a50105660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 -- 172.21.15.78:0/1678957783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a50107080 msgr2=0x7f7a50105660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:17.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a50107080 0x7f7a50105660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 -- 172.21.15.78:0/1678957783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a501a8200 con 0x7f7a501030c0 2024-05-21T15:15:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a55926700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501030c0 0x7f7a50105120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000d980 tx=0x7f7a4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a477fe700 1 -- 172.21.15.78:0/1678957783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40016070 con 0x7f7a501030c0 2024-05-21T15:15:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a477fe700 1 -- 172.21.15.78:0/1678957783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a40004d10 con 0x7f7a501030c0 2024-05-21T15:15:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a501a84f0 con 0x7f7a501030c0 2024-05-21T15:15:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.628+0000 7f7a477fe700 1 -- 172.21.15.78:0/1678957783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400055f0 con 0x7f7a501030c0 2024-05-21T15:15:17.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.629+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a501a8a30 con 0x7f7a501030c0 2024-05-21T15:15:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.630+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a50050170 con 0x7f7a501030c0 2024-05-21T15:15:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.631+0000 7f7a477fe700 1 -- 172.21.15.78:0/1678957783 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a40004750 con 0x7f7a501030c0 2024-05-21T15:15:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.631+0000 7f7a477fe700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7a3c0a2cb0 0x7f7a3c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.632+0000 7f7a56127700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7a3c0a2cb0 0x7f7a3c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:17.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.632+0000 7f7a477fe700 1 -- 172.21.15.78:0/1678957783 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f7a400cf020 con 0x7f7a501030c0 2024-05-21T15:15:17.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.632+0000 7f7a56127700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7a3c0a2cb0 0x7f7a3c0a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0059d0 tx=0x7f7a4c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:17.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.636+0000 7f7a477fe700 1 -- 172.21.15.78:0/1678957783 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7a4001f020 con 0x7f7a501030c0 2024-05-21T15:15:17.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:17 smithi084 ceph-mon[36334]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T15:15:17.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:17 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3320641723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T15:15:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:17 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1237364240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T15:15:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:17 smithi084 ceph-mon[36334]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:15:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:17 smithi172 ceph-mon[36402]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T15:15:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:17 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3320641723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T15:15:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:17 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1237364240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T15:15:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:17 smithi172 ceph-mon[36402]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:15:17.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.838+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7a5002dc60 con 0x7f7a501030c0 2024-05-21T15:15:17.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.838+0000 7f7a477fe700 1 -- 172.21.15.78:0/1678957783 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a4009be00 con 0x7f7a501030c0 2024-05-21T15:15:17.842 INFO:teuthology.orchestra.run.smithi078.stdout:107374182466 2024-05-21T15:15:17.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7a3c0a2cb0 msgr2=0x7f7a3c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:17.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7a3c0a2cb0 0x7f7a3c0a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0059d0 tx=0x7f7a4c01a040 comp rx=0 tx=0).stop 2024-05-21T15:15:17.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501030c0 msgr2=0x7f7a50105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:17.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501030c0 0x7f7a50105120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000d980 tx=0x7f7a4000dc90 comp rx=0 tx=0).stop 2024-05-21T15:15:17.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 shutdown_connections 2024-05-21T15:15:17.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7a3c0a2cb0 0x7f7a3c0a51c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a50107080 0x7f7a50105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a50102770 0x7f7a50104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 --2- 172.21.15.78:0/1678957783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a501030c0 0x7f7a50105120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:17.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 >> 172.21.15.78:0/1678957783 conn(0x7f7a500fd570 msgr2=0x7f7a501019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:17.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 shutdown_connections 2024-05-21T15:15:17.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:17.840+0000 7f7a5838b700 1 -- 172.21.15.78:0/1678957783 wait complete. 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.045+0000 7f2929df7700 1 -- 172.21.15.78:0/2924217777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29241075c0 msgr2=0x7f292410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.045+0000 7f2929df7700 1 --2- 172.21.15.78:0/2924217777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29241075c0 0x7f292410ef20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f291c00b0d0 tx=0x7f291c00b3e0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.045+0000 7f2929df7700 1 -- 172.21.15.78:0/2924217777 shutdown_connections 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.045+0000 7f2929df7700 1 --2- 172.21.15.78:0/2924217777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29241075c0 0x7f292410ef20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.045+0000 7f2929df7700 1 --2- 172.21.15.78:0/2924217777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2924102340 0x7f2924102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.045+0000 7f2929df7700 1 --2- 172.21.15.78:0/2924217777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2924106c70 0x7f2924107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.045+0000 7f2929df7700 1 -- 172.21.15.78:0/2924217777 >> 172.21.15.78:0/2924217777 conn(0x7f29240fdc70 msgr2=0x7f2924100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.046+0000 7f2929df7700 1 -- 172.21.15.78:0/2924217777 shutdown_connections 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.046+0000 7f2929df7700 1 -- 172.21.15.78:0/2924217777 wait complete. 2024-05-21T15:15:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.046+0000 7f2929df7700 1 Processor -- start 2024-05-21T15:15:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.046+0000 7f2929df7700 1 -- start start 2024-05-21T15:15:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2929df7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2924102340 0x7f292419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2929df7700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2924106c70 0x7f292419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2929df7700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29241075c0 0x7f2924199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2929df7700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113490 con 0x7f2924106c70 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2929df7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113310 con 0x7f2924102340 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2929df7700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113610 con 0x7f29241075c0 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2924106c70 0x7f292419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2924106c70 0x7f292419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44846/0 (socket says 172.21.15.78:44846) 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 -- 172.21.15.78:0/3202467112 learned_addr learned my addr 172.21.15.78:0/3202467112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2923fff700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29241075c0 0x7f2924199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f29237fe700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2924102340 0x7f292419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 -- 172.21.15.78:0/3202467112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29241075c0 msgr2=0x7f2924199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29241075c0 0x7f2924199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 -- 172.21.15.78:0/3202467112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2924102340 msgr2=0x7f292419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2924102340 0x7f292419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 -- 172.21.15.78:0/3202467112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292419a260 con 0x7f2924106c70 2024-05-21T15:15:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2922ffd700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2924106c70 0x7f292419fee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f291400d950 tx=0x7f291400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.047+0000 7f2920ff9700 1 -- 172.21.15.78:0/3202467112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914016070 con 0x7f2924106c70 2024-05-21T15:15:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.048+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292419a550 con 0x7f2924106c70 2024-05-21T15:15:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.048+0000 7f2920ff9700 1 -- 172.21.15.78:0/3202467112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2914004d10 con 0x7f2924106c70 2024-05-21T15:15:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.048+0000 7f2920ff9700 1 -- 172.21.15.78:0/3202467112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140055e0 con 0x7f2924106c70 2024-05-21T15:15:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.048+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29241a6750 con 0x7f2924106c70 2024-05-21T15:15:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.049+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29241037f0 con 0x7f2924106c70 2024-05-21T15:15:18.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.050+0000 7f2920ff9700 1 -- 172.21.15.78:0/3202467112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2914004750 con 0x7f2924106c70 2024-05-21T15:15:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.051+0000 7f2920ff9700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f290c0a2cb0 0x7f290c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.051+0000 7f29237fe700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f290c0a2cb0 0x7f290c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.051+0000 7f2920ff9700 1 -- 172.21.15.78:0/3202467112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f29140cef10 con 0x7f2924106c70 2024-05-21T15:15:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.051+0000 7f29237fe700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f290c0a2cb0 0x7f290c0a51c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2918005950 tx=0x7f29180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.055+0000 7f2920ff9700 1 -- 172.21.15.78:0/3202467112 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f291401f020 con 0x7f2924106c70 2024-05-21T15:15:18.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.268+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f292402dc60 con 0x7f2924106c70 2024-05-21T15:15:18.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.268+0000 7f2920ff9700 1 -- 172.21.15.78:0/3202467112 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f291409bcf0 con 0x7f2924106c70 2024-05-21T15:15:18.272 INFO:teuthology.orchestra.run.smithi078.stdout:197568495667 2024-05-21T15:15:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f290c0a2cb0 msgr2=0x7f290c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f290c0a2cb0 0x7f290c0a51c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2918005950 tx=0x7f29180058e0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2924106c70 msgr2=0x7f292419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2924106c70 0x7f292419fee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f291400d950 tx=0x7f291400dc60 comp rx=0 tx=0).stop 2024-05-21T15:15:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 shutdown_connections 2024-05-21T15:15:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f290c0a2cb0 0x7f290c0a51c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29241075c0 0x7f2924199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2924102340 0x7f292419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 --2- 172.21.15.78:0/3202467112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2924106c70 0x7f292419fee0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 >> 172.21.15.78:0/3202467112 conn(0x7f29240fdc70 msgr2=0x7f29241063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 shutdown_connections 2024-05-21T15:15:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.270+0000 7f2929df7700 1 -- 172.21.15.78:0/3202467112 wait complete. 2024-05-21T15:15:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:18 smithi078 ceph-mon[31997]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-21T15:15:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:18 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1678957783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T15:15:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:18 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3202467112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T15:15:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:18 smithi084 ceph-mon[36334]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-21T15:15:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:18 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1678957783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T15:15:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:18 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3202467112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T15:15:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:18 smithi172 ceph-mon[36402]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-21T15:15:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:18 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1678957783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T15:15:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:18 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3202467112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T15:15:18.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182459 got 107374182466 for osd.3 2024-05-21T15:15:18.978 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:19.571 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:19 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/732300649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T15:15:19.571 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:19 smithi078 ceph-mon[31997]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:15:19.601 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645335 got 309237645341 for osd.10 2024-05-21T15:15:19.601 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:19 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/732300649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T15:15:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:19 smithi084 ceph-mon[36334]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:15:19.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:19 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/732300649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T15:15:19.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:19 smithi172 ceph-mon[36402]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-21T15:15:20.191 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808863 got 287762808868 for osd.9 2024-05-21T15:15:20.191 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 -- 172.21.15.78:0/741679061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c1028e0 msgr2=0x7fc81c10f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 --2- 172.21.15.78:0/741679061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c1028e0 0x7fc81c10f600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc810009990 tx=0x7fc810009ca0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 -- 172.21.15.78:0/741679061 shutdown_connections 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 --2- 172.21.15.78:0/741679061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc81c10fb40 0x7fc81c111f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 --2- 172.21.15.78:0/741679061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc81c101f90 0x7fc81c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 --2- 172.21.15.78:0/741679061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c1028e0 0x7fc81c10f600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 -- 172.21.15.78:0/741679061 >> 172.21.15.78:0/741679061 conn(0x7fc81c077c80 msgr2=0x7fc81c0780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 -- 172.21.15.78:0/741679061 shutdown_connections 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 -- 172.21.15.78:0/741679061 wait complete. 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.498+0000 7fc821fd3700 1 Processor -- start 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc821fd3700 1 -- start start 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc821fd3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c101f90 0x7fc81c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc821fd3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc81c1028e0 0x7fc81c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc821fd3700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc81c10fb40 0x7fc81c1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc821fd3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c115670 con 0x7fc81c101f90 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc821fd3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1154f0 con 0x7fc81c1028e0 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc821fd3700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1157f0 con 0x7fc81c10fb40 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c101f90 0x7fc81c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c101f90 0x7fc81c19bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44870/0 (socket says 172.21.15.78:44870) 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 -- 172.21.15.78:0/732300649 learned_addr learned my addr 172.21.15.78:0/732300649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81bfff700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc81c10fb40 0x7fc81c1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81affd700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc81c1028e0 0x7fc81c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 -- 172.21.15.78:0/732300649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc81c10fb40 msgr2=0x7fc81c1a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc81c10fb40 0x7fc81c1a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 -- 172.21.15.78:0/732300649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc81c1028e0 msgr2=0x7fc81c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:20.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc81c1028e0 0x7fc81c19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 -- 172.21.15.78:0/732300649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c1a0e50 con 0x7fc81c101f90 2024-05-21T15:15:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.499+0000 7fc81b7fe700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c101f90 0x7fc81c19bf80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00d950 tx=0x7fc80c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.500+0000 7fc818ff9700 1 -- 172.21.15.78:0/732300649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c016070 con 0x7fc81c101f90 2024-05-21T15:15:20.319 INFO:teuthology.orchestra.run.smithi078.stdout:163208757304 2024-05-21T15:15:20.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.500+0000 7fc818ff9700 1 -- 172.21.15.78:0/732300649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc80c00fd70 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.500+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c1a1140 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.500+0000 7fc818ff9700 1 -- 172.21.15.78:0/732300649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c00e610 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.500+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c1a8a60 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.501+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc81c050170 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.502+0000 7fc818ff9700 1 -- 172.21.15.78:0/732300649 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc80c019070 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.503+0000 7fc818ff9700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc8040a2d30 0x7fc8040a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.503+0000 7fc81affd700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc8040a2d30 0x7fc8040a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.503+0000 7fc818ff9700 1 -- 172.21.15.78:0/732300649 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc80c0cec70 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.503+0000 7fc81affd700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc8040a2d30 0x7fc8040a5240 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc810006010 tx=0x7fc81001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.507+0000 7fc818ff9700 1 -- 172.21.15.78:0/732300649 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc80c09ba50 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.708+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc81c02dc60 con 0x7fc81c101f90 2024-05-21T15:15:20.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.708+0000 7fc818ff9700 1 -- 172.21.15.78:0/732300649 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc80c09b350 con 0x7fc81c101f90 2024-05-21T15:15:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc8040a2d30 msgr2=0x7fc8040a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc8040a2d30 0x7fc8040a5240 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc810006010 tx=0x7fc81001a040 comp rx=0 tx=0).stop 2024-05-21T15:15:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c101f90 msgr2=0x7fc81c19bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c101f90 0x7fc81c19bf80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00d950 tx=0x7fc80c00dc60 comp rx=0 tx=0).stop 2024-05-21T15:15:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 shutdown_connections 2024-05-21T15:15:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc8040a2d30 0x7fc8040a5240 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc81c10fb40 0x7fc81c1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc81c101f90 0x7fc81c19bf80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 --2- 172.21.15.78:0/732300649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc81c1028e0 0x7fc81c19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 >> 172.21.15.78:0/732300649 conn(0x7fc81c077c80 msgr2=0x7fc81c111020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 shutdown_connections 2024-05-21T15:15:20.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:18.710+0000 7fc821fd3700 1 -- 172.21.15.78:0/732300649 wait complete. 2024-05-21T15:15:20.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757304 for osd.5 2024-05-21T15:15:20.895 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:21.321 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495667 for osd.6 2024-05-21T15:15:21.322 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:21.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:21 smithi078 ceph-mon[31997]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 83 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:21 smithi078 ceph-mon[31997]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-21T15:15:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:21 smithi078 ceph-mon[31997]: Cluster is now healthy 2024-05-21T15:15:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:21 smithi084 ceph-mon[36334]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 83 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:21 smithi084 ceph-mon[36334]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-21T15:15:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:21 smithi084 ceph-mon[36334]: Cluster is now healthy 2024-05-21T15:15:21.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:21 smithi172 ceph-mon[36402]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 83 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:21.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:21 smithi172 ceph-mon[36402]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-21T15:15:21.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:21 smithi172 ceph-mon[36402]: Cluster is now healthy 2024-05-21T15:15:21.813 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332141 for osd.7 2024-05-21T15:15:21.813 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:22.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 -- 172.21.15.78:0/2455575895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 msgr2=0x7f5d20101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 --2- 172.21.15.78:0/2455575895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 0x7f5d20101d70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0099c0 tx=0x7f5d1c009cd0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 -- 172.21.15.78:0/2455575895 shutdown_connections 2024-05-21T15:15:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 --2- 172.21.15.78:0/2455575895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d201086d0 0x7f5d20110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 --2- 172.21.15.78:0/2455575895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 0x7f5d20101d70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 --2- 172.21.15.78:0/2455575895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d20107d80 0x7f5d20108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 -- 172.21.15.78:0/2455575895 >> 172.21.15.78:0/2455575895 conn(0x7f5d200fd360 msgr2=0x7f5d200ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 -- 172.21.15.78:0/2455575895 shutdown_connections 2024-05-21T15:15:22.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.589+0000 7f5d28448700 1 -- 172.21.15.78:0/2455575895 wait complete. 2024-05-21T15:15:22.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d28448700 1 Processor -- start 2024-05-21T15:15:22.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d28448700 1 -- start start 2024-05-21T15:15:22.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d28448700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 0x7f5d20199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:22.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d28448700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d20107d80 0x7f5d2019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:22.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d28448700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d201086d0 0x7f5d2019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d28448700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d201134d0 con 0x7f5d20101960 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d28448700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20113350 con 0x7f5d20107d80 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d28448700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20113650 con 0x7f5d201086d0 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d261e4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 0x7f5d20199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.590+0000 7f5d261e4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 0x7f5d20199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43842/0 (socket says 172.21.15.78:43842) 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d261e4700 1 -- 172.21.15.78:0/1652554045 learned_addr learned my addr 172.21.15.78:0/1652554045 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d259e3700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d20107d80 0x7f5d2019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d261e4700 1 -- 172.21.15.78:0/1652554045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d201086d0 msgr2=0x7f5d2019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d269e5700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d201086d0 0x7f5d2019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d261e4700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d201086d0 0x7f5d2019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d261e4700 1 -- 172.21.15.78:0/1652554045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d20107d80 msgr2=0x7f5d2019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d261e4700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d20107d80 0x7f5d2019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d261e4700 1 -- 172.21.15.78:0/1652554045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2019ec90 con 0x7f5d20101960 2024-05-21T15:15:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d259e3700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d20107d80 0x7f5d2019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d261e4700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 0x7f5d20199dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009fd0 tx=0x7f5d1c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d177fe700 1 -- 172.21.15.78:0/1652554045 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c027070 con 0x7f5d20101960 2024-05-21T15:15:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2019ef20 con 0x7f5d20101960 2024-05-21T15:15:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d177fe700 1 -- 172.21.15.78:0/1652554045 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c004150 con 0x7f5d20101960 2024-05-21T15:15:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d177fe700 1 -- 172.21.15.78:0/1652554045 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00f460 con 0x7f5d20101960 2024-05-21T15:15:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.591+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2019f3d0 con 0x7f5d20101960 2024-05-21T15:15:22.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.592+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d20050170 con 0x7f5d20101960 2024-05-21T15:15:22.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.593+0000 7f5d177fe700 1 -- 172.21.15.78:0/1652554045 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5d1c02a070 con 0x7f5d20101960 2024-05-21T15:15:22.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.594+0000 7f5d177fe700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5d0c0a2cb0 0x7f5d0c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:22.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.594+0000 7f5d259e3700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5d0c0a2cb0 0x7f5d0c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:22.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.594+0000 7f5d177fe700 1 -- 172.21.15.78:0/1652554045 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f5d1c0d0830 con 0x7f5d20101960 2024-05-21T15:15:22.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.595+0000 7f5d259e3700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5d0c0a2cb0 0x7f5d0c0a51c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d10005fd0 tx=0x7f5d10005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:22.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.598+0000 7f5d177fe700 1 -- 172.21.15.78:0/1652554045 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5d1c031020 con 0x7f5d20101960 2024-05-21T15:15:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.798+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5d2002dc60 con 0x7f5d20101960 2024-05-21T15:15:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.799+0000 7f5d177fe700 1 -- 172.21.15.78:0/1652554045 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d1c013070 con 0x7f5d20101960 2024-05-21T15:15:22.803 INFO:teuthology.orchestra.run.smithi078.stdout:253403070505 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5d0c0a2cb0 msgr2=0x7f5d0c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5d0c0a2cb0 0x7f5d0c0a51c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d10005fd0 tx=0x7f5d10005dc0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 msgr2=0x7f5d20199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 0x7f5d20199dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009fd0 tx=0x7f5d1c003730 comp rx=0 tx=0).stop 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 shutdown_connections 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f5d0c0a2cb0 0x7f5d0c0a51c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d201086d0 0x7f5d2019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d20101960 0x7f5d20199dc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 --2- 172.21.15.78:0/1652554045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d20107d80 0x7f5d2019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 >> 172.21.15.78:0/1652554045 conn(0x7f5d200fd360 msgr2=0x7f5d201074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 shutdown_connections 2024-05-21T15:15:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:22.801+0000 7f5d28448700 1 -- 172.21.15.78:0/1652554045 wait complete. 2024-05-21T15:15:23.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:22 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1652554045' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T15:15:23.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:22 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1652554045' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T15:15:23.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:22 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1652554045' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T15:15:23.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070498 got 253403070505 for osd.8 2024-05-21T15:15:23.582 DEBUG:teuthology.parallel:result is None 2024-05-21T15:15:23.582 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T15:15:23.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T15:15:23.857 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:24.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:23 smithi084 ceph-mon[36334]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 5 op/s; 83 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:24.179 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:23 smithi078 ceph-mon[31997]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 5 op/s; 83 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:24.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:23 smithi172 ceph-mon[36402]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 5 op/s; 83 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:25.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.457+0000 7febb4fe4700 1 -- 172.21.15.78:0/3275240364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb0106c70 msgr2=0x7febb0107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:25.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.457+0000 7febb4fe4700 1 --2- 172.21.15.78:0/3275240364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb0106c70 0x7febb0107080 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feba40099c0 tx=0x7feba4009cd0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.458+0000 7febb4fe4700 1 -- 172.21.15.78:0/3275240364 shutdown_connections 2024-05-21T15:15:25.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.458+0000 7febb4fe4700 1 --2- 172.21.15.78:0/3275240364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb01075c0 0x7febb010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.458+0000 7febb4fe4700 1 --2- 172.21.15.78:0/3275240364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febb0102340 0x7febb0102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.458+0000 7febb4fe4700 1 --2- 172.21.15.78:0/3275240364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb0106c70 0x7febb0107080 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.458+0000 7febb4fe4700 1 -- 172.21.15.78:0/3275240364 >> 172.21.15.78:0/3275240364 conn(0x7febb00fdc40 msgr2=0x7febb0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.458+0000 7febb4fe4700 1 -- 172.21.15.78:0/3275240364 shutdown_connections 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.458+0000 7febb4fe4700 1 -- 172.21.15.78:0/3275240364 wait complete. 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.458+0000 7febb4fe4700 1 Processor -- start 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febb4fe4700 1 -- start start 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febb4fe4700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb0102340 0x7febb019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febb4fe4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febb0106c70 0x7febb019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febb4fe4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb01075c0 0x7febb0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febb4fe4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febb0112370 con 0x7febb01075c0 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febb4fe4700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7febb01121f0 con 0x7febb0106c70 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febb4fe4700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7febb01124f0 con 0x7febb0102340 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febaed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb01075c0 0x7febb0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febae59c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb0102340 0x7febb019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febaed9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb01075c0 0x7febb0198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43862/0 (socket says 172.21.15.78:43862) 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febae59c700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb0102340 0x7febb019e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:53594/0 (socket says 172.21.15.78:53594) 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febaed9d700 1 -- 172.21.15.78:0/2568177179 learned_addr learned my addr 172.21.15.78:0/2568177179 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febadd9b700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febb0106c70 0x7febb019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febaed9d700 1 -- 172.21.15.78:0/2568177179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb0102340 msgr2=0x7febb019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febaed9d700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb0102340 0x7febb019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febaed9d700 1 -- 172.21.15.78:0/2568177179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febb0106c70 msgr2=0x7febb019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febaed9d700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febb0106c70 0x7febb019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.459+0000 7febaed9d700 1 -- 172.21.15.78:0/2568177179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb01991d0 con 0x7febb01075c0 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.460+0000 7febaed9d700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb01075c0 0x7febb0198b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feba8007d70 tx=0x7feba800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.460+0000 7feb9f7fe700 1 -- 172.21.15.78:0/2568177179 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8016020 con 0x7febb01075c0 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.460+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb01994c0 con 0x7febb01075c0 2024-05-21T15:15:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.460+0000 7feb9f7fe700 1 -- 172.21.15.78:0/2568177179 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba800ac90 con 0x7febb01075c0 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.460+0000 7feb9f7fe700 1 -- 172.21.15.78:0/2568177179 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba80115c0 con 0x7febb01075c0 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.460+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb0199980 con 0x7febb01075c0 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.461+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb01037f0 con 0x7febb01075c0 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.462+0000 7feb9f7fe700 1 -- 172.21.15.78:0/2568177179 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feba8019020 con 0x7febb01075c0 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.463+0000 7feb9f7fe700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feb980a2cb0 0x7feb980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.463+0000 7febae59c700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feb980a2cb0 0x7feb980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.463+0000 7feb9f7fe700 1 -- 172.21.15.78:0/2568177179 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7feba80cf000 con 0x7febb01075c0 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.463+0000 7febae59c700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feb980a2cb0 0x7feb980a51c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feba0005950 tx=0x7feba000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.468+0000 7feb9f7fe700 1 -- 172.21.15.78:0/2568177179 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feba8020020 con 0x7febb01075c0 2024-05-21T15:15:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.670+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7febb010a160 con 0x7feb980a2cb0 2024-05-21T15:15:25.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.677+0000 7feb9f7fe700 1 -- 172.21.15.78:0/2568177179 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7febb010a160 con 0x7feb980a2cb0 2024-05-21T15:15:25.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feb980a2cb0 msgr2=0x7feb980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feb980a2cb0 0x7feb980a51c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feba0005950 tx=0x7feba000b410 comp rx=0 tx=0).stop 2024-05-21T15:15:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb01075c0 msgr2=0x7febb0198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb01075c0 0x7febb0198b30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feba8007d70 tx=0x7feba800a340 comp rx=0 tx=0).stop 2024-05-21T15:15:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 shutdown_connections 2024-05-21T15:15:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7feb980a2cb0 0x7feb980a51c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb01075c0 0x7febb0198b30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb0102340 0x7febb019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 --2- 172.21.15.78:0/2568177179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febb0106c70 0x7febb019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.681+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 >> 172.21.15.78:0/2568177179 conn(0x7febb00fdc40 msgr2=0x7febb0106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.682+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 shutdown_connections 2024-05-21T15:15:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:25.682+0000 7febb4fe4700 1 -- 172.21.15.78:0/2568177179 wait complete. 2024-05-21T15:15:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-05-21T15:15:26.332 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:26 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.332 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:26 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.332 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:26 smithi078 ceph-mon[31997]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s; 67 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:26.332 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:26 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:15:26.332 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:26 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:15:26.332 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:26 smithi078 ceph-mon[31997]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.358 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-05-21T15:15:25.269418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657438,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":825,"num_read_kb":531,"num_write":420,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181408,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358472,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583352,"statfs":{"total":1151940427776,"available":1151573352448,"internally_reserved":0,"allocated":13725696,"data_stored":7441782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":33,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":672,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.905201"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370540+0000","last_change":"2024-05-21T15:14:12.254239+0000","last_active":"2024-05-21T15:15:16.370540+0000","last_peered":"2024-05-21T15:15:16.370540+0000","last_clean":"2024-05-21T15:15:16.370540+0000","last_became_active":"2024-05-21T15:14:12.253885+0000","last_became_peered":"2024-05-21T15:14:12.253885+0000","last_unstale":"2024-05-21T15:15:16.370540+0000","last_undegraded":"2024-05-21T15:15:16.370540+0000","last_fullsized":"2024-05-21T15:15:16.370540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:35.126836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366300+0000","last_change":"2024-05-21T15:14:12.262984+0000","last_active":"2024-05-21T15:15:16.366300+0000","last_peered":"2024-05-21T15:15:16.366300+0000","last_clean":"2024-05-21T15:15:16.366300+0000","last_became_active":"2024-05-21T15:14:12.262667+0000","last_became_peered":"2024-05-21T15:14:12.262667+0000","last_unstale":"2024-05-21T15:15:16.366300+0000","last_undegraded":"2024-05-21T15:15:16.366300+0000","last_fullsized":"2024-05-21T15:15:16.366300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:47:47.434314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366237+0000","last_change":"2024-05-21T15:14:12.260573+0000","last_active":"2024-05-21T15:15:16.366237+0000","last_peered":"2024-05-21T15:15:16.366237+0000","last_clean":"2024-05-21T15:15:16.366237+0000","last_became_active":"2024-05-21T15:14:12.259073+0000","last_became_peered":"2024-05-21T15:14:12.259073+0000","last_unstale":"2024-05-21T15:15:16.366237+0000","last_undegraded":"2024-05-21T15:15:16.366237+0000","last_fullsized":"2024-05-21T15:15:16.366237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:15:01.919464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.615011+0000","last_change":"2024-05-21T15:14:12.256992+0000","last_active":"2024-05-21T15:15:16.615011+0000","last_peered":"2024-05-21T15:15:16.615011+0000","last_clean":"2024-05-21T15:15:16.615011+0000","last_became_active":"2024-05-21T15:14:12.256448+0000","last_became_peered":"2024-05-21T15:14:12.256448+0000","last_unstale":"2024-05-21T15:15:16.615011+0000","last_undegraded":"2024-05-21T15:15:16.615011+0000","last_fullsized":"2024-05-21T15:15:16.615011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:40.569263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367125+0000","last_change":"2024-05-21T15:14:12.251533+0000","last_active":"2024-05-21T15:15:16.367125+0000","last_peered":"2024-05-21T15:15:16.367125+0000","last_clean":"2024-05-21T15:15:16.367125+0000","last_became_active":"2024-05-21T15:14:12.251210+0000","last_became_peered":"2024-05-21T15:14:12.251210+0000","last_unstale":"2024-05-21T15:15:16.367125+0000","last_undegraded":"2024-05-21T15:15:16.367125+0000","last_fullsized":"2024-05-21T15:15:16.367125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:43:34.109179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376308+0000","last_change":"2024-05-21T15:14:12.251767+0000","last_active":"2024-05-21T15:15:16.376308+0000","last_peered":"2024-05-21T15:15:16.376308+0000","last_clean":"2024-05-21T15:15:16.376308+0000","last_became_active":"2024-05-21T15:14:12.251427+0000","last_became_peered":"2024-05-21T15:14:12.251427+0000","last_unstale":"2024-05-21T15:15:16.376308+0000","last_undegraded":"2024-05-21T15:15:16.376308+0000","last_fullsized":"2024-05-21T15:15:16.376308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:48:41.804074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370565+0000","last_change":"2024-05-21T15:14:12.253472+0000","last_active":"2024-05-21T15:15:16.370565+0000","last_peered":"2024-05-21T15:15:16.370565+0000","last_clean":"2024-05-21T15:15:16.370565+0000","last_became_active":"2024-05-21T15:14:12.252762+0000","last_became_peered":"2024-05-21T15:14:12.252762+0000","last_unstale":"2024-05-21T15:15:16.370565+0000","last_undegraded":"2024-05-21T15:15:16.370565+0000","last_fullsized":"2024-05-21T15:15:16.370565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:11:44.497404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370791+0000","last_change":"2024-05-21T15:14:12.258627+0000","last_active":"2024-05-21T15:15:16.370791+0000","last_peered":"2024-05-21T15:15:16.370791+0000","last_clean":"2024-05-21T15:15:16.370791+0000","last_became_active":"2024-05-21T15:14:12.258118+0000","last_became_peered":"2024-05-21T15:14:12.258118+0000","last_unstale":"2024-05-21T15:15:16.370791+0000","last_undegraded":"2024-05-21T15:15:16.370791+0000","last_fullsized":"2024-05-21T15:15:16.370791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:56.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.386192+0000","last_change":"2024-05-21T15:14:12.255998+0000","last_active":"2024-05-21T15:15:16.386192+0000","last_peered":"2024-05-21T15:15:16.386192+0000","last_clean":"2024-05-21T15:15:16.386192+0000","last_became_active":"2024-05-21T15:14:12.255658+0000","last_became_peered":"2024-05-21T15:14:12.255658+0000","last_unstale":"2024-05-21T15:15:16.386192+0000","last_undegraded":"2024-05-21T15:15:16.386192+0000","last_fullsized":"2024-05-21T15:15:16.386192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:44:42.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376642+0000","last_change":"2024-05-21T15:14:12.257617+0000","last_active":"2024-05-21T15:15:16.376642+0000","last_peered":"2024-05-21T15:15:16.376642+0000","last_clean":"2024-05-21T15:15:16.376642+0000","last_became_active":"2024-05-21T15:14:12.256947+0000","last_became_peered":"2024-05-21T15:14:12.256947+0000","last_unstale":"2024-05-21T15:15:16.376642+0000","last_undegraded":"2024-05-21T15:15:16.376642+0000","last_fullsized":"2024-05-21T15:15:16.376642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:45:58.174954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371137+0000","last_change":"2024-05-21T15:14:12.260454+0000","last_active":"2024-05-21T15:15:16.371137+0000","last_peered":"2024-05-21T15:15:16.371137+0000","last_clean":"2024-05-21T15:15:16.371137+0000","last_became_active":"2024-05-21T15:14:12.260126+0000","last_became_peered":"2024-05-21T15:14:12.260126+0000","last_unstale":"2024-05-21T15:15:16.371137+0000","last_undegraded":"2024-05-21T15:15:16.371137+0000","last_fullsized":"2024-05-21T15:15:16.371137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:49:45.212050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.614872+0000","last_change":"2024-05-21T15:14:12.252071+0000","last_active":"2024-05-21T15:15:16.614872+0000","last_peered":"2024-05-21T15:15:16.614872+0000","last_clean":"2024-05-21T15:15:16.614872+0000","last_became_active":"2024-05-21T15:14:12.251706+0000","last_became_peered":"2024-05-21T15:14:12.251706+0000","last_unstale":"2024-05-21T15:15:16.614872+0000","last_undegraded":"2024-05-21T15:15:16.614872+0000","last_fullsized":"2024-05-21T15:15:16.614872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:02:44.349368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014102+0000","last_change":"2024-05-21T15:14:12.239877+0000","last_active":"2024-05-21T15:15:17.014102+0000","last_peered":"2024-05-21T15:15:17.014102+0000","last_clean":"2024-05-21T15:15:17.014102+0000","last_became_active":"2024-05-21T15:14:12.239615+0000","last_became_peered":"2024-05-21T15:14:12.239615+0000","last_unstale":"2024-05-21T15:15:17.014102+0000","last_undegraded":"2024-05-21T15:15:17.014102+0000","last_fullsized":"2024-05-21T15:15:17.014102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:23.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366047+0000","last_change":"2024-05-21T15:14:12.260349+0000","last_active":"2024-05-21T15:15:16.366047+0000","last_peered":"2024-05-21T15:15:16.366047+0000","last_clean":"2024-05-21T15:15:16.366047+0000","last_became_active":"2024-05-21T15:14:12.259658+0000","last_became_peered":"2024-05-21T15:14:12.259658+0000","last_unstale":"2024-05-21T15:15:16.366047+0000","last_undegraded":"2024-05-21T15:15:16.366047+0000","last_fullsized":"2024-05-21T15:15:16.366047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:59:32.326571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.015535+0000","last_change":"2024-05-21T15:15:17.015535+0000","last_active":"2024-05-21T15:15:17.015535+0000","last_peered":"2024-05-21T15:15:17.015535+0000","last_clean":"2024-05-21T15:15:17.015535+0000","last_became_active":"2024-05-21T15:15:17.015057+0000","last_became_peered":"2024-05-21T15:15:17.015057+0000","last_unstale":"2024-05-21T15:15:17.015535+0000","last_undegraded":"2024-05-21T15:15:17.015535+0000","last_fullsized":"2024-05-21T15:15:17.015535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:48:14.652870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385395+0000","last_change":"2024-05-21T15:14:12.264667+0000","last_active":"2024-05-21T15:15:16.385395+0000","last_peered":"2024-05-21T15:15:16.385395+0000","last_clean":"2024-05-21T15:15:16.385395+0000","last_became_active":"2024-05-21T15:14:12.264284+0000","last_became_peered":"2024-05-21T15:14:12.264284+0000","last_unstale":"2024-05-21T15:15:16.385395+0000","last_undegraded":"2024-05-21T15:15:16.385395+0000","last_fullsized":"2024-05-21T15:15:16.385395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:24.861321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370655+0000","last_change":"2024-05-21T15:14:12.259912+0000","last_active":"2024-05-21T15:15:16.370655+0000","last_peered":"2024-05-21T15:15:16.370655+0000","last_clean":"2024-05-21T15:15:16.370655+0000","last_became_active":"2024-05-21T15:14:12.259578+0000","last_became_peered":"2024-05-21T15:14:12.259578+0000","last_unstale":"2024-05-21T15:15:16.370655+0000","last_undegraded":"2024-05-21T15:15:16.370655+0000","last_fullsized":"2024-05-21T15:15:16.370655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:05:33.372481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369455+0000","last_change":"2024-05-21T15:14:12.258355+0000","last_active":"2024-05-21T15:15:16.369455+0000","last_peered":"2024-05-21T15:15:16.369455+0000","last_clean":"2024-05-21T15:15:16.369455+0000","last_became_active":"2024-05-21T15:14:12.257991+0000","last_became_peered":"2024-05-21T15:14:12.257991+0000","last_unstale":"2024-05-21T15:15:16.369455+0000","last_undegraded":"2024-05-21T15:15:16.369455+0000","last_fullsized":"2024-05-21T15:15:16.369455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:54.014553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372836+0000","last_change":"2024-05-21T15:14:12.258873+0000","last_active":"2024-05-21T15:15:16.372836+0000","last_peered":"2024-05-21T15:15:16.372836+0000","last_clean":"2024-05-21T15:15:16.372836+0000","last_became_active":"2024-05-21T15:14:12.258548+0000","last_became_peered":"2024-05-21T15:14:12.258548+0000","last_unstale":"2024-05-21T15:15:16.372836+0000","last_undegraded":"2024-05-21T15:15:16.372836+0000","last_fullsized":"2024-05-21T15:15:16.372836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:00.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:22.625145+0000","last_change":"2024-05-21T15:15:16.626507+0000","last_active":"2024-05-21T15:15:22.625145+0000","last_peered":"2024-05-21T15:15:22.625145+0000","last_clean":"2024-05-21T15:15:22.625145+0000","last_became_active":"2024-05-21T15:15:16.615945+0000","last_became_peered":"2024-05-21T15:15:16.615945+0000","last_unstale":"2024-05-21T15:15:22.625145+0000","last_undegraded":"2024-05-21T15:15:22.625145+0000","last_fullsized":"2024-05-21T15:15:22.625145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:16:01.229449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371331+0000","last_change":"2024-05-21T15:14:12.261632+0000","last_active":"2024-05-21T15:15:16.371331+0000","last_peered":"2024-05-21T15:15:16.371331+0000","last_clean":"2024-05-21T15:15:16.371331+0000","last_became_active":"2024-05-21T15:14:12.261149+0000","last_became_peered":"2024-05-21T15:14:12.261149+0000","last_unstale":"2024-05-21T15:15:16.371331+0000","last_undegraded":"2024-05-21T15:15:16.371331+0000","last_fullsized":"2024-05-21T15:15:16.371331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:11.349395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371454+0000","last_change":"2024-05-21T15:14:11.567859+0000","last_active":"2024-05-21T15:15:16.371454+0000","last_peered":"2024-05-21T15:15:16.371454+0000","last_clean":"2024-05-21T15:15:16.371454+0000","last_became_active":"2024-05-21T15:14:11.566855+0000","last_became_peered":"2024-05-21T15:14:11.566855+0000","last_unstale":"2024-05-21T15:15:16.371454+0000","last_undegraded":"2024-05-21T15:15:16.371454+0000","last_fullsized":"2024-05-21T15:15:16.371454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:21.568436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366374+0000","last_change":"2024-05-21T15:14:12.262153+0000","last_active":"2024-05-21T15:15:16.366374+0000","last_peered":"2024-05-21T15:15:16.366374+0000","last_clean":"2024-05-21T15:15:16.366374+0000","last_became_active":"2024-05-21T15:14:12.261780+0000","last_became_peered":"2024-05-21T15:14:12.261780+0000","last_unstale":"2024-05-21T15:15:16.366374+0000","last_undegraded":"2024-05-21T15:15:16.366374+0000","last_fullsized":"2024-05-21T15:15:16.366374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:21:25.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.368000+0000","last_change":"2024-05-21T15:14:11.570316+0000","last_active":"2024-05-21T15:15:16.368000+0000","last_peered":"2024-05-21T15:15:16.368000+0000","last_clean":"2024-05-21T15:15:16.368000+0000","last_became_active":"2024-05-21T15:14:11.569967+0000","last_became_peered":"2024-05-21T15:14:11.569967+0000","last_unstale":"2024-05-21T15:15:16.368000+0000","last_undegraded":"2024-05-21T15:15:16.368000+0000","last_fullsized":"2024-05-21T15:15:16.368000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:30:27.364742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385649+0000","last_change":"2024-05-21T15:14:12.261716+0000","last_active":"2024-05-21T15:15:16.385649+0000","last_peered":"2024-05-21T15:15:16.385649+0000","last_clean":"2024-05-21T15:15:16.385649+0000","last_became_active":"2024-05-21T15:14:12.261341+0000","last_became_peered":"2024-05-21T15:14:12.261341+0000","last_unstale":"2024-05-21T15:15:16.385649+0000","last_undegraded":"2024-05-21T15:15:16.385649+0000","last_fullsized":"2024-05-21T15:15:16.385649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:40:10.464974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367543+0000","last_change":"2024-05-21T15:14:11.573305+0000","last_active":"2024-05-21T15:15:16.367543+0000","last_peered":"2024-05-21T15:15:16.367543+0000","last_clean":"2024-05-21T15:15:16.367543+0000","last_became_active":"2024-05-21T15:14:11.572655+0000","last_became_peered":"2024-05-21T15:14:11.572655+0000","last_unstale":"2024-05-21T15:15:16.367543+0000","last_undegraded":"2024-05-21T15:15:16.367543+0000","last_fullsized":"2024-05-21T15:15:16.367543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:37.268440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372322+0000","last_change":"2024-05-21T15:14:12.253788+0000","last_active":"2024-05-21T15:15:16.372322+0000","last_peered":"2024-05-21T15:15:16.372322+0000","last_clean":"2024-05-21T15:15:16.372322+0000","last_became_active":"2024-05-21T15:14:12.253450+0000","last_became_peered":"2024-05-21T15:14:12.253450+0000","last_unstale":"2024-05-21T15:15:16.372322+0000","last_undegraded":"2024-05-21T15:15:16.372322+0000","last_fullsized":"2024-05-21T15:15:16.372322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:57.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:22.359820+0000","last_change":"2024-05-21T15:14:11.567400+0000","last_active":"2024-05-21T15:15:22.359820+0000","last_peered":"2024-05-21T15:15:22.359820+0000","last_clean":"2024-05-21T15:15:22.359820+0000","last_became_active":"2024-05-21T15:14:11.565995+0000","last_became_peered":"2024-05-21T15:14:11.565995+0000","last_unstale":"2024-05-21T15:15:22.359820+0000","last_undegraded":"2024-05-21T15:15:22.359820+0000","last_fullsized":"2024-05-21T15:15:22.359820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:50.463339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367049+0000","last_change":"2024-05-21T15:14:12.251565+0000","last_active":"2024-05-21T15:15:16.367049+0000","last_peered":"2024-05-21T15:15:16.367049+0000","last_clean":"2024-05-21T15:15:16.367049+0000","last_became_active":"2024-05-21T15:14:12.251289+0000","last_became_peered":"2024-05-21T15:14:12.251289+0000","last_unstale":"2024-05-21T15:15:16.367049+0000","last_undegraded":"2024-05-21T15:15:16.367049+0000","last_fullsized":"2024-05-21T15:15:16.367049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:44.619800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376264+0000","last_change":"2024-05-21T15:14:11.575193+0000","last_active":"2024-05-21T15:15:16.376264+0000","last_peered":"2024-05-21T15:15:16.376264+0000","last_clean":"2024-05-21T15:15:16.376264+0000","last_became_active":"2024-05-21T15:14:11.574861+0000","last_became_peered":"2024-05-21T15:14:11.574861+0000","last_unstale":"2024-05-21T15:15:16.376264+0000","last_undegraded":"2024-05-21T15:15:16.376264+0000","last_fullsized":"2024-05-21T15:15:16.376264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:54.548408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.015620+0000","last_change":"2024-05-21T15:15:17.015620+0000","last_active":"2024-05-21T15:15:17.015620+0000","last_peered":"2024-05-21T15:15:17.015620+0000","last_clean":"2024-05-21T15:15:17.015620+0000","last_became_active":"2024-05-21T15:15:17.015086+0000","last_became_peered":"2024-05-21T15:15:17.015086+0000","last_unstale":"2024-05-21T15:15:17.015620+0000","last_undegraded":"2024-05-21T15:15:17.015620+0000","last_fullsized":"2024-05-21T15:15:17.015620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:10.856246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014195+0000","last_change":"2024-05-21T15:14:11.572087+0000","last_active":"2024-05-21T15:15:17.014195+0000","last_peered":"2024-05-21T15:15:17.014195+0000","last_clean":"2024-05-21T15:15:17.014195+0000","last_became_active":"2024-05-21T15:14:11.571716+0000","last_became_peered":"2024-05-21T15:14:11.571716+0000","last_unstale":"2024-05-21T15:15:17.014195+0000","last_undegraded":"2024-05-21T15:15:17.014195+0000","last_fullsized":"2024-05-21T15:15:17.014195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:07.082766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370919+0000","last_change":"2024-05-21T15:14:12.252233+0000","last_active":"2024-05-21T15:15:16.370919+0000","last_peered":"2024-05-21T15:15:16.370919+0000","last_clean":"2024-05-21T15:15:16.370919+0000","last_became_active":"2024-05-21T15:14:12.251901+0000","last_became_peered":"2024-05-21T15:14:12.251901+0000","last_unstale":"2024-05-21T15:15:16.370919+0000","last_undegraded":"2024-05-21T15:15:16.370919+0000","last_fullsized":"2024-05-21T15:15:16.370919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:05.107436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371590+0000","last_change":"2024-05-21T15:14:11.565946+0000","last_active":"2024-05-21T15:15:16.371590+0000","last_peered":"2024-05-21T15:15:16.371590+0000","last_clean":"2024-05-21T15:15:16.371590+0000","last_became_active":"2024-05-21T15:14:11.565601+0000","last_became_peered":"2024-05-21T15:14:11.565601+0000","last_unstale":"2024-05-21T15:15:16.371590+0000","last_undegraded":"2024-05-21T15:15:16.371590+0000","last_fullsized":"2024-05-21T15:15:16.371590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:41:55.243473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369411+0000","last_change":"2024-05-21T15:14:12.255188+0000","last_active":"2024-05-21T15:15:16.369411+0000","last_peered":"2024-05-21T15:15:16.369411+0000","last_clean":"2024-05-21T15:15:16.369411+0000","last_became_active":"2024-05-21T15:14:12.254503+0000","last_became_peered":"2024-05-21T15:14:12.254503+0000","last_unstale":"2024-05-21T15:15:16.369411+0000","last_undegraded":"2024-05-21T15:15:16.369411+0000","last_fullsized":"2024-05-21T15:15:16.369411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:22:16.336740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":157,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:23.360511+0000","last_change":"2024-05-21T15:14:11.569882+0000","last_active":"2024-05-21T15:15:23.360511+0000","last_peered":"2024-05-21T15:15:23.360511+0000","last_clean":"2024-05-21T15:15:23.360511+0000","last_became_active":"2024-05-21T15:14:11.569532+0000","last_became_peered":"2024-05-21T15:14:11.569532+0000","last_unstale":"2024-05-21T15:15:23.360511+0000","last_undegraded":"2024-05-21T15:15:23.360511+0000","last_fullsized":"2024-05-21T15:15:23.360511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:58:50.081709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385949+0000","last_change":"2024-05-21T15:14:12.262691+0000","last_active":"2024-05-21T15:15:16.385949+0000","last_peered":"2024-05-21T15:15:16.385949+0000","last_clean":"2024-05-21T15:15:16.385949+0000","last_became_active":"2024-05-21T15:14:12.261492+0000","last_became_peered":"2024-05-21T15:14:12.261492+0000","last_unstale":"2024-05-21T15:15:16.385949+0000","last_undegraded":"2024-05-21T15:15:16.385949+0000","last_fullsized":"2024-05-21T15:15:16.385949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:35:00.121256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370695+0000","last_change":"2024-05-21T15:14:11.569231+0000","last_active":"2024-05-21T15:15:16.370695+0000","last_peered":"2024-05-21T15:15:16.370695+0000","last_clean":"2024-05-21T15:15:16.370695+0000","last_became_active":"2024-05-21T15:14:11.568916+0000","last_became_peered":"2024-05-21T15:14:11.568916+0000","last_unstale":"2024-05-21T15:15:16.370695+0000","last_undegraded":"2024-05-21T15:15:16.370695+0000","last_fullsized":"2024-05-21T15:15:16.370695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:26:13.411854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366276+0000","last_change":"2024-05-21T15:14:12.256307+0000","last_active":"2024-05-21T15:15:16.366276+0000","last_peered":"2024-05-21T15:15:16.366276+0000","last_clean":"2024-05-21T15:15:16.366276+0000","last_became_active":"2024-05-21T15:14:12.255828+0000","last_became_peered":"2024-05-21T15:14:12.255828+0000","last_unstale":"2024-05-21T15:15:16.366276+0000","last_undegraded":"2024-05-21T15:15:16.366276+0000","last_fullsized":"2024-05-21T15:15:16.366276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:11:40.068779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372517+0000","last_change":"2024-05-21T15:14:11.568239+0000","last_active":"2024-05-21T15:15:16.372517+0000","last_peered":"2024-05-21T15:15:16.372517+0000","last_clean":"2024-05-21T15:15:16.372517+0000","last_became_active":"2024-05-21T15:14:11.567836+0000","last_became_peered":"2024-05-21T15:14:11.567836+0000","last_unstale":"2024-05-21T15:15:16.372517+0000","last_undegraded":"2024-05-21T15:15:16.372517+0000","last_fullsized":"2024-05-21T15:15:16.372517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:10.244172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385787+0000","last_change":"2024-05-21T15:14:12.257762+0000","last_active":"2024-05-21T15:15:16.385787+0000","last_peered":"2024-05-21T15:15:16.385787+0000","last_clean":"2024-05-21T15:15:16.385787+0000","last_became_active":"2024-05-21T15:14:12.256335+0000","last_became_peered":"2024-05-21T15:14:12.256335+0000","last_unstale":"2024-05-21T15:15:16.385787+0000","last_undegraded":"2024-05-21T15:15:16.385787+0000","last_fullsized":"2024-05-21T15:15:16.385787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:35.219673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":194,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:23.388091+0000","last_change":"2024-05-21T15:14:11.564407+0000","last_active":"2024-05-21T15:15:23.388091+0000","last_peered":"2024-05-21T15:15:23.388091+0000","last_clean":"2024-05-21T15:15:23.388091+0000","last_became_active":"2024-05-21T15:14:11.564012+0000","last_became_peered":"2024-05-21T15:14:11.564012+0000","last_unstale":"2024-05-21T15:15:23.388091+0000","last_undegraded":"2024-05-21T15:15:23.388091+0000","last_fullsized":"2024-05-21T15:15:23.388091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:34:54.177844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":144,"num_read_kb":130,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367652+0000","last_change":"2024-05-21T15:14:12.253820+0000","last_active":"2024-05-21T15:15:16.367652+0000","last_peered":"2024-05-21T15:15:16.367652+0000","last_clean":"2024-05-21T15:15:16.367652+0000","last_became_active":"2024-05-21T15:14:12.253360+0000","last_became_peered":"2024-05-21T15:14:12.253360+0000","last_unstale":"2024-05-21T15:15:16.367652+0000","last_undegraded":"2024-05-21T15:15:16.367652+0000","last_fullsized":"2024-05-21T15:15:16.367652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:38:19.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372744+0000","last_change":"2024-05-21T15:14:11.571161+0000","last_active":"2024-05-21T15:15:16.372744+0000","last_peered":"2024-05-21T15:15:16.372744+0000","last_clean":"2024-05-21T15:15:16.372744+0000","last_became_active":"2024-05-21T15:14:11.570837+0000","last_became_peered":"2024-05-21T15:14:11.570837+0000","last_unstale":"2024-05-21T15:15:16.372744+0000","last_undegraded":"2024-05-21T15:15:16.372744+0000","last_fullsized":"2024-05-21T15:15:16.372744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:24:55.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.013982+0000","last_change":"2024-05-21T15:14:12.255713+0000","last_active":"2024-05-21T15:15:17.013982+0000","last_peered":"2024-05-21T15:15:17.013982+0000","last_clean":"2024-05-21T15:15:17.013982+0000","last_became_active":"2024-05-21T15:14:12.254458+0000","last_became_peered":"2024-05-21T15:14:12.254458+0000","last_unstale":"2024-05-21T15:15:17.013982+0000","last_undegraded":"2024-05-21T15:15:17.013982+0000","last_fullsized":"2024-05-21T15:15:17.013982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:55:24.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371715+0000","last_change":"2024-05-21T15:14:11.565916+0000","last_active":"2024-05-21T15:15:16.371715+0000","last_peered":"2024-05-21T15:15:16.371715+0000","last_clean":"2024-05-21T15:15:16.371715+0000","last_became_active":"2024-05-21T15:14:11.565460+0000","last_became_peered":"2024-05-21T15:14:11.565460+0000","last_unstale":"2024-05-21T15:15:16.371715+0000","last_undegraded":"2024-05-21T15:15:16.371715+0000","last_fullsized":"2024-05-21T15:15:16.371715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:09.982208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371049+0000","last_change":"2024-05-21T15:14:12.251856+0000","last_active":"2024-05-21T15:15:16.371049+0000","last_peered":"2024-05-21T15:15:16.371049+0000","last_clean":"2024-05-21T15:15:16.371049+0000","last_became_active":"2024-05-21T15:14:12.251496+0000","last_became_peered":"2024-05-21T15:14:12.251496+0000","last_unstale":"2024-05-21T15:15:16.371049+0000","last_undegraded":"2024-05-21T15:15:16.371049+0000","last_fullsized":"2024-05-21T15:15:16.371049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:17.566276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:23.360499+0000","last_change":"2024-05-21T15:14:11.565485+0000","last_active":"2024-05-21T15:15:23.360499+0000","last_peered":"2024-05-21T15:15:23.360499+0000","last_clean":"2024-05-21T15:15:23.360499+0000","last_became_active":"2024-05-21T15:14:11.565092+0000","last_became_peered":"2024-05-21T15:14:11.565092+0000","last_unstale":"2024-05-21T15:15:23.360499+0000","last_undegraded":"2024-05-21T15:15:23.360499+0000","last_fullsized":"2024-05-21T15:15:23.360499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:01:16.101793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370404+0000","last_change":"2024-05-21T15:14:12.262283+0000","last_active":"2024-05-21T15:15:16.370404+0000","last_peered":"2024-05-21T15:15:16.370404+0000","last_clean":"2024-05-21T15:15:16.370404+0000","last_became_active":"2024-05-21T15:14:12.261610+0000","last_became_peered":"2024-05-21T15:14:12.261610+0000","last_unstale":"2024-05-21T15:15:16.370404+0000","last_undegraded":"2024-05-21T15:15:16.370404+0000","last_fullsized":"2024-05-21T15:15:16.370404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:44:33.087115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366991+0000","last_change":"2024-05-21T15:14:11.569905+0000","last_active":"2024-05-21T15:15:16.366991+0000","last_peered":"2024-05-21T15:15:16.366991+0000","last_clean":"2024-05-21T15:15:16.366991+0000","last_became_active":"2024-05-21T15:14:11.569221+0000","last_became_peered":"2024-05-21T15:14:11.569221+0000","last_unstale":"2024-05-21T15:15:16.366991+0000","last_undegraded":"2024-05-21T15:15:16.366991+0000","last_fullsized":"2024-05-21T15:15:16.366991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:11:04.875890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.386384+0000","last_change":"2024-05-21T15:15:16.386384+0000","last_active":"2024-05-21T15:15:16.386384+0000","last_peered":"2024-05-21T15:15:16.386384+0000","last_clean":"2024-05-21T15:15:16.386384+0000","last_became_active":"2024-05-21T15:15:16.385810+0000","last_became_peered":"2024-05-21T15:15:16.385810+0000","last_unstale":"2024-05-21T15:15:16.386384+0000","last_undegraded":"2024-05-21T15:15:16.386384+0000","last_fullsized":"2024-05-21T15:15:16.386384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:49:45.031393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367815+0000","last_change":"2024-05-21T15:14:11.569670+0000","last_active":"2024-05-21T15:15:16.367815+0000","last_peered":"2024-05-21T15:15:16.367815+0000","last_clean":"2024-05-21T15:15:16.367815+0000","last_became_active":"2024-05-21T15:14:11.569257+0000","last_became_peered":"2024-05-21T15:14:11.569257+0000","last_unstale":"2024-05-21T15:15:16.367815+0000","last_undegraded":"2024-05-21T15:15:16.367815+0000","last_fullsized":"2024-05-21T15:15:16.367815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:18.584920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.374845+0000","last_change":"2024-05-21T15:15:16.374845+0000","last_active":"2024-05-21T15:15:16.374845+0000","last_peered":"2024-05-21T15:15:16.374845+0000","last_clean":"2024-05-21T15:15:16.374845+0000","last_became_active":"2024-05-21T15:15:16.374393+0000","last_became_peered":"2024-05-21T15:15:16.374393+0000","last_unstale":"2024-05-21T15:15:16.374845+0000","last_undegraded":"2024-05-21T15:15:16.374845+0000","last_fullsized":"2024-05-21T15:15:16.374845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:38:21.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369371+0000","last_change":"2024-05-21T15:14:11.567849+0000","last_active":"2024-05-21T15:15:16.369371+0000","last_peered":"2024-05-21T15:15:16.369371+0000","last_clean":"2024-05-21T15:15:16.369371+0000","last_became_active":"2024-05-21T15:14:11.567476+0000","last_became_peered":"2024-05-21T15:14:11.567476+0000","last_unstale":"2024-05-21T15:15:16.369371+0000","last_undegraded":"2024-05-21T15:15:16.369371+0000","last_fullsized":"2024-05-21T15:15:16.369371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:13.516765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376527+0000","last_change":"2024-05-21T15:14:12.255165+0000","last_active":"2024-05-21T15:15:16.376527+0000","last_peered":"2024-05-21T15:15:16.376527+0000","last_clean":"2024-05-21T15:15:16.376527+0000","last_became_active":"2024-05-21T15:14:12.254743+0000","last_became_peered":"2024-05-21T15:14:12.254743+0000","last_unstale":"2024-05-21T15:15:16.376527+0000","last_undegraded":"2024-05-21T15:15:16.376527+0000","last_fullsized":"2024-05-21T15:15:16.376527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:53:11.059597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:20.622508+0000","last_change":"2024-05-21T15:14:11.569745+0000","last_active":"2024-05-21T15:15:20.622508+0000","last_peered":"2024-05-21T15:15:20.622508+0000","last_clean":"2024-05-21T15:15:20.622508+0000","last_became_active":"2024-05-21T15:14:11.569409+0000","last_became_peered":"2024-05-21T15:14:11.569409+0000","last_unstale":"2024-05-21T15:15:20.622508+0000","last_undegraded":"2024-05-21T15:15:20.622508+0000","last_fullsized":"2024-05-21T15:15:20.622508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:00:37.963797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376429+0000","last_change":"2024-05-21T15:14:12.254631+0000","last_active":"2024-05-21T15:15:16.376429+0000","last_peered":"2024-05-21T15:15:16.376429+0000","last_clean":"2024-05-21T15:15:16.376429+0000","last_became_active":"2024-05-21T15:14:12.254239+0000","last_became_peered":"2024-05-21T15:14:12.254239+0000","last_unstale":"2024-05-21T15:15:16.376429+0000","last_undegraded":"2024-05-21T15:15:16.376429+0000","last_fullsized":"2024-05-21T15:15:16.376429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:55:17.135550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.616309+0000","last_change":"2024-05-21T15:15:16.615801+0000","last_active":"2024-05-21T15:15:16.616309+0000","last_peered":"2024-05-21T15:15:16.616309+0000","last_clean":"2024-05-21T15:15:16.616309+0000","last_became_active":"2024-05-21T15:15:16.615323+0000","last_became_peered":"2024-05-21T15:15:16.615323+0000","last_unstale":"2024-05-21T15:15:16.616309+0000","last_undegraded":"2024-05-21T15:15:16.616309+0000","last_fullsized":"2024-05-21T15:15:16.616309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:15.233980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:21.356730+0000","last_change":"2024-05-21T15:14:11.574352+0000","last_active":"2024-05-21T15:15:21.356730+0000","last_peered":"2024-05-21T15:15:21.356730+0000","last_clean":"2024-05-21T15:15:21.356730+0000","last_became_active":"2024-05-21T15:14:11.574041+0000","last_became_peered":"2024-05-21T15:14:11.574041+0000","last_unstale":"2024-05-21T15:15:21.356730+0000","last_undegraded":"2024-05-21T15:15:21.356730+0000","last_fullsized":"2024-05-21T15:15:21.356730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.563218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371236+0000","last_change":"2024-05-21T15:14:12.257886+0000","last_active":"2024-05-21T15:15:16.371236+0000","last_peered":"2024-05-21T15:15:16.371236+0000","last_clean":"2024-05-21T15:15:16.371236+0000","last_became_active":"2024-05-21T15:14:12.257546+0000","last_became_peered":"2024-05-21T15:14:12.257546+0000","last_unstale":"2024-05-21T15:15:16.371236+0000","last_undegraded":"2024-05-21T15:15:16.371236+0000","last_fullsized":"2024-05-21T15:15:16.371236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:20.953421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.614562+0000","last_change":"2024-05-21T15:14:11.568524+0000","last_active":"2024-05-21T15:15:16.614562+0000","last_peered":"2024-05-21T15:15:16.614562+0000","last_clean":"2024-05-21T15:15:16.614562+0000","last_became_active":"2024-05-21T15:14:11.567981+0000","last_became_peered":"2024-05-21T15:14:11.567981+0000","last_unstale":"2024-05-21T15:15:16.614562+0000","last_undegraded":"2024-05-21T15:15:16.614562+0000","last_fullsized":"2024-05-21T15:15:16.614562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370486+0000","last_change":"2024-05-21T15:13:02.249260+0000","last_active":"2024-05-21T15:15:16.370486+0000","last_peered":"2024-05-21T15:15:16.370486+0000","last_clean":"2024-05-21T15:15:16.370486+0000","last_became_active":"2024-05-21T15:13:02.248637+0000","last_became_peered":"2024-05-21T15:13:02.248637+0000","last_unstale":"2024-05-21T15:15:16.370486+0000","last_undegraded":"2024-05-21T15:15:16.370486+0000","last_fullsized":"2024-05-21T15:15:16.370486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_clean_scrub_stamp":"2024-05-21T15:09:41.506463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:26.250910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014280+0000","last_change":"2024-05-21T15:14:12.259861+0000","last_active":"2024-05-21T15:15:17.014280+0000","last_peered":"2024-05-21T15:15:17.014280+0000","last_clean":"2024-05-21T15:15:17.014280+0000","last_became_active":"2024-05-21T15:14:12.259510+0000","last_became_peered":"2024-05-21T15:14:12.259510+0000","last_unstale":"2024-05-21T15:15:17.014280+0000","last_undegraded":"2024-05-21T15:15:17.014280+0000","last_fullsized":"2024-05-21T15:15:17.014280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:49.548366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370650+0000","last_change":"2024-05-21T15:14:11.570705+0000","last_active":"2024-05-21T15:15:16.370650+0000","last_peered":"2024-05-21T15:15:16.370650+0000","last_clean":"2024-05-21T15:15:16.370650+0000","last_became_active":"2024-05-21T15:14:11.570403+0000","last_became_peered":"2024-05-21T15:14:11.570403+0000","last_unstale":"2024-05-21T15:15:16.370650+0000","last_undegraded":"2024-05-21T15:15:16.370650+0000","last_fullsized":"2024-05-21T15:15:16.370650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:02:34.156258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367655+0000","last_change":"2024-05-21T15:14:11.572311+0000","last_active":"2024-05-21T15:15:16.367655+0000","last_peered":"2024-05-21T15:15:16.367655+0000","last_clean":"2024-05-21T15:15:16.367655+0000","last_became_active":"2024-05-21T15:14:11.571949+0000","last_became_peered":"2024-05-21T15:14:11.571949+0000","last_unstale":"2024-05-21T15:15:16.367655+0000","last_undegraded":"2024-05-21T15:15:16.367655+0000","last_fullsized":"2024-05-21T15:15:16.367655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:57:59.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014340+0000","last_change":"2024-05-21T15:14:12.260717+0000","last_active":"2024-05-21T15:15:17.014340+0000","last_peered":"2024-05-21T15:15:17.014340+0000","last_clean":"2024-05-21T15:15:17.014340+0000","last_became_active":"2024-05-21T15:14:12.259911+0000","last_became_peered":"2024-05-21T15:14:12.259911+0000","last_unstale":"2024-05-21T15:15:17.014340+0000","last_undegraded":"2024-05-21T15:15:17.014340+0000","last_fullsized":"2024-05-21T15:15:17.014340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:02.609299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.381952+0000","last_change":"2024-05-21T15:15:16.381952+0000","last_active":"2024-05-21T15:15:16.381952+0000","last_peered":"2024-05-21T15:15:16.381952+0000","last_clean":"2024-05-21T15:15:16.381952+0000","last_became_active":"2024-05-21T15:15:16.373966+0000","last_became_peered":"2024-05-21T15:15:16.373966+0000","last_unstale":"2024-05-21T15:15:16.381952+0000","last_undegraded":"2024-05-21T15:15:16.381952+0000","last_fullsized":"2024-05-21T15:15:16.381952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:49:05.368066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371210+0000","last_change":"2024-05-21T15:14:12.253634+0000","last_active":"2024-05-21T15:15:16.371210+0000","last_peered":"2024-05-21T15:15:16.371210+0000","last_clean":"2024-05-21T15:15:16.371210+0000","last_became_active":"2024-05-21T15:14:12.253284+0000","last_became_peered":"2024-05-21T15:14:12.253284+0000","last_unstale":"2024-05-21T15:15:16.371210+0000","last_undegraded":"2024-05-21T15:15:16.371210+0000","last_fullsized":"2024-05-21T15:15:16.371210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:28:07.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.614992+0000","last_change":"2024-05-21T15:14:11.574147+0000","last_active":"2024-05-21T15:15:16.614992+0000","last_peered":"2024-05-21T15:15:16.614992+0000","last_clean":"2024-05-21T15:15:16.614992+0000","last_became_active":"2024-05-21T15:14:11.573721+0000","last_became_peered":"2024-05-21T15:14:11.573721+0000","last_unstale":"2024-05-21T15:15:16.614992+0000","last_undegraded":"2024-05-21T15:15:16.614992+0000","last_fullsized":"2024-05-21T15:15:16.614992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:10.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366391+0000","last_change":"2024-05-21T15:14:12.258632+0000","last_active":"2024-05-21T15:15:16.366391+0000","last_peered":"2024-05-21T15:15:16.366391+0000","last_clean":"2024-05-21T15:15:16.366391+0000","last_became_active":"2024-05-21T15:14:12.257908+0000","last_became_peered":"2024-05-21T15:14:12.257908+0000","last_unstale":"2024-05-21T15:15:16.366391+0000","last_undegraded":"2024-05-21T15:15:16.366391+0000","last_fullsized":"2024-05-21T15:15:16.366391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:12.062387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385475+0000","last_change":"2024-05-21T15:14:11.562592+0000","last_active":"2024-05-21T15:15:16.385475+0000","last_peered":"2024-05-21T15:15:16.385475+0000","last_clean":"2024-05-21T15:15:16.385475+0000","last_became_active":"2024-05-21T15:14:11.562265+0000","last_became_peered":"2024-05-21T15:14:11.562265+0000","last_unstale":"2024-05-21T15:15:16.385475+0000","last_undegraded":"2024-05-21T15:15:16.385475+0000","last_fullsized":"2024-05-21T15:15:16.385475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:24.662576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367004+0000","last_change":"2024-05-21T15:14:12.249207+0000","last_active":"2024-05-21T15:15:16.367004+0000","last_peered":"2024-05-21T15:15:16.367004+0000","last_clean":"2024-05-21T15:15:16.367004+0000","last_became_active":"2024-05-21T15:14:12.248819+0000","last_became_peered":"2024-05-21T15:14:12.248819+0000","last_unstale":"2024-05-21T15:15:16.367004+0000","last_undegraded":"2024-05-21T15:15:16.367004+0000","last_fullsized":"2024-05-21T15:15:16.367004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:38.208636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370878+0000","last_change":"2024-05-21T15:14:11.572044+0000","last_active":"2024-05-21T15:15:16.370878+0000","last_peered":"2024-05-21T15:15:16.370878+0000","last_clean":"2024-05-21T15:15:16.370878+0000","last_became_active":"2024-05-21T15:14:11.570721+0000","last_became_peered":"2024-05-21T15:14:11.570721+0000","last_unstale":"2024-05-21T15:15:16.370878+0000","last_undegraded":"2024-05-21T15:15:16.370878+0000","last_fullsized":"2024-05-21T15:15:16.370878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:34:08.413781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369772+0000","last_change":"2024-05-21T15:14:12.256327+0000","last_active":"2024-05-21T15:15:16.369772+0000","last_peered":"2024-05-21T15:15:16.369772+0000","last_clean":"2024-05-21T15:15:16.369772+0000","last_became_active":"2024-05-21T15:14:12.255962+0000","last_became_peered":"2024-05-21T15:14:12.255962+0000","last_unstale":"2024-05-21T15:15:16.369772+0000","last_undegraded":"2024-05-21T15:15:16.369772+0000","last_fullsized":"2024-05-21T15:15:16.369772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:05.076323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.386278+0000","last_change":"2024-05-21T15:15:16.385658+0000","last_active":"2024-05-21T15:15:16.386278+0000","last_peered":"2024-05-21T15:15:16.386278+0000","last_clean":"2024-05-21T15:15:16.386278+0000","last_became_active":"2024-05-21T15:15:16.375122+0000","last_became_peered":"2024-05-21T15:15:16.375122+0000","last_unstale":"2024-05-21T15:15:16.386278+0000","last_undegraded":"2024-05-21T15:15:16.386278+0000","last_fullsized":"2024-05-21T15:15:16.386278+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:57:59.599504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371212+0000","last_change":"2024-05-21T15:14:12.256993+0000","last_active":"2024-05-21T15:15:16.371212+0000","last_peered":"2024-05-21T15:15:16.371212+0000","last_clean":"2024-05-21T15:15:16.371212+0000","last_became_active":"2024-05-21T15:14:12.256179+0000","last_became_peered":"2024-05-21T15:14:12.256179+0000","last_unstale":"2024-05-21T15:15:16.371212+0000","last_undegraded":"2024-05-21T15:15:16.371212+0000","last_fullsized":"2024-05-21T15:15:16.371212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:01:09.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.391272+0000","last_change":"2024-05-21T15:15:16.391272+0000","last_active":"2024-05-21T15:15:16.391272+0000","last_peered":"2024-05-21T15:15:16.391272+0000","last_clean":"2024-05-21T15:15:16.391272+0000","last_became_active":"2024-05-21T15:15:16.386921+0000","last_became_peered":"2024-05-21T15:15:16.386921+0000","last_unstale":"2024-05-21T15:15:16.391272+0000","last_undegraded":"2024-05-21T15:15:16.391272+0000","last_fullsized":"2024-05-21T15:15:16.391272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:01:55.466989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370986+0000","last_change":"2024-05-21T15:14:12.260506+0000","last_active":"2024-05-21T15:15:16.370986+0000","last_peered":"2024-05-21T15:15:16.370986+0000","last_clean":"2024-05-21T15:15:16.370986+0000","last_became_active":"2024-05-21T15:14:12.260168+0000","last_became_peered":"2024-05-21T15:14:12.260168+0000","last_unstale":"2024-05-21T15:15:16.370986+0000","last_undegraded":"2024-05-21T15:15:16.370986+0000","last_fullsized":"2024-05-21T15:15:16.370986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:11.321622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:19.399258+0000","last_change":"2024-05-21T15:14:11.568630+0000","last_active":"2024-05-21T15:15:19.399258+0000","last_peered":"2024-05-21T15:15:19.399258+0000","last_clean":"2024-05-21T15:15:19.399258+0000","last_became_active":"2024-05-21T15:14:11.568229+0000","last_became_peered":"2024-05-21T15:14:11.568229+0000","last_unstale":"2024-05-21T15:15:19.399258+0000","last_undegraded":"2024-05-21T15:15:19.399258+0000","last_fullsized":"2024-05-21T15:15:19.399258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:21.372395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371147+0000","last_change":"2024-05-21T15:14:12.259408+0000","last_active":"2024-05-21T15:15:16.371147+0000","last_peered":"2024-05-21T15:15:16.371147+0000","last_clean":"2024-05-21T15:15:16.371147+0000","last_became_active":"2024-05-21T15:14:12.259082+0000","last_became_peered":"2024-05-21T15:14:12.259082+0000","last_unstale":"2024-05-21T15:15:16.371147+0000","last_undegraded":"2024-05-21T15:15:16.371147+0000","last_fullsized":"2024-05-21T15:15:16.371147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:32:09.059307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372199+0000","last_change":"2024-05-21T15:14:11.571077+0000","last_active":"2024-05-21T15:15:16.372199+0000","last_peered":"2024-05-21T15:15:16.372199+0000","last_clean":"2024-05-21T15:15:16.372199+0000","last_became_active":"2024-05-21T15:14:11.570664+0000","last_became_peered":"2024-05-21T15:14:11.570664+0000","last_unstale":"2024-05-21T15:15:16.372199+0000","last_undegraded":"2024-05-21T15:15:16.372199+0000","last_fullsized":"2024-05-21T15:15:16.372199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:37:16.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370637+0000","last_change":"2024-05-21T15:14:12.252972+0000","last_active":"2024-05-21T15:15:16.370637+0000","last_peered":"2024-05-21T15:15:16.370637+0000","last_clean":"2024-05-21T15:15:16.370637+0000","last_became_active":"2024-05-21T15:14:12.252591+0000","last_became_peered":"2024-05-21T15:14:12.252591+0000","last_unstale":"2024-05-21T15:15:16.370637+0000","last_undegraded":"2024-05-21T15:15:16.370637+0000","last_fullsized":"2024-05-21T15:15:16.370637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:14:16.430371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014320+0000","last_change":"2024-05-21T15:14:11.577138+0000","last_active":"2024-05-21T15:15:17.014320+0000","last_peered":"2024-05-21T15:15:17.014320+0000","last_clean":"2024-05-21T15:15:17.014320+0000","last_became_active":"2024-05-21T15:14:11.576629+0000","last_became_peered":"2024-05-21T15:14:11.576629+0000","last_unstale":"2024-05-21T15:15:17.014320+0000","last_undegraded":"2024-05-21T15:15:17.014320+0000","last_fullsized":"2024-05-21T15:15:17.014320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:20:20.134764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371001+0000","last_change":"2024-05-21T15:14:12.256174+0000","last_active":"2024-05-21T15:15:16.371001+0000","last_peered":"2024-05-21T15:15:16.371001+0000","last_clean":"2024-05-21T15:15:16.371001+0000","last_became_active":"2024-05-21T15:14:12.255839+0000","last_became_peered":"2024-05-21T15:14:12.255839+0000","last_unstale":"2024-05-21T15:15:16.371001+0000","last_undegraded":"2024-05-21T15:15:16.371001+0000","last_fullsized":"2024-05-21T15:15:16.371001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:12:40.743118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369583+0000","last_change":"2024-05-21T15:14:12.257028+0000","last_active":"2024-05-21T15:15:16.369583+0000","last_peered":"2024-05-21T15:15:16.369583+0000","last_clean":"2024-05-21T15:15:16.369583+0000","last_became_active":"2024-05-21T15:14:12.256611+0000","last_became_peered":"2024-05-21T15:14:12.256611+0000","last_unstale":"2024-05-21T15:15:16.369583+0000","last_undegraded":"2024-05-21T15:15:16.369583+0000","last_fullsized":"2024-05-21T15:15:16.369583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:08:43.881073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370885+0000","last_change":"2024-05-21T15:14:12.255724+0000","last_active":"2024-05-21T15:15:16.370885+0000","last_peered":"2024-05-21T15:15:16.370885+0000","last_clean":"2024-05-21T15:15:16.370885+0000","last_became_active":"2024-05-21T15:14:12.255272+0000","last_became_peered":"2024-05-21T15:14:12.255272+0000","last_unstale":"2024-05-21T15:15:16.370885+0000","last_undegraded":"2024-05-21T15:15:16.370885+0000","last_fullsized":"2024-05-21T15:15:16.370885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:07:07.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370522+0000","last_change":"2024-05-21T15:14:12.262508+0000","last_active":"2024-05-21T15:15:16.370522+0000","last_peered":"2024-05-21T15:15:16.370522+0000","last_clean":"2024-05-21T15:15:16.370522+0000","last_became_active":"2024-05-21T15:14:12.262155+0000","last_became_peered":"2024-05-21T15:14:12.262155+0000","last_unstale":"2024-05-21T15:15:16.370522+0000","last_undegraded":"2024-05-21T15:15:16.370522+0000","last_fullsized":"2024-05-21T15:15:16.370522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:54:19.982044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366130+0000","last_change":"2024-05-21T15:14:12.251715+0000","last_active":"2024-05-21T15:15:16.366130+0000","last_peered":"2024-05-21T15:15:16.366130+0000","last_clean":"2024-05-21T15:15:16.366130+0000","last_became_active":"2024-05-21T15:14:12.251337+0000","last_became_peered":"2024-05-21T15:14:12.251337+0000","last_unstale":"2024-05-21T15:15:16.366130+0000","last_undegraded":"2024-05-21T15:15:16.366130+0000","last_fullsized":"2024-05-21T15:15:16.366130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:03.562179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014242+0000","last_change":"2024-05-21T15:14:12.253166+0000","last_active":"2024-05-21T15:15:17.014242+0000","last_peered":"2024-05-21T15:15:17.014242+0000","last_clean":"2024-05-21T15:15:17.014242+0000","last_became_active":"2024-05-21T15:14:12.251830+0000","last_became_peered":"2024-05-21T15:14:12.251830+0000","last_unstale":"2024-05-21T15:15:17.014242+0000","last_undegraded":"2024-05-21T15:15:17.014242+0000","last_fullsized":"2024-05-21T15:15:17.014242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:11:58.679568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370077+0000","last_change":"2024-05-21T15:14:12.256131+0000","last_active":"2024-05-21T15:15:16.370077+0000","last_peered":"2024-05-21T15:15:16.370077+0000","last_clean":"2024-05-21T15:15:16.370077+0000","last_became_active":"2024-05-21T15:14:12.255772+0000","last_became_peered":"2024-05-21T15:14:12.255772+0000","last_unstale":"2024-05-21T15:15:16.370077+0000","last_undegraded":"2024-05-21T15:15:16.370077+0000","last_fullsized":"2024-05-21T15:15:16.370077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:26:48.047278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.373186+0000","last_change":"2024-05-21T15:14:12.260829+0000","last_active":"2024-05-21T15:15:16.373186+0000","last_peered":"2024-05-21T15:15:16.373186+0000","last_clean":"2024-05-21T15:15:16.373186+0000","last_became_active":"2024-05-21T15:14:12.259686+0000","last_became_peered":"2024-05-21T15:14:12.259686+0000","last_unstale":"2024-05-21T15:15:16.373186+0000","last_undegraded":"2024-05-21T15:15:16.373186+0000","last_fullsized":"2024-05-21T15:15:16.373186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:48.480814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371069+0000","last_change":"2024-05-21T15:14:12.262892+0000","last_active":"2024-05-21T15:15:16.371069+0000","last_peered":"2024-05-21T15:15:16.371069+0000","last_clean":"2024-05-21T15:15:16.371069+0000","last_became_active":"2024-05-21T15:14:12.262532+0000","last_became_peered":"2024-05-21T15:14:12.262532+0000","last_unstale":"2024-05-21T15:15:16.371069+0000","last_undegraded":"2024-05-21T15:15:16.371069+0000","last_fullsized":"2024-05-21T15:15:16.371069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:22.424783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376162+0000","last_change":"2024-05-21T15:15:16.375027+0000","last_active":"2024-05-21T15:15:16.376162+0000","last_peered":"2024-05-21T15:15:16.376162+0000","last_clean":"2024-05-21T15:15:16.376162+0000","last_became_active":"2024-05-21T15:15:16.374542+0000","last_became_peered":"2024-05-21T15:15:16.374542+0000","last_unstale":"2024-05-21T15:15:16.376162+0000","last_undegraded":"2024-05-21T15:15:16.376162+0000","last_fullsized":"2024-05-21T15:15:16.376162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:55:14.399237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370953+0000","last_change":"2024-05-21T15:14:12.259845+0000","last_active":"2024-05-21T15:15:16.370953+0000","last_peered":"2024-05-21T15:15:16.370953+0000","last_clean":"2024-05-21T15:15:16.370953+0000","last_became_active":"2024-05-21T15:14:12.259455+0000","last_became_peered":"2024-05-21T15:14:12.259455+0000","last_unstale":"2024-05-21T15:15:16.370953+0000","last_undegraded":"2024-05-21T15:15:16.370953+0000","last_fullsized":"2024-05-21T15:15:16.370953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:24:59.655178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385581+0000","last_change":"2024-05-21T15:14:12.259713+0000","last_active":"2024-05-21T15:15:16.385581+0000","last_peered":"2024-05-21T15:15:16.385581+0000","last_clean":"2024-05-21T15:15:16.385581+0000","last_became_active":"2024-05-21T15:14:12.259057+0000","last_became_peered":"2024-05-21T15:14:12.259057+0000","last_unstale":"2024-05-21T15:15:16.385581+0000","last_undegraded":"2024-05-21T15:15:16.385581+0000","last_fullsized":"2024-05-21T15:15:16.385581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:46:22.377652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014185+0000","last_change":"2024-05-21T15:14:12.254389+0000","last_active":"2024-05-21T15:15:17.014185+0000","last_peered":"2024-05-21T15:15:17.014185+0000","last_clean":"2024-05-21T15:15:17.014185+0000","last_became_active":"2024-05-21T15:14:12.253846+0000","last_became_peered":"2024-05-21T15:14:12.253846+0000","last_unstale":"2024-05-21T15:15:17.014185+0000","last_undegraded":"2024-05-21T15:15:17.014185+0000","last_fullsized":"2024-05-21T15:15:17.014185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:32.932316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369609+0000","last_change":"2024-05-21T15:14:12.252292+0000","last_active":"2024-05-21T15:15:16.369609+0000","last_peered":"2024-05-21T15:15:16.369609+0000","last_clean":"2024-05-21T15:15:16.369609+0000","last_became_active":"2024-05-21T15:14:12.251816+0000","last_became_peered":"2024-05-21T15:14:12.251816+0000","last_unstale":"2024-05-21T15:15:16.369609+0000","last_undegraded":"2024-05-21T15:15:16.369609+0000","last_fullsized":"2024-05-21T15:15:16.369609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:59:14.332010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66836,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":689,"num_read_kb":412,"num_write":237,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":672,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1108033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":456911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":456617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1049807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":456628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":517564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":457967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":518175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":456453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":457161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1049485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:15:26.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T15:15:26.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:26 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:26 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:26 smithi084 ceph-mon[36334]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s; 67 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:26.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:26 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:15:26.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:26 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:15:26.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:26 smithi084 ceph-mon[36334]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:26 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:26 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:26 smithi172 ceph-mon[36402]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s; 67 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:26.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:26 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:15:26.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:26 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:15:26.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:26 smithi172 ceph-mon[36402]: from='mgr.14152 172.21.15.78:0/3294387832' entity='mgr.x' 2024-05-21T15:15:26.603 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:27.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:27 smithi078 ceph-mon[31997]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:15:27.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:27 smithi084 ceph-mon[36334]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:15:27.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:27 smithi172 ceph-mon[36402]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:15:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.209+0000 7ff4268fa700 1 -- 172.21.15.78:0/75218585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 msgr2=0x7ff420102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.209+0000 7ff4268fa700 1 --2- 172.21.15.78:0/75218585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 0x7ff420102ea0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4080099c0 tx=0x7ff408009cd0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.209+0000 7ff4268fa700 1 -- 172.21.15.78:0/75218585 shutdown_connections 2024-05-21T15:15:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.209+0000 7ff4268fa700 1 --2- 172.21.15.78:0/75218585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4201077f0 0x7ff42010f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.209+0000 7ff4268fa700 1 --2- 172.21.15.78:0/75218585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 0x7ff420102ea0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.209+0000 7ff4268fa700 1 --2- 172.21.15.78:0/75218585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff420106ea0 0x7ff4201072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.209+0000 7ff4268fa700 1 -- 172.21.15.78:0/75218585 >> 172.21.15.78:0/75218585 conn(0x7ff4200fd3a0 msgr2=0x7ff4200ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.209+0000 7ff4268fa700 1 -- 172.21.15.78:0/75218585 shutdown_connections 2024-05-21T15:15:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.210+0000 7ff4268fa700 1 -- 172.21.15.78:0/75218585 wait complete. 2024-05-21T15:15:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.210+0000 7ff4268fa700 1 Processor -- start 2024-05-21T15:15:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.210+0000 7ff4268fa700 1 -- start start 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff4268fa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 0x7ff42019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff4268fa700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff420106ea0 0x7ff42019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff4268fa700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4201077f0 0x7ff42019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff4268fa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff420112570 con 0x7ff420102a90 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff4268fa700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201123f0 con 0x7ff4201077f0 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff4268fa700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201126f0 con 0x7ff420106ea0 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 0x7ff42019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 0x7ff42019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43884/0 (socket says 172.21.15.78:43884) 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 -- 172.21.15.78:0/2906686973 learned_addr learned my addr 172.21.15.78:0/2906686973 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff424e97700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4201077f0 0x7ff42019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41f7fe700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff420106ea0 0x7ff42019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 -- 172.21.15.78:0/2906686973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff420106ea0 msgr2=0x7ff42019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff420106ea0 0x7ff42019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 -- 172.21.15.78:0/2906686973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4201077f0 msgr2=0x7ff42019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4201077f0 0x7ff42019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 -- 172.21.15.78:0/2906686973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4201a60e0 con 0x7ff420102a90 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff424e97700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4201077f0 0x7ff42019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.211+0000 7ff41ffff700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 0x7ff42019ad70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff4080059d0 tx=0x7ff40800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.212+0000 7ff41d7fa700 1 -- 172.21.15.78:0/2906686973 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408027070 con 0x7ff420102a90 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.212+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4201a6370 con 0x7ff420102a90 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.212+0000 7ff41d7fa700 1 -- 172.21.15.78:0/2906686973 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff408003680 con 0x7ff420102a90 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.212+0000 7ff41d7fa700 1 -- 172.21.15.78:0/2906686973 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408017650 con 0x7ff420102a90 2024-05-21T15:15:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.212+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4201a6820 con 0x7ff420102a90 2024-05-21T15:15:28.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.213+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff420050170 con 0x7ff420102a90 2024-05-21T15:15:28.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.214+0000 7ff41d7fa700 1 -- 172.21.15.78:0/2906686973 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff40802a070 con 0x7ff420102a90 2024-05-21T15:15:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.215+0000 7ff41d7fa700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff40c0a2cb0 0x7ff40c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.215+0000 7ff41f7fe700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff40c0a2cb0 0x7ff40c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.215+0000 7ff41d7fa700 1 -- 172.21.15.78:0/2906686973 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff4080d0380 con 0x7ff420102a90 2024-05-21T15:15:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.215+0000 7ff41f7fe700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff40c0a2cb0 0x7ff40c0a51c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff42006a8e0 tx=0x7ff410009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:28.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.218+0000 7ff41d7fa700 1 -- 172.21.15.78:0/2906686973 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff408013070 con 0x7ff420102a90 2024-05-21T15:15:28.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:28 smithi078 ceph-mon[31997]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:28.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:28 smithi084 ceph-mon[36334]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:28.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.413+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff42006af80 con 0x7ff40c0a2cb0 2024-05-21T15:15:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.420+0000 7ff41d7fa700 1 -- 172.21.15.78:0/2906686973 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258768 (secure 0 0 0) 0x7ff42006af80 con 0x7ff40c0a2cb0 2024-05-21T15:15:28.426 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff40c0a2cb0 msgr2=0x7ff40c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff40c0a2cb0 0x7ff40c0a51c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff42006a8e0 tx=0x7ff410009260 comp rx=0 tx=0).stop 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 msgr2=0x7ff42019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 0x7ff42019ad70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff4080059d0 tx=0x7ff40800fa00 comp rx=0 tx=0).stop 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 shutdown_connections 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff40c0a2cb0 0x7ff40c0a51c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4201077f0 0x7ff42019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff420102a90 0x7ff42019ad70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 --2- 172.21.15.78:0/2906686973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff420106ea0 0x7ff42019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 >> 172.21.15.78:0/2906686973 conn(0x7ff4200fd3a0 msgr2=0x7ff420106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:28.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 shutdown_connections 2024-05-21T15:15:28.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:28.423+0000 7ff4268fa700 1 -- 172.21.15.78:0/2906686973 wait complete. 2024-05-21T15:15:28.429 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-05-21T15:15:28.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:28 smithi172 ceph-mon[36402]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-05-21T15:15:29.166 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-05-21T15:15:27.270253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657438,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":830,"num_read_kb":531,"num_write":420,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181408,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358472,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583352,"statfs":{"total":1151940427776,"available":1151573352448,"internally_reserved":0,"allocated":13725696,"data_stored":7441782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":33,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":672,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.905329"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370540+0000","last_change":"2024-05-21T15:14:12.254239+0000","last_active":"2024-05-21T15:15:16.370540+0000","last_peered":"2024-05-21T15:15:16.370540+0000","last_clean":"2024-05-21T15:15:16.370540+0000","last_became_active":"2024-05-21T15:14:12.253885+0000","last_became_peered":"2024-05-21T15:14:12.253885+0000","last_unstale":"2024-05-21T15:15:16.370540+0000","last_undegraded":"2024-05-21T15:15:16.370540+0000","last_fullsized":"2024-05-21T15:15:16.370540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:35.126836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366300+0000","last_change":"2024-05-21T15:14:12.262984+0000","last_active":"2024-05-21T15:15:16.366300+0000","last_peered":"2024-05-21T15:15:16.366300+0000","last_clean":"2024-05-21T15:15:16.366300+0000","last_became_active":"2024-05-21T15:14:12.262667+0000","last_became_peered":"2024-05-21T15:14:12.262667+0000","last_unstale":"2024-05-21T15:15:16.366300+0000","last_undegraded":"2024-05-21T15:15:16.366300+0000","last_fullsized":"2024-05-21T15:15:16.366300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:47:47.434314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366237+0000","last_change":"2024-05-21T15:14:12.260573+0000","last_active":"2024-05-21T15:15:16.366237+0000","last_peered":"2024-05-21T15:15:16.366237+0000","last_clean":"2024-05-21T15:15:16.366237+0000","last_became_active":"2024-05-21T15:14:12.259073+0000","last_became_peered":"2024-05-21T15:14:12.259073+0000","last_unstale":"2024-05-21T15:15:16.366237+0000","last_undegraded":"2024-05-21T15:15:16.366237+0000","last_fullsized":"2024-05-21T15:15:16.366237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:15:01.919464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.615011+0000","last_change":"2024-05-21T15:14:12.256992+0000","last_active":"2024-05-21T15:15:16.615011+0000","last_peered":"2024-05-21T15:15:16.615011+0000","last_clean":"2024-05-21T15:15:16.615011+0000","last_became_active":"2024-05-21T15:14:12.256448+0000","last_became_peered":"2024-05-21T15:14:12.256448+0000","last_unstale":"2024-05-21T15:15:16.615011+0000","last_undegraded":"2024-05-21T15:15:16.615011+0000","last_fullsized":"2024-05-21T15:15:16.615011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:40.569263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367125+0000","last_change":"2024-05-21T15:14:12.251533+0000","last_active":"2024-05-21T15:15:16.367125+0000","last_peered":"2024-05-21T15:15:16.367125+0000","last_clean":"2024-05-21T15:15:16.367125+0000","last_became_active":"2024-05-21T15:14:12.251210+0000","last_became_peered":"2024-05-21T15:14:12.251210+0000","last_unstale":"2024-05-21T15:15:16.367125+0000","last_undegraded":"2024-05-21T15:15:16.367125+0000","last_fullsized":"2024-05-21T15:15:16.367125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:43:34.109179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376308+0000","last_change":"2024-05-21T15:14:12.251767+0000","last_active":"2024-05-21T15:15:16.376308+0000","last_peered":"2024-05-21T15:15:16.376308+0000","last_clean":"2024-05-21T15:15:16.376308+0000","last_became_active":"2024-05-21T15:14:12.251427+0000","last_became_peered":"2024-05-21T15:14:12.251427+0000","last_unstale":"2024-05-21T15:15:16.376308+0000","last_undegraded":"2024-05-21T15:15:16.376308+0000","last_fullsized":"2024-05-21T15:15:16.376308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:48:41.804074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370565+0000","last_change":"2024-05-21T15:14:12.253472+0000","last_active":"2024-05-21T15:15:16.370565+0000","last_peered":"2024-05-21T15:15:16.370565+0000","last_clean":"2024-05-21T15:15:16.370565+0000","last_became_active":"2024-05-21T15:14:12.252762+0000","last_became_peered":"2024-05-21T15:14:12.252762+0000","last_unstale":"2024-05-21T15:15:16.370565+0000","last_undegraded":"2024-05-21T15:15:16.370565+0000","last_fullsized":"2024-05-21T15:15:16.370565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:11:44.497404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370791+0000","last_change":"2024-05-21T15:14:12.258627+0000","last_active":"2024-05-21T15:15:16.370791+0000","last_peered":"2024-05-21T15:15:16.370791+0000","last_clean":"2024-05-21T15:15:16.370791+0000","last_became_active":"2024-05-21T15:14:12.258118+0000","last_became_peered":"2024-05-21T15:14:12.258118+0000","last_unstale":"2024-05-21T15:15:16.370791+0000","last_undegraded":"2024-05-21T15:15:16.370791+0000","last_fullsized":"2024-05-21T15:15:16.370791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:56.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.386192+0000","last_change":"2024-05-21T15:14:12.255998+0000","last_active":"2024-05-21T15:15:16.386192+0000","last_peered":"2024-05-21T15:15:16.386192+0000","last_clean":"2024-05-21T15:15:16.386192+0000","last_became_active":"2024-05-21T15:14:12.255658+0000","last_became_peered":"2024-05-21T15:14:12.255658+0000","last_unstale":"2024-05-21T15:15:16.386192+0000","last_undegraded":"2024-05-21T15:15:16.386192+0000","last_fullsized":"2024-05-21T15:15:16.386192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:44:42.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376642+0000","last_change":"2024-05-21T15:14:12.257617+0000","last_active":"2024-05-21T15:15:16.376642+0000","last_peered":"2024-05-21T15:15:16.376642+0000","last_clean":"2024-05-21T15:15:16.376642+0000","last_became_active":"2024-05-21T15:14:12.256947+0000","last_became_peered":"2024-05-21T15:14:12.256947+0000","last_unstale":"2024-05-21T15:15:16.376642+0000","last_undegraded":"2024-05-21T15:15:16.376642+0000","last_fullsized":"2024-05-21T15:15:16.376642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:45:58.174954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371137+0000","last_change":"2024-05-21T15:14:12.260454+0000","last_active":"2024-05-21T15:15:16.371137+0000","last_peered":"2024-05-21T15:15:16.371137+0000","last_clean":"2024-05-21T15:15:16.371137+0000","last_became_active":"2024-05-21T15:14:12.260126+0000","last_became_peered":"2024-05-21T15:14:12.260126+0000","last_unstale":"2024-05-21T15:15:16.371137+0000","last_undegraded":"2024-05-21T15:15:16.371137+0000","last_fullsized":"2024-05-21T15:15:16.371137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:49:45.212050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.614872+0000","last_change":"2024-05-21T15:14:12.252071+0000","last_active":"2024-05-21T15:15:16.614872+0000","last_peered":"2024-05-21T15:15:16.614872+0000","last_clean":"2024-05-21T15:15:16.614872+0000","last_became_active":"2024-05-21T15:14:12.251706+0000","last_became_peered":"2024-05-21T15:14:12.251706+0000","last_unstale":"2024-05-21T15:15:16.614872+0000","last_undegraded":"2024-05-21T15:15:16.614872+0000","last_fullsized":"2024-05-21T15:15:16.614872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:02:44.349368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014102+0000","last_change":"2024-05-21T15:14:12.239877+0000","last_active":"2024-05-21T15:15:17.014102+0000","last_peered":"2024-05-21T15:15:17.014102+0000","last_clean":"2024-05-21T15:15:17.014102+0000","last_became_active":"2024-05-21T15:14:12.239615+0000","last_became_peered":"2024-05-21T15:14:12.239615+0000","last_unstale":"2024-05-21T15:15:17.014102+0000","last_undegraded":"2024-05-21T15:15:17.014102+0000","last_fullsized":"2024-05-21T15:15:17.014102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:23.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366047+0000","last_change":"2024-05-21T15:14:12.260349+0000","last_active":"2024-05-21T15:15:16.366047+0000","last_peered":"2024-05-21T15:15:16.366047+0000","last_clean":"2024-05-21T15:15:16.366047+0000","last_became_active":"2024-05-21T15:14:12.259658+0000","last_became_peered":"2024-05-21T15:14:12.259658+0000","last_unstale":"2024-05-21T15:15:16.366047+0000","last_undegraded":"2024-05-21T15:15:16.366047+0000","last_fullsized":"2024-05-21T15:15:16.366047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:59:32.326571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.015535+0000","last_change":"2024-05-21T15:15:17.015535+0000","last_active":"2024-05-21T15:15:17.015535+0000","last_peered":"2024-05-21T15:15:17.015535+0000","last_clean":"2024-05-21T15:15:17.015535+0000","last_became_active":"2024-05-21T15:15:17.015057+0000","last_became_peered":"2024-05-21T15:15:17.015057+0000","last_unstale":"2024-05-21T15:15:17.015535+0000","last_undegraded":"2024-05-21T15:15:17.015535+0000","last_fullsized":"2024-05-21T15:15:17.015535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:48:14.652870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385395+0000","last_change":"2024-05-21T15:14:12.264667+0000","last_active":"2024-05-21T15:15:16.385395+0000","last_peered":"2024-05-21T15:15:16.385395+0000","last_clean":"2024-05-21T15:15:16.385395+0000","last_became_active":"2024-05-21T15:14:12.264284+0000","last_became_peered":"2024-05-21T15:14:12.264284+0000","last_unstale":"2024-05-21T15:15:16.385395+0000","last_undegraded":"2024-05-21T15:15:16.385395+0000","last_fullsized":"2024-05-21T15:15:16.385395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:24.861321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370655+0000","last_change":"2024-05-21T15:14:12.259912+0000","last_active":"2024-05-21T15:15:16.370655+0000","last_peered":"2024-05-21T15:15:16.370655+0000","last_clean":"2024-05-21T15:15:16.370655+0000","last_became_active":"2024-05-21T15:14:12.259578+0000","last_became_peered":"2024-05-21T15:14:12.259578+0000","last_unstale":"2024-05-21T15:15:16.370655+0000","last_undegraded":"2024-05-21T15:15:16.370655+0000","last_fullsized":"2024-05-21T15:15:16.370655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:05:33.372481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369455+0000","last_change":"2024-05-21T15:14:12.258355+0000","last_active":"2024-05-21T15:15:16.369455+0000","last_peered":"2024-05-21T15:15:16.369455+0000","last_clean":"2024-05-21T15:15:16.369455+0000","last_became_active":"2024-05-21T15:14:12.257991+0000","last_became_peered":"2024-05-21T15:14:12.257991+0000","last_unstale":"2024-05-21T15:15:16.369455+0000","last_undegraded":"2024-05-21T15:15:16.369455+0000","last_fullsized":"2024-05-21T15:15:16.369455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:54.014553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372836+0000","last_change":"2024-05-21T15:14:12.258873+0000","last_active":"2024-05-21T15:15:16.372836+0000","last_peered":"2024-05-21T15:15:16.372836+0000","last_clean":"2024-05-21T15:15:16.372836+0000","last_became_active":"2024-05-21T15:14:12.258548+0000","last_became_peered":"2024-05-21T15:14:12.258548+0000","last_unstale":"2024-05-21T15:15:16.372836+0000","last_undegraded":"2024-05-21T15:15:16.372836+0000","last_fullsized":"2024-05-21T15:15:16.372836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:00.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:22.625145+0000","last_change":"2024-05-21T15:15:16.626507+0000","last_active":"2024-05-21T15:15:22.625145+0000","last_peered":"2024-05-21T15:15:22.625145+0000","last_clean":"2024-05-21T15:15:22.625145+0000","last_became_active":"2024-05-21T15:15:16.615945+0000","last_became_peered":"2024-05-21T15:15:16.615945+0000","last_unstale":"2024-05-21T15:15:22.625145+0000","last_undegraded":"2024-05-21T15:15:22.625145+0000","last_fullsized":"2024-05-21T15:15:22.625145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:16:01.229449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371331+0000","last_change":"2024-05-21T15:14:12.261632+0000","last_active":"2024-05-21T15:15:16.371331+0000","last_peered":"2024-05-21T15:15:16.371331+0000","last_clean":"2024-05-21T15:15:16.371331+0000","last_became_active":"2024-05-21T15:14:12.261149+0000","last_became_peered":"2024-05-21T15:14:12.261149+0000","last_unstale":"2024-05-21T15:15:16.371331+0000","last_undegraded":"2024-05-21T15:15:16.371331+0000","last_fullsized":"2024-05-21T15:15:16.371331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:11.349395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371454+0000","last_change":"2024-05-21T15:14:11.567859+0000","last_active":"2024-05-21T15:15:16.371454+0000","last_peered":"2024-05-21T15:15:16.371454+0000","last_clean":"2024-05-21T15:15:16.371454+0000","last_became_active":"2024-05-21T15:14:11.566855+0000","last_became_peered":"2024-05-21T15:14:11.566855+0000","last_unstale":"2024-05-21T15:15:16.371454+0000","last_undegraded":"2024-05-21T15:15:16.371454+0000","last_fullsized":"2024-05-21T15:15:16.371454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:21.568436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366374+0000","last_change":"2024-05-21T15:14:12.262153+0000","last_active":"2024-05-21T15:15:16.366374+0000","last_peered":"2024-05-21T15:15:16.366374+0000","last_clean":"2024-05-21T15:15:16.366374+0000","last_became_active":"2024-05-21T15:14:12.261780+0000","last_became_peered":"2024-05-21T15:14:12.261780+0000","last_unstale":"2024-05-21T15:15:16.366374+0000","last_undegraded":"2024-05-21T15:15:16.366374+0000","last_fullsized":"2024-05-21T15:15:16.366374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:21:25.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.368000+0000","last_change":"2024-05-21T15:14:11.570316+0000","last_active":"2024-05-21T15:15:16.368000+0000","last_peered":"2024-05-21T15:15:16.368000+0000","last_clean":"2024-05-21T15:15:16.368000+0000","last_became_active":"2024-05-21T15:14:11.569967+0000","last_became_peered":"2024-05-21T15:14:11.569967+0000","last_unstale":"2024-05-21T15:15:16.368000+0000","last_undegraded":"2024-05-21T15:15:16.368000+0000","last_fullsized":"2024-05-21T15:15:16.368000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:30:27.364742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385649+0000","last_change":"2024-05-21T15:14:12.261716+0000","last_active":"2024-05-21T15:15:16.385649+0000","last_peered":"2024-05-21T15:15:16.385649+0000","last_clean":"2024-05-21T15:15:16.385649+0000","last_became_active":"2024-05-21T15:14:12.261341+0000","last_became_peered":"2024-05-21T15:14:12.261341+0000","last_unstale":"2024-05-21T15:15:16.385649+0000","last_undegraded":"2024-05-21T15:15:16.385649+0000","last_fullsized":"2024-05-21T15:15:16.385649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:40:10.464974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367543+0000","last_change":"2024-05-21T15:14:11.573305+0000","last_active":"2024-05-21T15:15:16.367543+0000","last_peered":"2024-05-21T15:15:16.367543+0000","last_clean":"2024-05-21T15:15:16.367543+0000","last_became_active":"2024-05-21T15:14:11.572655+0000","last_became_peered":"2024-05-21T15:14:11.572655+0000","last_unstale":"2024-05-21T15:15:16.367543+0000","last_undegraded":"2024-05-21T15:15:16.367543+0000","last_fullsized":"2024-05-21T15:15:16.367543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:37.268440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372322+0000","last_change":"2024-05-21T15:14:12.253788+0000","last_active":"2024-05-21T15:15:16.372322+0000","last_peered":"2024-05-21T15:15:16.372322+0000","last_clean":"2024-05-21T15:15:16.372322+0000","last_became_active":"2024-05-21T15:14:12.253450+0000","last_became_peered":"2024-05-21T15:14:12.253450+0000","last_unstale":"2024-05-21T15:15:16.372322+0000","last_undegraded":"2024-05-21T15:15:16.372322+0000","last_fullsized":"2024-05-21T15:15:16.372322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:57.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:22.359820+0000","last_change":"2024-05-21T15:14:11.567400+0000","last_active":"2024-05-21T15:15:22.359820+0000","last_peered":"2024-05-21T15:15:22.359820+0000","last_clean":"2024-05-21T15:15:22.359820+0000","last_became_active":"2024-05-21T15:14:11.565995+0000","last_became_peered":"2024-05-21T15:14:11.565995+0000","last_unstale":"2024-05-21T15:15:22.359820+0000","last_undegraded":"2024-05-21T15:15:22.359820+0000","last_fullsized":"2024-05-21T15:15:22.359820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:50.463339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367049+0000","last_change":"2024-05-21T15:14:12.251565+0000","last_active":"2024-05-21T15:15:16.367049+0000","last_peered":"2024-05-21T15:15:16.367049+0000","last_clean":"2024-05-21T15:15:16.367049+0000","last_became_active":"2024-05-21T15:14:12.251289+0000","last_became_peered":"2024-05-21T15:14:12.251289+0000","last_unstale":"2024-05-21T15:15:16.367049+0000","last_undegraded":"2024-05-21T15:15:16.367049+0000","last_fullsized":"2024-05-21T15:15:16.367049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:44.619800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376264+0000","last_change":"2024-05-21T15:14:11.575193+0000","last_active":"2024-05-21T15:15:16.376264+0000","last_peered":"2024-05-21T15:15:16.376264+0000","last_clean":"2024-05-21T15:15:16.376264+0000","last_became_active":"2024-05-21T15:14:11.574861+0000","last_became_peered":"2024-05-21T15:14:11.574861+0000","last_unstale":"2024-05-21T15:15:16.376264+0000","last_undegraded":"2024-05-21T15:15:16.376264+0000","last_fullsized":"2024-05-21T15:15:16.376264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:54.548408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.015620+0000","last_change":"2024-05-21T15:15:17.015620+0000","last_active":"2024-05-21T15:15:17.015620+0000","last_peered":"2024-05-21T15:15:17.015620+0000","last_clean":"2024-05-21T15:15:17.015620+0000","last_became_active":"2024-05-21T15:15:17.015086+0000","last_became_peered":"2024-05-21T15:15:17.015086+0000","last_unstale":"2024-05-21T15:15:17.015620+0000","last_undegraded":"2024-05-21T15:15:17.015620+0000","last_fullsized":"2024-05-21T15:15:17.015620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:10.856246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014195+0000","last_change":"2024-05-21T15:14:11.572087+0000","last_active":"2024-05-21T15:15:17.014195+0000","last_peered":"2024-05-21T15:15:17.014195+0000","last_clean":"2024-05-21T15:15:17.014195+0000","last_became_active":"2024-05-21T15:14:11.571716+0000","last_became_peered":"2024-05-21T15:14:11.571716+0000","last_unstale":"2024-05-21T15:15:17.014195+0000","last_undegraded":"2024-05-21T15:15:17.014195+0000","last_fullsized":"2024-05-21T15:15:17.014195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:07.082766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370919+0000","last_change":"2024-05-21T15:14:12.252233+0000","last_active":"2024-05-21T15:15:16.370919+0000","last_peered":"2024-05-21T15:15:16.370919+0000","last_clean":"2024-05-21T15:15:16.370919+0000","last_became_active":"2024-05-21T15:14:12.251901+0000","last_became_peered":"2024-05-21T15:14:12.251901+0000","last_unstale":"2024-05-21T15:15:16.370919+0000","last_undegraded":"2024-05-21T15:15:16.370919+0000","last_fullsized":"2024-05-21T15:15:16.370919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:05.107436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371590+0000","last_change":"2024-05-21T15:14:11.565946+0000","last_active":"2024-05-21T15:15:16.371590+0000","last_peered":"2024-05-21T15:15:16.371590+0000","last_clean":"2024-05-21T15:15:16.371590+0000","last_became_active":"2024-05-21T15:14:11.565601+0000","last_became_peered":"2024-05-21T15:14:11.565601+0000","last_unstale":"2024-05-21T15:15:16.371590+0000","last_undegraded":"2024-05-21T15:15:16.371590+0000","last_fullsized":"2024-05-21T15:15:16.371590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:41:55.243473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369411+0000","last_change":"2024-05-21T15:14:12.255188+0000","last_active":"2024-05-21T15:15:16.369411+0000","last_peered":"2024-05-21T15:15:16.369411+0000","last_clean":"2024-05-21T15:15:16.369411+0000","last_became_active":"2024-05-21T15:14:12.254503+0000","last_became_peered":"2024-05-21T15:14:12.254503+0000","last_unstale":"2024-05-21T15:15:16.369411+0000","last_undegraded":"2024-05-21T15:15:16.369411+0000","last_fullsized":"2024-05-21T15:15:16.369411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:22:16.336740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":157,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:23.360511+0000","last_change":"2024-05-21T15:14:11.569882+0000","last_active":"2024-05-21T15:15:23.360511+0000","last_peered":"2024-05-21T15:15:23.360511+0000","last_clean":"2024-05-21T15:15:23.360511+0000","last_became_active":"2024-05-21T15:14:11.569532+0000","last_became_peered":"2024-05-21T15:14:11.569532+0000","last_unstale":"2024-05-21T15:15:23.360511+0000","last_undegraded":"2024-05-21T15:15:23.360511+0000","last_fullsized":"2024-05-21T15:15:23.360511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:58:50.081709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385949+0000","last_change":"2024-05-21T15:14:12.262691+0000","last_active":"2024-05-21T15:15:16.385949+0000","last_peered":"2024-05-21T15:15:16.385949+0000","last_clean":"2024-05-21T15:15:16.385949+0000","last_became_active":"2024-05-21T15:14:12.261492+0000","last_became_peered":"2024-05-21T15:14:12.261492+0000","last_unstale":"2024-05-21T15:15:16.385949+0000","last_undegraded":"2024-05-21T15:15:16.385949+0000","last_fullsized":"2024-05-21T15:15:16.385949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:35:00.121256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370695+0000","last_change":"2024-05-21T15:14:11.569231+0000","last_active":"2024-05-21T15:15:16.370695+0000","last_peered":"2024-05-21T15:15:16.370695+0000","last_clean":"2024-05-21T15:15:16.370695+0000","last_became_active":"2024-05-21T15:14:11.568916+0000","last_became_peered":"2024-05-21T15:14:11.568916+0000","last_unstale":"2024-05-21T15:15:16.370695+0000","last_undegraded":"2024-05-21T15:15:16.370695+0000","last_fullsized":"2024-05-21T15:15:16.370695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:26:13.411854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366276+0000","last_change":"2024-05-21T15:14:12.256307+0000","last_active":"2024-05-21T15:15:16.366276+0000","last_peered":"2024-05-21T15:15:16.366276+0000","last_clean":"2024-05-21T15:15:16.366276+0000","last_became_active":"2024-05-21T15:14:12.255828+0000","last_became_peered":"2024-05-21T15:14:12.255828+0000","last_unstale":"2024-05-21T15:15:16.366276+0000","last_undegraded":"2024-05-21T15:15:16.366276+0000","last_fullsized":"2024-05-21T15:15:16.366276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:11:40.068779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372517+0000","last_change":"2024-05-21T15:14:11.568239+0000","last_active":"2024-05-21T15:15:16.372517+0000","last_peered":"2024-05-21T15:15:16.372517+0000","last_clean":"2024-05-21T15:15:16.372517+0000","last_became_active":"2024-05-21T15:14:11.567836+0000","last_became_peered":"2024-05-21T15:14:11.567836+0000","last_unstale":"2024-05-21T15:15:16.372517+0000","last_undegraded":"2024-05-21T15:15:16.372517+0000","last_fullsized":"2024-05-21T15:15:16.372517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:10.244172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385787+0000","last_change":"2024-05-21T15:14:12.257762+0000","last_active":"2024-05-21T15:15:16.385787+0000","last_peered":"2024-05-21T15:15:16.385787+0000","last_clean":"2024-05-21T15:15:16.385787+0000","last_became_active":"2024-05-21T15:14:12.256335+0000","last_became_peered":"2024-05-21T15:14:12.256335+0000","last_unstale":"2024-05-21T15:15:16.385787+0000","last_undegraded":"2024-05-21T15:15:16.385787+0000","last_fullsized":"2024-05-21T15:15:16.385787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:35.219673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":194,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:23.388091+0000","last_change":"2024-05-21T15:14:11.564407+0000","last_active":"2024-05-21T15:15:23.388091+0000","last_peered":"2024-05-21T15:15:23.388091+0000","last_clean":"2024-05-21T15:15:23.388091+0000","last_became_active":"2024-05-21T15:14:11.564012+0000","last_became_peered":"2024-05-21T15:14:11.564012+0000","last_unstale":"2024-05-21T15:15:23.388091+0000","last_undegraded":"2024-05-21T15:15:23.388091+0000","last_fullsized":"2024-05-21T15:15:23.388091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:34:54.177844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":144,"num_read_kb":130,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367652+0000","last_change":"2024-05-21T15:14:12.253820+0000","last_active":"2024-05-21T15:15:16.367652+0000","last_peered":"2024-05-21T15:15:16.367652+0000","last_clean":"2024-05-21T15:15:16.367652+0000","last_became_active":"2024-05-21T15:14:12.253360+0000","last_became_peered":"2024-05-21T15:14:12.253360+0000","last_unstale":"2024-05-21T15:15:16.367652+0000","last_undegraded":"2024-05-21T15:15:16.367652+0000","last_fullsized":"2024-05-21T15:15:16.367652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:38:19.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372744+0000","last_change":"2024-05-21T15:14:11.571161+0000","last_active":"2024-05-21T15:15:16.372744+0000","last_peered":"2024-05-21T15:15:16.372744+0000","last_clean":"2024-05-21T15:15:16.372744+0000","last_became_active":"2024-05-21T15:14:11.570837+0000","last_became_peered":"2024-05-21T15:14:11.570837+0000","last_unstale":"2024-05-21T15:15:16.372744+0000","last_undegraded":"2024-05-21T15:15:16.372744+0000","last_fullsized":"2024-05-21T15:15:16.372744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:24:55.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.013982+0000","last_change":"2024-05-21T15:14:12.255713+0000","last_active":"2024-05-21T15:15:17.013982+0000","last_peered":"2024-05-21T15:15:17.013982+0000","last_clean":"2024-05-21T15:15:17.013982+0000","last_became_active":"2024-05-21T15:14:12.254458+0000","last_became_peered":"2024-05-21T15:14:12.254458+0000","last_unstale":"2024-05-21T15:15:17.013982+0000","last_undegraded":"2024-05-21T15:15:17.013982+0000","last_fullsized":"2024-05-21T15:15:17.013982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:55:24.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371715+0000","last_change":"2024-05-21T15:14:11.565916+0000","last_active":"2024-05-21T15:15:16.371715+0000","last_peered":"2024-05-21T15:15:16.371715+0000","last_clean":"2024-05-21T15:15:16.371715+0000","last_became_active":"2024-05-21T15:14:11.565460+0000","last_became_peered":"2024-05-21T15:14:11.565460+0000","last_unstale":"2024-05-21T15:15:16.371715+0000","last_undegraded":"2024-05-21T15:15:16.371715+0000","last_fullsized":"2024-05-21T15:15:16.371715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:09.982208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371049+0000","last_change":"2024-05-21T15:14:12.251856+0000","last_active":"2024-05-21T15:15:16.371049+0000","last_peered":"2024-05-21T15:15:16.371049+0000","last_clean":"2024-05-21T15:15:16.371049+0000","last_became_active":"2024-05-21T15:14:12.251496+0000","last_became_peered":"2024-05-21T15:14:12.251496+0000","last_unstale":"2024-05-21T15:15:16.371049+0000","last_undegraded":"2024-05-21T15:15:16.371049+0000","last_fullsized":"2024-05-21T15:15:16.371049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:17.566276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:23.360499+0000","last_change":"2024-05-21T15:14:11.565485+0000","last_active":"2024-05-21T15:15:23.360499+0000","last_peered":"2024-05-21T15:15:23.360499+0000","last_clean":"2024-05-21T15:15:23.360499+0000","last_became_active":"2024-05-21T15:14:11.565092+0000","last_became_peered":"2024-05-21T15:14:11.565092+0000","last_unstale":"2024-05-21T15:15:23.360499+0000","last_undegraded":"2024-05-21T15:15:23.360499+0000","last_fullsized":"2024-05-21T15:15:23.360499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:01:16.101793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370404+0000","last_change":"2024-05-21T15:14:12.262283+0000","last_active":"2024-05-21T15:15:16.370404+0000","last_peered":"2024-05-21T15:15:16.370404+0000","last_clean":"2024-05-21T15:15:16.370404+0000","last_became_active":"2024-05-21T15:14:12.261610+0000","last_became_peered":"2024-05-21T15:14:12.261610+0000","last_unstale":"2024-05-21T15:15:16.370404+0000","last_undegraded":"2024-05-21T15:15:16.370404+0000","last_fullsized":"2024-05-21T15:15:16.370404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:44:33.087115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366991+0000","last_change":"2024-05-21T15:14:11.569905+0000","last_active":"2024-05-21T15:15:16.366991+0000","last_peered":"2024-05-21T15:15:16.366991+0000","last_clean":"2024-05-21T15:15:16.366991+0000","last_became_active":"2024-05-21T15:14:11.569221+0000","last_became_peered":"2024-05-21T15:14:11.569221+0000","last_unstale":"2024-05-21T15:15:16.366991+0000","last_undegraded":"2024-05-21T15:15:16.366991+0000","last_fullsized":"2024-05-21T15:15:16.366991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:11:04.875890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.386384+0000","last_change":"2024-05-21T15:15:16.386384+0000","last_active":"2024-05-21T15:15:16.386384+0000","last_peered":"2024-05-21T15:15:16.386384+0000","last_clean":"2024-05-21T15:15:16.386384+0000","last_became_active":"2024-05-21T15:15:16.385810+0000","last_became_peered":"2024-05-21T15:15:16.385810+0000","last_unstale":"2024-05-21T15:15:16.386384+0000","last_undegraded":"2024-05-21T15:15:16.386384+0000","last_fullsized":"2024-05-21T15:15:16.386384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:49:45.031393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367815+0000","last_change":"2024-05-21T15:14:11.569670+0000","last_active":"2024-05-21T15:15:16.367815+0000","last_peered":"2024-05-21T15:15:16.367815+0000","last_clean":"2024-05-21T15:15:16.367815+0000","last_became_active":"2024-05-21T15:14:11.569257+0000","last_became_peered":"2024-05-21T15:14:11.569257+0000","last_unstale":"2024-05-21T15:15:16.367815+0000","last_undegraded":"2024-05-21T15:15:16.367815+0000","last_fullsized":"2024-05-21T15:15:16.367815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:18.584920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.374845+0000","last_change":"2024-05-21T15:15:16.374845+0000","last_active":"2024-05-21T15:15:16.374845+0000","last_peered":"2024-05-21T15:15:16.374845+0000","last_clean":"2024-05-21T15:15:16.374845+0000","last_became_active":"2024-05-21T15:15:16.374393+0000","last_became_peered":"2024-05-21T15:15:16.374393+0000","last_unstale":"2024-05-21T15:15:16.374845+0000","last_undegraded":"2024-05-21T15:15:16.374845+0000","last_fullsized":"2024-05-21T15:15:16.374845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:38:21.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369371+0000","last_change":"2024-05-21T15:14:11.567849+0000","last_active":"2024-05-21T15:15:16.369371+0000","last_peered":"2024-05-21T15:15:16.369371+0000","last_clean":"2024-05-21T15:15:16.369371+0000","last_became_active":"2024-05-21T15:14:11.567476+0000","last_became_peered":"2024-05-21T15:14:11.567476+0000","last_unstale":"2024-05-21T15:15:16.369371+0000","last_undegraded":"2024-05-21T15:15:16.369371+0000","last_fullsized":"2024-05-21T15:15:16.369371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:13.516765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376527+0000","last_change":"2024-05-21T15:14:12.255165+0000","last_active":"2024-05-21T15:15:16.376527+0000","last_peered":"2024-05-21T15:15:16.376527+0000","last_clean":"2024-05-21T15:15:16.376527+0000","last_became_active":"2024-05-21T15:14:12.254743+0000","last_became_peered":"2024-05-21T15:14:12.254743+0000","last_unstale":"2024-05-21T15:15:16.376527+0000","last_undegraded":"2024-05-21T15:15:16.376527+0000","last_fullsized":"2024-05-21T15:15:16.376527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:53:11.059597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:20.622508+0000","last_change":"2024-05-21T15:14:11.569745+0000","last_active":"2024-05-21T15:15:20.622508+0000","last_peered":"2024-05-21T15:15:20.622508+0000","last_clean":"2024-05-21T15:15:20.622508+0000","last_became_active":"2024-05-21T15:14:11.569409+0000","last_became_peered":"2024-05-21T15:14:11.569409+0000","last_unstale":"2024-05-21T15:15:20.622508+0000","last_undegraded":"2024-05-21T15:15:20.622508+0000","last_fullsized":"2024-05-21T15:15:20.622508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:00:37.963797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376429+0000","last_change":"2024-05-21T15:14:12.254631+0000","last_active":"2024-05-21T15:15:16.376429+0000","last_peered":"2024-05-21T15:15:16.376429+0000","last_clean":"2024-05-21T15:15:16.376429+0000","last_became_active":"2024-05-21T15:14:12.254239+0000","last_became_peered":"2024-05-21T15:14:12.254239+0000","last_unstale":"2024-05-21T15:15:16.376429+0000","last_undegraded":"2024-05-21T15:15:16.376429+0000","last_fullsized":"2024-05-21T15:15:16.376429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:55:17.135550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.616309+0000","last_change":"2024-05-21T15:15:16.615801+0000","last_active":"2024-05-21T15:15:16.616309+0000","last_peered":"2024-05-21T15:15:16.616309+0000","last_clean":"2024-05-21T15:15:16.616309+0000","last_became_active":"2024-05-21T15:15:16.615323+0000","last_became_peered":"2024-05-21T15:15:16.615323+0000","last_unstale":"2024-05-21T15:15:16.616309+0000","last_undegraded":"2024-05-21T15:15:16.616309+0000","last_fullsized":"2024-05-21T15:15:16.616309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:15.233980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:21.356730+0000","last_change":"2024-05-21T15:14:11.574352+0000","last_active":"2024-05-21T15:15:21.356730+0000","last_peered":"2024-05-21T15:15:21.356730+0000","last_clean":"2024-05-21T15:15:21.356730+0000","last_became_active":"2024-05-21T15:14:11.574041+0000","last_became_peered":"2024-05-21T15:14:11.574041+0000","last_unstale":"2024-05-21T15:15:21.356730+0000","last_undegraded":"2024-05-21T15:15:21.356730+0000","last_fullsized":"2024-05-21T15:15:21.356730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.563218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371236+0000","last_change":"2024-05-21T15:14:12.257886+0000","last_active":"2024-05-21T15:15:16.371236+0000","last_peered":"2024-05-21T15:15:16.371236+0000","last_clean":"2024-05-21T15:15:16.371236+0000","last_became_active":"2024-05-21T15:14:12.257546+0000","last_became_peered":"2024-05-21T15:14:12.257546+0000","last_unstale":"2024-05-21T15:15:16.371236+0000","last_undegraded":"2024-05-21T15:15:16.371236+0000","last_fullsized":"2024-05-21T15:15:16.371236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:20.953421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.614562+0000","last_change":"2024-05-21T15:14:11.568524+0000","last_active":"2024-05-21T15:15:16.614562+0000","last_peered":"2024-05-21T15:15:16.614562+0000","last_clean":"2024-05-21T15:15:16.614562+0000","last_became_active":"2024-05-21T15:14:11.567981+0000","last_became_peered":"2024-05-21T15:14:11.567981+0000","last_unstale":"2024-05-21T15:15:16.614562+0000","last_undegraded":"2024-05-21T15:15:16.614562+0000","last_fullsized":"2024-05-21T15:15:16.614562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370486+0000","last_change":"2024-05-21T15:13:02.249260+0000","last_active":"2024-05-21T15:15:16.370486+0000","last_peered":"2024-05-21T15:15:16.370486+0000","last_clean":"2024-05-21T15:15:16.370486+0000","last_became_active":"2024-05-21T15:13:02.248637+0000","last_became_peered":"2024-05-21T15:13:02.248637+0000","last_unstale":"2024-05-21T15:15:16.370486+0000","last_undegraded":"2024-05-21T15:15:16.370486+0000","last_fullsized":"2024-05-21T15:15:16.370486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_clean_scrub_stamp":"2024-05-21T15:09:41.506463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:26.250910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014280+0000","last_change":"2024-05-21T15:14:12.259861+0000","last_active":"2024-05-21T15:15:17.014280+0000","last_peered":"2024-05-21T15:15:17.014280+0000","last_clean":"2024-05-21T15:15:17.014280+0000","last_became_active":"2024-05-21T15:14:12.259510+0000","last_became_peered":"2024-05-21T15:14:12.259510+0000","last_unstale":"2024-05-21T15:15:17.014280+0000","last_undegraded":"2024-05-21T15:15:17.014280+0000","last_fullsized":"2024-05-21T15:15:17.014280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:49.548366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370650+0000","last_change":"2024-05-21T15:14:11.570705+0000","last_active":"2024-05-21T15:15:16.370650+0000","last_peered":"2024-05-21T15:15:16.370650+0000","last_clean":"2024-05-21T15:15:16.370650+0000","last_became_active":"2024-05-21T15:14:11.570403+0000","last_became_peered":"2024-05-21T15:14:11.570403+0000","last_unstale":"2024-05-21T15:15:16.370650+0000","last_undegraded":"2024-05-21T15:15:16.370650+0000","last_fullsized":"2024-05-21T15:15:16.370650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:02:34.156258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367655+0000","last_change":"2024-05-21T15:14:11.572311+0000","last_active":"2024-05-21T15:15:16.367655+0000","last_peered":"2024-05-21T15:15:16.367655+0000","last_clean":"2024-05-21T15:15:16.367655+0000","last_became_active":"2024-05-21T15:14:11.571949+0000","last_became_peered":"2024-05-21T15:14:11.571949+0000","last_unstale":"2024-05-21T15:15:16.367655+0000","last_undegraded":"2024-05-21T15:15:16.367655+0000","last_fullsized":"2024-05-21T15:15:16.367655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:57:59.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014340+0000","last_change":"2024-05-21T15:14:12.260717+0000","last_active":"2024-05-21T15:15:17.014340+0000","last_peered":"2024-05-21T15:15:17.014340+0000","last_clean":"2024-05-21T15:15:17.014340+0000","last_became_active":"2024-05-21T15:14:12.259911+0000","last_became_peered":"2024-05-21T15:14:12.259911+0000","last_unstale":"2024-05-21T15:15:17.014340+0000","last_undegraded":"2024-05-21T15:15:17.014340+0000","last_fullsized":"2024-05-21T15:15:17.014340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:02.609299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.381952+0000","last_change":"2024-05-21T15:15:16.381952+0000","last_active":"2024-05-21T15:15:16.381952+0000","last_peered":"2024-05-21T15:15:16.381952+0000","last_clean":"2024-05-21T15:15:16.381952+0000","last_became_active":"2024-05-21T15:15:16.373966+0000","last_became_peered":"2024-05-21T15:15:16.373966+0000","last_unstale":"2024-05-21T15:15:16.381952+0000","last_undegraded":"2024-05-21T15:15:16.381952+0000","last_fullsized":"2024-05-21T15:15:16.381952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:49:05.368066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371210+0000","last_change":"2024-05-21T15:14:12.253634+0000","last_active":"2024-05-21T15:15:16.371210+0000","last_peered":"2024-05-21T15:15:16.371210+0000","last_clean":"2024-05-21T15:15:16.371210+0000","last_became_active":"2024-05-21T15:14:12.253284+0000","last_became_peered":"2024-05-21T15:14:12.253284+0000","last_unstale":"2024-05-21T15:15:16.371210+0000","last_undegraded":"2024-05-21T15:15:16.371210+0000","last_fullsized":"2024-05-21T15:15:16.371210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:28:07.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.614992+0000","last_change":"2024-05-21T15:14:11.574147+0000","last_active":"2024-05-21T15:15:16.614992+0000","last_peered":"2024-05-21T15:15:16.614992+0000","last_clean":"2024-05-21T15:15:16.614992+0000","last_became_active":"2024-05-21T15:14:11.573721+0000","last_became_peered":"2024-05-21T15:14:11.573721+0000","last_unstale":"2024-05-21T15:15:16.614992+0000","last_undegraded":"2024-05-21T15:15:16.614992+0000","last_fullsized":"2024-05-21T15:15:16.614992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:10.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366391+0000","last_change":"2024-05-21T15:14:12.258632+0000","last_active":"2024-05-21T15:15:16.366391+0000","last_peered":"2024-05-21T15:15:16.366391+0000","last_clean":"2024-05-21T15:15:16.366391+0000","last_became_active":"2024-05-21T15:14:12.257908+0000","last_became_peered":"2024-05-21T15:14:12.257908+0000","last_unstale":"2024-05-21T15:15:16.366391+0000","last_undegraded":"2024-05-21T15:15:16.366391+0000","last_fullsized":"2024-05-21T15:15:16.366391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:12.062387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385475+0000","last_change":"2024-05-21T15:14:11.562592+0000","last_active":"2024-05-21T15:15:16.385475+0000","last_peered":"2024-05-21T15:15:16.385475+0000","last_clean":"2024-05-21T15:15:16.385475+0000","last_became_active":"2024-05-21T15:14:11.562265+0000","last_became_peered":"2024-05-21T15:14:11.562265+0000","last_unstale":"2024-05-21T15:15:16.385475+0000","last_undegraded":"2024-05-21T15:15:16.385475+0000","last_fullsized":"2024-05-21T15:15:16.385475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:24.662576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.367004+0000","last_change":"2024-05-21T15:14:12.249207+0000","last_active":"2024-05-21T15:15:16.367004+0000","last_peered":"2024-05-21T15:15:16.367004+0000","last_clean":"2024-05-21T15:15:16.367004+0000","last_became_active":"2024-05-21T15:14:12.248819+0000","last_became_peered":"2024-05-21T15:14:12.248819+0000","last_unstale":"2024-05-21T15:15:16.367004+0000","last_undegraded":"2024-05-21T15:15:16.367004+0000","last_fullsized":"2024-05-21T15:15:16.367004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:38.208636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370878+0000","last_change":"2024-05-21T15:14:11.572044+0000","last_active":"2024-05-21T15:15:16.370878+0000","last_peered":"2024-05-21T15:15:16.370878+0000","last_clean":"2024-05-21T15:15:16.370878+0000","last_became_active":"2024-05-21T15:14:11.570721+0000","last_became_peered":"2024-05-21T15:14:11.570721+0000","last_unstale":"2024-05-21T15:15:16.370878+0000","last_undegraded":"2024-05-21T15:15:16.370878+0000","last_fullsized":"2024-05-21T15:15:16.370878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:34:08.413781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369772+0000","last_change":"2024-05-21T15:14:12.256327+0000","last_active":"2024-05-21T15:15:16.369772+0000","last_peered":"2024-05-21T15:15:16.369772+0000","last_clean":"2024-05-21T15:15:16.369772+0000","last_became_active":"2024-05-21T15:14:12.255962+0000","last_became_peered":"2024-05-21T15:14:12.255962+0000","last_unstale":"2024-05-21T15:15:16.369772+0000","last_undegraded":"2024-05-21T15:15:16.369772+0000","last_fullsized":"2024-05-21T15:15:16.369772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:05.076323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.386278+0000","last_change":"2024-05-21T15:15:16.385658+0000","last_active":"2024-05-21T15:15:16.386278+0000","last_peered":"2024-05-21T15:15:16.386278+0000","last_clean":"2024-05-21T15:15:16.386278+0000","last_became_active":"2024-05-21T15:15:16.375122+0000","last_became_peered":"2024-05-21T15:15:16.375122+0000","last_unstale":"2024-05-21T15:15:16.386278+0000","last_undegraded":"2024-05-21T15:15:16.386278+0000","last_fullsized":"2024-05-21T15:15:16.386278+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:57:59.599504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371212+0000","last_change":"2024-05-21T15:14:12.256993+0000","last_active":"2024-05-21T15:15:16.371212+0000","last_peered":"2024-05-21T15:15:16.371212+0000","last_clean":"2024-05-21T15:15:16.371212+0000","last_became_active":"2024-05-21T15:14:12.256179+0000","last_became_peered":"2024-05-21T15:14:12.256179+0000","last_unstale":"2024-05-21T15:15:16.371212+0000","last_undegraded":"2024-05-21T15:15:16.371212+0000","last_fullsized":"2024-05-21T15:15:16.371212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:01:09.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.391272+0000","last_change":"2024-05-21T15:15:16.391272+0000","last_active":"2024-05-21T15:15:16.391272+0000","last_peered":"2024-05-21T15:15:16.391272+0000","last_clean":"2024-05-21T15:15:16.391272+0000","last_became_active":"2024-05-21T15:15:16.386921+0000","last_became_peered":"2024-05-21T15:15:16.386921+0000","last_unstale":"2024-05-21T15:15:16.391272+0000","last_undegraded":"2024-05-21T15:15:16.391272+0000","last_fullsized":"2024-05-21T15:15:16.391272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:01:55.466989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370986+0000","last_change":"2024-05-21T15:14:12.260506+0000","last_active":"2024-05-21T15:15:16.370986+0000","last_peered":"2024-05-21T15:15:16.370986+0000","last_clean":"2024-05-21T15:15:16.370986+0000","last_became_active":"2024-05-21T15:14:12.260168+0000","last_became_peered":"2024-05-21T15:14:12.260168+0000","last_unstale":"2024-05-21T15:15:16.370986+0000","last_undegraded":"2024-05-21T15:15:16.370986+0000","last_fullsized":"2024-05-21T15:15:16.370986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:11.321622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:24.408807+0000","last_change":"2024-05-21T15:14:11.568630+0000","last_active":"2024-05-21T15:15:24.408807+0000","last_peered":"2024-05-21T15:15:24.408807+0000","last_clean":"2024-05-21T15:15:24.408807+0000","last_became_active":"2024-05-21T15:14:11.568229+0000","last_became_peered":"2024-05-21T15:14:11.568229+0000","last_unstale":"2024-05-21T15:15:24.408807+0000","last_undegraded":"2024-05-21T15:15:24.408807+0000","last_fullsized":"2024-05-21T15:15:24.408807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:21.372395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371147+0000","last_change":"2024-05-21T15:14:12.259408+0000","last_active":"2024-05-21T15:15:16.371147+0000","last_peered":"2024-05-21T15:15:16.371147+0000","last_clean":"2024-05-21T15:15:16.371147+0000","last_became_active":"2024-05-21T15:14:12.259082+0000","last_became_peered":"2024-05-21T15:14:12.259082+0000","last_unstale":"2024-05-21T15:15:16.371147+0000","last_undegraded":"2024-05-21T15:15:16.371147+0000","last_fullsized":"2024-05-21T15:15:16.371147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:32:09.059307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.372199+0000","last_change":"2024-05-21T15:14:11.571077+0000","last_active":"2024-05-21T15:15:16.372199+0000","last_peered":"2024-05-21T15:15:16.372199+0000","last_clean":"2024-05-21T15:15:16.372199+0000","last_became_active":"2024-05-21T15:14:11.570664+0000","last_became_peered":"2024-05-21T15:14:11.570664+0000","last_unstale":"2024-05-21T15:15:16.372199+0000","last_undegraded":"2024-05-21T15:15:16.372199+0000","last_fullsized":"2024-05-21T15:15:16.372199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:37:16.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370637+0000","last_change":"2024-05-21T15:14:12.252972+0000","last_active":"2024-05-21T15:15:16.370637+0000","last_peered":"2024-05-21T15:15:16.370637+0000","last_clean":"2024-05-21T15:15:16.370637+0000","last_became_active":"2024-05-21T15:14:12.252591+0000","last_became_peered":"2024-05-21T15:14:12.252591+0000","last_unstale":"2024-05-21T15:15:16.370637+0000","last_undegraded":"2024-05-21T15:15:16.370637+0000","last_fullsized":"2024-05-21T15:15:16.370637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:14:16.430371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014320+0000","last_change":"2024-05-21T15:14:11.577138+0000","last_active":"2024-05-21T15:15:17.014320+0000","last_peered":"2024-05-21T15:15:17.014320+0000","last_clean":"2024-05-21T15:15:17.014320+0000","last_became_active":"2024-05-21T15:14:11.576629+0000","last_became_peered":"2024-05-21T15:14:11.576629+0000","last_unstale":"2024-05-21T15:15:17.014320+0000","last_undegraded":"2024-05-21T15:15:17.014320+0000","last_fullsized":"2024-05-21T15:15:17.014320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:20:20.134764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371001+0000","last_change":"2024-05-21T15:14:12.256174+0000","last_active":"2024-05-21T15:15:16.371001+0000","last_peered":"2024-05-21T15:15:16.371001+0000","last_clean":"2024-05-21T15:15:16.371001+0000","last_became_active":"2024-05-21T15:14:12.255839+0000","last_became_peered":"2024-05-21T15:14:12.255839+0000","last_unstale":"2024-05-21T15:15:16.371001+0000","last_undegraded":"2024-05-21T15:15:16.371001+0000","last_fullsized":"2024-05-21T15:15:16.371001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:12:40.743118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369583+0000","last_change":"2024-05-21T15:14:12.257028+0000","last_active":"2024-05-21T15:15:16.369583+0000","last_peered":"2024-05-21T15:15:16.369583+0000","last_clean":"2024-05-21T15:15:16.369583+0000","last_became_active":"2024-05-21T15:14:12.256611+0000","last_became_peered":"2024-05-21T15:14:12.256611+0000","last_unstale":"2024-05-21T15:15:16.369583+0000","last_undegraded":"2024-05-21T15:15:16.369583+0000","last_fullsized":"2024-05-21T15:15:16.369583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:08:43.881073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370885+0000","last_change":"2024-05-21T15:14:12.255724+0000","last_active":"2024-05-21T15:15:16.370885+0000","last_peered":"2024-05-21T15:15:16.370885+0000","last_clean":"2024-05-21T15:15:16.370885+0000","last_became_active":"2024-05-21T15:14:12.255272+0000","last_became_peered":"2024-05-21T15:14:12.255272+0000","last_unstale":"2024-05-21T15:15:16.370885+0000","last_undegraded":"2024-05-21T15:15:16.370885+0000","last_fullsized":"2024-05-21T15:15:16.370885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:07:07.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370522+0000","last_change":"2024-05-21T15:14:12.262508+0000","last_active":"2024-05-21T15:15:16.370522+0000","last_peered":"2024-05-21T15:15:16.370522+0000","last_clean":"2024-05-21T15:15:16.370522+0000","last_became_active":"2024-05-21T15:14:12.262155+0000","last_became_peered":"2024-05-21T15:14:12.262155+0000","last_unstale":"2024-05-21T15:15:16.370522+0000","last_undegraded":"2024-05-21T15:15:16.370522+0000","last_fullsized":"2024-05-21T15:15:16.370522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:54:19.982044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.366130+0000","last_change":"2024-05-21T15:14:12.251715+0000","last_active":"2024-05-21T15:15:16.366130+0000","last_peered":"2024-05-21T15:15:16.366130+0000","last_clean":"2024-05-21T15:15:16.366130+0000","last_became_active":"2024-05-21T15:14:12.251337+0000","last_became_peered":"2024-05-21T15:14:12.251337+0000","last_unstale":"2024-05-21T15:15:16.366130+0000","last_undegraded":"2024-05-21T15:15:16.366130+0000","last_fullsized":"2024-05-21T15:15:16.366130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:03.562179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014242+0000","last_change":"2024-05-21T15:14:12.253166+0000","last_active":"2024-05-21T15:15:17.014242+0000","last_peered":"2024-05-21T15:15:17.014242+0000","last_clean":"2024-05-21T15:15:17.014242+0000","last_became_active":"2024-05-21T15:14:12.251830+0000","last_became_peered":"2024-05-21T15:14:12.251830+0000","last_unstale":"2024-05-21T15:15:17.014242+0000","last_undegraded":"2024-05-21T15:15:17.014242+0000","last_fullsized":"2024-05-21T15:15:17.014242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:11:58.679568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370077+0000","last_change":"2024-05-21T15:14:12.256131+0000","last_active":"2024-05-21T15:15:16.370077+0000","last_peered":"2024-05-21T15:15:16.370077+0000","last_clean":"2024-05-21T15:15:16.370077+0000","last_became_active":"2024-05-21T15:14:12.255772+0000","last_became_peered":"2024-05-21T15:14:12.255772+0000","last_unstale":"2024-05-21T15:15:16.370077+0000","last_undegraded":"2024-05-21T15:15:16.370077+0000","last_fullsized":"2024-05-21T15:15:16.370077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:26:48.047278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.373186+0000","last_change":"2024-05-21T15:14:12.260829+0000","last_active":"2024-05-21T15:15:16.373186+0000","last_peered":"2024-05-21T15:15:16.373186+0000","last_clean":"2024-05-21T15:15:16.373186+0000","last_became_active":"2024-05-21T15:14:12.259686+0000","last_became_peered":"2024-05-21T15:14:12.259686+0000","last_unstale":"2024-05-21T15:15:16.373186+0000","last_undegraded":"2024-05-21T15:15:16.373186+0000","last_fullsized":"2024-05-21T15:15:16.373186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:48.480814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.371069+0000","last_change":"2024-05-21T15:14:12.262892+0000","last_active":"2024-05-21T15:15:16.371069+0000","last_peered":"2024-05-21T15:15:16.371069+0000","last_clean":"2024-05-21T15:15:16.371069+0000","last_became_active":"2024-05-21T15:14:12.262532+0000","last_became_peered":"2024-05-21T15:14:12.262532+0000","last_unstale":"2024-05-21T15:15:16.371069+0000","last_undegraded":"2024-05-21T15:15:16.371069+0000","last_fullsized":"2024-05-21T15:15:16.371069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:22.424783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.376162+0000","last_change":"2024-05-21T15:15:16.375027+0000","last_active":"2024-05-21T15:15:16.376162+0000","last_peered":"2024-05-21T15:15:16.376162+0000","last_clean":"2024-05-21T15:15:16.376162+0000","last_became_active":"2024-05-21T15:15:16.374542+0000","last_became_peered":"2024-05-21T15:15:16.374542+0000","last_unstale":"2024-05-21T15:15:16.376162+0000","last_undegraded":"2024-05-21T15:15:16.376162+0000","last_fullsized":"2024-05-21T15:15:16.376162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:55:14.399237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.370953+0000","last_change":"2024-05-21T15:14:12.259845+0000","last_active":"2024-05-21T15:15:16.370953+0000","last_peered":"2024-05-21T15:15:16.370953+0000","last_clean":"2024-05-21T15:15:16.370953+0000","last_became_active":"2024-05-21T15:14:12.259455+0000","last_became_peered":"2024-05-21T15:14:12.259455+0000","last_unstale":"2024-05-21T15:15:16.370953+0000","last_undegraded":"2024-05-21T15:15:16.370953+0000","last_fullsized":"2024-05-21T15:15:16.370953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:24:59.655178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.385581+0000","last_change":"2024-05-21T15:14:12.259713+0000","last_active":"2024-05-21T15:15:16.385581+0000","last_peered":"2024-05-21T15:15:16.385581+0000","last_clean":"2024-05-21T15:15:16.385581+0000","last_became_active":"2024-05-21T15:14:12.259057+0000","last_became_peered":"2024-05-21T15:14:12.259057+0000","last_unstale":"2024-05-21T15:15:16.385581+0000","last_undegraded":"2024-05-21T15:15:16.385581+0000","last_fullsized":"2024-05-21T15:15:16.385581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:46:22.377652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:17.014185+0000","last_change":"2024-05-21T15:14:12.254389+0000","last_active":"2024-05-21T15:15:17.014185+0000","last_peered":"2024-05-21T15:15:17.014185+0000","last_clean":"2024-05-21T15:15:17.014185+0000","last_became_active":"2024-05-21T15:14:12.253846+0000","last_became_peered":"2024-05-21T15:14:12.253846+0000","last_unstale":"2024-05-21T15:15:17.014185+0000","last_undegraded":"2024-05-21T15:15:17.014185+0000","last_fullsized":"2024-05-21T15:15:17.014185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:32.932316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-21T15:15:16.369609+0000","last_change":"2024-05-21T15:14:12.252292+0000","last_active":"2024-05-21T15:15:16.369609+0000","last_peered":"2024-05-21T15:15:16.369609+0000","last_clean":"2024-05-21T15:15:16.369609+0000","last_became_active":"2024-05-21T15:14:12.251816+0000","last_became_peered":"2024-05-21T15:14:12.251816+0000","last_unstale":"2024-05-21T15:15:16.369609+0000","last_undegraded":"2024-05-21T15:15:16.369609+0000","last_fullsized":"2024-05-21T15:15:16.369609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:59:14.332010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66836,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":694,"num_read_kb":412,"num_write":237,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":672,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1108033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":456911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":456617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1049807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":456628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":517564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":457967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":518175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":456453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":457161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1049485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:15:29.170 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T15:15:29.171 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-21T15:15:29.171 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-21T15:15:29.171 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-21T15:15:29.415 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:30.476 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:30 smithi078 ceph-mon[31997]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:15:30.476 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:30 smithi078 ceph-mon[31997]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 55 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T15:15:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:30 smithi084 ceph-mon[36334]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:15:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:30 smithi084 ceph-mon[36334]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 55 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T15:15:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:30 smithi172 ceph-mon[36402]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:15:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:30 smithi172 ceph-mon[36402]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 55 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T15:15:30.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 -- 172.21.15.78:0/354574248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4107da0 msgr2=0x7ff2b41081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:30.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 --2- 172.21.15.78:0/354574248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4107da0 0x7ff2b41081b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40099c0 tx=0x7ff2a4009cd0 comp rx=0 tx=0).stop 2024-05-21T15:15:30.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 -- 172.21.15.78:0/354574248 shutdown_connections 2024-05-21T15:15:30.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 --2- 172.21.15.78:0/354574248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2b41086f0 0x7ff2b4110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:30.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 --2- 172.21.15.78:0/354574248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4103470 0x7ff2b4103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:30.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 --2- 172.21.15.78:0/354574248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4107da0 0x7ff2b41081b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:30.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 -- 172.21.15.78:0/354574248 >> 172.21.15.78:0/354574248 conn(0x7ff2b40feec0 msgr2=0x7ff2b41012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:30.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 -- 172.21.15.78:0/354574248 shutdown_connections 2024-05-21T15:15:30.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 -- 172.21.15.78:0/354574248 wait complete. 2024-05-21T15:15:30.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.970+0000 7ff2b9813700 1 Processor -- start 2024-05-21T15:15:30.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b9813700 1 -- start start 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b9813700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4103470 0x7ff2b4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b9813700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2b4107da0 0x7ff2b419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b9813700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b41086f0 0x7ff2b419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b9813700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4113470 con 0x7ff2b41086f0 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b9813700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41132f0 con 0x7ff2b4103470 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b9813700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41135f0 con 0x7ff2b4107da0 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b41086f0 0x7ff2b419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b41086f0 0x7ff2b419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39858/0 (socket says 172.21.15.78:39858) 2024-05-21T15:15:30.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 -- 172.21.15.78:0/743133268 learned_addr learned my addr 172.21.15.78:0/743133268 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b2ffd700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4103470 0x7ff2b4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b27fc700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2b4107da0 0x7ff2b419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 -- 172.21.15.78:0/743133268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2b4107da0 msgr2=0x7ff2b419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2b4107da0 0x7ff2b419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 -- 172.21.15.78:0/743133268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4103470 msgr2=0x7ff2b4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4103470 0x7ff2b4199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 -- 172.21.15.78:0/743133268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b419ec90 con 0x7ff2b41086f0 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b2ffd700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4103470 0x7ff2b4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.971+0000 7ff2b37fe700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b41086f0 0x7ff2b419e5f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00ada0 tx=0x7ff2ac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.972+0000 7ff2b8811700 1 -- 172.21.15.78:0/743133268 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac017020 con 0x7ff2b41086f0 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.972+0000 7ff2b8811700 1 -- 172.21.15.78:0/743133268 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac004bb0 con 0x7ff2b41086f0 2024-05-21T15:15:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.972+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b419ef80 con 0x7ff2b41086f0 2024-05-21T15:15:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.972+0000 7ff2b8811700 1 -- 172.21.15.78:0/743133268 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac005520 con 0x7ff2b41086f0 2024-05-21T15:15:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.972+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41a68a0 con 0x7ff2b41086f0 2024-05-21T15:15:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.973+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b4050170 con 0x7ff2b41086f0 2024-05-21T15:15:30.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.975+0000 7ff2b8811700 1 -- 172.21.15.78:0/743133268 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff2ac0056c0 con 0x7ff2b41086f0 2024-05-21T15:15:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.975+0000 7ff2b8811700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff2a00a2d30 0x7ff2a00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.976+0000 7ff2b2ffd700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff2a00a2d30 0x7ff2a00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.976+0000 7ff2b8811700 1 -- 172.21.15.78:0/743133268 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff2ac0cee60 con 0x7ff2b41086f0 2024-05-21T15:15:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.976+0000 7ff2b2ffd700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff2a00a2d30 0x7ff2a00a5240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00bc10 tx=0x7ff29c00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:30.979+0000 7ff2b8811700 1 -- 172.21.15.78:0/743133268 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff2ac021020 con 0x7ff2b41086f0 2024-05-21T15:15:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.221+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff2b402dc60 con 0x7ff2b41086f0 2024-05-21T15:15:31.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.221+0000 7ff2b8811700 1 -- 172.21.15.78:0/743133268 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff2ac09bcd0 con 0x7ff2b41086f0 2024-05-21T15:15:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:31.225 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff2a00a2d30 msgr2=0x7ff2a00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff2a00a2d30 0x7ff2a00a5240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00bc10 tx=0x7ff29c00b5f0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b41086f0 msgr2=0x7ff2b419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b41086f0 0x7ff2b419e5f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00ada0 tx=0x7ff2ac00c330 comp rx=0 tx=0).stop 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 shutdown_connections 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff2a00a2d30 0x7ff2a00a5240 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b41086f0 0x7ff2b419e5f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2b4103470 0x7ff2b4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 --2- 172.21.15.78:0/743133268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2b4107da0 0x7ff2b419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 >> 172.21.15.78:0/743133268 conn(0x7ff2b40feec0 msgr2=0x7ff2b41074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 shutdown_connections 2024-05-21T15:15:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.224+0000 7ff2b9813700 1 -- 172.21.15.78:0/743133268 wait complete. 2024-05-21T15:15:31.601 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:31 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/743133268' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T15:15:31.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:31 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/743133268' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T15:15:31.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:31 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/743133268' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T15:15:31.845 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-21T15:15:31.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:15:31.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 -- 172.21.15.78:0/1405349971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 msgr2=0x7f34e8102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:31.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 --2- 172.21.15.78:0/1405349971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 0x7f34e8102fc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009900 tx=0x7f34dc02a410 comp rx=0 tx=0).stop 2024-05-21T15:15:31.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 -- 172.21.15.78:0/1405349971 shutdown_connections 2024-05-21T15:15:31.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 --2- 172.21.15.78:0/1405349971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34e81078b0 0x7f34e8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 --2- 172.21.15.78:0/1405349971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 0x7f34e8102fc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 --2- 172.21.15.78:0/1405349971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e8101970 0x7f34e8102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 -- 172.21.15.78:0/1405349971 >> 172.21.15.78:0/1405349971 conn(0x7f34e80fcb80 msgr2=0x7f34e80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:31.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 -- 172.21.15.78:0/1405349971 shutdown_connections 2024-05-21T15:15:31.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.958+0000 7f34f0667640 1 -- 172.21.15.78:0/1405349971 wait complete. 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34f0667640 1 Processor -- start 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34f0667640 1 -- start start 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34f0667640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e8101970 0x7f34e8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34f0667640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 0x7f34e81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34f0667640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34e81078b0 0x7f34e81a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34f0667640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34e810e750 con 0x7f34e8102bb0 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34f0667640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f34e810e5d0 con 0x7f34e8101970 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34f0667640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34e810e8d0 con 0x7f34e81078b0 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34edc02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 0x7f34e81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:31.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34edc02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 0x7f34e81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39884/0 (socket says 172.21.15.78:39884) 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34edc02640 1 -- 172.21.15.78:0/789027665 learned_addr learned my addr 172.21.15.78:0/789027665 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34eec04640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34e81078b0 0x7f34e81a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34ee403640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e8101970 0x7f34e8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34edc02640 1 -- 172.21.15.78:0/789027665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34e81078b0 msgr2=0x7f34e81a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34edc02640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34e81078b0 0x7f34e81a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34edc02640 1 -- 172.21.15.78:0/789027665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e8101970 msgr2=0x7f34e8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34edc02640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e8101970 0x7f34e8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.959+0000 7f34edc02640 1 -- 172.21.15.78:0/789027665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e81a0dc0 con 0x7f34e8102bb0 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.960+0000 7f34edc02640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 0x7f34e81997c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f34dc031440 tx=0x7f34dc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.960+0000 7f34e77fe640 1 -- 172.21.15.78:0/789027665 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc03f030 con 0x7f34e8102bb0 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.960+0000 7f34e77fe640 1 -- 172.21.15.78:0/789027665 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34dc003ce0 con 0x7f34e8102bb0 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.960+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e81a1050 con 0x7f34e8102bb0 2024-05-21T15:15:31.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.960+0000 7f34e77fe640 1 -- 172.21.15.78:0/789027665 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc035070 con 0x7f34e8102bb0 2024-05-21T15:15:31.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.960+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e81a1540 con 0x7f34e8102bb0 2024-05-21T15:15:31.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.961+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e81066c0 con 0x7f34e8102bb0 2024-05-21T15:15:31.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.962+0000 7f34e77fe640 1 -- 172.21.15.78:0/789027665 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f34dc042030 con 0x7f34e8102bb0 2024-05-21T15:15:31.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.963+0000 7f34e77fe640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f34d00a2c20 0x7f34d00a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:31.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.963+0000 7f34ee403640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f34d00a2c20 0x7f34d00a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.963+0000 7f34e77fe640 1 -- 172.21.15.78:0/789027665 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f34dc0efa90 con 0x7f34e8102bb0 2024-05-21T15:15:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.963+0000 7f34ee403640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f34d00a2c20 0x7f34d00a5130 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34e8102570 tx=0x7f34d8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:31.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:31.967+0000 7f34e77fe640 1 -- 172.21.15.78:0/789027665 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f34dc0bc870 con 0x7f34e8102bb0 2024-05-21T15:15:32.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.144+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34e8062750 con 0x7f34e8102bb0 2024-05-21T15:15:32.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.145+0000 7f34e77fe640 1 -- 172.21.15.78:0/789027665 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22616 (secure 0 0 0) 0x7f34dc0bc170 con 0x7f34e8102bb0 2024-05-21T15:15:32.149 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:32.149 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":87,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:15:16.363058+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f34d00a2c20 msgr2=0x7f34d00a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f34d00a2c20 0x7f34d00a5130 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34e8102570 tx=0x7f34d8009500 comp rx=0 tx=0).stop 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 msgr2=0x7f34e81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 0x7f34e81997c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f34dc031440 tx=0x7f34dc032ce0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 shutdown_connections 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f34d00a2c20 0x7f34d00a5130 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34e81078b0 0x7f34e81a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34e8102bb0 0x7f34e81997c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 --2- 172.21.15.78:0/789027665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f34e8101970 0x7f34e8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 >> 172.21.15.78:0/789027665 conn(0x7f34e80fcb80 msgr2=0x7f34e8103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:32.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 shutdown_connections 2024-05-21T15:15:32.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.148+0000 7f34f0667640 1 -- 172.21.15.78:0/789027665 wait complete. 2024-05-21T15:15:32.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:15:32.163 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:15:32.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 -- 172.21.15.78:0/798949948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 msgr2=0x7f1518104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 --2- 172.21.15.78:0/798949948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 0x7f1518104460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1508009900 tx=0x7f150802a410 comp rx=0 tx=0).stop 2024-05-21T15:15:32.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 -- 172.21.15.78:0/798949948 shutdown_connections 2024-05-21T15:15:32.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 --2- 172.21.15.78:0/798949948 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1518101200 0x7f151810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 --2- 172.21.15.78:0/798949948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1518100700 0x7f1518100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 --2- 172.21.15.78:0/798949948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 0x7f1518104460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 -- 172.21.15.78:0/798949948 >> 172.21.15.78:0/798949948 conn(0x7f15180fc300 msgr2=0x7f15180fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:32.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 -- 172.21.15.78:0/798949948 shutdown_connections 2024-05-21T15:15:32.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.274+0000 7f151d95e640 1 -- 172.21.15.78:0/798949948 wait complete. 2024-05-21T15:15:32.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f151d95e640 1 Processor -- start 2024-05-21T15:15:32.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f151d95e640 1 -- start start 2024-05-21T15:15:32.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f151d95e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1518100700 0x7f15181992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f151d95e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1518101200 0x7f1518199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f151d95e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 0x7f15181a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f151d95e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f151810e710 con 0x7f1518104050 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f151d95e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f151810e590 con 0x7f1518101200 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f151d95e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f151810e890 con 0x7f1518100700 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f15177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 0x7f15181a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f15177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 0x7f15181a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39910/0 (socket says 172.21.15.78:39910) 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.275+0000 7f15177fe640 1 -- 172.21.15.78:0/3456222638 learned_addr learned my addr 172.21.15.78:0/3456222638 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f15167fc640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1518101200 0x7f1518199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f1516ffd640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1518100700 0x7f15181992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f15177fe640 1 -- 172.21.15.78:0/3456222638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1518100700 msgr2=0x7f15181992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f15177fe640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1518100700 0x7f15181992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f15177fe640 1 -- 172.21.15.78:0/3456222638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1518101200 msgr2=0x7f1518199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f15177fe640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1518101200 0x7f1518199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f15177fe640 1 -- 172.21.15.78:0/3456222638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15181a0e00 con 0x7f1518104050 2024-05-21T15:15:32.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f15167fc640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1518101200 0x7f1518199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:32.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f15177fe640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 0x7f15181a0790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f151000eb10 tx=0x7f151000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:32.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f151c95c640 1 -- 172.21.15.78:0/3456222638 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510004b20 con 0x7f1518104050 2024-05-21T15:15:32.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f151c95c640 1 -- 172.21.15.78:0/3456222638 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1510004cc0 con 0x7f1518104050 2024-05-21T15:15:32.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f151c95c640 1 -- 172.21.15.78:0/3456222638 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510010760 con 0x7f1518104050 2024-05-21T15:15:32.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15181a10f0 con 0x7f1518104050 2024-05-21T15:15:32.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.276+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15181a15f0 con 0x7f1518104050 2024-05-21T15:15:32.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.277+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15181079e0 con 0x7f1518104050 2024-05-21T15:15:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.278+0000 7f151c95c640 1 -- 172.21.15.78:0/3456222638 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1510005020 con 0x7f1518104050 2024-05-21T15:15:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.279+0000 7f151c95c640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15000a2c20 0x7f15000a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.279+0000 7f1516ffd640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15000a2c20 0x7f15000a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.279+0000 7f151c95c640 1 -- 172.21.15.78:0/3456222638 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f15100cf8d0 con 0x7f1518104050 2024-05-21T15:15:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.280+0000 7f1516ffd640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15000a2c20 0x7f15000a5130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1508031610 tx=0x7f1508000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.284+0000 7f151c95c640 1 -- 172.21.15.78:0/3456222638 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f151009c6b0 con 0x7f1518104050 2024-05-21T15:15:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.461+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1518062750 con 0x7f1518104050 2024-05-21T15:15:32.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.462+0000 7f151c95c640 1 -- 172.21.15.78:0/3456222638 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f151009bfb0 con 0x7f1518104050 2024-05-21T15:15:32.465 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15000a2c20 msgr2=0x7f15000a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15000a2c20 0x7f15000a5130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1508031610 tx=0x7f1508000bc0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 msgr2=0x7f15181a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 0x7f15181a0790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f151000eb10 tx=0x7f151000c2d0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 shutdown_connections 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f15000a2c20 0x7f15000a5130 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1518101200 0x7f1518199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1518104050 0x7f15181a0790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 --2- 172.21.15.78:0/3456222638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1518100700 0x7f15181992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 >> 172.21.15.78:0/3456222638 conn(0x7f15180fc300 msgr2=0x7f1518104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 shutdown_connections 2024-05-21T15:15:32.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.464+0000 7f151d95e640 1 -- 172.21.15.78:0/3456222638 wait complete. 2024-05-21T15:15:32.477 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:32 smithi078 ceph-mon[31997]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s; 55 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T15:15:32.477 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:32 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/789027665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:32.478 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:15:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 -- 172.21.15.78:0/503200274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0201019b0 msgr2=0x7fe0201026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 --2- 172.21.15.78:0/503200274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0201019b0 0x7fe0201026b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe010009900 tx=0x7fe01002a410 comp rx=0 tx=0).stop 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 -- 172.21.15.78:0/503200274 shutdown_connections 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 --2- 172.21.15.78:0/503200274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe020102bf0 0x7fe020103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 --2- 172.21.15.78:0/503200274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0201078f0 0x7fe020109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 --2- 172.21.15.78:0/503200274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0201019b0 0x7fe0201026b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 -- 172.21.15.78:0/503200274 >> 172.21.15.78:0/503200274 conn(0x7fe0200fcba0 msgr2=0x7fe0200ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 -- 172.21.15.78:0/503200274 shutdown_connections 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.590+0000 7fe02634c640 1 -- 172.21.15.78:0/503200274 wait complete. 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe02634c640 1 Processor -- start 2024-05-21T15:15:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe02634c640 1 -- start start 2024-05-21T15:15:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe02634c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0201019b0 0x7fe020074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe02634c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe020102bf0 0x7fe02019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe02634c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0201078f0 0x7fe0201a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe02634c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe02010d600 con 0x7fe020102bf0 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe02634c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe02010d480 con 0x7fe0201019b0 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe02634c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe02010d780 con 0x7fe0201078f0 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe020102bf0 0x7fe02019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe020102bf0 0x7fe02019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39926/0 (socket says 172.21.15.78:39926) 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe01f7fe640 1 -- 172.21.15.78:0/3025361390 learned_addr learned my addr 172.21.15.78:0/3025361390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.591+0000 7fe01ffff640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0201019b0 0x7fe020074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe0248e9640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0201078f0 0x7fe0201a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01f7fe640 1 -- 172.21.15.78:0/3025361390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0201078f0 msgr2=0x7fe0201a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01f7fe640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0201078f0 0x7fe0201a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01f7fe640 1 -- 172.21.15.78:0/3025361390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0201019b0 msgr2=0x7fe020074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01f7fe640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0201019b0 0x7fe020074810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01f7fe640 1 -- 172.21.15.78:0/3025361390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0201a0df0 con 0x7fe020102bf0 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01f7fe640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe020102bf0 0x7fe02019e060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe01400c890 tx=0x7fe01400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01d7fa640 1 -- 172.21.15.78:0/3025361390 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014004220 con 0x7fe020102bf0 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01d7fa640 1 -- 172.21.15.78:0/3025361390 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0140043c0 con 0x7fe020102bf0 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0201a10e0 con 0x7fe020102bf0 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe01d7fa640 1 -- 172.21.15.78:0/3025361390 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140049e0 con 0x7fe020102bf0 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.592+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0201a1630 con 0x7fe020102bf0 2024-05-21T15:15:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.593+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe020106700 con 0x7fe020102bf0 2024-05-21T15:15:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.595+0000 7fe01d7fa640 1 -- 172.21.15.78:0/3025361390 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe0140079b0 con 0x7fe020102bf0 2024-05-21T15:15:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.595+0000 7fe01d7fa640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe0080a2c20 0x7fe0080a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.595+0000 7fe01ffff640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe0080a2c20 0x7fe0080a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.595+0000 7fe01d7fa640 1 -- 172.21.15.78:0/3025361390 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe0140cf280 con 0x7fe020102bf0 2024-05-21T15:15:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.596+0000 7fe01ffff640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe0080a2c20 0x7fe0080a5130 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe010005ab0 tx=0x7fe010031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.599+0000 7fe01d7fa640 1 -- 172.21.15.78:0/3025361390 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe014014030 con 0x7fe020102bf0 2024-05-21T15:15:32.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:32 smithi084 ceph-mon[36334]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s; 55 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T15:15:32.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:32 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/789027665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:32 smithi172 ceph-mon[36402]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s; 55 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T15:15:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:32 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/789027665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.776+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe020062750 con 0x7fe020102bf0 2024-05-21T15:15:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.776+0000 7fe01d7fa640 1 -- 172.21.15.78:0/3025361390 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fe01409c060 con 0x7fe020102bf0 2024-05-21T15:15:32.779 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe0080a2c20 msgr2=0x7fe0080a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe0080a2c20 0x7fe0080a5130 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe010005ab0 tx=0x7fe010031400 comp rx=0 tx=0).stop 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe020102bf0 msgr2=0x7fe02019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe020102bf0 0x7fe02019e060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe01400c890 tx=0x7fe01400cd50 comp rx=0 tx=0).stop 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 shutdown_connections 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe0080a2c20 0x7fe0080a5130 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0201078f0 0x7fe0201a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe020102bf0 0x7fe02019e060 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 --2- 172.21.15.78:0/3025361390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0201019b0 0x7fe020074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 >> 172.21.15.78:0/3025361390 conn(0x7fe0200fcba0 msgr2=0x7fe0201038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 shutdown_connections 2024-05-21T15:15:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.778+0000 7fe02634c640 1 -- 172.21.15.78:0/3025361390 wait complete. 2024-05-21T15:15:32.793 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:15:32.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/2456733628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4103590 msgr2=0x7f6bd41059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/2456733628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4103590 0x7f6bd41059b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009900 tx=0x7f6bc402a410 comp rx=0 tx=0).stop 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/2456733628 shutdown_connections 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/2456733628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd4108850 0x7f6bd410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/2456733628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bd4105ef0 0x7f6bd4108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/2456733628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4103590 0x7f6bd41059b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/2456733628 >> 172.21.15.78:0/2456733628 conn(0x7f6bd40fcb70 msgr2=0x7f6bd40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/2456733628 shutdown_connections 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/2456733628 wait complete. 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 Processor -- start 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.905+0000 7f6bdb6ca640 1 -- start start 2024-05-21T15:15:32.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bdb6ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd4103590 0x7f6bd41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bdb6ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bd4105ef0 0x7f6bd41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bdb6ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4108850 0x7f6bd41a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bdb6ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd410e6d0 con 0x7f6bd4108850 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bdb6ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd410e550 con 0x7f6bd4103590 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bdb6ca640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd410e850 con 0x7f6bd4105ef0 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4108850 0x7f6bd41a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4108850 0x7f6bd41a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39952/0 (socket says 172.21.15.78:39952) 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 -- 172.21.15.78:0/950535862 learned_addr learned my addr 172.21.15.78:0/950535862 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd8c65640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bd4105ef0 0x7f6bd41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9466640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd4103590 0x7f6bd41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 -- 172.21.15.78:0/950535862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bd4105ef0 msgr2=0x7f6bd41997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bd4105ef0 0x7f6bd41997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 -- 172.21.15.78:0/950535862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd4103590 msgr2=0x7f6bd41992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd4103590 0x7f6bd41992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 -- 172.21.15.78:0/950535862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd41a0df0 con 0x7f6bd4108850 2024-05-21T15:15:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd9c67640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4108850 0x7f6bd41a0780 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00dac0 tx=0x7f6bcc00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd27fc640 1 -- 172.21.15.78:0/950535862 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc017070 con 0x7f6bd4108850 2024-05-21T15:15:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd41a10e0 con 0x7f6bd4108850 2024-05-21T15:15:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd27fc640 1 -- 172.21.15.78:0/950535862 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc00be30 con 0x7f6bd4108850 2024-05-21T15:15:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bd27fc640 1 -- 172.21.15.78:0/950535862 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc00f890 con 0x7f6bd4108850 2024-05-21T15:15:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.906+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd41a15e0 con 0x7f6bd4108850 2024-05-21T15:15:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.907+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd40fe690 con 0x7f6bd4108850 2024-05-21T15:15:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.909+0000 7f6bd27fc640 1 -- 172.21.15.78:0/950535862 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6bcc004a40 con 0x7f6bd4108850 2024-05-21T15:15:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.909+0000 7f6bd27fc640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6bbc0a2c20 0x7f6bbc0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.910+0000 7f6bd9466640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6bbc0a2c20 0x7f6bbc0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.910+0000 7f6bd27fc640 1 -- 172.21.15.78:0/950535862 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6bcc0cf1f0 con 0x7f6bd4108850 2024-05-21T15:15:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.910+0000 7f6bd9466640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6bbc0a2c20 0x7f6bbc0a5130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4000c00 tx=0x7f6bc4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:32.913+0000 7f6bd27fc640 1 -- 172.21.15.78:0/950535862 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bcc09bfd0 con 0x7f6bd4108850 2024-05-21T15:15:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.085+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6bd4062750 con 0x7f6bd4108850 2024-05-21T15:15:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.086+0000 7f6bd27fc640 1 -- 172.21.15.78:0/950535862 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f6bcc09b8d0 con 0x7f6bd4108850 2024-05-21T15:15:33.089 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6bbc0a2c20 msgr2=0x7f6bbc0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6bbc0a2c20 0x7f6bbc0a5130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4000c00 tx=0x7f6bc4031400 comp rx=0 tx=0).stop 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4108850 msgr2=0x7f6bd41a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4108850 0x7f6bd41a0780 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00dac0 tx=0x7f6bcc00df80 comp rx=0 tx=0).stop 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 shutdown_connections 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6bbc0a2c20 0x7f6bbc0a5130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bd4108850 0x7f6bd41a0780 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bd4105ef0 0x7f6bd41997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 --2- 172.21.15.78:0/950535862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bd4103590 0x7f6bd41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 >> 172.21.15.78:0/950535862 conn(0x7f6bd40fcb70 msgr2=0x7f6bd4104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 shutdown_connections 2024-05-21T15:15:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.088+0000 7f6bdb6ca640 1 -- 172.21.15.78:0/950535862 wait complete. 2024-05-21T15:15:33.103 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:15:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 -- 172.21.15.78:0/1387641605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd028103cf0 msgr2=0x7fd028104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/1387641605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd028103cf0 0x7fd028104100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd010009900 tx=0x7fd01002a410 comp rx=0 tx=0).stop 2024-05-21T15:15:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 -- 172.21.15.78:0/1387641605 shutdown_connections 2024-05-21T15:15:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/1387641605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0281089f0 0x7fd02810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/1387641605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd028103cf0 0x7fd028104100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/1387641605 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd028102ab0 0x7fd0281037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 -- 172.21.15.78:0/1387641605 >> 172.21.15.78:0/1387641605 conn(0x7fd0280fdcc0 msgr2=0x7fd028100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 -- 172.21.15.78:0/1387641605 shutdown_connections 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.210+0000 7fd02d5ce640 1 -- 172.21.15.78:0/1387641605 wait complete. 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd02d5ce640 1 Processor -- start 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd02d5ce640 1 -- start start 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd02d5ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd028102ab0 0x7fd0281992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd02d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd028103cf0 0x7fd0281997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd02d5ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0281089f0 0x7fd0281a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd02d5ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd02810e730 con 0x7fd0281089f0 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd02d5ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd02810e5b0 con 0x7fd028103cf0 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd02d5ce640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd02810e8b0 con 0x7fd028102ab0 2024-05-21T15:15:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd0277fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0281089f0 0x7fd0281a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd0277fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0281089f0 0x7fd0281a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39964/0 (socket says 172.21.15.78:39964) 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.211+0000 7fd0277fe640 1 -- 172.21.15.78:0/4039999377 learned_addr learned my addr 172.21.15.78:0/4039999377 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd0267fc640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd028103cf0 0x7fd0281997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd026ffd640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd028102ab0 0x7fd0281992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd0277fe640 1 -- 172.21.15.78:0/4039999377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd028102ab0 msgr2=0x7fd0281992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd0277fe640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd028102ab0 0x7fd0281992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd0277fe640 1 -- 172.21.15.78:0/4039999377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd028103cf0 msgr2=0x7fd0281997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd0277fe640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd028103cf0 0x7fd0281997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd0277fe640 1 -- 172.21.15.78:0/4039999377 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0281a0de0 con 0x7fd0281089f0 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd0277fe640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0281089f0 0x7fd0281a0770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd02000d750 tx=0x7fd02000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd01ffff640 1 -- 172.21.15.78:0/4039999377 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020017070 con 0x7fd0281089f0 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0281a10d0 con 0x7fd0281089f0 2024-05-21T15:15:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd01ffff640 1 -- 172.21.15.78:0/4039999377 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd020005500 con 0x7fd0281089f0 2024-05-21T15:15:33.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd01ffff640 1 -- 172.21.15.78:0/4039999377 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020004d90 con 0x7fd0281089f0 2024-05-21T15:15:33.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.212+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0281a1620 con 0x7fd0281089f0 2024-05-21T15:15:33.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.213+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd028107800 con 0x7fd0281089f0 2024-05-21T15:15:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.214+0000 7fd01ffff640 1 -- 172.21.15.78:0/4039999377 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd020004f30 con 0x7fd0281089f0 2024-05-21T15:15:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.215+0000 7fd01ffff640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd0140a2c20 0x7fd0140a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.215+0000 7fd026ffd640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd0140a2c20 0x7fd0140a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.215+0000 7fd01ffff640 1 -- 172.21.15.78:0/4039999377 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd0200ceb10 con 0x7fd0281089f0 2024-05-21T15:15:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.216+0000 7fd026ffd640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd0140a2c20 0x7fd0140a5130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd0281036b0 tx=0x7fd01800a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:33.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.220+0000 7fd01ffff640 1 -- 172.21.15.78:0/4039999377 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd02009b2a0 con 0x7fd0281089f0 2024-05-21T15:15:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.437+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd028062750 con 0x7fd0281089f0 2024-05-21T15:15:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.438+0000 7fd01ffff640 1 -- 172.21.15.78:0/4039999377 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11925 (secure 0 0 0) 0x7fd02009f5a0 con 0x7fd0281089f0 2024-05-21T15:15:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:33.442 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:14:18.291660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:15:33.442 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-05-21T15:15:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd0140a2c20 msgr2=0x7fd0140a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd0140a2c20 0x7fd0140a5130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd0281036b0 tx=0x7fd01800a590 comp rx=0 tx=0).stop 2024-05-21T15:15:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0281089f0 msgr2=0x7fd0281a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0281089f0 0x7fd0281a0770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd02000d750 tx=0x7fd02000dc10 comp rx=0 tx=0).stop 2024-05-21T15:15:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 shutdown_connections 2024-05-21T15:15:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd0140a2c20 0x7fd0140a5130 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd028103cf0 0x7fd0281997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0281089f0 0x7fd0281a0770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 --2- 172.21.15.78:0/4039999377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd028102ab0 0x7fd0281992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 >> 172.21.15.78:0/4039999377 conn(0x7fd0280fdcc0 msgr2=0x7fd0281049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 shutdown_connections 2024-05-21T15:15:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.440+0000 7fd02d5ce640 1 -- 172.21.15.78:0/4039999377 wait complete. 2024-05-21T15:15:33.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:33 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3456222638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:33 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3025361390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:33 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/950535862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:15:33.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 -- 172.21.15.78:0/3827441490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a70101990 msgr2=0x7f6a70102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 --2- 172.21.15.78:0/3827441490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a70101990 0x7f6a70102690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009900 tx=0x7f6a6002a410 comp rx=0 tx=0).stop 2024-05-21T15:15:33.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 -- 172.21.15.78:0/3827441490 shutdown_connections 2024-05-21T15:15:33.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 --2- 172.21.15.78:0/3827441490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a701078d0 0x7f6a70109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 --2- 172.21.15.78:0/3827441490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a70102bd0 0x7f6a70102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 --2- 172.21.15.78:0/3827441490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a70101990 0x7f6a70102690 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 -- 172.21.15.78:0/3827441490 >> 172.21.15.78:0/3827441490 conn(0x7f6a700fcb80 msgr2=0x7f6a700ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:33.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 -- 172.21.15.78:0/3827441490 shutdown_connections 2024-05-21T15:15:33.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.563+0000 7f6a749ea640 1 -- 172.21.15.78:0/3827441490 wait complete. 2024-05-21T15:15:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a749ea640 1 Processor -- start 2024-05-21T15:15:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a749ea640 1 -- start start 2024-05-21T15:15:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a749ea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a70101990 0x7f6a70198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a749ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a70102bd0 0x7f6a70198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a749ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a701078d0 0x7f6a7019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a749ea640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010d5e0 con 0x7f6a701078d0 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a749ea640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010d460 con 0x7f6a70102bd0 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a749ea640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010d760 con 0x7f6a70101990 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a701078d0 0x7f6a7019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a701078d0 0x7f6a7019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39994/0 (socket says 172.21.15.78:39994) 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6ed9d640 1 -- 172.21.15.78:0/4176535279 learned_addr learned my addr 172.21.15.78:0/4176535279 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6dd9b640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a70102bd0 0x7f6a70198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6e59c640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a70101990 0x7f6a70198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6ed9d640 1 -- 172.21.15.78:0/4176535279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a70101990 msgr2=0x7f6a70198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6ed9d640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a70101990 0x7f6a70198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6ed9d640 1 -- 172.21.15.78:0/4176535279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a70102bd0 msgr2=0x7f6a70198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6ed9d640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a70102bd0 0x7f6a70198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6ed9d640 1 -- 172.21.15.78:0/4176535279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7019fcd0 con 0x7f6a701078d0 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.564+0000 7f6a6e59c640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a70101990 0x7f6a70198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.565+0000 7f6a6ed9d640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a701078d0 0x7f6a7019f710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800cce0 tx=0x7f6a6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.565+0000 7f6a677fe640 1 -- 172.21.15.78:0/4176535279 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6800f450 con 0x7f6a701078d0 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.565+0000 7f6a677fe640 1 -- 172.21.15.78:0/4176535279 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a6800fad0 con 0x7f6a701078d0 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.565+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7019ffc0 con 0x7f6a701078d0 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.565+0000 7f6a677fe640 1 -- 172.21.15.78:0/4176535279 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68018c10 con 0x7f6a701078d0 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.565+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a701a0420 con 0x7f6a701078d0 2024-05-21T15:15:33.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.566+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a70069140 con 0x7f6a701078d0 2024-05-21T15:15:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.567+0000 7f6a677fe640 1 -- 172.21.15.78:0/4176535279 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6a6800f5f0 con 0x7f6a701078d0 2024-05-21T15:15:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.568+0000 7f6a677fe640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6a5c0a2ca0 0x7f6a5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.568+0000 7f6a6e59c640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6a5c0a2ca0 0x7f6a5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.568+0000 7f6a677fe640 1 -- 172.21.15.78:0/4176535279 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6a680cf8e0 con 0x7f6a701078d0 2024-05-21T15:15:33.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.568+0000 7f6a6e59c640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6a5c0a2ca0 0x7f6a5c0a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a60000c00 tx=0x7f6a60032c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:33.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.572+0000 7f6a677fe640 1 -- 172.21.15.78:0/4176535279 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a68014030 con 0x7f6a701078d0 2024-05-21T15:15:33.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:33 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3456222638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:33 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3025361390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:33 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/950535862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:33 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3456222638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:33 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3025361390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:33 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/950535862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:33.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.748+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a7002d2a0 con 0x7f6a701078d0 2024-05-21T15:15:33.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.749+0000 7f6a677fe640 1 -- 172.21.15.78:0/4176535279 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22616 (secure 0 0 0) 0x7f6a6809c6c0 con 0x7f6a701078d0 2024-05-21T15:15:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:33.753 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":87,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:15:16.363058+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:15:33.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6a5c0a2ca0 msgr2=0x7f6a5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6a5c0a2ca0 0x7f6a5c0a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a60000c00 tx=0x7f6a60032c30 comp rx=0 tx=0).stop 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a701078d0 msgr2=0x7f6a7019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a701078d0 0x7f6a7019f710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800cce0 tx=0x7f6a6800a620 comp rx=0 tx=0).stop 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 shutdown_connections 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f6a5c0a2ca0 0x7f6a5c0a51b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a70102bd0 0x7f6a70198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a701078d0 0x7f6a7019f710 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 --2- 172.21.15.78:0/4176535279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a70101990 0x7f6a70198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 >> 172.21.15.78:0/4176535279 conn(0x7f6a700fcb80 msgr2=0x7f6a701038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 shutdown_connections 2024-05-21T15:15:33.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.751+0000 7f6a749ea640 1 -- 172.21.15.78:0/4176535279 wait complete. 2024-05-21T15:15:33.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:15:33.767 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:15:33.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 -- 172.21.15.78:0/546837257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8102aa0 msgr2=0x7fafb81037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 --2- 172.21.15.78:0/546837257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8102aa0 0x7fafb81037a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fafac009900 tx=0x7fafac02a410 comp rx=0 tx=0).stop 2024-05-21T15:15:33.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 -- 172.21.15.78:0/546837257 shutdown_connections 2024-05-21T15:15:33.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 --2- 172.21.15.78:0/546837257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb81089e0 0x7fafb810ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 --2- 172.21.15.78:0/546837257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafb8103ce0 0x7fafb81040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 --2- 172.21.15.78:0/546837257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8102aa0 0x7fafb81037a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 -- 172.21.15.78:0/546837257 >> 172.21.15.78:0/546837257 conn(0x7fafb80fdc90 msgr2=0x7fafb8100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:33.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 -- 172.21.15.78:0/546837257 shutdown_connections 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.878+0000 7fafbeee2640 1 -- 172.21.15.78:0/546837257 wait complete. 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafbeee2640 1 Processor -- start 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafbeee2640 1 -- start start 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafbeee2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb8102aa0 0x7fafb8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafbeee2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8103ce0 0x7fafb8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafbeee2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafb81089e0 0x7fafb81a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafbeee2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810e6d0 con 0x7fafb8103ce0 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafbeee2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810e550 con 0x7fafb8102aa0 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafbeee2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fafb810e850 con 0x7fafb81089e0 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8103ce0 0x7fafb8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8103ce0 0x7fafb8199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40004/0 (socket says 172.21.15.78:40004) 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafb7fff640 1 -- 172.21.15.78:0/397839987 learned_addr learned my addr 172.21.15.78:0/397839987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:33.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafb7fff640 1 -- 172.21.15.78:0/397839987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafb81089e0 msgr2=0x7fafb81a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafb7fff640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafb81089e0 0x7fafb81a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafb7fff640 1 -- 172.21.15.78:0/397839987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb8102aa0 msgr2=0x7fafb8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafb7fff640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb8102aa0 0x7fafb8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.879+0000 7fafb7fff640 1 -- 172.21.15.78:0/397839987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb81a0df0 con 0x7fafb8103ce0 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.880+0000 7fafb7fff640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8103ce0 0x7fafb8199860 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fafa400c890 tx=0x7fafa400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.880+0000 7fafb5ffb640 1 -- 172.21.15.78:0/397839987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004220 con 0x7fafb8103ce0 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.880+0000 7fafb5ffb640 1 -- 172.21.15.78:0/397839987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa40043c0 con 0x7fafb8103ce0 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.880+0000 7fafb5ffb640 1 -- 172.21.15.78:0/397839987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa40049e0 con 0x7fafb8103ce0 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.880+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a10e0 con 0x7fafb8103ce0 2024-05-21T15:15:33.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.880+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a1680 con 0x7fafb8103ce0 2024-05-21T15:15:33.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.881+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb81077f0 con 0x7fafb8103ce0 2024-05-21T15:15:33.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.882+0000 7fafb5ffb640 1 -- 172.21.15.78:0/397839987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fafa40079b0 con 0x7fafb8103ce0 2024-05-21T15:15:33.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.883+0000 7fafb5ffb640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fafa80a2ca0 0x7fafa80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:33.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.883+0000 7fafbcc7e640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fafa80a2ca0 0x7fafa80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:33.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.883+0000 7fafb5ffb640 1 -- 172.21.15.78:0/397839987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fafa40cf320 con 0x7fafb8103ce0 2024-05-21T15:15:33.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.883+0000 7fafbcc7e640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fafa80a2ca0 0x7fafa80a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafac0096c0 tx=0x7fafac031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:33.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:33.888+0000 7fafb5ffb640 1 -- 172.21.15.78:0/397839987 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fafa4014030 con 0x7fafb8103ce0 2024-05-21T15:15:34.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.063+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafb8062780 con 0x7fafb8103ce0 2024-05-21T15:15:34.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.063+0000 7fafb5ffb640 1 -- 172.21.15.78:0/397839987 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fafa409c0a0 con 0x7fafb8103ce0 2024-05-21T15:15:34.066 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:15:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fafa80a2ca0 msgr2=0x7fafa80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fafa80a2ca0 0x7fafa80a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafac0096c0 tx=0x7fafac031400 comp rx=0 tx=0).stop 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8103ce0 msgr2=0x7fafb8199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8103ce0 0x7fafb8199860 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fafa400c890 tx=0x7fafa400cd50 comp rx=0 tx=0).stop 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 shutdown_connections 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fafa80a2ca0 0x7fafa80a51b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafb81089e0 0x7fafb81a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb8103ce0 0x7fafb8199860 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 --2- 172.21.15.78:0/397839987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafb8102aa0 0x7fafb8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 >> 172.21.15.78:0/397839987 conn(0x7fafb80fdc90 msgr2=0x7fafb81049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 shutdown_connections 2024-05-21T15:15:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.065+0000 7fafbeee2640 1 -- 172.21.15.78:0/397839987 wait complete. 2024-05-21T15:15:34.081 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:15:34.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 -- 172.21.15.78:0/1328463351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 msgr2=0x7f395c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 --2- 172.21.15.78:0/1328463351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 0x7f395c109d10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3954009ca0 tx=0x7f395402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 -- 172.21.15.78:0/1328463351 shutdown_connections 2024-05-21T15:15:34.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 --2- 172.21.15.78:0/1328463351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 0x7f395c109d10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 --2- 172.21.15.78:0/1328463351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f395c102bf0 0x7f395c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 --2- 172.21.15.78:0/1328463351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395c1019b0 0x7f395c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 -- 172.21.15.78:0/1328463351 >> 172.21.15.78:0/1328463351 conn(0x7f395c0fcba0 msgr2=0x7f395c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:34.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 -- 172.21.15.78:0/1328463351 shutdown_connections 2024-05-21T15:15:34.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.193+0000 7f39616a2640 1 -- 172.21.15.78:0/1328463351 wait complete. 2024-05-21T15:15:34.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.194+0000 7f39616a2640 1 Processor -- start 2024-05-21T15:15:34.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.194+0000 7f39616a2640 1 -- start start 2024-05-21T15:15:34.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.194+0000 7f39616a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395c1019b0 0x7f395c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.194+0000 7f39616a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f395c102bf0 0x7f395c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.194+0000 7f39616a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 0x7f395c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.194+0000 7f39616a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10d630 con 0x7f395c1078f0 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.194+0000 7f39616a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10d4b0 con 0x7f395c1019b0 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.194+0000 7f39616a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f395c10d7b0 con 0x7f395c102bf0 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f395c102bf0 0x7f395c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 0x7f395c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f395c102bf0 0x7f395c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:58950/0 (socket says 172.21.15.78:58950) 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395a7fc640 1 -- 172.21.15.78:0/1290730446 learned_addr learned my addr 172.21.15.78:0/1290730446 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395affd640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395c1019b0 0x7f395c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395b7fe640 1 -- 172.21.15.78:0/1290730446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f395c102bf0 msgr2=0x7f395c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395b7fe640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f395c102bf0 0x7f395c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395b7fe640 1 -- 172.21.15.78:0/1290730446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395c1019b0 msgr2=0x7f395c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395b7fe640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395c1019b0 0x7f395c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395b7fe640 1 -- 172.21.15.78:0/1290730446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395c0fd030 con 0x7f395c1078f0 2024-05-21T15:15:34.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f395b7fe640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 0x7f395c1a0790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f395402ade0 tx=0x7f3954031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:34.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f3943fff640 1 -- 172.21.15.78:0/1290730446 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954003c30 con 0x7f395c1078f0 2024-05-21T15:15:34.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395c0fd2c0 con 0x7f395c1078f0 2024-05-21T15:15:34.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f3943fff640 1 -- 172.21.15.78:0/1290730446 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3954003dd0 con 0x7f395c1078f0 2024-05-21T15:15:34.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.195+0000 7f3943fff640 1 -- 172.21.15.78:0/1290730446 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954004770 con 0x7f395c1078f0 2024-05-21T15:15:34.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.196+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395c0fd810 con 0x7f395c1078f0 2024-05-21T15:15:34.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.197+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395c0fda70 con 0x7f395c1078f0 2024-05-21T15:15:34.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.198+0000 7f3943fff640 1 -- 172.21.15.78:0/1290730446 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f395402f710 con 0x7f395c1078f0 2024-05-21T15:15:34.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.199+0000 7f3943fff640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f39440a2c20 0x7f39440a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.199+0000 7f395affd640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f39440a2c20 0x7f39440a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.199+0000 7f3943fff640 1 -- 172.21.15.78:0/1290730446 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f39540f04e0 con 0x7f395c1078f0 2024-05-21T15:15:34.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.199+0000 7f395affd640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f39440a2c20 0x7f39440a5130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f395c1025b0 tx=0x7f394c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:34.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.202+0000 7f3943fff640 1 -- 172.21.15.78:0/1290730446 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f39540bd460 con 0x7f395c1078f0 2024-05-21T15:15:34.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.388+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f395c0fe320 con 0x7f395c1078f0 2024-05-21T15:15:34.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.388+0000 7f3943fff640 1 -- 172.21.15.78:0/1290730446 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f39540bcd60 con 0x7f395c1078f0 2024-05-21T15:15:34.391 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:15:34.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f39440a2c20 msgr2=0x7f39440a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f39440a2c20 0x7f39440a5130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f395c1025b0 tx=0x7f394c005f90 comp rx=0 tx=0).stop 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 msgr2=0x7f395c1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 0x7f395c1a0790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f395402ade0 tx=0x7f3954031ee0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 shutdown_connections 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f39440a2c20 0x7f39440a5130 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395c1078f0 0x7f395c1a0790 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f395c102bf0 0x7f395c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 --2- 172.21.15.78:0/1290730446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395c1019b0 0x7f395c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 >> 172.21.15.78:0/1290730446 conn(0x7f395c0fcba0 msgr2=0x7f395c108cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:34.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 shutdown_connections 2024-05-21T15:15:34.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.390+0000 7f39616a2640 1 -- 172.21.15.78:0/1290730446 wait complete. 2024-05-21T15:15:34.405 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:15:34.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 -- 172.21.15.78:0/1538748954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc101990 msgr2=0x7f01dc102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/1538748954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc101990 0x7f01dc102690 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009900 tx=0x7f01cc02a410 comp rx=0 tx=0).stop 2024-05-21T15:15:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 -- 172.21.15.78:0/1538748954 shutdown_connections 2024-05-21T15:15:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/1538748954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01dc1078d0 0x7f01dc109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/1538748954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc102bd0 0x7f01dc102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/1538748954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc101990 0x7f01dc102690 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 -- 172.21.15.78:0/1538748954 >> 172.21.15.78:0/1538748954 conn(0x7f01dc0fcb80 msgr2=0x7f01dc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:34.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 -- 172.21.15.78:0/1538748954 shutdown_connections 2024-05-21T15:15:34.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.530+0000 7f01e1ab1640 1 -- 172.21.15.78:0/1538748954 wait complete. 2024-05-21T15:15:34.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01e1ab1640 1 Processor -- start 2024-05-21T15:15:34.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01e1ab1640 1 -- start start 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01e1ab1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01dc101990 0x7f01dc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01e1ab1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc102bd0 0x7f01dc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01e1ab1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1078d0 0x7f01dc1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01e1ab1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc10e6d0 con 0x7f01dc102bd0 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01e1ab1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc10e550 con 0x7f01dc101990 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01e1ab1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc10e850 con 0x7f01dc1078d0 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc102bd0 0x7f01dc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc102bd0 0x7f01dc199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40040/0 (socket says 172.21.15.78:40040) 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.531+0000 7f01daffd640 1 -- 172.21.15.78:0/3405835743 learned_addr learned my addr 172.21.15.78:0/3405835743 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01dbfff640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1078d0 0x7f01dc1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01db7fe640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01dc101990 0x7f01dc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01daffd640 1 -- 172.21.15.78:0/3405835743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1078d0 msgr2=0x7f01dc1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01daffd640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1078d0 0x7f01dc1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01daffd640 1 -- 172.21.15.78:0/3405835743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01dc101990 msgr2=0x7f01dc199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01daffd640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01dc101990 0x7f01dc199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01daffd640 1 -- 172.21.15.78:0/3405835743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc1a1140 con 0x7f01dc102bd0 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01dbfff640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1078d0 0x7f01dc1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01db7fe640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01dc101990 0x7f01dc199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01daffd640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc102bd0 0x7f01dc199b40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f01d000d950 tx=0x7f01d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01d8ff9640 1 -- 172.21.15.78:0/3405835743 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0017030 con 0x7f01dc102bd0 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc1a1430 con 0x7f01dc102bd0 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01d8ff9640 1 -- 172.21.15.78:0/3405835743 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d0004020 con 0x7f01dc102bd0 2024-05-21T15:15:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01d8ff9640 1 -- 172.21.15.78:0/3405835743 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d00047a0 con 0x7f01dc102bd0 2024-05-21T15:15:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.532+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc1a1980 con 0x7f01dc102bd0 2024-05-21T15:15:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.533+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc1066e0 con 0x7f01dc102bd0 2024-05-21T15:15:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.535+0000 7f01d8ff9640 1 -- 172.21.15.78:0/3405835743 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f01d00041d0 con 0x7f01dc102bd0 2024-05-21T15:15:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.535+0000 7f01d8ff9640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01c40a2c20 0x7f01c40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.535+0000 7f01db7fe640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01c40a2c20 0x7f01c40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.535+0000 7f01d8ff9640 1 -- 172.21.15.78:0/3405835743 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f01d00ced10 con 0x7f01dc102bd0 2024-05-21T15:15:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.536+0000 7f01db7fe640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01c40a2c20 0x7f01c40a5130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01cc000c00 tx=0x7f01cc031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:34.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.544+0000 7f01d8ff9640 1 -- 172.21.15.78:0/3405835743 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f01d009baf0 con 0x7f01dc102bd0 2024-05-21T15:15:34.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:34 smithi078 ceph-mon[31997]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:34.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:34 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4039999377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:34 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4176535279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:34.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:34 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/397839987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:34.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:34 smithi084 ceph-mon[36334]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:34 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4039999377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:34 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4176535279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:34 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/397839987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:34 smithi172 ceph-mon[36402]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:34 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4039999377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:34.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:34 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4176535279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:34.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:34 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/397839987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:34.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.726+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f01dc02d3f0 con 0x7f01dc102bd0 2024-05-21T15:15:34.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.727+0000 7f01d8ff9640 1 -- 172.21.15.78:0/3405835743 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f01d009b3f0 con 0x7f01dc102bd0 2024-05-21T15:15:34.730 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:15:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01c40a2c20 msgr2=0x7f01c40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01c40a2c20 0x7f01c40a5130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01cc000c00 tx=0x7f01cc031400 comp rx=0 tx=0).stop 2024-05-21T15:15:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc102bd0 msgr2=0x7f01dc199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc102bd0 0x7f01dc199b40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f01d000d950 tx=0x7f01d000de10 comp rx=0 tx=0).stop 2024-05-21T15:15:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 shutdown_connections 2024-05-21T15:15:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f01c40a2c20 0x7f01c40a5130 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01dc1078d0 0x7f01dc1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01dc102bd0 0x7f01dc199b40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 --2- 172.21.15.78:0/3405835743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01dc101990 0x7f01dc199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 >> 172.21.15.78:0/3405835743 conn(0x7f01dc0fcb80 msgr2=0x7f01dc1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:34.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 shutdown_connections 2024-05-21T15:15:34.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.729+0000 7f01e1ab1640 1 -- 172.21.15.78:0/3405835743 wait complete. 2024-05-21T15:15:34.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:15:34.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.855+0000 7f73340a6640 1 -- 172.21.15.78:0/3801314259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 msgr2=0x7f732c101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.855+0000 7f73340a6640 1 --2- 172.21.15.78:0/3801314259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 0x7f732c101760 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7328009900 tx=0x7f732802a410 comp rx=0 tx=0).stop 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.855+0000 7f73340a6640 1 -- 172.21.15.78:0/3801314259 shutdown_connections 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.855+0000 7f73340a6640 1 --2- 172.21.15.78:0/3801314259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f732c103360 0x7f732c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.855+0000 7f73340a6640 1 --2- 172.21.15.78:0/3801314259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f732c1079b0 0x7f732c109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.855+0000 7f73340a6640 1 --2- 172.21.15.78:0/3801314259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 0x7f732c101760 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.855+0000 7f73340a6640 1 -- 172.21.15.78:0/3801314259 >> 172.21.15.78:0/3801314259 conn(0x7f732c0fc2e0 msgr2=0x7f732c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 -- 172.21.15.78:0/3801314259 shutdown_connections 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 -- 172.21.15.78:0/3801314259 wait complete. 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 Processor -- start 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 -- start start 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 0x7f732c1010e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f732c1079b0 0x7f732c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f732c19e130 0x7f732c1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10d5e0 con 0x7f732c102e80 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10d460 con 0x7f732c19e130 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f73340a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10d760 con 0x7f732c1079b0 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.856+0000 7f7331e42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 0x7f732c1010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331e42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 0x7f732c1010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40052/0 (socket says 172.21.15.78:40052) 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331e42640 1 -- 172.21.15.78:0/87572711 learned_addr learned my addr 172.21.15.78:0/87572711 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331641640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f732c1079b0 0x7f732c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7332643640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f732c19e130 0x7f732c1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331e42640 1 -- 172.21.15.78:0/87572711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f732c1079b0 msgr2=0x7f732c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331e42640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f732c1079b0 0x7f732c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331e42640 1 -- 172.21.15.78:0/87572711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f732c19e130 msgr2=0x7f732c1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331e42640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f732c19e130 0x7f732c1a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331e42640 1 -- 172.21.15.78:0/87572711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c1a0d90 con 0x7f732c102e80 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7331e42640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 0x7f732c1010e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7328005ab0 tx=0x7f7328004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7322ffd640 1 -- 172.21.15.78:0/87572711 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732803f030 con 0x7f732c102e80 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c1a1020 con 0x7f732c102e80 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7322ffd640 1 -- 172.21.15.78:0/87572711 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7328003ad0 con 0x7f732c102e80 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f7322ffd640 1 -- 172.21.15.78:0/87572711 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328035070 con 0x7f732c102e80 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.857+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c1a1510 con 0x7f732c102e80 2024-05-21T15:15:34.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.858+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732c1060b0 con 0x7f732c102e80 2024-05-21T15:15:34.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.860+0000 7f7322ffd640 1 -- 172.21.15.78:0/87572711 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7328042030 con 0x7f732c102e80 2024-05-21T15:15:34.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.861+0000 7f7322ffd640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f73180a2c20 0x7f73180a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:34.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.861+0000 7f7331641640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f73180a2c20 0x7f73180a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:34.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.861+0000 7f7322ffd640 1 -- 172.21.15.78:0/87572711 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f73280efba0 con 0x7f732c102e80 2024-05-21T15:15:34.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.861+0000 7f7331641640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f73180a2c20 0x7f73180a5130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f732c100a80 tx=0x7f731c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:34.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:34.864+0000 7f7322ffd640 1 -- 172.21.15.78:0/87572711 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73280bc960 con 0x7f732c102e80 2024-05-21T15:15:35.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.083+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f732c02d400 con 0x7f732c102e80 2024-05-21T15:15:35.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.084+0000 7f7322ffd640 1 -- 172.21.15.78:0/87572711 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11925 (secure 0 0 0) 0x7f73280bc260 con 0x7f732c102e80 2024-05-21T15:15:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:35.087 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:14:18.291660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:15:35.087 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f73180a2c20 msgr2=0x7f73180a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f73180a2c20 0x7f73180a5130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f732c100a80 tx=0x7f731c005f90 comp rx=0 tx=0).stop 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 msgr2=0x7f732c1010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 0x7f732c1010e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7328005ab0 tx=0x7f7328004270 comp rx=0 tx=0).stop 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 shutdown_connections 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f73180a2c20 0x7f73180a5130 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f732c19e130 0x7f732c1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f732c1079b0 0x7f732c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 --2- 172.21.15.78:0/87572711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f732c102e80 0x7f732c1010e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 >> 172.21.15.78:0/87572711 conn(0x7f732c0fc2e0 msgr2=0x7f732c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 shutdown_connections 2024-05-21T15:15:35.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.086+0000 7f73340a6640 1 -- 172.21.15.78:0/87572711 wait complete. 2024-05-21T15:15:35.101 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:15:35.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/3132032169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 msgr2=0x7fb8e810ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/3132032169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 0x7fb8e810ac90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009ca0 tx=0x7fb8e002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/3132032169 shutdown_connections 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/3132032169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 0x7fb8e810ac90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/3132032169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8105f10 0x7fb8e8108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/3132032169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e81035b0 0x7fb8e81059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/3132032169 >> 172.21.15.78:0/3132032169 conn(0x7fb8e80fcbb0 msgr2=0x7fb8e80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/3132032169 shutdown_connections 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.210+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/3132032169 wait complete. 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ef2d7640 1 Processor -- start 2024-05-21T15:15:35.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ef2d7640 1 -- start start 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ef2d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e81035b0 0x7fb8e81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ef2d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8105f10 0x7fb8e81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ef2d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 0x7fb8e81a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ef2d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810e740 con 0x7fb8e8108870 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ef2d7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810e5c0 con 0x7fb8e81035b0 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ef2d7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810e8c0 con 0x7fb8e8105f10 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed874640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 0x7fb8e81a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed874640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 0x7fb8e81a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40074/0 (socket says 172.21.15.78:40074) 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed874640 1 -- 172.21.15.78:0/4017502139 learned_addr learned my addr 172.21.15.78:0/4017502139 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ec872640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8105f10 0x7fb8e81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed073640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e81035b0 0x7fb8e81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed874640 1 -- 172.21.15.78:0/4017502139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8105f10 msgr2=0x7fb8e81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed874640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8105f10 0x7fb8e81997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed874640 1 -- 172.21.15.78:0/4017502139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e81035b0 msgr2=0x7fb8e81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed874640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e81035b0 0x7fb8e81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ed874640 1 -- 172.21.15.78:0/4017502139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e81a0df0 con 0x7fb8e8108870 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.211+0000 7fb8ec872640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8105f10 0x7fb8e81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.212+0000 7fb8ed874640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 0x7fb8e81a0780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb8e002f540 tx=0x7fb8e0005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.212+0000 7fb8de7fc640 1 -- 172.21.15.78:0/4017502139 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0003b70 con 0x7fb8e8108870 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.212+0000 7fb8de7fc640 1 -- 172.21.15.78:0/4017502139 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e0003d10 con 0x7fb8e8108870 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.212+0000 7fb8de7fc640 1 -- 172.21.15.78:0/4017502139 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e002fc00 con 0x7fb8e8108870 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.212+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e81a1080 con 0x7fb8e8108870 2024-05-21T15:15:35.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.212+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81a15d0 con 0x7fb8e8108870 2024-05-21T15:15:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.213+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e80fe6d0 con 0x7fb8e8108870 2024-05-21T15:15:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.215+0000 7fb8de7fc640 1 -- 172.21.15.78:0/4017502139 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb8e0004b40 con 0x7fb8e8108870 2024-05-21T15:15:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.215+0000 7fb8de7fc640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8d40a2c20 0x7fb8d40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.215+0000 7fb8de7fc640 1 -- 172.21.15.78:0/4017502139 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb8e00f0400 con 0x7fb8e8108870 2024-05-21T15:15:35.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.215+0000 7fb8ed073640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8d40a2c20 0x7fb8d40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.216+0000 7fb8ed073640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8d40a2c20 0x7fb8d40a5130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8e806ac40 tx=0x7fb8e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.219+0000 7fb8de7fc640 1 -- 172.21.15.78:0/4017502139 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8e00bd2f0 con 0x7fb8e8108870 2024-05-21T15:15:35.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:35 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1290730446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:35.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:35 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3405835743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:35.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:35 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/87572711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:35.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.404+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8e8062750 con 0x7fb8e8108870 2024-05-21T15:15:35.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.406+0000 7fb8de7fc640 1 -- 172.21.15.78:0/4017502139 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22616 (secure 0 0 0) 0x7fb8e00bcbf0 con 0x7fb8e8108870 2024-05-21T15:15:35.409 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:35.409 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":87,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:15:16.363058+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:15:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8d40a2c20 msgr2=0x7fb8d40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8d40a2c20 0x7fb8d40a5130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8e806ac40 tx=0x7fb8e4005f90 comp rx=0 tx=0).stop 2024-05-21T15:15:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 msgr2=0x7fb8e81a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 0x7fb8e81a0780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb8e002f540 tx=0x7fb8e0005df0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 shutdown_connections 2024-05-21T15:15:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb8d40a2c20 0x7fb8d40a5130 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8e8108870 0x7fb8e81a0780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8e8105f10 0x7fb8e81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 --2- 172.21.15.78:0/4017502139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8e81035b0 0x7fb8e81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 >> 172.21.15.78:0/4017502139 conn(0x7fb8e80fcbb0 msgr2=0x7fb8e8104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 shutdown_connections 2024-05-21T15:15:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.408+0000 7fb8ef2d7640 1 -- 172.21.15.78:0/4017502139 wait complete. 2024-05-21T15:15:35.423 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:15:35.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 -- 172.21.15.78:0/2229272342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 msgr2=0x7fa3e0109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/2229272342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 0x7fa3e0109cf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009ca0 tx=0x7fa3d802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 -- 172.21.15.78:0/2229272342 shutdown_connections 2024-05-21T15:15:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/2229272342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 0x7fa3e0109cf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/2229272342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e0102bd0 0x7fa3e0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/2229272342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e0101990 0x7fa3e0102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 -- 172.21.15.78:0/2229272342 >> 172.21.15.78:0/2229272342 conn(0x7fa3e00fcb80 msgr2=0x7fa3e00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 -- 172.21.15.78:0/2229272342 shutdown_connections 2024-05-21T15:15:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 -- 172.21.15.78:0/2229272342 wait complete. 2024-05-21T15:15:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.532+0000 7fa3e5db7640 1 Processor -- start 2024-05-21T15:15:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3e5db7640 1 -- start start 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3e5db7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e0101990 0x7fa3e0074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3e5db7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e0102bd0 0x7fa3e019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3e5db7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 0x7fa3e01a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3e5db7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e010d660 con 0x7fa3e01078d0 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3e5db7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e010d4e0 con 0x7fa3e0101990 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3e5db7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e010d7e0 con 0x7fa3e0102bd0 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 0x7fa3e01a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 0x7fa3e01a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40090/0 (socket says 172.21.15.78:40090) 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 -- 172.21.15.78:0/3658784656 learned_addr learned my addr 172.21.15.78:0/3658784656 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3deffd640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e0102bd0 0x7fa3e019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3df7fe640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e0101990 0x7fa3e0074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 -- 172.21.15.78:0/3658784656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e0102bd0 msgr2=0x7fa3e019e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e0102bd0 0x7fa3e019e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 -- 172.21.15.78:0/3658784656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e0101990 msgr2=0x7fa3e0074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e0101990 0x7fa3e0074810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 -- 172.21.15.78:0/3658784656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e01a0dc0 con 0x7fa3e01078d0 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3df7fe640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e0101990 0x7fa3e0074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.533+0000 7fa3dffff640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 0x7fa3e01a05a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8003f40 tx=0x7fa3d8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.534+0000 7fa3dcff9640 1 -- 172.21.15.78:0/3658784656 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80316d0 con 0x7fa3e01078d0 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.534+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01a1050 con 0x7fa3e01078d0 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.534+0000 7fa3dcff9640 1 -- 172.21.15.78:0/3658784656 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8031870 con 0x7fa3e01078d0 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.534+0000 7fa3dcff9640 1 -- 172.21.15.78:0/3658784656 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d802fbb0 con 0x7fa3e01078d0 2024-05-21T15:15:35.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.534+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e01a15a0 con 0x7fa3e01078d0 2024-05-21T15:15:35.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.535+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e01066e0 con 0x7fa3e01078d0 2024-05-21T15:15:35.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.536+0000 7fa3dcff9640 1 -- 172.21.15.78:0/3658784656 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa3d8004040 con 0x7fa3e01078d0 2024-05-21T15:15:35.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.536+0000 7fa3dcff9640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa3c80a2c20 0x7fa3c80a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.537+0000 7fa3df7fe640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa3c80a2c20 0x7fa3c80a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.537+0000 7fa3dcff9640 1 -- 172.21.15.78:0/3658784656 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa3d80f06d0 con 0x7fa3e01078d0 2024-05-21T15:15:35.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.537+0000 7fa3df7fe640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa3c80a2c20 0x7fa3c80a5130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0005d00 tx=0x7fa3d0005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.541+0000 7fa3dcff9640 1 -- 172.21.15.78:0/3658784656 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa3d80bd5c0 con 0x7fa3e01078d0 2024-05-21T15:15:35.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:35 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1290730446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:35.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:35 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3405835743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:35.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:35 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/87572711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:35.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:35 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1290730446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:35.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:35 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3405835743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:35.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:35 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/87572711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:35.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.754+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3e002d2a0 con 0x7fa3e01078d0 2024-05-21T15:15:35.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.755+0000 7fa3dcff9640 1 -- 172.21.15.78:0/3658784656 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11925 (secure 0 0 0) 0x7fa3d80bcec0 con 0x7fa3e01078d0 2024-05-21T15:15:35.758 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:35.758 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:14:18.291660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:15:35.758 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-05-21T15:15:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa3c80a2c20 msgr2=0x7fa3c80a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa3c80a2c20 0x7fa3c80a5130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0005d00 tx=0x7fa3d0005c90 comp rx=0 tx=0).stop 2024-05-21T15:15:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 msgr2=0x7fa3e01a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 0x7fa3e01a05a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8003f40 tx=0x7fa3d8003f70 comp rx=0 tx=0).stop 2024-05-21T15:15:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 shutdown_connections 2024-05-21T15:15:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa3c80a2c20 0x7fa3c80a5130 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3e01078d0 0x7fa3e01a05a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e0102bd0 0x7fa3e019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 --2- 172.21.15.78:0/3658784656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3e0101990 0x7fa3e0074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 >> 172.21.15.78:0/3658784656 conn(0x7fa3e00fcb80 msgr2=0x7fa3e01038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 shutdown_connections 2024-05-21T15:15:35.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.757+0000 7fa3e5db7640 1 -- 172.21.15.78:0/3658784656 wait complete. 2024-05-21T15:15:35.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T15:14:11.918954+0000', 'modified': '2024-05-21T15:14:18.291660+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24526, 'mds_2': 24502, 'mds_3': 24448, 'mds_4': 24419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14667': {'gid': 14667, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/3881579182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3881579182}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3881579182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14679': {'gid': 14679, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6841/2631303889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 2631303889}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 2631303889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'k', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.84:6833/502215426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6832', 'nonce': 502215426}, {'type': 'v1', 'addr': '172.21.15.84:6833', 'nonce': 502215426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.172:6833/1961121180', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 1961121180}, {'type': 'v1', 'addr': '172.21.15.172:6833', 'nonce': 1961121180}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6835/4071213416', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 4071213416}, {'type': 'v1', 'addr': '172.21.15.84:6835', 'nonce': 4071213416}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6837/3570369413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 3570369413}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 3570369413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/204442136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 204442136}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 204442136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.172:6837/3093965276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 3093965276}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 3093965276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.172:6839/3459432480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 3459432480}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 3459432480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6839/652490614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6838', 'nonce': 652490614}, {'type': 'v1', 'addr': '172.21.15.84:6839', 'nonce': 652490614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24526, 24527, 24502, 24448, 24419]} 2024-05-21T15:15:35.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T15:15:35.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.882+0000 7f30b4cbc640 1 -- 172.21.15.78:0/1627560153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0100710 msgr2=0x7f30b0100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.882+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/1627560153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0100710 0x7f30b0100b80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3098009900 tx=0x7f309802a410 comp rx=0 tx=0).stop 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.882+0000 7f30b4cbc640 1 -- 172.21.15.78:0/1627560153 shutdown_connections 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.882+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/1627560153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30b0101210 0x7f30b010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.882+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/1627560153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0100710 0x7f30b0100b80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.882+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/1627560153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b0104060 0x7f30b0104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.882+0000 7f30b4cbc640 1 -- 172.21.15.78:0/1627560153 >> 172.21.15.78:0/1627560153 conn(0x7f30b00fc330 msgr2=0x7f30b00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.882+0000 7f30b4cbc640 1 -- 172.21.15.78:0/1627560153 shutdown_connections 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.883+0000 7f30b4cbc640 1 -- 172.21.15.78:0/1627560153 wait complete. 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.883+0000 7f30b4cbc640 1 Processor -- start 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.883+0000 7f30b4cbc640 1 -- start start 2024-05-21T15:15:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30b4cbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b0100710 0x7f30b01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30b4cbc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0101210 0x7f30b01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30b4cbc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30b0104060 0x7f30b01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30b4cbc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30b010e770 con 0x7f30b0101210 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30b4cbc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f30b010e5f0 con 0x7f30b0104060 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30b4cbc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30b010e8f0 con 0x7f30b0100710 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0101210 0x7f30b01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0101210 0x7f30b01997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40104/0 (socket says 172.21.15.78:40104) 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 -- 172.21.15.78:0/3242969445 learned_addr learned my addr 172.21.15.78:0/3242969445 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30ae59c640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b0100710 0x7f30b01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30aed9d640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30b0104060 0x7f30b01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 -- 172.21.15.78:0/3242969445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b0100710 msgr2=0x7f30b01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b0100710 0x7f30b01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 -- 172.21.15.78:0/3242969445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30b0104060 msgr2=0x7f30b01a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30b0104060 0x7f30b01a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 -- 172.21.15.78:0/3242969445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b01a0df0 con 0x7f30b0101210 2024-05-21T15:15:35.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30aed9d640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30b0104060 0x7f30b01a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:35.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a7fff640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0101210 0x7f30b01997f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3098031440 tx=0x7f3098032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a77fe640 1 -- 172.21.15.78:0/3242969445 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309803f030 con 0x7f30b0101210 2024-05-21T15:15:35.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a77fe640 1 -- 172.21.15.78:0/3242969445 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098003ce0 con 0x7f30b0101210 2024-05-21T15:15:35.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30b4cbc640 1 -- 172.21.15.78:0/3242969445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b01a1080 con 0x7f30b0101210 2024-05-21T15:15:35.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30a77fe640 1 -- 172.21.15.78:0/3242969445 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098035070 con 0x7f30b0101210 2024-05-21T15:15:35.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.884+0000 7f30b4cbc640 1 -- 172.21.15.78:0/3242969445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b01a1570 con 0x7f30b0101210 2024-05-21T15:15:35.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.887+0000 7f30a77fe640 1 -- 172.21.15.78:0/3242969445 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3098042030 con 0x7f30b0101210 2024-05-21T15:15:35.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.887+0000 7f30bbc36740 1 Processor -- start 2024-05-21T15:15:35.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.887+0000 7f30bbc36740 1 -- start start 2024-05-21T15:15:35.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 0x55731e588470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1bf3b0 0x55731e5889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1c7590 0x55731e58f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30bbc36740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55731e4ec4d0 con 0x55731e1c7590 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30bbc36740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55731e4943e0 con 0x55731e1bf3b0 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30bbc36740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55731e4e8490 con 0x55731e1e8270 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1c7590 0x55731e58f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1c7590 0x55731e58f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40118/0 (socket says 172.21.15.78:40118) 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 -- 172.21.15.78:0/2780772956 learned_addr learned my addr 172.21.15.78:0/2780772956 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30aed9d640 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 0x55731e588470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a77fe640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f309c0a2c20 0x7f309c0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30ae59c640 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1bf3b0 0x55731e5889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 -- 172.21.15.78:0/2780772956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 msgr2=0x55731e588470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:35.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 0x55731e588470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30aed9d640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f309c0a2c20 0x7f309c0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 -- 172.21.15.78:0/2780772956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1bf3b0 msgr2=0x55731e5889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1bf3b0 0x55731e5889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 -- 172.21.15.78:0/2780772956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55731e2002d0 con 0x55731e1c7590 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30ae59c640 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1bf3b0 0x55731e5889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a77fe640 1 -- 172.21.15.78:0/3242969445 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f30980ef7b0 con 0x7f30b0101210 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1c7590 0x55731e58f8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f30980ed080 tx=0x7f309802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b045c68ca8c80c1d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30aed9d640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f309c0a2c20 0x7f309c0a5130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55731e589640 tx=0x7f30a8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.888+0000 7f30a4ff9640 1 -- 172.21.15.78:0/2780772956 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098049070 con 0x55731e1c7590 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30a4ff9640 1 -- 172.21.15.78:0/2780772956 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098005de0 con 0x55731e1c7590 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30a4ff9640 1 -- 172.21.15.78:0/2780772956 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30980c0240 con 0x55731e1c7590 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30bbc36740 1 -- 172.21.15.78:0/2780772956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1c7590 msgr2=0x55731e58f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1c7590 0x55731e58f8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f30980ed080 tx=0x7f309802a410 comp rx=0 tx=0).stop 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30bbc36740 1 -- 172.21.15.78:0/2780772956 shutdown_connections 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1bf3b0 0x55731e5889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1c7590 0x55731e58f8f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2780772956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 0x55731e588470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.889+0000 7f30bbc36740 1 -- 172.21.15.78:0/2780772956 >> 172.21.15.78:0/2780772956 conn(0x55731e1dc3c0 msgr2=0x55731e583e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 -- 172.21.15.78:0/2780772956 shutdown_connections 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 -- 172.21.15.78:0/2780772956 wait complete. 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 -- start start 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 Processor -- start 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e591fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e592500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 0x55731e599460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55731e590100 con 0x55731e1bf3b0 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55731e4a78e0 con 0x55731e1c7590 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55731e590280 con 0x55731e1e8270 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30bbc36740 10 client.0 _ll_register_callbacks cb 0x55731e4749c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30ae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e591fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30ae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e591fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40120/0 (socket says 172.21.15.78:40120) 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.890+0000 7f30ae59c640 1 -- 172.21.15.78:0/2906636877 learned_addr learned my addr 172.21.15.78:0/2906636877 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e592500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30aed9d640 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 0x55731e599460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30ae59c640 1 -- 172.21.15.78:0/2906636877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 msgr2=0x55731e599460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30ae59c640 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 0x55731e599460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30ae59c640 1 -- 172.21.15.78:0/2906636877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 msgr2=0x55731e592500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30ae59c640 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e592500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30ae59c640 1 -- 172.21.15.78:0/2906636877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55731e2ca6b0 con 0x55731e1bf3b0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30ae59c640 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e591fc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f30a000be60 tx=0x7f30a0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5d5613ec08e491 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f3095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f3095ffb640 1 -- 172.21.15.78:0/2906636877 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a0004220 con 0x55731e1bf3b0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f3095ffb640 1 -- 172.21.15.78:0/2906636877 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f30a00043c0 con 0x55731e1bf3b0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30bbc36740 1 -- 172.21.15.78:0/2906636877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55731e4f9c00 con 0x55731e1bf3b0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f3095ffb640 1 -- 172.21.15.78:0/2906636877 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30a0005020 con 0x55731e1bf3b0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30bbc36740 10 client.15099 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f30bbc36740 1 -- 172.21.15.78:0/2906636877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55731e59c460 con 0x55731e1bf3b0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f3095ffb640 1 -- 172.21.15.78:0/2906636877 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55731e4f9c00 con 0x55731e1bf3b0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.891+0000 7f3095ffb640 1 -- 172.21.15.78:0/2906636877 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f30a00107d0 con 0x55731e1bf3b0 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f30bbc36740 10 client.15099 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f30bbc36740 10 client.15099 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-21T15:15:35.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f30bbc36740 20 client.15099 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55731e1a6e40 0x55731e59ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f30bbc36740 4 client.15099 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f30bbc36740 1 -- 172.21.15.78:0/2906636877 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55731e4f3f50 con 0x55731e1a6e40 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55731e1a6e40 0x55731e59ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f3095ffb640 10 client.15099 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.892+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55731e1a6e40 0x55731e59ffd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd95e1ddffeab3d4 server_cookie=2cc53f8de08aba45 in_seq=0 out_seq=0 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f3095ffb640 1 -- 172.21.15.78:0/2906636877 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55731e4f3f50 con 0x55731e1a6e40 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f3095ffb640 10 client.15099 handle_command_reply: tid=0 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 client.15099 shutdown 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 -- 172.21.15.78:0/2906636877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 msgr2=0x55731e591fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e591fc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f30a000be60 tx=0x7f30a0012820 comp rx=0 tx=0).stop 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 -- 172.21.15.78:0/2906636877 shutdown_connections 2024-05-21T15:15:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55731e1a6e40 0x55731e59ffd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e592500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f3095ffb640 0 client.15099 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:15:35.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1e8270 0x55731e599460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2906636877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e591fc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 -- 172.21.15.78:0/2906636877 >> 172.21.15.78:0/2906636877 conn(0x55731e1dc3c0 msgr2=0x55731e582ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 -- 172.21.15.78:0/2906636877 shutdown_connections 2024-05-21T15:15:35.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 1 -- 172.21.15.78:0/2906636877 wait complete. 2024-05-21T15:15:35.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.893+0000 7f30bbc36740 20 client.15099 trim_cache size 0 max 16384 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30bbc36740 1 Processor -- start 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30bbc36740 1 -- start start 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1a6e40 0x55731e5905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e590af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e58b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30bbc36740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55731e59e310 con 0x55731e1bf3b0 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30bbc36740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55731e59e9e0 con 0x55731e1c7590 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30bbc36740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55731e59f0b0 con 0x55731e1a6e40 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e590af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e590af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40126/0 (socket says 172.21.15.78:40126) 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30a7fff640 1 -- 172.21.15.78:0/2992511899 learned_addr learned my addr 172.21.15.78:0/2992511899 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30a7fff640 1 -- 172.21.15.78:0/2992511899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1a6e40 msgr2=0x55731e5905b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2992511899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1a6e40 0x55731e5905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30a7fff640 1 -- 172.21.15.78:0/2992511899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 msgr2=0x55731e58b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2992511899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e58b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.912+0000 7f30a7fff640 1 -- 172.21.15.78:0/2992511899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55731e588020 con 0x55731e1bf3b0 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f30a7fff640 1 --2- 172.21.15.78:0/2992511899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e590af0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30980b8860 tx=0x7f30980f5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8500f103eefd23 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f3097fff640 1 -- 172.21.15.78:0/2992511899 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098049040 con 0x55731e1bf3b0 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f3097fff640 1 -- 172.21.15.78:0/2992511899 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980c0240 con 0x55731e1bf3b0 2024-05-21T15:15:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f3097fff640 1 -- 172.21.15.78:0/2992511899 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30980b8500 con 0x55731e1bf3b0 2024-05-21T15:15:35.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f30bbc36740 1 -- 172.21.15.78:0/2992511899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 msgr2=0x55731e590af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2992511899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e590af0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30980b8860 tx=0x7f30980f5530 comp rx=0 tx=0).stop 2024-05-21T15:15:35.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f30bbc36740 1 -- 172.21.15.78:0/2992511899 shutdown_connections 2024-05-21T15:15:35.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2992511899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e58b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2992511899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1bf3b0 0x55731e590af0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f30bbc36740 1 --2- 172.21.15.78:0/2992511899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1a6e40 0x55731e5905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.913+0000 7f30bbc36740 1 -- 172.21.15.78:0/2992511899 >> 172.21.15.78:0/2992511899 conn(0x55731e1dc3c0 msgr2=0x55731e58c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 -- 172.21.15.78:0/2992511899 shutdown_connections 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 -- 172.21.15.78:0/2992511899 wait complete. 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 -- start start 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 Processor -- start 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1a6e40 0x55731e5936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1bf3b0 0x55731e593c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e606e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55731e59fe50 con 0x55731e1a6e40 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55731e59f780 con 0x55731e1c7590 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55731e5a0520 con 0x55731e1bf3b0 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30bbc36740 10 client.0 _ll_register_callbacks cb 0x7f30a000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30ae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1a6e40 0x55731e5936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30ae59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1a6e40 0x55731e5936f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40134/0 (socket says 172.21.15.78:40134) 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30ae59c640 1 -- 172.21.15.78:0/3768775077 learned_addr learned my addr 172.21.15.78:0/3768775077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30ae59c640 1 -- 172.21.15.78:0/3768775077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1bf3b0 msgr2=0x55731e593c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30ae59c640 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1bf3b0 0x55731e593c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30ae59c640 1 -- 172.21.15.78:0/3768775077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 msgr2=0x55731e606e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30ae59c640 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e606e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.914+0000 7f30ae59c640 1 -- 172.21.15.78:0/3768775077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55731e598c00 con 0x55731e1a6e40 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30ae59c640 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1a6e40 0x55731e5936f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30a000be60 tx=0x7f30a00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538229453ebfdad9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f3095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f3095ffb640 1 -- 172.21.15.78:0/3768775077 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a002b070 con 0x55731e1a6e40 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f3095ffb640 1 -- 172.21.15.78:0/3768775077 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f30a00055d0 con 0x55731e1a6e40 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 1 -- 172.21.15.78:0/3768775077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30980c0240 con 0x55731e1a6e40 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f3095ffb640 1 -- 172.21.15.78:0/3768775077 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30a000b480 con 0x55731e1a6e40 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 10 client.15105 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 1 -- 172.21.15.78:0/3768775077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55731e608280 con 0x55731e1a6e40 2024-05-21T15:15:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f3095ffb640 1 -- 172.21.15.78:0/3768775077 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980c0240 con 0x55731e1a6e40 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f3095ffb640 1 -- 172.21.15.78:0/3768775077 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f30a003a540 con 0x55731e1a6e40 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 10 client.15105 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 10 client.15105 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 20 client.15105 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55731e1e8270 0x55731e60bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 4 client.15105 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.915+0000 7f30bbc36740 1 -- 172.21.15.78:0/3768775077 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "status"}) -- 0x55731e471b80 con 0x55731e1e8270 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f30a7fff640 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55731e1e8270 0x55731e60bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f3095ffb640 10 client.15105 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f30a7fff640 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55731e1e8270 0x55731e60bc10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=433d266cba57ec6e server_cookie=eaf42250d825218e in_seq=0 out_seq=0 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f3095ffb640 1 -- 172.21.15.78:0/3768775077 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55731e471b80 con 0x55731e1e8270 2024-05-21T15:15:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f3095ffb640 10 client.15105 handle_command_reply: tid=0 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f30bbc36740 1 client.15105 shutdown 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f30bbc36740 1 -- 172.21.15.78:0/3768775077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1a6e40 msgr2=0x55731e5936f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f30bbc36740 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1a6e40 0x55731e5936f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30a000be60 tx=0x7f30a00058d0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.916+0000 7f30bbc36740 1 -- 172.21.15.78:0/3768775077 shutdown_connections 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30bbc36740 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55731e1e8270 0x55731e60bc10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30bbc36740 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55731e1c7590 0x55731e606e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f3095ffb640 0 client.15105 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30bbc36740 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55731e1bf3b0 0x55731e593c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30bbc36740 1 --2- 172.21.15.78:0/3768775077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55731e1a6e40 0x55731e5936f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30bbc36740 1 -- 172.21.15.78:0/3768775077 >> 172.21.15.78:0/3768775077 conn(0x55731e1dc3c0 msgr2=0x55731e58b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30bbc36740 1 -- 172.21.15.78:0/3768775077 shutdown_connections 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30bbc36740 1 -- 172.21.15.78:0/3768775077 wait complete. 2024-05-21T15:15:35.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30bbc36740 20 client.15105 trim_cache size 0 max 16384 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24419, 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 78.642750739999997, 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:15:35.926 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:15:35.927 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 117.085593276 2024-05-21T15:15:35.927 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:15:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 -- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f309c0a2c20 msgr2=0x7f309c0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f309c0a2c20 0x7f309c0a5130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55731e589640 tx=0x7f30a8004ab0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 -- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0101210 msgr2=0x7f30b01997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0101210 0x7f30b01997f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3098031440 tx=0x7f3098032ce0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 -- 172.21.15.78:0/3242969445 shutdown_connections 2024-05-21T15:15:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f309c0a2c20 0x7f309c0a5130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30b0104060 0x7f30b01a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30b0101210 0x7f30b01997f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 --2- 172.21.15.78:0/3242969445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b0100710 0x7f30b01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 -- 172.21.15.78:0/3242969445 >> 172.21.15.78:0/3242969445 conn(0x7f30b00fc330 msgr2=0x7f30b0104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 -- 172.21.15.78:0/3242969445 shutdown_connections 2024-05-21T15:15:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:35.917+0000 7f30b4cbc640 1 -- 172.21.15.78:0/3242969445 wait complete. 2024-05-21T15:15:35.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.039+0000 7f525345d640 1 -- 172.21.15.78:0/63172999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c102ac0 msgr2=0x7f524c1037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.039+0000 7f525345d640 1 --2- 172.21.15.78:0/63172999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c102ac0 0x7f524c1037c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f523c009900 tx=0x7f523c02a410 comp rx=0 tx=0).stop 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.039+0000 7f525345d640 1 -- 172.21.15.78:0/63172999 shutdown_connections 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.039+0000 7f525345d640 1 --2- 172.21.15.78:0/63172999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f524c103d00 0x7f524c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.039+0000 7f525345d640 1 --2- 172.21.15.78:0/63172999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c108a00 0x7f524c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.039+0000 7f525345d640 1 --2- 172.21.15.78:0/63172999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c102ac0 0x7f524c1037c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.039+0000 7f525345d640 1 -- 172.21.15.78:0/63172999 >> 172.21.15.78:0/63172999 conn(0x7f524c0fdcf0 msgr2=0x7f524c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 -- 172.21.15.78:0/63172999 shutdown_connections 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 -- 172.21.15.78:0/63172999 wait complete. 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 Processor -- start 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 -- start start 2024-05-21T15:15:36.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f524c102ac0 0x7f524c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c103d00 0x7f524c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c108a00 0x7f524c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f524c10e740 con 0x7f524c108a00 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f524c10e5c0 con 0x7f524c102ac0 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f525345d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f524c10e8c0 con 0x7f524c103d00 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.040+0000 7f52519fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c108a00 0x7f524c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52519fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c108a00 0x7f524c1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40160/0 (socket says 172.21.15.78:40160) 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52519fa640 1 -- 172.21.15.78:0/3189861274 learned_addr learned my addr 172.21.15.78:0/3189861274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52509f8640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c103d00 0x7f524c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52511f9640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f524c102ac0 0x7f524c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52519fa640 1 -- 172.21.15.78:0/3189861274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c103d00 msgr2=0x7f524c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52519fa640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c103d00 0x7f524c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52519fa640 1 -- 172.21.15.78:0/3189861274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f524c102ac0 msgr2=0x7f524c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52519fa640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f524c102ac0 0x7f524c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52519fa640 1 -- 172.21.15.78:0/3189861274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c1a0eb0 con 0x7f524c108a00 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52509f8640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c103d00 0x7f524c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52511f9640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f524c102ac0 0x7f524c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52519fa640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c108a00 0x7f524c1a0840 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f524400cad0 tx=0x7f524400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52427fc640 1 -- 172.21.15.78:0/3189861274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524400ae20 con 0x7f524c108a00 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52427fc640 1 -- 172.21.15.78:0/3189861274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5244005020 con 0x7f524c108a00 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f525345d640 1 -- 172.21.15.78:0/3189861274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c1a11a0 con 0x7f524c108a00 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f52427fc640 1 -- 172.21.15.78:0/3189861274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524400f6f0 con 0x7f524c108a00 2024-05-21T15:15:36.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.041+0000 7f525345d640 1 -- 172.21.15.78:0/3189861274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c1a16a0 con 0x7f524c108a00 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.044+0000 7f525a3d8740 1 Processor -- start 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.044+0000 7f52427fc640 1 -- 172.21.15.78:0/3189861274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f52440051d0 con 0x7f524c108a00 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f525a3d8740 1 -- start start 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 0x55b43a636880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a636e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a63dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f525a3d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b43a5964d0 con 0x55b43a271590 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f525a3d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b43a53e3e0 con 0x55b43a2693b0 2024-05-21T15:15:36.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f525a3d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b43a592490 con 0x55b43a292270 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a63dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a63dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40174/0 (socket says 172.21.15.78:40174) 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 -- 172.21.15.78:0/1105287514 learned_addr learned my addr 172.21.15.78:0/1105287514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 0x55b43a636880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52511f9640 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a636e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52427fc640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52380a2c20 0x7f52380a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 -- 172.21.15.78:0/1105287514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 msgr2=0x55b43a636880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 0x55b43a636880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 -- 172.21.15.78:0/1105287514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 msgr2=0x55b43a636e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a636e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 -- 172.21.15.78:0/1105287514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43a2aa2d0 con 0x55b43a271590 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52519fa640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52380a2c20 0x7f52380a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.045+0000 7f52509f8640 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a63dd00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5248012810 tx=0x7f5248009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd75e4acd81c14a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.046+0000 7f52427fc640 1 -- 172.21.15.78:0/3189861274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f52440cf340 con 0x7f524c108a00 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.046+0000 7f52377fe640 1 -- 172.21.15.78:0/1105287514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248012d90 con 0x55b43a271590 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.046+0000 7f52377fe640 1 -- 172.21.15.78:0/1105287514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5248010cb0 con 0x55b43a271590 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.046+0000 7f52377fe640 1 -- 172.21.15.78:0/1105287514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524801ec20 con 0x55b43a271590 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 -- 172.21.15.78:0/1105287514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 msgr2=0x55b43a63dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f52519fa640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52380a2c20 0x7f52380a5130 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f524409b9b0 tx=0x7f5244005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a63dd00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5248012810 tx=0x7f5248009740 comp rx=0 tx=0).stop 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 -- 172.21.15.78:0/1105287514 shutdown_connections 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a636e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a63dd00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 --2- 172.21.15.78:0/1105287514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 0x55b43a636880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 -- 172.21.15.78:0/1105287514 >> 172.21.15.78:0/1105287514 conn(0x55b43a2863c0 msgr2=0x55b43a62de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 -- 172.21.15.78:0/1105287514 shutdown_connections 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 -- 172.21.15.78:0/1105287514 wait complete. 2024-05-21T15:15:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 -- start start 2024-05-21T15:15:36.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 Processor -- start 2024-05-21T15:15:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.047+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a6405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a640b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 0x55b43a647a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b43a63e750 con 0x55b43a271590 2024-05-21T15:15:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b43a5518e0 con 0x55b43a2693b0 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b43a63e8d0 con 0x55b43a292270 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 10 client.0 _ll_register_callbacks cb 0x55b43a51e9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a640b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a640b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40190/0 (socket says 172.21.15.78:40190) 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 -- 172.21.15.78:0/3966782571 learned_addr learned my addr 172.21.15.78:0/3966782571 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52509f8640 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a6405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52509f8640 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 0x55b43a647a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 -- 172.21.15.78:0/3966782571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 msgr2=0x55b43a647a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 0x55b43a647a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 -- 172.21.15.78:0/3966782571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 msgr2=0x55b43a6405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a6405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 -- 172.21.15.78:0/3966782571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b43a3746b0 con 0x55b43a271590 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f52511f9640 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a640b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f523c006010 tx=0x7f523c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8734c2c517c69 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f5234ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f5234ff9640 1 -- 172.21.15.78:0/3966782571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c035030 con 0x55b43a271590 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f5234ff9640 1 -- 172.21.15.78:0/3966782571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f523c0316b0 con 0x55b43a271590 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 1 -- 172.21.15.78:0/3966782571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5248010cb0 con 0x55b43a271590 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f5234ff9640 1 -- 172.21.15.78:0/3966782571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f523c04a070 con 0x55b43a271590 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 10 client.15117 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.048+0000 7f525a3d8740 1 -- 172.21.15.78:0/3966782571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b43a64aa30 con 0x55b43a271590 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f5234ff9640 1 -- 172.21.15.78:0/3966782571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5248010cb0 con 0x55b43a271590 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f5234ff9640 1 -- 172.21.15.78:0/3966782571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f523c004c00 con 0x55b43a271590 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f525a3d8740 10 client.15117 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f525a3d8740 10 client.15117 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f525a3d8740 20 client.15117 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55b43a250e40 0x55b43a64e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f525a3d8740 4 client.15117 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f525a3d8740 1 -- 172.21.15.78:0/3966782571 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b43a5a3c00 con 0x55b43a250e40 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.049+0000 7f52509f8640 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55b43a250e40 0x55b43a64e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.050+0000 7f5234ff9640 10 client.15117 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.050+0000 7f52509f8640 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55b43a250e40 0x55b43a64e510 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=912aa7feca4e6d29 server_cookie=97b61c010afb0273 in_seq=0 out_seq=0 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.050+0000 7f5234ff9640 1 -- 172.21.15.78:0/3966782571 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b43a5a3c00 con 0x55b43a250e40 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.050+0000 7f5234ff9640 10 client.15117 handle_command_reply: tid=0 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.050+0000 7f525a3d8740 1 client.15117 shutdown 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 -- 172.21.15.78:0/3966782571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 msgr2=0x55b43a640b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a640b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f523c006010 tx=0x7f523c005040 comp rx=0 tx=0).stop 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 -- 172.21.15.78:0/3966782571 shutdown_connections 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55b43a250e40 0x55b43a64e510 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a292270 0x55b43a647a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a271590 0x55b43a640b30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f5234ff9640 0 client.15117 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:15:36.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3966782571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a6405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 -- 172.21.15.78:0/3966782571 >> 172.21.15.78:0/3966782571 conn(0x55b43a2863c0 msgr2=0x55b43a62cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 -- 172.21.15.78:0/3966782571 shutdown_connections 2024-05-21T15:15:36.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 1 -- 172.21.15.78:0/3966782571 wait complete. 2024-05-21T15:15:36.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.051+0000 7f525a3d8740 20 client.15117 trim_cache size 0 max 16384 2024-05-21T15:15:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f525a3d8740 1 Processor -- start 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f525a3d8740 1 -- start start 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a63aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a2693b0 0x55b43a6479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a271590 0x55b43a63af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f525a3d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b43a64c850 con 0x55b43a250e40 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f525a3d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b43a64cf20 con 0x55b43a271590 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f525a3d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b43a64d5f0 con 0x55b43a2693b0 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52509f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a63aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52509f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a63aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40196/0 (socket says 172.21.15.78:40196) 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52509f8640 1 -- 172.21.15.78:0/3373339047 learned_addr learned my addr 172.21.15.78:0/3373339047 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52511f9640 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a2693b0 0x55b43a6479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52509f8640 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a271590 0x55b43a63af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52509f8640 1 -- 172.21.15.78:0/3373339047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a2693b0 msgr2=0x55b43a6479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52509f8640 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a2693b0 0x55b43a6479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52509f8640 1 -- 172.21.15.78:0/3373339047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a271590 msgr2=0x55b43a63af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.069+0000 7f52509f8640 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a271590 0x55b43a63af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f52509f8640 1 -- 172.21.15.78:0/3373339047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43a64e170 con 0x55b43a250e40 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f52511f9640 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a2693b0 0x55b43a6479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f52509f8640 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a63aa20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f52480096b0 tx=0x7f524801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523d7dd13d29ee04 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f5236ffd640 1 -- 172.21.15.78:0/3373339047 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248010820 con 0x55b43a250e40 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f5236ffd640 1 -- 172.21.15.78:0/3373339047 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5248025040 con 0x55b43a250e40 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f5236ffd640 1 -- 172.21.15.78:0/3373339047 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524802e7a0 con 0x55b43a250e40 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f525a3d8740 1 -- 172.21.15.78:0/3373339047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 msgr2=0x55b43a63aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a63aa20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f52480096b0 tx=0x7f524801b040 comp rx=0 tx=0).stop 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f525a3d8740 1 -- 172.21.15.78:0/3373339047 shutdown_connections 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.070+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a271590 0x55b43a63af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a2693b0 0x55b43a6479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3373339047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a63aa20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 -- 172.21.15.78:0/3373339047 >> 172.21.15.78:0/3373339047 conn(0x55b43a2863c0 msgr2=0x55b43a62d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 -- 172.21.15.78:0/3373339047 shutdown_connections 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 -- 172.21.15.78:0/3373339047 wait complete. 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 -- start start 2024-05-21T15:15:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 Processor -- start 2024-05-21T15:15:36.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a642e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a6433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a271590 0x55b43a6462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b43a64e390 con 0x55b43a250e40 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b43a64dcc0 con 0x55b43a2693b0 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b43a64ea60 con 0x55b43a271590 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f525a3d8740 10 client.0 _ll_register_callbacks cb 0x7f5248005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f52509f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a642e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.071+0000 7f52509f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a642e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40198/0 (socket says 172.21.15.78:40198) 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f52509f8640 1 -- 172.21.15.78:0/3133393504 learned_addr learned my addr 172.21.15.78:0/3133393504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f52511f9640 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a6433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f52509f8640 1 -- 172.21.15.78:0/3133393504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a271590 msgr2=0x55b43a6462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f52509f8640 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a271590 0x55b43a6462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f52509f8640 1 -- 172.21.15.78:0/3133393504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 msgr2=0x55b43a6433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f52509f8640 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a6433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f52509f8640 1 -- 172.21.15.78:0/3133393504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b43a646870 con 0x55b43a250e40 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f52509f8640 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a642e80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5248030000 tx=0x7f52480096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c427432f4abdd247 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f5234ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f5234ff9640 1 -- 172.21.15.78:0/3133393504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524802e520 con 0x55b43a250e40 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f5234ff9640 1 -- 172.21.15.78:0/3133393504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f524802e6c0 con 0x55b43a250e40 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f525a3d8740 1 -- 172.21.15.78:0/3133393504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5248025040 con 0x55b43a250e40 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f5234ff9640 1 -- 172.21.15.78:0/3133393504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f524802e8c0 con 0x55b43a250e40 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f525a3d8740 10 client.15123 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f525a3d8740 1 -- 172.21.15.78:0/3133393504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b43a647af0 con 0x55b43a250e40 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.072+0000 7f5234ff9640 1 -- 172.21.15.78:0/3133393504 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5248025040 con 0x55b43a250e40 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f5234ff9640 1 -- 172.21.15.78:0/3133393504 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f52480038f0 con 0x55b43a250e40 2024-05-21T15:15:36.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f525a3d8740 10 client.15123 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f525a3d8740 10 client.15123 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f525a3d8740 20 client.15123 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55b43a292270 0x55b43a6ba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f525a3d8740 4 client.15123 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f525a3d8740 1 -- 172.21.15.78:0/3133393504 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "status"}) -- 0x55b43a59df50 con 0x55b43a292270 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f52511f9640 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55b43a292270 0x55b43a6ba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f5234ff9640 10 client.15123 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.073+0000 7f52511f9640 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55b43a292270 0x55b43a6ba120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=198edfb27fc55cf7 server_cookie=9e0a8e0995be9a8c in_seq=0 out_seq=0 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f5234ff9640 1 -- 172.21.15.78:0/3133393504 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b43a59df50 con 0x55b43a292270 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f5234ff9640 10 client.15123 handle_command_reply: tid=0 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 client.15123 shutdown 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 -- 172.21.15.78:0/3133393504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 msgr2=0x55b43a642e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a642e80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5248030000 tx=0x7f52480096e0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 -- 172.21.15.78:0/3133393504 shutdown_connections 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55b43a292270 0x55b43a6ba120 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b43a2693b0 0x55b43a6433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b43a271590 0x55b43a6462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f5234ff9640 0 client.15123 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 --2- 172.21.15.78:0/3133393504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b43a250e40 0x55b43a642e80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 -- 172.21.15.78:0/3133393504 >> 172.21.15.78:0/3133393504 conn(0x55b43a2863c0 msgr2=0x55b43a62d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 -- 172.21.15.78:0/3133393504 shutdown_connections 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 1 -- 172.21.15.78:0/3133393504 wait complete. 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525a3d8740 20 client.15123 trim_cache size 0 max 16384 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:15:36.083 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24448, 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 79.809767492999995, 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 120.088542487 2024-05-21T15:15:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 -- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52380a2c20 msgr2=0x7f52380a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52380a2c20 0x7f52380a5130 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f524409b9b0 tx=0x7f5244005960 comp rx=0 tx=0).stop 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 -- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c108a00 msgr2=0x7f524c1a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c108a00 0x7f524c1a0840 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f524400cad0 tx=0x7f524400cf90 comp rx=0 tx=0).stop 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 -- 172.21.15.78:0/3189861274 shutdown_connections 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52380a2c20 0x7f52380a5130 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524c108a00 0x7f524c1a0840 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c103d00 0x7f524c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 --2- 172.21.15.78:0/3189861274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f524c102ac0 0x7f524c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.074+0000 7f525345d640 1 -- 172.21.15.78:0/3189861274 >> 172.21.15.78:0/3189861274 conn(0x7f524c0fdcf0 msgr2=0x7f524c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.075+0000 7f525345d640 1 -- 172.21.15.78:0/3189861274 shutdown_connections 2024-05-21T15:15:36.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.075+0000 7f525345d640 1 -- 172.21.15.78:0/3189861274 wait complete. 2024-05-21T15:15:36.088 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-21T15:15:36.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.196+0000 7fe684c1b640 1 -- 172.21.15.78:0/242304684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 msgr2=0x7fe6801037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.196+0000 7fe684c1b640 1 --2- 172.21.15.78:0/242304684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 0x7fe6801037b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe670009900 tx=0x7fe67002a410 comp rx=0 tx=0).stop 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.196+0000 7fe684c1b640 1 -- 172.21.15.78:0/242304684 shutdown_connections 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.196+0000 7fe684c1b640 1 --2- 172.21.15.78:0/242304684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe680103cf0 0x7fe680104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.196+0000 7fe684c1b640 1 --2- 172.21.15.78:0/242304684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6801089f0 0x7fe68010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.196+0000 7fe684c1b640 1 --2- 172.21.15.78:0/242304684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 0x7fe6801037b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.196+0000 7fe684c1b640 1 -- 172.21.15.78:0/242304684 >> 172.21.15.78:0/242304684 conn(0x7fe6800fdcc0 msgr2=0x7fe680100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.196+0000 7fe684c1b640 1 -- 172.21.15.78:0/242304684 shutdown_connections 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.197+0000 7fe684c1b640 1 -- 172.21.15.78:0/242304684 wait complete. 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.197+0000 7fe684c1b640 1 Processor -- start 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.197+0000 7fe684c1b640 1 -- start start 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.197+0000 7fe684c1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 0x7fe680199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.197+0000 7fe684c1b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe680103cf0 0x7fe680199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.197+0000 7fe684c1b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6801089f0 0x7fe6801a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe684c1b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe68010e6e0 con 0x7fe680102ab0 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe684c1b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe68010e560 con 0x7fe680103cf0 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe684c1b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe68010e860 con 0x7fe6801089f0 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 0x7fe680199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 0x7fe680199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40204/0 (socket says 172.21.15.78:40204) 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 -- 172.21.15.78:0/48829087 learned_addr learned my addr 172.21.15.78:0/48829087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6801089f0 0x7fe6801a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe677fff640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe680103cf0 0x7fe680199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 -- 172.21.15.78:0/48829087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6801089f0 msgr2=0x7fe6801a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6801089f0 0x7fe6801a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 -- 172.21.15.78:0/48829087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe680103cf0 msgr2=0x7fe680199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe680103cf0 0x7fe680199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 -- 172.21.15.78:0/48829087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6801a0e80 con 0x7fe680102ab0 2024-05-21T15:15:36.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe67e59c640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 0x7fe680199340 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe670006010 tx=0x7fe6700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe6777fe640 1 -- 172.21.15.78:0/48829087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67003f030 con 0x7fe680102ab0 2024-05-21T15:15:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe684c1b640 1 -- 172.21.15.78:0/48829087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6801a1110 con 0x7fe680102ab0 2024-05-21T15:15:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe6777fe640 1 -- 172.21.15.78:0/48829087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe670031a20 con 0x7fe680102ab0 2024-05-21T15:15:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe6777fe640 1 -- 172.21.15.78:0/48829087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670035070 con 0x7fe680102ab0 2024-05-21T15:15:36.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.198+0000 7fe684c1b640 1 -- 172.21.15.78:0/48829087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6801a1600 con 0x7fe680102ab0 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe6777fe640 1 -- 172.21.15.78:0/48829087 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe670042030 con 0x7fe680102ab0 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe68bb9e740 1 Processor -- start 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe68bb9e740 1 -- start start 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471239270 0x5594715dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715dde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471218590 0x5594715e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55947153d4d0 con 0x559471218590 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5594714e53e0 con 0x559471239270 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559471539490 con 0x5594712103b0 2024-05-21T15:15:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471218590 0x5594715e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471218590 0x5594715e4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40206/0 (socket says 172.21.15.78:40206) 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.201+0000 7fe677fff640 1 -- 172.21.15.78:0/2604842501 learned_addr learned my addr 172.21.15.78:0/2604842501 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471239270 0x5594715dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe67e59c640 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715dde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe6777fe640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe66c0a2c20 0x7fe66c0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe66c0a2c20 0x7fe66c0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe677fff640 1 -- 172.21.15.78:0/2604842501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 msgr2=0x5594715dde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe677fff640 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715dde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe677fff640 1 -- 172.21.15.78:0/2604842501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471239270 msgr2=0x5594715dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe677fff640 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471239270 0x5594715dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe677fff640 1 -- 172.21.15.78:0/2604842501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594712512d0 con 0x559471218590 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471239270 0x5594715dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe67e59c640 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:36.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe6777fe640 1 -- 172.21.15.78:0/48829087 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe6700ef5b0 con 0x7fe680102ab0 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe677fff640 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471218590 0x5594715e4d00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe668009c90 tx=0x7fe668011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dabf2acf10a38f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe66c0a2c20 0x7fe66c0a5130 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5594715dea50 tx=0x7fe678004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe674ff9640 1 -- 172.21.15.78:0/2604842501 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668005220 con 0x559471218590 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe674ff9640 1 -- 172.21.15.78:0/2604842501 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe668004c80 con 0x559471218590 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.202+0000 7fe674ff9640 1 -- 172.21.15.78:0/2604842501 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668019710 con 0x559471218590 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2604842501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471218590 msgr2=0x5594715e4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471218590 0x5594715e4d00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe668009c90 tx=0x7fe668011810 comp rx=0 tx=0).stop 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2604842501 shutdown_connections 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471218590 0x5594715e4d00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715dde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2604842501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471239270 0x5594715dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2604842501 >> 172.21.15.78:0/2604842501 conn(0x55947122d3c0 msgr2=0x5594715d4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2604842501 shutdown_connections 2024-05-21T15:15:36.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.203+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2604842501 wait complete. 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 1 -- start start 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 1 Processor -- start 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471218590 0x5594715e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471239270 0x5594715eeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594715e5750 con 0x559471239270 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5594714f88e0 con 0x559471218590 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594715e58d0 con 0x5594712103b0 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe68bb9e740 10 client.0 _ll_register_callbacks cb 0x5594714c59c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471239270 0x5594715eeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471239270 0x5594715eeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40210/0 (socket says 172.21.15.78:40210) 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67ed9d640 1 -- 172.21.15.78:0/2508598699 learned_addr learned my addr 172.21.15.78:0/2508598699 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe677fff640 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471218590 0x5594715e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67e59c640 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67ed9d640 1 -- 172.21.15.78:0/2508598699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 msgr2=0x5594715e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67ed9d640 1 -- 172.21.15.78:0/2508598699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471218590 msgr2=0x5594715e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471218590 0x5594715e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.204+0000 7fe67ed9d640 1 -- 172.21.15.78:0/2508598699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947131b6b0 con 0x559471239270 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471239270 0x5594715eeb50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe678009bf0 tx=0x7fe678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16607194d2006cad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe665ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe665ffb640 1 -- 172.21.15.78:0/2508598699 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780129d0 con 0x559471239270 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2508598699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe668004c80 con 0x559471239270 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe665ffb640 1 -- 172.21.15.78:0/2508598699 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe678012b70 con 0x559471239270 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe665ffb640 1 -- 172.21.15.78:0/2508598699 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe678012d70 con 0x559471239270 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 10 client.15135 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2508598699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5594715f1b50 con 0x559471239270 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe665ffb640 1 -- 172.21.15.78:0/2508598699 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe668004c80 con 0x559471239270 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe665ffb640 1 -- 172.21.15.78:0/2508598699 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe67802a490 con 0x559471239270 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 10 client.15135 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 10 client.15135 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 20 client.15135 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594711f7e40 0x5594715f5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 4 client.15135 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.205+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2508598699 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55947154ac00 con 0x5594711f7e40 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.206+0000 7fe67e59c640 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594711f7e40 0x5594715f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.206+0000 7fe665ffb640 10 client.15135 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.206+0000 7fe67e59c640 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594711f7e40 0x5594715f5700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a89d34af2f73406 server_cookie=40e750d92140437 in_seq=0 out_seq=0 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe665ffb640 1 -- 172.21.15.78:0/2508598699 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55947154ac00 con 0x5594711f7e40 2024-05-21T15:15:36.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe665ffb640 10 client.15135 handle_command_reply: tid=0 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 client.15135 shutdown 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2508598699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471239270 msgr2=0x5594715eeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471239270 0x5594715eeb50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe678009bf0 tx=0x7fe678004a40 comp rx=0 tx=0).stop 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2508598699 shutdown_connections 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594711f7e40 0x5594715f5700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559471218590 0x5594715e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559471239270 0x5594715eeb50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe665ffb640 0 client.15135 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/2508598699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594712103b0 0x5594715e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2508598699 >> 172.21.15.78:0/2508598699 conn(0x55947122d3c0 msgr2=0x5594715d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2508598699 shutdown_connections 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 1 -- 172.21.15.78:0/2508598699 wait complete. 2024-05-21T15:15:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.207+0000 7fe68bb9e740 20 client.15135 trim_cache size 0 max 16384 2024-05-21T15:15:36.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.225+0000 7fe68bb9e740 1 Processor -- start 2024-05-21T15:15:36.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.225+0000 7fe68bb9e740 1 -- start start 2024-05-21T15:15:36.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.225+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 0x5594715e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 0x5594715e2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594715f3a40 con 0x5594712103b0 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5594715f4110 con 0x5594711f7e40 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594715f47e0 con 0x559471218590 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715eea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40212/0 (socket says 172.21.15.78:40212) 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 -- 172.21.15.78:0/4089405085 learned_addr learned my addr 172.21.15.78:0/4089405085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe67e59c640 1 --2- 172.21.15.78:0/4089405085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 0x5594715e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 -- 172.21.15.78:0/4089405085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 msgr2=0x5594715e2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 --2- 172.21.15.78:0/4089405085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 0x5594715e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 -- 172.21.15.78:0/4089405085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 msgr2=0x5594715e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 --2- 172.21.15.78:0/4089405085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 0x5594715e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 -- 172.21.15.78:0/4089405085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594715e82b0 con 0x5594712103b0 2024-05-21T15:15:36.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe677fff640 1 --2- 172.21.15.78:0/4089405085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715eea70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe668009be0 tx=0x7fe6680099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654ccad393ff927 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe667fff640 1 -- 172.21.15.78:0/4089405085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668028070 con 0x5594712103b0 2024-05-21T15:15:36.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe667fff640 1 -- 172.21.15.78:0/4089405085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe668005200 con 0x5594712103b0 2024-05-21T15:15:36.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.226+0000 7fe667fff640 1 -- 172.21.15.78:0/4089405085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66802d6c0 con 0x5594712103b0 2024-05-21T15:15:36.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 -- 172.21.15.78:0/4089405085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 msgr2=0x5594715eea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/4089405085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715eea70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe668009be0 tx=0x7fe6680099e0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 -- 172.21.15.78:0/4089405085 shutdown_connections 2024-05-21T15:15:36.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/4089405085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 0x5594715e2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/4089405085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715eea70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/4089405085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 0x5594715e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 -- 172.21.15.78:0/4089405085 >> 172.21.15.78:0/4089405085 conn(0x55947122d3c0 msgr2=0x5594715e0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 -- 172.21.15.78:0/4089405085 shutdown_connections 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 -- 172.21.15.78:0/4089405085 wait complete. 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 -- start start 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.227+0000 7fe68bb9e740 1 Processor -- start 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 0x5594715e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715ea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 0x5594715ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594715f5580 con 0x5594712103b0 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5594715f4eb0 con 0x5594711f7e40 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594715f5c50 con 0x559471218590 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 10 client.0 _ll_register_callbacks cb 0x7fe678009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715ea110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40220/0 (socket says 172.21.15.78:40220) 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 -- 172.21.15.78:0/3324723421 learned_addr learned my addr 172.21.15.78:0/3324723421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe67e59c640 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 0x5594715e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe67ed9d640 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 0x5594715ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 -- 172.21.15.78:0/3324723421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 msgr2=0x5594715ed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 0x5594715ed050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 -- 172.21.15.78:0/3324723421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 msgr2=0x5594715e9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 0x5594715e9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 -- 172.21.15.78:0/3324723421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594715ed610 con 0x5594712103b0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe677fff640 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715ea110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe668000c00 tx=0x7fe668005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afff8a18a38b364d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe665ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe665ffb640 1 -- 172.21.15.78:0/3324723421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66803b060 con 0x5594712103b0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe665ffb640 1 -- 172.21.15.78:0/3324723421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe66802d670 con 0x5594712103b0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 1 -- 172.21.15.78:0/3324723421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe668005200 con 0x5594712103b0 2024-05-21T15:15:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe665ffb640 1 -- 172.21.15.78:0/3324723421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe668028040 con 0x5594712103b0 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 10 client.15141 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.228+0000 7fe68bb9e740 1 -- 172.21.15.78:0/3324723421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5594715ee890 con 0x5594712103b0 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe665ffb640 1 -- 172.21.15.78:0/3324723421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe668005200 con 0x5594712103b0 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe665ffb640 1 -- 172.21.15.78:0/3324723421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe668019550 con 0x5594712103b0 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe68bb9e740 10 client.15141 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe68bb9e740 10 client.15141 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe68bb9e740 20 client.15141 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x559471239270 0x559471661060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe68bb9e740 4 client.15141 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe68bb9e740 1 -- 172.21.15.78:0/3324723421 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "status"}) -- 0x559471544f50 con 0x559471239270 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.229+0000 7fe67e59c640 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x559471239270 0x559471661060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe665ffb640 10 client.15141 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe67e59c640 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x559471239270 0x559471661060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e245ad758feafeb3 server_cookie=56f94acb35572123 in_seq=0 out_seq=0 2024-05-21T15:15:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe665ffb640 1 -- 172.21.15.78:0/3324723421 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559471544f50 con 0x559471239270 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe665ffb640 10 client.15141 handle_command_reply: tid=0 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 client.15141 shutdown 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 -- 172.21.15.78:0/3324723421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 msgr2=0x5594715ea110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715ea110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe668000c00 tx=0x7fe668005d30 comp rx=0 tx=0).stop 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 -- 172.21.15.78:0/3324723421 shutdown_connections 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x559471239270 0x559471661060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559471218590 0x5594715ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594712103b0 0x5594715ea110 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 --2- 172.21.15.78:0/3324723421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594711f7e40 0x5594715e9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe665ffb640 0 client.15141 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 -- 172.21.15.78:0/3324723421 >> 172.21.15.78:0/3324723421 conn(0x55947122d3c0 msgr2=0x5594715e1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 -- 172.21.15.78:0/3324723421 shutdown_connections 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 1 -- 172.21.15.78:0/3324723421 wait complete. 2024-05-21T15:15:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.230+0000 7fe68bb9e740 20 client.15141 trim_cache size 0 max 16384 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24502, 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 80.652979264999999, 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 103.489472342 2024-05-21T15:15:36.240 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:15:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 -- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe66c0a2c20 msgr2=0x7fe66c0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe66c0a2c20 0x7fe66c0a5130 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5594715dea50 tx=0x7fe678004ab0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 -- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 msgr2=0x7fe680199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 0x7fe680199340 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe670006010 tx=0x7fe6700042f0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 -- 172.21.15.78:0/48829087 shutdown_connections 2024-05-21T15:15:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe66c0a2c20 0x7fe66c0a5130 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe680103cf0 0x7fe680199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6801089f0 0x7fe6801a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 --2- 172.21.15.78:0/48829087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe680102ab0 0x7fe680199340 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 -- 172.21.15.78:0/48829087 >> 172.21.15.78:0/48829087 conn(0x7fe6800fdcc0 msgr2=0x7fe6801049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 -- 172.21.15.78:0/48829087 shutdown_connections 2024-05-21T15:15:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.231+0000 7fe684c1b640 1 -- 172.21.15.78:0/48829087 wait complete. 2024-05-21T15:15:36.245 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-21T15:15:36.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 -- 172.21.15.78:0/3193689544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 msgr2=0x7fc07c104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 --2- 172.21.15.78:0/3193689544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 0x7fc07c104470 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc070009900 tx=0x7fc07002a410 comp rx=0 tx=0).stop 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 -- 172.21.15.78:0/3193689544 shutdown_connections 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 --2- 172.21.15.78:0/3193689544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101210 0x7fc07c10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 --2- 172.21.15.78:0/3193689544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc07c100710 0x7fc07c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 --2- 172.21.15.78:0/3193689544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 0x7fc07c104470 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 -- 172.21.15.78:0/3193689544 >> 172.21.15.78:0/3193689544 conn(0x7fc07c0fc330 msgr2=0x7fc07c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 -- 172.21.15.78:0/3193689544 shutdown_connections 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 -- 172.21.15.78:0/3193689544 wait complete. 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.354+0000 7fc0830bc640 1 Processor -- start 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc0830bc640 1 -- start start 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc0830bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc07c100710 0x7fc07c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc0830bc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101210 0x7fc07c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc0830bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 0x7fc07c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc0830bc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c10e720 con 0x7fc07c104060 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc0830bc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c10e5a0 con 0x7fc07c101210 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc0830bc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c10e8a0 con 0x7fc07c100710 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 0x7fc07c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 0x7fc07c1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40250/0 (socket says 172.21.15.78:40250) 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 -- 172.21.15.78:0/2902102572 learned_addr learned my addr 172.21.15.78:0/2902102572 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc080e58640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc07c100710 0x7fc07c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc07bfff640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101210 0x7fc07c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 -- 172.21.15.78:0/2902102572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc07c100710 msgr2=0x7fc07c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc07c100710 0x7fc07c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 -- 172.21.15.78:0/2902102572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101210 msgr2=0x7fc07c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101210 0x7fc07c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 -- 172.21.15.78:0/2902102572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c1a0e00 con 0x7fc07c104060 2024-05-21T15:15:36.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc081659640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 0x7fc07c1a0790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc07400eb10 tx=0x7fc07400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc079ffb640 1 -- 172.21.15.78:0/2902102572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074004b20 con 0x7fc07c104060 2024-05-21T15:15:36.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc079ffb640 1 -- 172.21.15.78:0/2902102572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc074004cc0 con 0x7fc07c104060 2024-05-21T15:15:36.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.355+0000 7fc079ffb640 1 -- 172.21.15.78:0/2902102572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074010760 con 0x7fc07c104060 2024-05-21T15:15:36.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.356+0000 7fc0830bc640 1 -- 172.21.15.78:0/2902102572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07c1a10f0 con 0x7fc07c104060 2024-05-21T15:15:36.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.356+0000 7fc0830bc640 1 -- 172.21.15.78:0/2902102572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07c1a15f0 con 0x7fc07c104060 2024-05-21T15:15:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc08a036740 1 Processor -- start 2024-05-21T15:15:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc079ffb640 1 -- 172.21.15.78:0/2902102572 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc074005020 con 0x7fc07c104060 2024-05-21T15:15:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc08a036740 1 -- start start 2024-05-21T15:15:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc08a036740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbbae270 0x55e2cbf4e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc08a036740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc08a036740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc08a036740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbeb24d0 con 0x55e2cbb853b0 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc08a036740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbe5a3e0 con 0x55e2cbbae270 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.358+0000 7fc08a036740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbeae490 con 0x55e2cbb8d590 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40256/0 (socket says 172.21.15.78:40256) 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 -- 172.21.15.78:0/2839686655 learned_addr learned my addr 172.21.15.78:0/2839686655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc07bfff640 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc07bfff640 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbbae270 0x55e2cbf4e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 -- 172.21.15.78:0/2839686655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 msgr2=0x55e2cbf558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 -- 172.21.15.78:0/2839686655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbbae270 msgr2=0x55e2cbf4e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbbae270 0x55e2cbf4e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 -- 172.21.15.78:0/2839686655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2cbbc62d0 con 0x55e2cbb853b0 2024-05-21T15:15:36.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc079ffb640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc06c0a2c20 0x7fc06c0a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc080e58640 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc070009230 tx=0x7fc070032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ea5e559c086d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc07bfff640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc06c0a2c20 0x7fc06c0a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc066ffd640 1 -- 172.21.15.78:0/2839686655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070040070 con 0x55e2cbb853b0 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc079ffb640 1 -- 172.21.15.78:0/2902102572 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc0740cf7a0 con 0x7fc07c104060 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc066ffd640 1 -- 172.21.15.78:0/2839686655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc070032e20 con 0x55e2cbb853b0 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.359+0000 7fc066ffd640 1 -- 172.21.15.78:0/2839686655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070035040 con 0x55e2cbb853b0 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 -- 172.21.15.78:0/2839686655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbb853b0 msgr2=0x55e2cbf4e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc07bfff640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc06c0a2c20 0x7fc06c0a5130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55e2cbf4f640 tx=0x7fc06800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc070009230 tx=0x7fc070032c30 comp rx=0 tx=0).stop 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 -- 172.21.15.78:0/2839686655 shutdown_connections 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e9f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 --2- 172.21.15.78:0/2839686655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbbae270 0x55e2cbf4e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 -- 172.21.15.78:0/2839686655 >> 172.21.15.78:0/2839686655 conn(0x55e2cbba23c0 msgr2=0x55e2cbf49e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 -- 172.21.15.78:0/2839686655 shutdown_connections 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.360+0000 7fc08a036740 1 -- 172.21.15.78:0/2839686655 wait complete. 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 1 -- start start 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 1 Processor -- start 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb853b0 0x55e2cbf58240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb8d590 0x55e2cbf58780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf5f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbf562e0 con 0x55e2cbbae270 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbe6d8e0 con 0x55e2cbb8d590 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbf56460 con 0x55e2cbb853b0 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc08a036740 10 client.0 _ll_register_callbacks cb 0x55e2cbe3a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:36.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc081659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf5f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc081659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf5f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40272/0 (socket says 172.21.15.78:40272) 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc081659640 1 -- 172.21.15.78:0/3473329750 learned_addr learned my addr 172.21.15.78:0/3473329750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc080e58640 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb853b0 0x55e2cbf58240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc081659640 1 -- 172.21.15.78:0/3473329750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb853b0 msgr2=0x55e2cbf58240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc081659640 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb853b0 0x55e2cbf58240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc081659640 1 -- 172.21.15.78:0/3473329750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb8d590 msgr2=0x55e2cbf58780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc081659640 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb8d590 0x55e2cbf58780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.361+0000 7fc081659640 1 -- 172.21.15.78:0/3473329750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2cbc906b0 con 0x55e2cbbae270 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc081659640 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf5f6e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc0740a45c0 tx=0x7fc07409b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875f4dc5e65b23a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc05bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc05bfff640 1 -- 172.21.15.78:0/3473329750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0740a4860 con 0x55e2cbbae270 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc05bfff640 1 -- 172.21.15.78:0/3473329750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc074005310 con 0x55e2cbbae270 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 1 -- 172.21.15.78:0/3473329750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc070032e20 con 0x55e2cbbae270 2024-05-21T15:15:36.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc05bfff640 1 -- 172.21.15.78:0/3473329750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0740b9030 con 0x55e2cbbae270 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 10 client.15153 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 1 -- 172.21.15.78:0/3473329750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2cbf62680 con 0x55e2cbbae270 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc05bfff640 1 -- 172.21.15.78:0/3473329750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc070032e20 con 0x55e2cbbae270 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc05bfff640 1 -- 172.21.15.78:0/3473329750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc07409b3f0 con 0x55e2cbbae270 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 10 client.15153 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 10 client.15153 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 20 client.15153 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e2cbb6ce40 0x55e2cbf66220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 4 client.15153 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.362+0000 7fc08a036740 1 -- 172.21.15.78:0/3473329750 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2cbebfc00 con 0x55e2cbb6ce40 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.363+0000 7fc080e58640 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e2cbb6ce40 0x55e2cbf66220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.363+0000 7fc080e58640 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e2cbb6ce40 0x55e2cbf66220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a85d47ffe6d189e4 server_cookie=47acb5ef19073434 in_seq=0 out_seq=0 2024-05-21T15:15:36.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.363+0000 7fc05bfff640 10 client.15153 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:15:36.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc05bfff640 1 -- 172.21.15.78:0/3473329750 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2cbebfc00 con 0x55e2cbb6ce40 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc05bfff640 10 client.15153 handle_command_reply: tid=0 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 client.15153 shutdown 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 -- 172.21.15.78:0/3473329750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 msgr2=0x55e2cbf5f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf5f6e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc0740a45c0 tx=0x7fc07409b7e0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 -- 172.21.15.78:0/3473329750 shutdown_connections 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e2cbb6ce40 0x55e2cbf66220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb8d590 0x55e2cbf58780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc05bfff640 0 client.15153 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf5f6e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 --2- 172.21.15.78:0/3473329750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb853b0 0x55e2cbf58240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 -- 172.21.15.78:0/3473329750 >> 172.21.15.78:0/3473329750 conn(0x55e2cbba23c0 msgr2=0x55e2cbf48ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 -- 172.21.15.78:0/3473329750 shutdown_connections 2024-05-21T15:15:36.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 1 -- 172.21.15.78:0/3473329750 wait complete. 2024-05-21T15:15:36.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.364+0000 7fc08a036740 20 client.15153 trim_cache size 0 max 16384 2024-05-21T15:15:36.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.382+0000 7fc08a036740 1 Processor -- start 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc08a036740 1 -- start start 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc08a036740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc08a036740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc08a036740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf49d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc08a036740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbf64560 con 0x55e2cbbae270 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc08a036740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbf64c30 con 0x55e2cbb853b0 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc08a036740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbf65300 con 0x55e2cbb8d590 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf49d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf49d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40286/0 (socket says 172.21.15.78:40286) 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 -- 172.21.15.78:0/2157629592 learned_addr learned my addr 172.21.15.78:0/2157629592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc080e58640 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc07bfff640 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 -- 172.21.15.78:0/2157629592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 msgr2=0x55e2cbf497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 -- 172.21.15.78:0/2157629592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 msgr2=0x55e2cbf4e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 -- 172.21.15.78:0/2157629592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2cbf4e020 con 0x55e2cbbae270 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc07bfff640 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc080e58640 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc081659640 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf49d30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc07409bb80 tx=0x7fc07409b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7106beef220bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc0667fc640 1 -- 172.21.15.78:0/2157629592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0740a0230 con 0x55e2cbbae270 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc0667fc640 1 -- 172.21.15.78:0/2157629592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0740a4af0 con 0x55e2cbbae270 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.383+0000 7fc0667fc640 1 -- 172.21.15.78:0/2157629592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07409b530 con 0x55e2cbbae270 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 -- 172.21.15.78:0/2157629592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 msgr2=0x55e2cbf49d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf49d30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc07409bb80 tx=0x7fc07409b910 comp rx=0 tx=0).stop 2024-05-21T15:15:36.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 -- 172.21.15.78:0/2157629592 shutdown_connections 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbf49d30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 --2- 172.21.15.78:0/2157629592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 0x55e2cbf4e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 -- 172.21.15.78:0/2157629592 >> 172.21.15.78:0/2157629592 conn(0x55e2cbba23c0 msgr2=0x55e2cbf58200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 -- 172.21.15.78:0/2157629592 shutdown_connections 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.384+0000 7fc08a036740 1 -- 172.21.15.78:0/2157629592 wait complete. 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 1 -- start start 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 1 Processor -- start 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 0x55e2cbf59b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf5a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbfcd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbf660a0 con 0x55e2cbbae270 2024-05-21T15:15:36.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbf659d0 con 0x55e2cbb853b0 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2cbf66770 con 0x55e2cbb8d590 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc08a036740 10 client.0 _ll_register_callbacks cb 0x7fc0740057d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc081659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbfcd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc081659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbfcd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40294/0 (socket says 172.21.15.78:40294) 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc081659640 1 -- 172.21.15.78:0/3140101228 learned_addr learned my addr 172.21.15.78:0/3140101228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc07bfff640 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf5a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc081659640 1 -- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 msgr2=0x55e2cbf5a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc081659640 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf5a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc081659640 1 -- 172.21.15.78:0/3140101228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 msgr2=0x55e2cbf59b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc081659640 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 0x55e2cbf59b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc081659640 1 -- 172.21.15.78:0/3140101228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2cbf5a710 con 0x55e2cbbae270 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.385+0000 7fc07bfff640 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf5a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:36.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc081659640 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbfcd090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc074098850 tx=0x7fc07400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855e27c2bb925a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc05bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc05bfff640 1 -- 172.21.15.78:0/3140101228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0740a0230 con 0x55e2cbbae270 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc05bfff640 1 -- 172.21.15.78:0/3140101228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc074085170 con 0x55e2cbbae270 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 1 -- 172.21.15.78:0/3140101228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0740a4af0 con 0x55e2cbbae270 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc05bfff640 1 -- 172.21.15.78:0/3140101228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc07409b2d0 con 0x55e2cbbae270 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 10 client.15159 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 1 -- 172.21.15.78:0/3140101228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2cbfce5a0 con 0x55e2cbbae270 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc05bfff640 1 -- 172.21.15.78:0/3140101228 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0740a4af0 con 0x55e2cbbae270 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc05bfff640 1 -- 172.21.15.78:0/3140101228 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc074081070 con 0x55e2cbbae270 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 10 client.15159 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 10 client.15159 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 20 client.15159 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e2cbb42280 0x55e2cbfd1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 4 client.15159 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc08a036740 1 -- 172.21.15.78:0/3140101228 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "status"}) -- 0x55e2cbeb9f50 con 0x55e2cbb42280 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.386+0000 7fc080e58640 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e2cbb42280 0x55e2cbfd1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc05bfff640 10 client.15159 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc080e58640 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e2cbb42280 0x55e2cbfd1a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd884a18c753a9f4 server_cookie=31138772adf5743c in_seq=0 out_seq=0 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc05bfff640 1 -- 172.21.15.78:0/3140101228 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e2cbeb9f50 con 0x55e2cbb42280 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc05bfff640 10 client.15159 handle_command_reply: tid=0 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 client.15159 shutdown 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 -- 172.21.15.78:0/3140101228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 msgr2=0x55e2cbfcd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbfcd090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc074098850 tx=0x7fc07400efd0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 -- 172.21.15.78:0/3140101228 shutdown_connections 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e2cbb42280 0x55e2cbfd1a90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2cbbae270 0x55e2cbfcd090 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2cbb8d590 0x55e2cbf5a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc05bfff640 0 client.15159 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 --2- 172.21.15.78:0/3140101228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e2cbb853b0 0x55e2cbf59b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 -- 172.21.15.78:0/3140101228 >> 172.21.15.78:0/3140101228 conn(0x55e2cbba23c0 msgr2=0x55e2cbf497f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 -- 172.21.15.78:0/3140101228 shutdown_connections 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.387+0000 7fc08a036740 1 -- 172.21.15.78:0/3140101228 wait complete. 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc08a036740 20 client.15159 trim_cache size 0 max 16384 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:15:36.398 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24526, 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 81.824884975000003, 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 98.051717518000004 2024-05-21T15:15:36.399 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:15:36.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 -- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc06c0a2c20 msgr2=0x7fc06c0a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc06c0a2c20 0x7fc06c0a5130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55e2cbf4f640 tx=0x7fc06800b390 comp rx=0 tx=0).stop 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 -- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 msgr2=0x7fc07c1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 0x7fc07c1a0790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc07400eb10 tx=0x7fc07400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 -- 172.21.15.78:0/2902102572 shutdown_connections 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc06c0a2c20 0x7fc06c0a5130 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101210 0x7fc07c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc07c104060 0x7fc07c1a0790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 --2- 172.21.15.78:0/2902102572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc07c100710 0x7fc07c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 -- 172.21.15.78:0/2902102572 >> 172.21.15.78:0/2902102572 conn(0x7fc07c0fc330 msgr2=0x7fc07c104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 -- 172.21.15.78:0/2902102572 shutdown_connections 2024-05-21T15:15:36.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.388+0000 7fc0830bc640 1 -- 172.21.15.78:0/2902102572 wait complete. 2024-05-21T15:15:36.402 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-21T15:15:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 -- 172.21.15.78:0/3688685915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968103cf0 msgr2=0x7f7968104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 --2- 172.21.15.78:0/3688685915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968103cf0 0x7f7968104100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f795002a410 comp rx=0 tx=0).stop 2024-05-21T15:15:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 -- 172.21.15.78:0/3688685915 shutdown_connections 2024-05-21T15:15:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 --2- 172.21.15.78:0/3688685915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79681089f0 0x7f796810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 --2- 172.21.15.78:0/3688685915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968103cf0 0x7f7968104100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 --2- 172.21.15.78:0/3688685915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7968102ab0 0x7f79681037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 -- 172.21.15.78:0/3688685915 >> 172.21.15.78:0/3688685915 conn(0x7f79680fdca0 msgr2=0x7f7968100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 -- 172.21.15.78:0/3688685915 shutdown_connections 2024-05-21T15:15:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 -- 172.21.15.78:0/3688685915 wait complete. 2024-05-21T15:15:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.511+0000 7f796d4c3640 1 Processor -- start 2024-05-21T15:15:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f796d4c3640 1 -- start start 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f796d4c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968102ab0 0x7f79681992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f796d4c3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7968103cf0 0x7f79681997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f796d4c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79681089f0 0x7f79681a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f796d4c3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f796810e700 con 0x7f7968102ab0 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f796d4c3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f796810e580 con 0x7f79681089f0 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f796d4c3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f796810e880 con 0x7f7968103cf0 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968102ab0 0x7f79681992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968102ab0 0x7f79681992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40310/0 (socket says 172.21.15.78:40310) 2024-05-21T15:15:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 -- 172.21.15.78:0/28003160 learned_addr learned my addr 172.21.15.78:0/28003160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f79677fe640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79681089f0 0x7f79681a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 -- 172.21.15.78:0/28003160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7968103cf0 msgr2=0x7f79681997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f79667fc640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7968103cf0 0x7f79681997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7968103cf0 0x7f79681997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 -- 172.21.15.78:0/28003160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79681089f0 msgr2=0x7f79681a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79681089f0 0x7f79681a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 -- 172.21.15.78:0/28003160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79681a0d30 con 0x7f7968102ab0 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f79677fe640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79681089f0 0x7f79681a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.512+0000 7f7966ffd640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968102ab0 0x7f79681992a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f795800c970 tx=0x7f795800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.513+0000 7f795ffff640 1 -- 172.21.15.78:0/28003160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958007c70 con 0x7f7968102ab0 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.513+0000 7f795ffff640 1 -- 172.21.15.78:0/28003160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7958007e10 con 0x7f7968102ab0 2024-05-21T15:15:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.513+0000 7f796d4c3640 1 -- 172.21.15.78:0/28003160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79681a0fc0 con 0x7f7968102ab0 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.513+0000 7f795ffff640 1 -- 172.21.15.78:0/28003160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580188c0 con 0x7f7968102ab0 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.513+0000 7f796d4c3640 1 -- 172.21.15.78:0/28003160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79681a1480 con 0x7f7968102ab0 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.515+0000 7f795ffff640 1 -- 172.21.15.78:0/28003160 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7958018a60 con 0x7f7968102ab0 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.515+0000 7f7974367740 1 Processor -- start 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.515+0000 7f7974367740 1 -- start start 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7974367740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e08880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7974367740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a3b3b0 0x5619a5e08e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7974367740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a43590 0x5619a5e0fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7974367740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619a5d684d0 con 0x5619a5a3b3b0 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7974367740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5619a5d103e0 con 0x5619a5a64270 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7974367740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619a5d64490 con 0x5619a5a43590 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7966ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a3b3b0 0x5619a5e08e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7966ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a3b3b0 0x5619a5e08e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40314/0 (socket says 172.21.15.78:40314) 2024-05-21T15:15:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7966ffd640 1 -- 172.21.15.78:0/2503923750 learned_addr learned my addr 172.21.15.78:0/2503923750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f79677fe640 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e08880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f79667fc640 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a43590 0x5619a5e0fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f795ffff640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79540a2c20 0x7f79540a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7966ffd640 1 -- 172.21.15.78:0/2503923750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a43590 msgr2=0x5619a5e0fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7966ffd640 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a43590 0x5619a5e0fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7966ffd640 1 -- 172.21.15.78:0/2503923750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 msgr2=0x5619a5e08880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7966ffd640 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e08880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f7966ffd640 1 -- 172.21.15.78:0/2503923750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a5a7c2d0 con 0x5619a5a3b3b0 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f79677fe640 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e08880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f79677fe640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79540a2c20 0x7f79540a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.516+0000 7f795ffff640 1 -- 172.21.15.78:0/28003160 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f79580cf2f0 con 0x7f7968102ab0 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.517+0000 7f7966ffd640 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a3b3b0 0x5619a5e08e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f79580ccbc0 tx=0x7f7958066650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93d4bacc973e026 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.517+0000 7f795d7fa640 1 -- 172.21.15.78:0/2503923750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795809b510 con 0x5619a5a3b3b0 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.517+0000 7f795d7fa640 1 -- 172.21.15.78:0/2503923750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7958070280 con 0x5619a5a3b3b0 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.517+0000 7f795d7fa640 1 -- 172.21.15.78:0/2503923750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958070470 con 0x5619a5a3b3b0 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- 172.21.15.78:0/2503923750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a3b3b0 msgr2=0x5619a5e08e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f79677fe640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79540a2c20 0x7f79540a5130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7960009fd0 tx=0x7f7960004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a3b3b0 0x5619a5e08e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f79580ccbc0 tx=0x7f7958066650 comp rx=0 tx=0).stop 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- 172.21.15.78:0/2503923750 shutdown_connections 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a43590 0x5619a5e0fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a3b3b0 0x5619a5e08e00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 --2- 172.21.15.78:0/2503923750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e08880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- 172.21.15.78:0/2503923750 >> 172.21.15.78:0/2503923750 conn(0x5619a5a583c0 msgr2=0x5619a5dffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- 172.21.15.78:0/2503923750 shutdown_connections 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- 172.21.15.78:0/2503923750 wait complete. 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- start start 2024-05-21T15:15:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 Processor -- start 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e12620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e12b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e19ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619a5e106f0 con 0x5619a5a43590 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5619a5d238e0 con 0x5619a5a3b3b0 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619a5e10870 con 0x5619a5a64270 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.518+0000 7f7974367740 10 client.0 _ll_register_callbacks cb 0x5619a5cf09c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e12b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e12b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40322/0 (socket says 172.21.15.78:40322) 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 -- 172.21.15.78:0/2871000740 learned_addr learned my addr 172.21.15.78:0/2871000740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79677fe640 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e19ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f7966ffd640 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e12620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 -- 172.21.15.78:0/2871000740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 msgr2=0x5619a5e19ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e19ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 -- 172.21.15.78:0/2871000740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 msgr2=0x5619a5e12620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e12620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 -- 172.21.15.78:0/2871000740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a5b466b0 con 0x5619a5a43590 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79677fe640 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e19ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f79667fc640 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e12b60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7950031440 tx=0x7f7950032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d50d28075154c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f794e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f794e7fc640 1 -- 172.21.15.78:0/2871000740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950040070 con 0x5619a5a43590 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f794e7fc640 1 -- 172.21.15.78:0/2871000740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f7950004240 con 0x5619a5a43590 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f7974367740 1 -- 172.21.15.78:0/2871000740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619a5d75c00 con 0x5619a5a43590 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f794e7fc640 1 -- 172.21.15.78:0/2871000740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f795004a070 con 0x5619a5a43590 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f7974367740 10 client.15171 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:36.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f7974367740 1 -- 172.21.15.78:0/2871000740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619a5e1ca60 con 0x5619a5a43590 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.519+0000 7f794e7fc640 1 -- 172.21.15.78:0/2871000740 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5619a5d75c00 con 0x5619a5a43590 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f794e7fc640 1 -- 172.21.15.78:0/2871000740 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7950004b10 con 0x5619a5a43590 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f7974367740 10 client.15171 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f7974367740 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f7974367740 20 client.15171 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f7974367740 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619a5a22e40 0x5619a5e20540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f7974367740 4 client.15171 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f7974367740 1 -- 172.21.15.78:0/2871000740 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619a5d6ff50 con 0x5619a5a22e40 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f7966ffd640 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619a5a22e40 0x5619a5e20540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f7966ffd640 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619a5a22e40 0x5619a5e20540 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedc1188289869dd server_cookie=d808dfd37d1d5bb9 in_seq=0 out_seq=0 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.520+0000 7f794e7fc640 10 client.15171 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f794e7fc640 1 -- 172.21.15.78:0/2871000740 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5619a5d6ff50 con 0x5619a5a22e40 2024-05-21T15:15:36.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f794e7fc640 10 client.15171 handle_command_reply: tid=0 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 client.15171 shutdown 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 -- 172.21.15.78:0/2871000740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 msgr2=0x5619a5e12b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e12b60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7950031440 tx=0x7f7950032f80 comp rx=0 tx=0).stop 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 -- 172.21.15.78:0/2871000740 shutdown_connections 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619a5a22e40 0x5619a5e20540 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e19ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e12b60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 --2- 172.21.15.78:0/2871000740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e12620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f794e7fc640 0 client.15171 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.521+0000 7f7974367740 1 -- 172.21.15.78:0/2871000740 >> 172.21.15.78:0/2871000740 conn(0x5619a5a583c0 msgr2=0x5619a5dfeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.522+0000 7f7974367740 1 -- 172.21.15.78:0/2871000740 shutdown_connections 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.522+0000 7f7974367740 1 -- 172.21.15.78:0/2871000740 wait complete. 2024-05-21T15:15:36.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.522+0000 7f7974367740 20 client.15171 trim_cache size 0 max 16384 2024-05-21T15:15:36.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f7974367740 1 Processor -- start 2024-05-21T15:15:36.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f7974367740 1 -- start start 2024-05-21T15:15:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f7974367740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a3b3b0 0x5619a5e10c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f7974367740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e11180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f7974367740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f7974367740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619a5e1e880 con 0x5619a5a43590 2024-05-21T15:15:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f7974367740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5619a5e1ef50 con 0x5619a5a64270 2024-05-21T15:15:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f7974367740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619a5e1f620 con 0x5619a5a3b3b0 2024-05-21T15:15:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.540+0000 7f79667fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e11180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79667fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e11180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40332/0 (socket says 172.21.15.78:40332) 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79667fc640 1 -- 172.21.15.78:0/3465664092 learned_addr learned my addr 172.21.15.78:0/3465664092 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79677fe640 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f7966ffd640 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a3b3b0 0x5619a5e10c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79667fc640 1 -- 172.21.15.78:0/3465664092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a3b3b0 msgr2=0x5619a5e10c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79667fc640 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a3b3b0 0x5619a5e10c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79667fc640 1 -- 172.21.15.78:0/3465664092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 msgr2=0x5619a5e199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79667fc640 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79667fc640 1 -- 172.21.15.78:0/3465664092 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a5e116c0 con 0x5619a5a43590 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79677fe640 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f79667fc640 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e11180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f795002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5bd773fbce09e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f795cff9640 1 -- 172.21.15.78:0/3465664092 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950040070 con 0x5619a5a43590 2024-05-21T15:15:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f795cff9640 1 -- 172.21.15.78:0/3465664092 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950005000 con 0x5619a5a43590 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.541+0000 7f795cff9640 1 -- 172.21.15.78:0/3465664092 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950035040 con 0x5619a5a43590 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 -- 172.21.15.78:0/3465664092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 msgr2=0x5619a5e11180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e11180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f795002a410 comp rx=0 tx=0).stop 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 -- 172.21.15.78:0/3465664092 shutdown_connections 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a64270 0x5619a5e199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e11180 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 --2- 172.21.15.78:0/3465664092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a3b3b0 0x5619a5e10c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 -- 172.21.15.78:0/3465664092 >> 172.21.15.78:0/3465664092 conn(0x5619a5a583c0 msgr2=0x5619a5e0cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 -- 172.21.15.78:0/3465664092 shutdown_connections 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 -- 172.21.15.78:0/3465664092 wait complete. 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 -- start start 2024-05-21T15:15:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.542+0000 7f7974367740 1 Processor -- start 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7974367740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7974367740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7974367740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e873b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7974367740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619a5e203c0 con 0x5619a5a43590 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7974367740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5619a5e1fcf0 con 0x5619a5a3b3b0 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7974367740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619a5e20a90 con 0x5619a5a64270 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7974367740 10 client.0 _ll_register_callbacks cb 0x7f7950009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e156e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40348/0 (socket says 172.21.15.78:40348) 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 -- 172.21.15.78:0/1692546562 learned_addr learned my addr 172.21.15.78:0/1692546562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79677fe640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e873b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7966ffd640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 -- 172.21.15.78:0/1692546562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 msgr2=0x5619a5e873b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e873b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 -- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 msgr2=0x5619a5e151a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e151a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 -- 172.21.15.78:0/1692546562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a5e878f0 con 0x5619a5a43590 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79677fe640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e873b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f7966ffd640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f79667fc640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e156e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f795002a930 tx=0x7f79500315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd686cbdafd87efe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f794e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.543+0000 7f794e7fc640 1 -- 172.21.15.78:0/1692546562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950040070 con 0x5619a5a43590 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f794e7fc640 1 -- 172.21.15.78:0/1692546562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f7950004ad0 con 0x5619a5a43590 2024-05-21T15:15:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 1 -- 172.21.15.78:0/1692546562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7950005000 con 0x5619a5a43590 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f794e7fc640 1 -- 172.21.15.78:0/1692546562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7950067070 con 0x5619a5a43590 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 10 client.15177 fetch_fsmap learned FSMap version 25 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 1 -- 172.21.15.78:0/1692546562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619a5e88a50 con 0x5619a5a43590 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f794e7fc640 1 -- 172.21.15.78:0/1692546562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950005000 con 0x5619a5a43590 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f794e7fc640 1 -- 172.21.15.78:0/1692546562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7950057070 con 0x5619a5a43590 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 10 client.15177 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 10 client.15177 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 20 client.15177 populate_metadata read hostname 'smithi078' 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619a59f8280 0x5619a5e8c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 4 client.15177 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7974367740 1 -- 172.21.15.78:0/1692546562 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "status"}) -- 0x5619a5cedb80 con 0x5619a59f8280 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.544+0000 7f7966ffd640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619a59f8280 0x5619a5e8c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:36.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7966ffd640 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619a59f8280 0x5619a5e8c140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c540d153afdd9775 server_cookie=eceb5ccaec8878d6 in_seq=0 out_seq=0 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f794e7fc640 10 client.15177 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f794e7fc640 1 -- 172.21.15.78:0/1692546562 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5619a5cedb80 con 0x5619a59f8280 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f794e7fc640 10 client.15177 handle_command_reply: tid=0 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 client.15177 shutdown 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 -- 172.21.15.78:0/1692546562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 msgr2=0x5619a5e156e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e156e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f795002a930 tx=0x7f79500315d0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 -- 172.21.15.78:0/1692546562 shutdown_connections 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619a59f8280 0x5619a5e8c140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619a5a64270 0x5619a5e873b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f794e7fc640 0 client.15177 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619a5a43590 0x5619a5e156e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 --2- 172.21.15.78:0/1692546562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619a5a3b3b0 0x5619a5e151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 -- 172.21.15.78:0/1692546562 >> 172.21.15.78:0/1692546562 conn(0x5619a5a583c0 msgr2=0x5619a5dff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 -- 172.21.15.78:0/1692546562 shutdown_connections 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 1 -- 172.21.15.78:0/1692546562 wait complete. 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.545+0000 7f7974367740 20 client.15177 trim_cache size 0 max 16384 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24527, 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:15:36.555 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:15:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 83.992539102999999, 2024-05-21T15:15:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:15:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:15:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:15:36.556 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 89.543822574000004 2024-05-21T15:15:36.556 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 -- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79540a2c20 msgr2=0x7f79540a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79540a2c20 0x7f79540a5130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7960009fd0 tx=0x7f7960004a40 comp rx=0 tx=0).stop 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 -- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968102ab0 msgr2=0x7f79681992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968102ab0 0x7f79681992a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f795800c970 tx=0x7f795800ce30 comp rx=0 tx=0).stop 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 -- 172.21.15.78:0/28003160 shutdown_connections 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f79540a2c20 0x7f79540a5130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79681089f0 0x7f79681a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7968103cf0 0x7f79681997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 --2- 172.21.15.78:0/28003160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7968102ab0 0x7f79681992a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 -- 172.21.15.78:0/28003160 >> 172.21.15.78:0/28003160 conn(0x7f79680fdca0 msgr2=0x7f79681049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 -- 172.21.15.78:0/28003160 shutdown_connections 2024-05-21T15:15:36.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:36.546+0000 7f796d4c3640 1 -- 172.21.15.78:0/28003160 wait complete. 2024-05-21T15:15:36.560 INFO:tasks.cephadm:Setup complete, yielding 2024-05-21T15:15:36.560 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T15:15:36.571 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:15:36.571 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-21T15:15:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:36 smithi084 ceph-mon[36334]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:36 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4017502139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:36 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3658784656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:36 smithi172 ceph-mon[36402]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:36 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4017502139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:36.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:36 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3658784656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:36.811 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:36.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:36 smithi078 ceph-mon[31997]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:36.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:36 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4017502139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:36.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:36 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3658784656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:38.392 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:38 smithi078 ceph-mon[31997]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 -- 172.21.15.78:0/1615201984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 msgr2=0x7fa558108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/1615201984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 0x7fa558108a40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009990 tx=0x7fa54c009ca0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 -- 172.21.15.78:0/1615201984 shutdown_connections 2024-05-21T15:15:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/1615201984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa558108f80 0x7fa55810b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/1615201984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa558103cc0 0x7fa5581060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/1615201984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 0x7fa558108a40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 -- 172.21.15.78:0/1615201984 >> 172.21.15.78:0/1615201984 conn(0x7fa5580fd700 msgr2=0x7fa5580ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 -- 172.21.15.78:0/1615201984 shutdown_connections 2024-05-21T15:15:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.395+0000 7fa55ca7a700 1 -- 172.21.15.78:0/1615201984 wait complete. 2024-05-21T15:15:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa55ca7a700 1 Processor -- start 2024-05-21T15:15:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa55ca7a700 1 -- start start 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa55ca7a700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa558103cc0 0x7fa558103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa55ca7a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 0x7fa558101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa55ca7a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa558108f80 0x7fa558102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa55ca7a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810ed80 con 0x7fa558106620 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa55ca7a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810ec00 con 0x7fa558108f80 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa55ca7a700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa55810ef00 con 0x7fa558103cc0 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa555d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 0x7fa558101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa555d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 0x7fa558101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40372/0 (socket says 172.21.15.78:40372) 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa555d9b700 1 -- 172.21.15.78:0/4287977183 learned_addr learned my addr 172.21.15.78:0/4287977183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.396+0000 7fa556d9d700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa558108f80 0x7fa558102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa55659c700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa558103cc0 0x7fa558103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa555d9b700 1 -- 172.21.15.78:0/4287977183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa558103cc0 msgr2=0x7fa558103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa555d9b700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa558103cc0 0x7fa558103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa555d9b700 1 -- 172.21.15.78:0/4287977183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa558108f80 msgr2=0x7fa558102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa555d9b700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa558108f80 0x7fa558102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa555d9b700 1 -- 172.21.15.78:0/4287977183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa558102870 con 0x7fa558106620 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa55659c700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa558103cc0 0x7fa558103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa556d9d700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa558108f80 0x7fa558102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa555d9b700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 0x7fa558101bd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0098e0 tx=0x7fa54c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa5477fe700 1 -- 172.21.15.78:0/4287977183 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c003c70 con 0x7fa558106620 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5581a3930 con 0x7fa558106620 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa5477fe700 1 -- 172.21.15.78:0/4287977183 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa54c004830 con 0x7fa558106620 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa5477fe700 1 -- 172.21.15.78:0/4287977183 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0054e0 con 0x7fa558106620 2024-05-21T15:15:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.397+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5581a3cd0 con 0x7fa558106620 2024-05-21T15:15:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.398+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa558050170 con 0x7fa558106620 2024-05-21T15:15:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.399+0000 7fa5477fe700 1 -- 172.21.15.78:0/4287977183 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa54c003e10 con 0x7fa558106620 2024-05-21T15:15:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.400+0000 7fa5477fe700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5400a2cb0 0x7fa5400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.400+0000 7fa55659c700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5400a2cb0 0x7fa5400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.400+0000 7fa5477fe700 1 -- 172.21.15.78:0/4287977183 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa54c0d02c0 con 0x7fa558106620 2024-05-21T15:15:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.400+0000 7fa55659c700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5400a2cb0 0x7fa5400a51c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa548005fd0 tx=0x7fa548005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.405+0000 7fa5477fe700 1 -- 172.21.15.78:0/4287977183 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa54c013070 con 0x7fa558106620 2024-05-21T15:15:38.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.617+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa55802e010 con 0x7fa5400a2cb0 2024-05-21T15:15:38.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.618+0000 7fa5477fe700 1 -- 172.21.15.78:0/4287977183 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa55802e010 con 0x7fa5400a2cb0 2024-05-21T15:15:38.621 INFO:teuthology.orchestra.run.smithi078.stdout:Backend: cephadm 2024-05-21T15:15:38.621 INFO:teuthology.orchestra.run.smithi078.stdout:Available: Yes 2024-05-21T15:15:38.621 INFO:teuthology.orchestra.run.smithi078.stdout:Paused: No 2024-05-21T15:15:38.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5400a2cb0 msgr2=0x7fa5400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:38.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5400a2cb0 0x7fa5400a51c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa548005fd0 tx=0x7fa548005ee0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 msgr2=0x7fa558101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:38.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 0x7fa558101bd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0098e0 tx=0x7fa54c0047b0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 shutdown_connections 2024-05-21T15:15:38.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa5400a2cb0 0x7fa5400a51c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa558108f80 0x7fa558102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa558103cc0 0x7fa558103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 --2- 172.21.15.78:0/4287977183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa558106620 0x7fa558101bd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:38.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 >> 172.21.15.78:0/4287977183 conn(0x7fa5580fd700 msgr2=0x7fa558104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:38.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 shutdown_connections 2024-05-21T15:15:38.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:38.620+0000 7fa55ca7a700 1 -- 172.21.15.78:0/4287977183 wait complete. 2024-05-21T15:15:38.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:38 smithi084 ceph-mon[36334]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:38 smithi172 ceph-mon[36402]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:39.399 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-21T15:15:39.641 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:40 smithi084 ceph-mon[36334]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:40 smithi084 ceph-mon[36334]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:40 smithi172 ceph-mon[36402]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:40 smithi172 ceph-mon[36402]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:40.736 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:40 smithi078 ceph-mon[31997]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:40.736 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:40 smithi078 ceph-mon[31997]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-21T15:15:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 -- 172.21.15.78:0/1794256687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 msgr2=0x7f9f7410a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1794256687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 0x7f9f7410a280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00b0a0 tx=0x7f9f6c00b3b0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 -- 172.21.15.78:0/1794256687 shutdown_connections 2024-05-21T15:15:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1794256687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 0x7f9f7410a280 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1794256687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f74101dc0 0x7f9f741021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1794256687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f74102fa0 0x7f9f741033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 -- 172.21.15.78:0/1794256687 >> 172.21.15.78:0/1794256687 conn(0x7f9f740fd510 msgr2=0x7f9f740ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 -- 172.21.15.78:0/1794256687 shutdown_connections 2024-05-21T15:15:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.258+0000 7f9f7999f700 1 -- 172.21.15.78:0/1794256687 wait complete. 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.259+0000 7f9f7999f700 1 Processor -- start 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.259+0000 7f9f7999f700 1 -- start start 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7999f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f74101dc0 0x7f9f7406aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7999f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f74102fa0 0x7f9f741a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7999f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 0x7f9f741a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7999f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7410db10 con 0x7f9f741038f0 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7999f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7410d990 con 0x7f9f74102fa0 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7999f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7410dc90 con 0x7f9f74101dc0 2024-05-21T15:15:41.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 0x7f9f741a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 0x7f9f741a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49148/0 (socket says 172.21.15.78:49148) 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 -- 172.21.15.78:0/1895282906 learned_addr learned my addr 172.21.15.78:0/1895282906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7899d700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f74101dc0 0x7f9f7406aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f73fff700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f74102fa0 0x7f9f741a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 -- 172.21.15.78:0/1895282906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f74101dc0 msgr2=0x7f9f7406aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f74101dc0 0x7f9f7406aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 -- 172.21.15.78:0/1895282906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f74102fa0 msgr2=0x7f9f741a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f74102fa0 0x7f9f741a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 -- 172.21.15.78:0/1895282906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f741a33f0 con 0x7f9f741038f0 2024-05-21T15:15:41.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f73fff700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f74102fa0 0x7f9f741a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:41.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f7919e700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 0x7f9f741a2d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009bb0 tx=0x7f9f6c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:41.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.260+0000 7f9f71ffb700 1 -- 172.21.15.78:0/1895282906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c021070 con 0x7f9f741038f0 2024-05-21T15:15:41.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.261+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f741a3680 con 0x7f9f741038f0 2024-05-21T15:15:41.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.261+0000 7f9f71ffb700 1 -- 172.21.15.78:0/1895282906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c007d00 con 0x7f9f741038f0 2024-05-21T15:15:41.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.261+0000 7f9f71ffb700 1 -- 172.21.15.78:0/1895282906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c01d930 con 0x7f9f741038f0 2024-05-21T15:15:41.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.261+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f741a3c00 con 0x7f9f741038f0 2024-05-21T15:15:41.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.262+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f74050170 con 0x7f9f741038f0 2024-05-21T15:15:41.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.263+0000 7f9f71ffb700 1 -- 172.21.15.78:0/1895282906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9f6c024070 con 0x7f9f741038f0 2024-05-21T15:15:41.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.264+0000 7f9f71ffb700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9f640a2cb0 0x7f9f640a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:41.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.264+0000 7f9f7899d700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9f640a2cb0 0x7f9f640a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:41.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.264+0000 7f9f71ffb700 1 -- 172.21.15.78:0/1895282906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9f6c00f020 con 0x7f9f741038f0 2024-05-21T15:15:41.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.264+0000 7f9f7899d700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9f640a2cb0 0x7f9f640a51c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f74101b80 tx=0x7f9f680091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:41.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.269+0000 7f9f71ffb700 1 -- 172.21.15.78:0/1895282906 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9f6c02b020 con 0x7f9f741038f0 2024-05-21T15:15:41.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.478+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9f74062a80 con 0x7f9f640a2cb0 2024-05-21T15:15:41.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.491+0000 7f9f71ffb700 1 -- 172.21.15.78:0/1895282906 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f9f74062a80 con 0x7f9f640a2cb0 2024-05-21T15:15:41.494 INFO:teuthology.orchestra.run.smithi078.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.a smithi078 running (97s) 16s ago 97s 85.9M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 4be056043a8e 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.b smithi084 running (94s) 28s ago 94s 93.5M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 57cd46193d8a 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.c smithi172 running (109s) 28s ago 108s 85.3M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 92ef669fa197 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.d smithi078 running (117s) 16s ago 116s 81.4M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 49275e609824 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.e smithi084 running (114s) 28s ago 114s 83.6M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 a93cd8b3a910 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.f smithi172 running (111s) 28s ago 111s 85.2M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 354413a5be18 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.g smithi078 running (100s) 16s ago 100s 84.4M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 4c0e61d3c68f 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.h smithi084 running (106s) 28s ago 105s 84.4M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 096aa407a45f 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.i smithi172 running (103s) 28s ago 103s 85.7M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 e57ba4470e22 2024-05-21T15:15:41.495 INFO:teuthology.orchestra.run.smithi078.stdout:mds.j smithi078 running (2m) 16s ago 119s 83.7M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 403f2890c906 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:mds.k smithi084 running (2m) 28s ago 2m 86.7M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 bffdc70a4ddc 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:mds.l smithi172 running (2m) 28s ago 2m 84.5M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 e7fb22fe21c2 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.x smithi078 *:9283,8765 running (9m) 16s ago 9m 583M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 a09b5065f38a 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.y smithi084 *:8443,8765 running (7m) 28s ago 7m 521M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 472ee36adf99 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.z smithi172 *:8443,8765 running (7m) 28s ago 7m 523M - 19.0.0-2744-gd2b507c7 cfcc61b2d476 3e525d029db1 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:mon.a smithi078 running (9m) 16s ago 9m 116M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 1cd58a23b211 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:mon.b smithi084 running (7m) 28s ago 7m 107M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 85123a736b78 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:mon.c smithi172 running (7m) 28s ago 7m 105M 2048M 19.0.0-2744-gd2b507c7 cfcc61b2d476 14e48ae40261 2024-05-21T15:15:41.496 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 smithi078 running (6m) 16s ago 6m 114M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 5830d917b4cf 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 smithi084 running (6m) 28s ago 6m 115M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 24cc1940280b 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 smithi172 running (6m) 28s ago 6m 124M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 12ea02df0e53 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 smithi078 running (5m) 16s ago 5m 114M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 6ed3e33ad927 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 smithi084 running (5m) 28s ago 5m 111M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 a714699a6b6e 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 smithi172 running (4m) 28s ago 4m 122M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 e4fd9406f407 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 smithi078 running (4m) 16s ago 4m 114M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 dccfa821094f 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 smithi084 running (4m) 28s ago 4m 123M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 bed83afc2773 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 smithi172 running (3m) 28s ago 3m 113M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 d7e486abab17 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 smithi078 running (3m) 16s ago 3m 121M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 d3f4652349fa 2024-05-21T15:15:41.497 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 smithi084 running (2m) 28s ago 2m 116M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 90ae1a8952b5 2024-05-21T15:15:41.498 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 smithi172 running (2m) 28s ago 2m 110M 4272M 19.0.0-2744-gd2b507c7 cfcc61b2d476 cb9c465bff35 2024-05-21T15:15:41.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9f640a2cb0 msgr2=0x7f9f640a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:41.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9f640a2cb0 0x7f9f640a51c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f74101b80 tx=0x7f9f680091c0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 msgr2=0x7f9f741a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:41.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 0x7f9f741a2d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009bb0 tx=0x7f9f6c004e20 comp rx=0 tx=0).stop 2024-05-21T15:15:41.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 shutdown_connections 2024-05-21T15:15:41.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9f640a2cb0 0x7f9f640a51c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f741038f0 0x7f9f741a2d80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f74101dc0 0x7f9f7406aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 --2- 172.21.15.78:0/1895282906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f74102fa0 0x7f9f741a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:41.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 >> 172.21.15.78:0/1895282906 conn(0x7f9f740fd510 msgr2=0x7f9f74106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:41.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 shutdown_connections 2024-05-21T15:15:41.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:41.494+0000 7f9f7999f700 1 -- 172.21.15.78:0/1895282906 wait complete. 2024-05-21T15:15:42.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-21T15:15:42.398 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:42.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:42 smithi078 ceph-mon[31997]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:42.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:42 smithi084 ceph-mon[36334]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:42 smithi172 ceph-mon[36402]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T15:15:43.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:43 smithi084 ceph-mon[36334]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:43.654 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:43 smithi078 ceph-mon[31997]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:43.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:43 smithi172 ceph-mon[36402]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:44.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 -- 172.21.15.78:0/3200716258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 msgr2=0x7f1ab810b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:44.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 --2- 172.21.15.78:0/3200716258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 0x7f1ab810b390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000b0a0 tx=0x7f1ab000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 -- 172.21.15.78:0/3200716258 shutdown_connections 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 --2- 172.21.15.78:0/3200716258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 0x7f1ab810b390 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 --2- 172.21.15.78:0/3200716258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ab8075530 0x7f1ab8105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 --2- 172.21.15.78:0/3200716258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ab8106310 0x7f1ab81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 -- 172.21.15.78:0/3200716258 >> 172.21.15.78:0/3200716258 conn(0x7f1ab80fd710 msgr2=0x7f1ab80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 -- 172.21.15.78:0/3200716258 shutdown_connections 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.003+0000 7f1abd669700 1 -- 172.21.15.78:0/3200716258 wait complete. 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1abd669700 1 Processor -- start 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1abd669700 1 -- start start 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1abd669700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ab8075530 0x7f1ab8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1abd669700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ab8106310 0x7f1ab8199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1abd669700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 0x7f1ab81a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:44.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1abd669700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab810ed70 con 0x7f1ab8108e80 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1abd669700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab810ebf0 con 0x7f1ab8106310 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1abd669700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab810eef0 con 0x7f1ab8075530 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1ab77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 0x7f1ab81a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.004+0000 7f1ab77fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 0x7f1ab81a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49166/0 (socket says 172.21.15.78:49166) 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab77fe700 1 -- 172.21.15.78:0/4015280000 learned_addr learned my addr 172.21.15.78:0/4015280000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab6ffd700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ab8075530 0x7f1ab8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1aaffff700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ab8106310 0x7f1ab8199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab77fe700 1 -- 172.21.15.78:0/4015280000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ab8075530 msgr2=0x7f1ab8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab77fe700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ab8075530 0x7f1ab8199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab77fe700 1 -- 172.21.15.78:0/4015280000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ab8106310 msgr2=0x7f1ab8199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:44.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab77fe700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ab8106310 0x7f1ab8199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab77fe700 1 -- 172.21.15.78:0/4015280000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab81a12e0 con 0x7f1ab8108e80 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab6ffd700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ab8075530 0x7f1ab8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1aaffff700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ab8106310 0x7f1ab8199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab77fe700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 0x7f1ab81a0c60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00062a0 tx=0x7f1ab0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab4ff9700 1 -- 172.21.15.78:0/4015280000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0021070 con 0x7f1ab8108e80 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab81a1570 con 0x7f1ab8108e80 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab4ff9700 1 -- 172.21.15.78:0/4015280000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab0004020 con 0x7f1ab8108e80 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1ab4ff9700 1 -- 172.21.15.78:0/4015280000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab001d6b0 con 0x7f1ab8108e80 2024-05-21T15:15:44.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.005+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab81a1a20 con 0x7f1ab8108e80 2024-05-21T15:15:44.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.006+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab8050170 con 0x7f1ab8108e80 2024-05-21T15:15:44.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.008+0000 7f1ab4ff9700 1 -- 172.21.15.78:0/4015280000 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1ab0004540 con 0x7f1ab8108e80 2024-05-21T15:15:44.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.008+0000 7f1ab4ff9700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1aa40a2cb0 0x7f1aa40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:44.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.008+0000 7f1ab6ffd700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1aa40a2cb0 0x7f1aa40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:44.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.009+0000 7f1ab4ff9700 1 -- 172.21.15.78:0/4015280000 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f1ab000f020 con 0x7f1ab8108e80 2024-05-21T15:15:44.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.009+0000 7f1ab6ffd700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1aa40a2cb0 0x7f1aa40a51c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005950 tx=0x7f1aa8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:44.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.012+0000 7f1ab4ff9700 1 -- 172.21.15.78:0/4015280000 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ab002a030 con 0x7f1ab8108e80 2024-05-21T15:15:44.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.214+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1ab8062a80 con 0x7f1aa40a2cb0 2024-05-21T15:15:44.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.220+0000 7f1ab4ff9700 1 -- 172.21.15.78:0/4015280000 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1ab8062a80 con 0x7f1aa40a2cb0 2024-05-21T15:15:44.223 INFO:teuthology.orchestra.run.smithi078.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T15:15:44.223 INFO:teuthology.orchestra.run.smithi078.stdout:mds.all 12/12 30s ago 2m smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi084=b;smithi084=e;smithi084=h;smithi084=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-05-21T15:15:44.223 INFO:teuthology.orchestra.run.smithi078.stdout:mgr 3/3 30s ago 7m smithi078=x;smithi084=y;smithi172=z;count:3 2024-05-21T15:15:44.223 INFO:teuthology.orchestra.run.smithi078.stdout:mon 3/3 30s ago 7m smithi078:172.21.15.78=a;smithi084:172.21.15.84=b;smithi172:172.21.15.172=c;count:3 2024-05-21T15:15:44.223 INFO:teuthology.orchestra.run.smithi078.stdout:osd 12 30s ago - 2024-05-21T15:15:44.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.222+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1aa40a2cb0 msgr2=0x7f1aa40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.222+0000 7f1abd669700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1aa40a2cb0 0x7f1aa40a51c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005950 tx=0x7f1aa8005f50 comp rx=0 tx=0).stop 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.222+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 msgr2=0x7f1ab81a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 0x7f1ab81a0c60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00062a0 tx=0x7f1ab0004e20 comp rx=0 tx=0).stop 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 shutdown_connections 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1aa40a2cb0 0x7f1aa40a51c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ab8108e80 0x7f1ab81a0c60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ab8075530 0x7f1ab8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 --2- 172.21.15.78:0/4015280000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ab8106310 0x7f1ab8199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 >> 172.21.15.78:0/4015280000 conn(0x7f1ab80fd710 msgr2=0x7f1ab8104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 shutdown_connections 2024-05-21T15:15:44.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:44.223+0000 7f1abd669700 1 -- 172.21.15.78:0/4015280000 wait complete. 2024-05-21T15:15:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:44 smithi084 ceph-mon[36334]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:44 smithi084 ceph-mon[36334]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:44 smithi172 ceph-mon[36402]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:44.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:44 smithi172 ceph-mon[36402]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:44.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:44 smithi078 ceph-mon[31997]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:44.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:44 smithi078 ceph-mon[31997]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:44.932 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-21T15:15:45.171 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:45.567 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:45 smithi078 ceph-mon[31997]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:45.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:45 smithi084 ceph-mon[36334]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:45.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:45 smithi172 ceph-mon[36402]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:15:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:46 smithi078 ceph-mon[31997]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-21T15:15:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:46 smithi084 ceph-mon[36334]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-21T15:15:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:46 smithi172 ceph-mon[36402]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-05-21T15:15:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 -- 172.21.15.78:0/2611923180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158103c40 msgr2=0x7f415810a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 --2- 172.21.15.78:0/2611923180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158103c40 0x7f415810a4f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f415000b0a0 tx=0x7f415000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:15:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 -- 172.21.15.78:0/2611923180 shutdown_connections 2024-05-21T15:15:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 --2- 172.21.15.78:0/2611923180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158103c40 0x7f415810a4f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:46.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 --2- 172.21.15.78:0/2611923180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4158102170 0x7f4158102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:46.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 --2- 172.21.15.78:0/2611923180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41581032a0 0x7f41581036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:46.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 -- 172.21.15.78:0/2611923180 >> 172.21.15.78:0/2611923180 conn(0x7f41580fd790 msgr2=0x7f41580ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:46.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 -- 172.21.15.78:0/2611923180 shutdown_connections 2024-05-21T15:15:46.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 -- 172.21.15.78:0/2611923180 wait complete. 2024-05-21T15:15:46.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 Processor -- start 2024-05-21T15:15:46.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.861+0000 7f416029b700 1 -- start start 2024-05-21T15:15:46.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f416029b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158102170 0x7f41581987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:46.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f416029b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41581032a0 0x7f4158198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:46.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f416029b700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4158103c40 0x7f415819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:46.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f416029b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f415810ddb0 con 0x7f4158102170 2024-05-21T15:15:46.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f416029b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f415810dc30 con 0x7f41581032a0 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f416029b700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f415810df30 con 0x7f4158103c40 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158102170 0x7f41581987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158102170 0x7f41581987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49188/0 (socket says 172.21.15.78:49188) 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 -- 172.21.15.78:0/942624712 learned_addr learned my addr 172.21.15.78:0/942624712 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415d836700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41581032a0 0x7f4158198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 -- 172.21.15.78:0/942624712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4158103c40 msgr2=0x7f415819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4158103c40 0x7f415819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 -- 172.21.15.78:0/942624712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41581032a0 msgr2=0x7f4158198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41581032a0 0x7f4158198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 -- 172.21.15.78:0/942624712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41581a0350 con 0x7f4158102170 2024-05-21T15:15:46.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.862+0000 7f415e037700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158102170 0x7f41581987f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f414c00b4d0 tx=0x7f414c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:46.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.863+0000 7f414b7fe700 1 -- 172.21.15.78:0/942624712 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c015020 con 0x7f4158102170 2024-05-21T15:15:46.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.863+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41581a0640 con 0x7f4158102170 2024-05-21T15:15:46.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.863+0000 7f414b7fe700 1 -- 172.21.15.78:0/942624712 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f414c011d10 con 0x7f4158102170 2024-05-21T15:15:46.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.863+0000 7f414b7fe700 1 -- 172.21.15.78:0/942624712 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c010e20 con 0x7f4158102170 2024-05-21T15:15:46.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.863+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41581a0b80 con 0x7f4158102170 2024-05-21T15:15:46.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.864+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4158050170 con 0x7f4158102170 2024-05-21T15:15:46.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.865+0000 7f414b7fe700 1 -- 172.21.15.78:0/942624712 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f414c018020 con 0x7f4158102170 2024-05-21T15:15:46.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.865+0000 7f414b7fe700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f41440a2ca0 0x7f41440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:46.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.866+0000 7f415d836700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f41440a2ca0 0x7f41440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:46.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.866+0000 7f414b7fe700 1 -- 172.21.15.78:0/942624712 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f414c0cecb0 con 0x7f4158102170 2024-05-21T15:15:46.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.866+0000 7f415d836700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f41440a2ca0 0x7f41440a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f41581998a0 tx=0x7f415400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:46.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:46.869+0000 7f414b7fe700 1 -- 172.21.15.78:0/942624712 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f414c01f020 con 0x7f4158102170 2024-05-21T15:15:47.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.068+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4158062a80 con 0x7f41440a2ca0 2024-05-21T15:15:47.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.070+0000 7f414b7fe700 1 -- 172.21.15.78:0/942624712 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4158062a80 con 0x7f41440a2ca0 2024-05-21T15:15:47.073 INFO:teuthology.orchestra.run.smithi078.stdout:HOST ADDR LABELS STATUS 2024-05-21T15:15:47.073 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 172.21.15.78 2024-05-21T15:15:47.074 INFO:teuthology.orchestra.run.smithi078.stdout:smithi084 172.21.15.84 2024-05-21T15:15:47.074 INFO:teuthology.orchestra.run.smithi078.stdout:smithi172 172.21.15.172 2024-05-21T15:15:47.074 INFO:teuthology.orchestra.run.smithi078.stdout:3 hosts in cluster 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f41440a2ca0 msgr2=0x7f41440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f41440a2ca0 0x7f41440a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f41581998a0 tx=0x7f415400b410 comp rx=0 tx=0).stop 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158102170 msgr2=0x7f41581987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158102170 0x7f41581987f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f414c00b4d0 tx=0x7f414c00b7e0 comp rx=0 tx=0).stop 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 shutdown_connections 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f41440a2ca0 0x7f41440a51b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4158103c40 0x7f415819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4158102170 0x7f41581987f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 --2- 172.21.15.78:0/942624712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41581032a0 0x7f4158198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 >> 172.21.15.78:0/942624712 conn(0x7f41580fd790 msgr2=0x7f41581064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 shutdown_connections 2024-05-21T15:15:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:47.072+0000 7f416029b700 1 -- 172.21.15.78:0/942624712 wait complete. 2024-05-21T15:15:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:47 smithi084 ceph-mon[36334]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:47 smithi084 ceph-mon[36334]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:47 smithi172 ceph-mon[36402]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:47 smithi172 ceph-mon[36402]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:47 smithi078 ceph-mon[31997]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:47 smithi078 ceph-mon[31997]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:15:47.723 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-21T15:15:47.965 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 -- 172.21.15.78:0/531194039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 msgr2=0x7f526810b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 --2- 172.21.15.78:0/531194039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 0x7f526810b390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f526000b0a0 tx=0x7f526000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 -- 172.21.15.78:0/531194039 shutdown_connections 2024-05-21T15:15:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 --2- 172.21.15.78:0/531194039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 0x7f526810b390 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 --2- 172.21.15.78:0/531194039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52680754f0 0x7f5268105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 --2- 172.21.15.78:0/531194039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268106350 0x7f52681087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 -- 172.21.15.78:0/531194039 >> 172.21.15.78:0/531194039 conn(0x7f52680fd710 msgr2=0x7f52680ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 -- 172.21.15.78:0/531194039 shutdown_connections 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 -- 172.21.15.78:0/531194039 wait complete. 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 Processor -- start 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.684+0000 7f526fba6700 1 -- start start 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526fba6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52680754f0 0x7f52681035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526fba6700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268106350 0x7f5268101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526fba6700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 0x7f52681021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526fba6700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f526810ed90 con 0x7f5268108e80 2024-05-21T15:15:49.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526fba6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f526810ec10 con 0x7f52680754f0 2024-05-21T15:15:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526fba6700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f526810ef10 con 0x7f5268106350 2024-05-21T15:15:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 0x7f52681021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 0x7f52681021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44278/0 (socket says 172.21.15.78:44278) 2024-05-21T15:15:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 -- 172.21.15.78:0/876606130 learned_addr learned my addr 172.21.15.78:0/876606130 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526d141700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268106350 0x7f5268101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526d942700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52680754f0 0x7f52681035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:49.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 -- 172.21.15.78:0/876606130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268106350 msgr2=0x7f5268101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268106350 0x7f5268101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 -- 172.21.15.78:0/876606130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52680754f0 msgr2=0x7f52681035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52680754f0 0x7f52681035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 -- 172.21.15.78:0/876606130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5268102890 con 0x7f5268108e80 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.685+0000 7f526e143700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 0x7f52681021c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f52600062a0 tx=0x7f5260004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.686+0000 7f525effd700 1 -- 172.21.15.78:0/876606130 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260021070 con 0x7f5268108e80 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.686+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52681a3870 con 0x7f5268108e80 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.686+0000 7f525effd700 1 -- 172.21.15.78:0/876606130 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5260007c40 con 0x7f5268108e80 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.686+0000 7f525effd700 1 -- 172.21.15.78:0/876606130 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526001d450 con 0x7f5268108e80 2024-05-21T15:15:49.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.686+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52681a3c10 con 0x7f5268108e80 2024-05-21T15:15:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.687+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5268050170 con 0x7f5268108e80 2024-05-21T15:15:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.689+0000 7f525effd700 1 -- 172.21.15.78:0/876606130 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f52600040b0 con 0x7f5268108e80 2024-05-21T15:15:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.689+0000 7f525effd700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52540a2c20 0x7f52540a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.689+0000 7f526d942700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52540a2c20 0x7f52540a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.690+0000 7f525effd700 1 -- 172.21.15.78:0/876606130 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f52600d0150 con 0x7f5268108e80 2024-05-21T15:15:49.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.690+0000 7f526d942700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52540a2c20 0x7f52540a5130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f52680751f0 tx=0x7f525800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:49.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.694+0000 7f525effd700 1 -- 172.21.15.78:0/876606130 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f526009d120 con 0x7f5268108e80 2024-05-21T15:15:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.897+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 --> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f526802e030 con 0x7f52540a2c20 2024-05-21T15:15:49.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.899+0000 7f525effd700 1 -- 172.21.15.78:0/876606130 <== mgr.14152 v2:172.21.15.78:6800/2228258510 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f526802e030 con 0x7f52540a2c20 2024-05-21T15:15:49.901 INFO:teuthology.orchestra.run.smithi078.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-21T15:15:49.902 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-21T15:15:49.902 INFO:teuthology.orchestra.run.smithi078.stdout:smithi084 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WF400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-21T15:15:49.902 INFO:teuthology.orchestra.run.smithi078.stdout:smithi172 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52540a2c20 msgr2=0x7f52540a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52540a2c20 0x7f52540a5130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f52680751f0 tx=0x7f525800a380 comp rx=0 tx=0).stop 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 msgr2=0x7f52681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 0x7f52681021c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f52600062a0 tx=0x7f5260004d70 comp rx=0 tx=0).stop 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 shutdown_connections 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f52540a2c20 0x7f52540a5130 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5268108e80 0x7f52681021c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52680754f0 0x7f52681035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 --2- 172.21.15.78:0/876606130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268106350 0x7f5268101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 >> 172.21.15.78:0/876606130 conn(0x7f52680fd710 msgr2=0x7f52681047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 shutdown_connections 2024-05-21T15:15:49.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:49.901+0000 7f526fba6700 1 -- 172.21.15.78:0/876606130 wait complete. 2024-05-21T15:15:50.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:50 smithi078 ceph-mon[31997]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:15:50.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:50 smithi084 ceph-mon[36334]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:15:50.664 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T15:15:50.678 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:15:50.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-21T15:15:50.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:50 smithi172 ceph-mon[36402]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:15:50.920 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:51.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:51 smithi078 ceph-mon[31997]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:51.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:51 smithi084 ceph-mon[36334]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:51.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:51 smithi172 ceph-mon[36402]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 -- 172.21.15.78:0/869215727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578103b40 msgr2=0x7fc57810a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 --2- 172.21.15.78:0/869215727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578103b40 0x7fc57810a5d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b0a0 tx=0x7fc57000b3b0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 -- 172.21.15.78:0/869215727 shutdown_connections 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 --2- 172.21.15.78:0/869215727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578103b40 0x7fc57810a5d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 --2- 172.21.15.78:0/869215727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578102600 0x7fc578102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 --2- 172.21.15.78:0/869215727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc578102fe0 0x7fc5781034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 -- 172.21.15.78:0/869215727 >> 172.21.15.78:0/869215727 conn(0x7fc5780fce90 msgr2=0x7fc5780ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 -- 172.21.15.78:0/869215727 shutdown_connections 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.572+0000 7fc57f74f700 1 -- 172.21.15.78:0/869215727 wait complete. 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57f74f700 1 Processor -- start 2024-05-21T15:15:52.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57f74f700 1 -- start start 2024-05-21T15:15:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57f74f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc578102600 0x7fc578198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57f74f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578102fe0 0x7fc578198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57f74f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578103b40 0x7fc57819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57f74f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc57810dc90 con 0x7fc578102fe0 2024-05-21T15:15:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57f74f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc57810db10 con 0x7fc578102600 2024-05-21T15:15:52.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57f74f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc57810de10 con 0x7fc578103b40 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.573+0000 7fc57ccea700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578102fe0 0x7fc578198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57ccea700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578102fe0 0x7fc578198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44296/0 (socket says 172.21.15.78:44296) 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57ccea700 1 -- 172.21.15.78:0/1457708541 learned_addr learned my addr 172.21.15.78:0/1457708541 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57dcec700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578103b40 0x7fc57819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57d4eb700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc578102600 0x7fc578198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57ccea700 1 -- 172.21.15.78:0/1457708541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578103b40 msgr2=0x7fc57819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57ccea700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578103b40 0x7fc57819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57ccea700 1 -- 172.21.15.78:0/1457708541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc578102600 msgr2=0x7fc578198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57ccea700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc578102600 0x7fc578198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57ccea700 1 -- 172.21.15.78:0/1457708541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5781a0280 con 0x7fc578102fe0 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57d4eb700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc578102600 0x7fc578198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57ccea700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578102fe0 0x7fc578198c60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc574007910 tx=0x7fc574007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:52.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc56e7fc700 1 -- 172.21.15.78:0/1457708541 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574007e60 con 0x7fc578102fe0 2024-05-21T15:15:52.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc56e7fc700 1 -- 172.21.15.78:0/1457708541 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc574004500 con 0x7fc578102fe0 2024-05-21T15:15:52.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc56e7fc700 1 -- 172.21.15.78:0/1457708541 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740053c0 con 0x7fc578102fe0 2024-05-21T15:15:52.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5781012c0 con 0x7fc578102fe0 2024-05-21T15:15:52.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.574+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5781017b0 con 0x7fc578102fe0 2024-05-21T15:15:52.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.575+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc578050170 con 0x7fc578102fe0 2024-05-21T15:15:52.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.577+0000 7fc56e7fc700 1 -- 172.21.15.78:0/1457708541 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc57401f020 con 0x7fc578102fe0 2024-05-21T15:15:52.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.577+0000 7fc56e7fc700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5640a2ca0 0x7fc5640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:52.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.577+0000 7fc57d4eb700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5640a2ca0 0x7fc5640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:52.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.577+0000 7fc56e7fc700 1 -- 172.21.15.78:0/1457708541 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc5740d0aa0 con 0x7fc578102fe0 2024-05-21T15:15:52.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.578+0000 7fc57d4eb700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5640a2ca0 0x7fc5640a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc568005950 tx=0x7fc568005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:52.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.581+0000 7fc56e7fc700 1 -- 172.21.15.78:0/1457708541 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc574026030 con 0x7fc578102fe0 2024-05-21T15:15:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:52 smithi078 ceph-mon[31997]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:15:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:52 smithi084 ceph-mon[36334]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:15:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:52 smithi172 ceph-mon[36402]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:15:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.823+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc5781a0510 con 0x7fc578102fe0 2024-05-21T15:15:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.823+0000 7fc56e7fc700 1 -- 172.21.15.78:0/1457708541 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3086 (secure 0 0 0) 0x7fc57409d9e0 con 0x7fc578102fe0 2024-05-21T15:15:52.827 INFO:teuthology.orchestra.run.smithi078.stdout:e25 2024-05-21T15:15:52.827 INFO:teuthology.orchestra.run.smithi078.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T15:15:52.827 INFO:teuthology.orchestra.run.smithi078.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T15:15:52.827 INFO:teuthology.orchestra.run.smithi078.stdout:legacy client fscid: 1 2024-05-21T15:15:52.827 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:52.827 INFO:teuthology.orchestra.run.smithi078.stdout:Filesystem 'cephfs' (1) 2024-05-21T15:15:52.827 INFO:teuthology.orchestra.run.smithi078.stdout:fs_name cephfs 2024-05-21T15:15:52.827 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 25 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-05-21T15:14:11.918954+0000 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-05-21T15:14:18.291660+0000 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:tableserver 0 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:root 0 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:session_timeout 300 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:session_autoclose 300 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:max_file_size 1099511627776 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:max_xattr_size 65536 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:required_client_features {} 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure 0 2024-05-21T15:15:52.828 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure_osd_epoch 0 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:max_mds 5 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:in 0,1,2,3,4 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:up {0=24527,1=24526,2=24502,3=24448,4=24419} 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:failed 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:damaged 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:stopped 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:data_pools [3] 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:metadata_pool 2 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:inline_data disabled 2024-05-21T15:15:52.829 INFO:teuthology.orchestra.run.smithi078.stdout:balancer 2024-05-21T15:15:52.830 INFO:teuthology.orchestra.run.smithi078.stdout:bal_rank_mask -1 2024-05-21T15:15:52.830 INFO:teuthology.orchestra.run.smithi078.stdout:standby_count_wanted 0 2024-05-21T15:15:52.830 INFO:teuthology.orchestra.run.smithi078.stdout:qdb_cluster leader: 24527 members: 24526,24527,24502,24448,24419 2024-05-21T15:15:52.830 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b{0:24527} state up:active seq 3 addr [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.830 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.f{0:24490} state up:standby-replay seq 1 addr [v2:172.21.15.172:6834/204442136,v1:172.21.15.172:6835/204442136] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.830 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.i{1:24526} state up:active seq 6 addr [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.830 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.h{1:24479} state up:standby-replay seq 1 addr [v2:172.21.15.84:6836/3570369413,v1:172.21.15.84:6837/3570369413] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.830 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c{2:24502} state up:active seq 7 addr [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.e{2:24455} state up:standby-replay seq 1 addr [v2:172.21.15.84:6834/4071213416,v1:172.21.15.84:6835/4071213416] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.l{3:24448} state up:active seq 12 addr [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a{3:14679} state up:standby-replay seq 1 addr [v2:172.21.15.78:6840/2631303889,v1:172.21.15.78:6841/2631303889] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.k{4:24419} state up:active seq 11 addr [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.g{4:14667} state up:standby-replay seq 1 addr [v2:172.21.15.78:6838/3881579182,v1:172.21.15.78:6839/3881579182] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout:Standby daemons: 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.j{-1:14574} state up:standby seq 1 addr [v2:172.21.15.78:6834/4003323899,v1:172.21.15.78:6835/4003323899] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.831 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d{-1:14592} state up:standby seq 1 addr [v2:172.21.15.78:6836/3299007119,v1:172.21.15.78:6837/3299007119] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5640a2ca0 msgr2=0x7fc5640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5640a2ca0 0x7fc5640a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc568005950 tx=0x7fc568005f50 comp rx=0 tx=0).stop 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578102fe0 msgr2=0x7fc578198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578102fe0 0x7fc578198c60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc574007910 tx=0x7fc574007c20 comp rx=0 tx=0).stop 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 shutdown_connections 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5640a2ca0 0x7fc5640a51b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578103b40 0x7fc57819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc578102600 0x7fc578198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 --2- 172.21.15.78:0/1457708541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc578102fe0 0x7fc578198c60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:52.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 >> 172.21.15.78:0/1457708541 conn(0x7fc5780fce90 msgr2=0x7fc5781065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:52.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 shutdown_connections 2024-05-21T15:15:52.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:52.826+0000 7fc57f74f700 1 -- 172.21.15.78:0/1457708541 wait complete. 2024-05-21T15:15:52.834 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-05-21T15:15:53.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:53 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1457708541' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T15:15:53.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-21T15:15:53.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:53 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1457708541' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T15:15:53.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:53 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1457708541' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T15:15:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:15:54.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:54 smithi078 ceph-mon[31997]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:15:54.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:54 smithi084 ceph-mon[36334]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:15:54.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:54 smithi172 ceph-mon[36402]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:15:55.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 -- 172.21.15.78:0/2412840886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 msgr2=0x7f9efc105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 --2- 172.21.15.78:0/2412840886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 0x7f9efc105d70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009990 tx=0x7f9eec009ca0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 -- 172.21.15.78:0/2412840886 shutdown_connections 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 --2- 172.21.15.78:0/2412840886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9efc108e70 0x7f9efc10b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 --2- 172.21.15.78:0/2412840886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 0x7f9efc105d70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 --2- 172.21.15.78:0/2412840886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc106340 0x7f9efc1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 -- 172.21.15.78:0/2412840886 >> 172.21.15.78:0/2412840886 conn(0x7f9efc0fd700 msgr2=0x7f9efc0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 -- 172.21.15.78:0/2412840886 shutdown_connections 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.422+0000 7f9f0228f700 1 -- 172.21.15.78:0/2412840886 wait complete. 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9f0228f700 1 Processor -- start 2024-05-21T15:15:55.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9f0228f700 1 -- start start 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9f0228f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 0x7f9efc1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9f0228f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc106340 0x7f9efc199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9f0228f700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9efc108e70 0x7f9efc1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9f0228f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10edd0 con 0x7f9efc06b110 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9f0228f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10ec50 con 0x7f9efc106340 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9f0228f700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10ef50 con 0x7f9efc108e70 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9efbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 0x7f9efc1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9efbfff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 0x7f9efc1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44300/0 (socket says 172.21.15.78:44300) 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.423+0000 7f9efbfff700 1 -- 172.21.15.78:0/1058505275 learned_addr learned my addr 172.21.15.78:0/1058505275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:55.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9efb7fe700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc106340 0x7f9efc199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9f0082c700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9efc108e70 0x7f9efc1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9efbfff700 1 -- 172.21.15.78:0/1058505275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9efc108e70 msgr2=0x7f9efc1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9efbfff700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9efc108e70 0x7f9efc1a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9efbfff700 1 -- 172.21.15.78:0/1058505275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc106340 msgr2=0x7f9efc199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9efbfff700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc106340 0x7f9efc199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9efbfff700 1 -- 172.21.15.78:0/1058505275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc1a1440 con 0x7f9efc06b110 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9efb7fe700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc106340 0x7f9efc199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9efbfff700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 0x7f9efc1998e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0098e0 tx=0x7f9eec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9ef97fa700 1 -- 172.21.15.78:0/1058505275 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec003c70 con 0x7f9efc06b110 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9ef97fa700 1 -- 172.21.15.78:0/1058505275 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eec004830 con 0x7f9efc06b110 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9efc1a16d0 con 0x7f9efc06b110 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9ef97fa700 1 -- 172.21.15.78:0/1058505275 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec005450 con 0x7f9efc06b110 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.424+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9efc1a1ba0 con 0x7f9efc06b110 2024-05-21T15:15:55.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.425+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9efc050170 con 0x7f9efc06b110 2024-05-21T15:15:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.426+0000 7f9ef97fa700 1 -- 172.21.15.78:0/1058505275 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9eec003e10 con 0x7f9efc06b110 2024-05-21T15:15:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.427+0000 7f9ef97fa700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9ee40a2ca0 0x7f9ee40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:55.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.427+0000 7f9efb7fe700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9ee40a2ca0 0x7f9ee40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.427+0000 7f9ef97fa700 1 -- 172.21.15.78:0/1058505275 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9eec0d04f0 con 0x7f9efc06b110 2024-05-21T15:15:55.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.427+0000 7f9efb7fe700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9ee40a2ca0 0x7f9ee40a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9efc19a990 tx=0x7f9ef0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:55.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.430+0000 7f9ef97fa700 1 -- 172.21.15.78:0/1058505275 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9eec013070 con 0x7f9efc06b110 2024-05-21T15:15:55.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.636+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f9efc063250 con 0x7f9efc06b110 2024-05-21T15:15:55.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.637+0000 7f9ef97fa700 1 -- 172.21.15.78:0/1058505275 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5309 (secure 0 0 0) 0x7f9eec09d4c0 con 0x7f9efc06b110 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 87 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:fsid 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-05-21T15:06:20.984577+0000 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-05-21T15:15:16.363058+0000 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:crush_version 26 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:full_ratio 0.95 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:backfillfull_ratio 0.9 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:nearfull_ratio 0.85 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:require_min_compat_client luminous 2024-05-21T15:15:55.640 INFO:teuthology.orchestra.run.smithi078.stdout:min_compat_client luminous 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:require_osd_release squid 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:stretch_mode_enabled false 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:max_osd 12 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6802/780655399,v1:172.21.15.78:6803/780655399] [v2:172.21.15.78:6804/780655399,v1:172.21.15.78:6805/780655399] exists,up d3fb7e10-ca25-4585-becc-d4ff4784cb9b 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6800/3339409975,v1:172.21.15.84:6801/3339409975] [v2:172.21.15.84:6802/3339409975,v1:172.21.15.84:6803/3339409975] exists,up 549cfd32-c874-4e14-9489-c7fb33b2ee68 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6800/55763200,v1:172.21.15.172:6801/55763200] [v2:172.21.15.172:6802/55763200,v1:172.21.15.172:6803/55763200] exists,up 233933e5-4dde-47f6-b007-015683591374 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6810/586567339,v1:172.21.15.78:6811/586567339] [v2:172.21.15.78:6812/586567339,v1:172.21.15.78:6813/586567339] exists,up 8bb0ba8c-de4c-4334-a4cf-dda60352271a 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6808/2716060044,v1:172.21.15.84:6809/2716060044] [v2:172.21.15.84:6810/2716060044,v1:172.21.15.84:6811/2716060044] exists,up 727cecb4-86c9-438c-9c09-7468e39aad27 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6808/3395704434,v1:172.21.15.172:6809/3395704434] [v2:172.21.15.172:6810/3395704434,v1:172.21.15.172:6811/3395704434] exists,up 2cc5c54e-742b-4406-80d2-e84e0ba50b8a 2024-05-21T15:15:55.641 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6818/4029644708,v1:172.21.15.78:6819/4029644708] [v2:172.21.15.78:6820/4029644708,v1:172.21.15.78:6821/4029644708] exists,up 2a3eb06b-7754-4293-bb69-301f02709e7d 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6816/429797253,v1:172.21.15.84:6817/429797253] [v2:172.21.15.84:6818/429797253,v1:172.21.15.84:6819/429797253] exists,up 9155447f-a10d-4e00-8f4f-a891a095b823 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6816/3263777193,v1:172.21.15.172:6817/3263777193] [v2:172.21.15.172:6818/3263777193,v1:172.21.15.172:6819/3263777193] exists,up 26bbfd7f-c916-46e3-8908-19f65e5350a3 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6826/2889072877,v1:172.21.15.78:6827/2889072877] [v2:172.21.15.78:6828/2889072877,v1:172.21.15.78:6829/2889072877] exists,up 92534fe2-a86c-45e6-b3f1-b436d49fb37d 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6824/1416740807,v1:172.21.15.84:6825/1416740807] [v2:172.21.15.84:6826/1416740807,v1:172.21.15.84:6827/1416740807] exists,up e378331e-ef86-4d80-a184-6acc0d9283b2 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6824/1663785983,v1:172.21.15.172:6825/1663785983] [v2:172.21.15.172:6826/1663785983,v1:172.21.15.172:6827/1663785983] exists,up 07bc1da2-963e-42e0-9b37-16a9a1be16c5 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.6 [8,11] 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.a [2,11] 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.b [2,1] 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.1f [8,4] 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.0 [5,9] 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.e [4,7] 2024-05-21T15:15:55.642 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.11 [4,8] 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.1b [3,6] 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.28 [5,8] 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.31 [4,6] 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/3694112952 expires 2024-05-22T15:07:15.158228+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1173534629 expires 2024-05-22T15:07:15.158228+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/3341206163 expires 2024-05-22T15:07:15.158228+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/888544333 expires 2024-05-22T15:06:47.981082+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/3694112952 expires 2024-05-22T15:07:15.158228+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1077949258 expires 2024-05-22T15:06:47.981082+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/888544333 expires 2024-05-22T15:06:47.981082+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2800502330 expires 2024-05-22T15:07:15.158228+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2303085949 expires 2024-05-22T15:06:47.981082+0000 2024-05-21T15:15:55.643 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2144862462 expires 2024-05-22T15:06:47.981082+0000 2024-05-21T15:15:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9ee40a2ca0 msgr2=0x7f9ee40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9ee40a2ca0 0x7f9ee40a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9efc19a990 tx=0x7f9ef0008040 comp rx=0 tx=0).stop 2024-05-21T15:15:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 msgr2=0x7f9efc1998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 0x7f9efc1998e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0098e0 tx=0x7f9eec0047b0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 shutdown_connections 2024-05-21T15:15:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f9ee40a2ca0 0x7f9ee40a51b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9efc108e70 0x7f9efc1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9efc06b110 0x7f9efc1998e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 --2- 172.21.15.78:0/1058505275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9efc106340 0x7f9efc199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 >> 172.21.15.78:0/1058505275 conn(0x7f9efc0fd700 msgr2=0x7f9efc104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 shutdown_connections 2024-05-21T15:15:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:55.640+0000 7f9f0228f700 1 -- 172.21.15.78:0/1058505275 wait complete. 2024-05-21T15:15:56.401 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-21T15:15:56.413 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:15:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 -- 172.21.15.78:0/781689582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c103cf0 msgr2=0x7f766c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 --2- 172.21.15.78:0/781689582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c103cf0 0x7f766c104100 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7660009900 tx=0x7f766002a410 comp rx=0 tx=0).stop 2024-05-21T15:15:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 -- 172.21.15.78:0/781689582 shutdown_connections 2024-05-21T15:15:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 --2- 172.21.15.78:0/781689582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f766c1089f0 0x7f766c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 --2- 172.21.15.78:0/781689582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c103cf0 0x7f766c104100 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 --2- 172.21.15.78:0/781689582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f766c102ab0 0x7f766c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 -- 172.21.15.78:0/781689582 >> 172.21.15.78:0/781689582 conn(0x7f766c0fdcc0 msgr2=0x7f766c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 -- 172.21.15.78:0/781689582 shutdown_connections 2024-05-21T15:15:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 -- 172.21.15.78:0/781689582 wait complete. 2024-05-21T15:15:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.524+0000 7f7671a6c640 1 Processor -- start 2024-05-21T15:15:56.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f7671a6c640 1 -- start start 2024-05-21T15:15:56.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f7671a6c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f766c102ab0 0x7f766c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:56.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f7671a6c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f766c103cf0 0x7f766c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f7671a6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c1089f0 0x7f766c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f7671a6c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f766c10e750 con 0x7f766c1089f0 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f7671a6c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f766c10e5d0 con 0x7f766c102ab0 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f7671a6c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f766c10e8d0 con 0x7f766c103cf0 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c1089f0 0x7f766c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c1089f0 0x7f766c1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44324/0 (socket says 172.21.15.78:44324) 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766bfff640 1 -- 172.21.15.78:0/1997114510 learned_addr learned my addr 172.21.15.78:0/1997114510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766affd640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f766c103cf0 0x7f766c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766b7fe640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f766c102ab0 0x7f766c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766bfff640 1 -- 172.21.15.78:0/1997114510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f766c103cf0 msgr2=0x7f766c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766bfff640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f766c103cf0 0x7f766c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766bfff640 1 -- 172.21.15.78:0/1997114510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f766c102ab0 msgr2=0x7f766c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766bfff640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f766c102ab0 0x7f766c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766bfff640 1 -- 172.21.15.78:0/1997114510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766c1a0cc0 con 0x7f766c1089f0 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.525+0000 7f766affd640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f766c103cf0 0x7f766c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.526+0000 7f766bfff640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c1089f0 0x7f766c1a0700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f766400c970 tx=0x7f766400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.526+0000 7f7668ff9640 1 -- 172.21.15.78:0/1997114510 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664007bb0 con 0x7f766c1089f0 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.526+0000 7f7668ff9640 1 -- 172.21.15.78:0/1997114510 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7664007d50 con 0x7f766c1089f0 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.526+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766c1a0fb0 con 0x7f766c1089f0 2024-05-21T15:15:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.526+0000 7f7668ff9640 1 -- 172.21.15.78:0/1997114510 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664018800 con 0x7f766c1089f0 2024-05-21T15:15:56.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.526+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f766c1a1410 con 0x7f766c1089f0 2024-05-21T15:15:56.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.527+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f766c107800 con 0x7f766c1089f0 2024-05-21T15:15:56.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.528+0000 7f7668ff9640 1 -- 172.21.15.78:0/1997114510 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f766400fa50 con 0x7f766c1089f0 2024-05-21T15:15:56.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.529+0000 7f7668ff9640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f76540a2b90 0x7f76540a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:56.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.529+0000 7f766b7fe640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f76540a2b90 0x7f76540a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:56.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.529+0000 7f7668ff9640 1 -- 172.21.15.78:0/1997114510 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f76640cf320 con 0x7f766c1089f0 2024-05-21T15:15:56.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.529+0000 7f766b7fe640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f76540a2b90 0x7f76540a50a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f765c005c40 tx=0x7f765c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:56.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.532+0000 7f7668ff9640 1 -- 172.21.15.78:0/1997114510 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7664014030 con 0x7f766c1089f0 2024-05-21T15:15:56.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:56 smithi078 ceph-mon[31997]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:15:56.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:56 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1058505275' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T15:15:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:56 smithi084 ceph-mon[36334]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:15:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:56 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1058505275' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T15:15:56.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:56 smithi172 ceph-mon[36402]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:15:56.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:56 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1058505275' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T15:15:56.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.708+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f766c02d4f0 con 0x7f766c1089f0 2024-05-21T15:15:56.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.710+0000 7f7668ff9640 1 -- 172.21.15.78:0/1997114510 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22616 (secure 0 0 0) 0x7f766409c260 con 0x7f766c1089f0 2024-05-21T15:15:56.713 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:56.713 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":87,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:15:16.363058+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:15:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f76540a2b90 msgr2=0x7f76540a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f76540a2b90 0x7f76540a50a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f765c005c40 tx=0x7f765c005bd0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c1089f0 msgr2=0x7f766c1a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c1089f0 0x7f766c1a0700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f766400c970 tx=0x7f766400ce30 comp rx=0 tx=0).stop 2024-05-21T15:15:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 shutdown_connections 2024-05-21T15:15:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f76540a2b90 0x7f76540a50a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f766c1089f0 0x7f766c1a0700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f766c103cf0 0x7f766c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 --2- 172.21.15.78:0/1997114510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f766c102ab0 0x7f766c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 >> 172.21.15.78:0/1997114510 conn(0x7f766c0fdcc0 msgr2=0x7f766c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 shutdown_connections 2024-05-21T15:15:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.712+0000 7f7671a6c640 1 -- 172.21.15.78:0/1997114510 wait complete. 2024-05-21T15:15:56.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:15:56.728 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:15:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.838+0000 7fd629102640 1 -- 172.21.15.78:0/670235768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624104060 msgr2=0x7fd624104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.838+0000 7fd629102640 1 --2- 172.21.15.78:0/670235768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624104060 0x7fd624104470 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd614009900 tx=0x7fd61402a410 comp rx=0 tx=0).stop 2024-05-21T15:15:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 -- 172.21.15.78:0/670235768 shutdown_connections 2024-05-21T15:15:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 --2- 172.21.15.78:0/670235768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624100710 0x7fd624100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 --2- 172.21.15.78:0/670235768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd624101210 0x7fd62410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 --2- 172.21.15.78:0/670235768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624104060 0x7fd624104470 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 -- 172.21.15.78:0/670235768 >> 172.21.15.78:0/670235768 conn(0x7fd6240fc330 msgr2=0x7fd6240fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 -- 172.21.15.78:0/670235768 shutdown_connections 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 -- 172.21.15.78:0/670235768 wait complete. 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 Processor -- start 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.839+0000 7fd629102640 1 -- start start 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd629102640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624100710 0x7fd624103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd629102640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd624101210 0x7fd6241018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd629102640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624104060 0x7fd624101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd629102640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd62410e750 con 0x7fd624100710 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd629102640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd62410e5d0 con 0x7fd624104060 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd629102640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd62410e8d0 con 0x7fd624101210 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624100710 0x7fd624103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624100710 0x7fd624103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44344/0 (socket says 172.21.15.78:44344) 2024-05-21T15:15:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 -- 172.21.15.78:0/1622191463 learned_addr learned my addr 172.21.15.78:0/1622191463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd61bfff640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd624101210 0x7fd6241018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd62359e640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624104060 0x7fd624101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 -- 172.21.15.78:0/1622191463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd624101210 msgr2=0x7fd6241018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd624101210 0x7fd6241018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 -- 172.21.15.78:0/1622191463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624104060 msgr2=0x7fd624101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624104060 0x7fd624101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 -- 172.21.15.78:0/1622191463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd624102520 con 0x7fd624100710 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd622d9d640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624100710 0x7fd624103210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd614031440 tx=0x7fd614032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd620d99640 1 -- 172.21.15.78:0/1622191463 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61403f030 con 0x7fd624100710 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd620d99640 1 -- 172.21.15.78:0/1622191463 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd614003ce0 con 0x7fd624100710 2024-05-21T15:15:56.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6241a33b0 con 0x7fd624100710 2024-05-21T15:15:56.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.840+0000 7fd620d99640 1 -- 172.21.15.78:0/1622191463 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614035070 con 0x7fd624100710 2024-05-21T15:15:56.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.841+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6241a37b0 con 0x7fd624100710 2024-05-21T15:15:56.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.841+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd624069110 con 0x7fd624100710 2024-05-21T15:15:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.843+0000 7fd620d99640 1 -- 172.21.15.78:0/1622191463 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd614042030 con 0x7fd624100710 2024-05-21T15:15:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.843+0000 7fd620d99640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd6100a2b90 0x7fd6100a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.843+0000 7fd61bfff640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd6100a2b90 0x7fd6100a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.844+0000 7fd620d99640 1 -- 172.21.15.78:0/1622191463 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd6140ef950 con 0x7fd624100710 2024-05-21T15:15:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.844+0000 7fd61bfff640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd6100a2b90 0x7fd6100a50a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd624102bb0 tx=0x7fd60c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:56.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:56.847+0000 7fd620d99640 1 -- 172.21.15.78:0/1622191463 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd6140bc890 con 0x7fd624100710 2024-05-21T15:15:57.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.021+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd62402d650 con 0x7fd624100710 2024-05-21T15:15:57.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.022+0000 7fd620d99640 1 -- 172.21.15.78:0/1622191463 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fd62402d650 con 0x7fd624100710 2024-05-21T15:15:57.025 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:15:57.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd6100a2b90 msgr2=0x7fd6100a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd6100a2b90 0x7fd6100a50a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd624102bb0 tx=0x7fd60c005f90 comp rx=0 tx=0).stop 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624100710 msgr2=0x7fd624103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624100710 0x7fd624103210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd614031440 tx=0x7fd614032ce0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 shutdown_connections 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd6100a2b90 0x7fd6100a50a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd624104060 0x7fd624101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd624101210 0x7fd6241018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 --2- 172.21.15.78:0/1622191463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd624100710 0x7fd624103210 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.023+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 >> 172.21.15.78:0/1622191463 conn(0x7fd6240fc330 msgr2=0x7fd624104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.024+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 shutdown_connections 2024-05-21T15:15:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.024+0000 7fd629102640 1 -- 172.21.15.78:0/1622191463 wait complete. 2024-05-21T15:15:57.038 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:15:57.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 -- 172.21.15.78:0/98845246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 msgr2=0x7f7c44109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 --2- 172.21.15.78:0/98845246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 0x7f7c44109cf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009900 tx=0x7f7c3c02a410 comp rx=0 tx=0).stop 2024-05-21T15:15:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 -- 172.21.15.78:0/98845246 shutdown_connections 2024-05-21T15:15:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 --2- 172.21.15.78:0/98845246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c44102bd0 0x7f7c44102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 --2- 172.21.15.78:0/98845246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 0x7f7c44109cf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 --2- 172.21.15.78:0/98845246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c44101990 0x7f7c44102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 -- 172.21.15.78:0/98845246 >> 172.21.15.78:0/98845246 conn(0x7f7c440fcb80 msgr2=0x7f7c440ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 -- 172.21.15.78:0/98845246 shutdown_connections 2024-05-21T15:15:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.148+0000 7f7c494e9640 1 -- 172.21.15.78:0/98845246 wait complete. 2024-05-21T15:15:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.149+0000 7f7c494e9640 1 Processor -- start 2024-05-21T15:15:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.149+0000 7f7c494e9640 1 -- start start 2024-05-21T15:15:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.149+0000 7f7c494e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c44101990 0x7f7c44198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.149+0000 7f7c494e9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c44102bd0 0x7f7c441989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c494e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 0x7f7c4419f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c494e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410d610 con 0x7f7c441078d0 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c494e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410d490 con 0x7f7c44101990 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c494e9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4410d790 con 0x7f7c44102bd0 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 0x7f7c4419f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 0x7f7c4419f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44374/0 (socket says 172.21.15.78:44374) 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 -- 172.21.15.78:0/275754366 learned_addr learned my addr 172.21.15.78:0/275754366 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c44101990 0x7f7c44198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 -- 172.21.15.78:0/275754366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c44102bd0 msgr2=0x7f7c441989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c427fc640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c44102bd0 0x7f7c441989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c44102bd0 0x7f7c441989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 -- 172.21.15.78:0/275754366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c44101990 msgr2=0x7f7c44198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c44101990 0x7f7c44198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 -- 172.21.15.78:0/275754366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4419ffc0 con 0x7f7c441078d0 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c44101990 0x7f7c44198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c437fe640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 0x7f7c4419f950 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009230 tx=0x7f7c3c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c2bfff640 1 -- 172.21.15.78:0/275754366 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c03f030 con 0x7f7c441078d0 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c441a0250 con 0x7f7c441078d0 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c2bfff640 1 -- 172.21.15.78:0/275754366 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c003ce0 con 0x7f7c441078d0 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c2bfff640 1 -- 172.21.15.78:0/275754366 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c035070 con 0x7f7c441078d0 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.150+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441a0790 con 0x7f7c441078d0 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.151+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c441066e0 con 0x7f7c441078d0 2024-05-21T15:15:57.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.153+0000 7f7c2bfff640 1 -- 172.21.15.78:0/275754366 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7c3c042030 con 0x7f7c441078d0 2024-05-21T15:15:57.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.154+0000 7f7c2bfff640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7c2c0a2b90 0x7f7c2c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.154+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7c2c0a2b90 0x7f7c2c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.154+0000 7f7c2bfff640 1 -- 172.21.15.78:0/275754366 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f7c3c0efc60 con 0x7f7c441078d0 2024-05-21T15:15:57.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.154+0000 7f7c42ffd640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7c2c0a2b90 0x7f7c2c0a50a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7c34005fd0 tx=0x7f7c34008310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:57.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.157+0000 7f7c2bfff640 1 -- 172.21.15.78:0/275754366 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c3c0bcba0 con 0x7f7c441078d0 2024-05-21T15:15:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.335+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c4406b250 con 0x7f7c441078d0 2024-05-21T15:15:57.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.336+0000 7f7c2bfff640 1 -- 172.21.15.78:0/275754366 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f7c3c0bc4a0 con 0x7f7c441078d0 2024-05-21T15:15:57.339 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.337+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7c2c0a2b90 msgr2=0x7f7c2c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.337+0000 7f7c494e9640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7c2c0a2b90 0x7f7c2c0a50a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7c34005fd0 tx=0x7f7c34008310 comp rx=0 tx=0).stop 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.337+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 msgr2=0x7f7c4419f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.337+0000 7f7c494e9640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 0x7f7c4419f950 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009230 tx=0x7f7c3c004430 comp rx=0 tx=0).stop 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.338+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 shutdown_connections 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.338+0000 7f7c494e9640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f7c2c0a2b90 0x7f7c2c0a50a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.338+0000 7f7c494e9640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c441078d0 0x7f7c4419f950 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.338+0000 7f7c494e9640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c44102bd0 0x7f7c441989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.338+0000 7f7c494e9640 1 --2- 172.21.15.78:0/275754366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c44101990 0x7f7c44198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.338+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 >> 172.21.15.78:0/275754366 conn(0x7f7c440fcb80 msgr2=0x7f7c441038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:57.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.338+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 shutdown_connections 2024-05-21T15:15:57.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.338+0000 7f7c494e9640 1 -- 172.21.15.78:0/275754366 wait complete. 2024-05-21T15:15:57.353 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 -- 172.21.15.78:0/2961908966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 msgr2=0x7f9230100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 --2- 172.21.15.78:0/2961908966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 0x7f9230100b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9218009900 tx=0x7f921802a410 comp rx=0 tx=0).stop 2024-05-21T15:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 -- 172.21.15.78:0/2961908966 shutdown_connections 2024-05-21T15:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 --2- 172.21.15.78:0/2961908966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92301011f0 0x7f923010ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 --2- 172.21.15.78:0/2961908966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 0x7f9230100b60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 --2- 172.21.15.78:0/2961908966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9230104040 0x7f9230104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 -- 172.21.15.78:0/2961908966 >> 172.21.15.78:0/2961908966 conn(0x7f92300fc2f0 msgr2=0x7f92300fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 -- 172.21.15.78:0/2961908966 shutdown_connections 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 -- 172.21.15.78:0/2961908966 wait complete. 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.467+0000 7f9235c92640 1 Processor -- start 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 -- start start 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 0x7f92301992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92301011f0 0x7f9230199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9230104040 0x7f92301a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f923010e700 con 0x7f92301006f0 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f923010e580 con 0x7f92301011f0 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f923010e880 con 0x7f9230104040 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 0x7f92301992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 0x7f92301992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44396/0 (socket says 172.21.15.78:44396) 2024-05-21T15:15:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 -- 172.21.15.78:0/547885738 learned_addr learned my addr 172.21.15.78:0/547885738 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922ffff640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9230104040 0x7f92301a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9227fff640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92301011f0 0x7f9230199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 -- 172.21.15.78:0/547885738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9230104040 msgr2=0x7f92301a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9230104040 0x7f92301a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 -- 172.21.15.78:0/547885738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92301011f0 msgr2=0x7f9230199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92301011f0 0x7f9230199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 -- 172.21.15.78:0/547885738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92301a0d60 con 0x7f92301006f0 2024-05-21T15:15:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9227fff640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92301011f0 0x7f9230199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922f7fe640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 0x7f92301992d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f922000e9f0 tx=0x7f922000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922d7fa640 1 -- 172.21.15.78:0/547885738 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922000ccc0 con 0x7f92301006f0 2024-05-21T15:15:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922d7fa640 1 -- 172.21.15.78:0/547885738 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92200035f0 con 0x7f92301006f0 2024-05-21T15:15:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 -- 172.21.15.78:0/547885738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92301a0ff0 con 0x7f92301006f0 2024-05-21T15:15:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f922d7fa640 1 -- 172.21.15.78:0/547885738 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922000ce60 con 0x7f92301006f0 2024-05-21T15:15:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.468+0000 7f9235c92640 1 -- 172.21.15.78:0/547885738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301a14b0 con 0x7f92301006f0 2024-05-21T15:15:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.469+0000 7f92267fc640 1 -- 172.21.15.78:0/547885738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9214005290 con 0x7f92301006f0 2024-05-21T15:15:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.471+0000 7f922d7fa640 1 -- 172.21.15.78:0/547885738 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f92200037a0 con 0x7f92301006f0 2024-05-21T15:15:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.471+0000 7f922d7fa640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f921c0a2b90 0x7f921c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.471+0000 7f9227fff640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f921c0a2b90 0x7f921c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.471+0000 7f922d7fa640 1 -- 172.21.15.78:0/547885738 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f92200cf490 con 0x7f92301006f0 2024-05-21T15:15:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.472+0000 7f9227fff640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f921c0a2b90 0x7f921c0a50a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9218000c00 tx=0x7f9218031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.475+0000 7f922d7fa640 1 -- 172.21.15.78:0/547885738 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f922009c3d0 con 0x7f92301006f0 2024-05-21T15:15:57.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:57 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1997114510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:57.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:57 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1622191463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:57.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:57 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/275754366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.650+0000 7f92267fc640 1 -- 172.21.15.78:0/547885738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9214005850 con 0x7f92301006f0 2024-05-21T15:15:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.650+0000 7f922d7fa640 1 -- 172.21.15.78:0/547885738 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f922009bcd0 con 0x7f92301006f0 2024-05-21T15:15:57.653 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:15:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 -- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f921c0a2b90 msgr2=0x7f921c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f921c0a2b90 0x7f921c0a50a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9218000c00 tx=0x7f9218031400 comp rx=0 tx=0).stop 2024-05-21T15:15:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 -- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 msgr2=0x7f92301992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 0x7f92301992d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f922000e9f0 tx=0x7f922000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 -- 172.21.15.78:0/547885738 shutdown_connections 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f921c0a2b90 0x7f921c0a50a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92301011f0 0x7f9230199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9230104040 0x7f92301a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 --2- 172.21.15.78:0/547885738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92301006f0 0x7f92301992d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 -- 172.21.15.78:0/547885738 >> 172.21.15.78:0/547885738 conn(0x7f92300fc2f0 msgr2=0x7f9230104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.652+0000 7f92267fc640 1 -- 172.21.15.78:0/547885738 shutdown_connections 2024-05-21T15:15:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.653+0000 7f92267fc640 1 -- 172.21.15.78:0/547885738 wait complete. 2024-05-21T15:15:57.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:57 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1997114510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:57.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:57 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1622191463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:57.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:57 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/275754366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:57.667 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:15:57.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:57 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1997114510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:57.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:57 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1622191463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:57.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:57 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/275754366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 -- 172.21.15.78:0/1081081333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8541078f0 msgr2=0x7fe854109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 --2- 172.21.15.78:0/1081081333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8541078f0 0x7fe854109d10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009ca0 tx=0x7fe84c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 -- 172.21.15.78:0/1081081333 shutdown_connections 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 --2- 172.21.15.78:0/1081081333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe854102bf0 0x7fe854103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 --2- 172.21.15.78:0/1081081333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8541078f0 0x7fe854109d10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 --2- 172.21.15.78:0/1081081333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8541019b0 0x7fe8541026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 -- 172.21.15.78:0/1081081333 >> 172.21.15.78:0/1081081333 conn(0x7fe8540fcba0 msgr2=0x7fe8540ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 -- 172.21.15.78:0/1081081333 shutdown_connections 2024-05-21T15:15:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.774+0000 7fe85a27a640 1 -- 172.21.15.78:0/1081081333 wait complete. 2024-05-21T15:15:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe85a27a640 1 Processor -- start 2024-05-21T15:15:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe85a27a640 1 -- start start 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe85a27a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8541019b0 0x7fe8540747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe85a27a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe854102bf0 0x7fe85419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe85a27a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8541078f0 0x7fe8541a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe85a27a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe85410d630 con 0x7fe854102bf0 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe85a27a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe85410d4b0 con 0x7fe8541078f0 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe85a27a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe85410d7b0 con 0x7fe8541019b0 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.775+0000 7fe8537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe854102bf0 0x7fe85419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe854102bf0 0x7fe85419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44412/0 (socket says 172.21.15.78:44412) 2024-05-21T15:15:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8537fe640 1 -- 172.21.15.78:0/289353873 learned_addr learned my addr 172.21.15.78:0/289353873 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe858817640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8541078f0 0x7fe8541a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe853fff640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8541019b0 0x7fe8540747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8537fe640 1 -- 172.21.15.78:0/289353873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8541019b0 msgr2=0x7fe8540747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8537fe640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8541019b0 0x7fe8540747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8537fe640 1 -- 172.21.15.78:0/289353873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8541078f0 msgr2=0x7fe8541a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8537fe640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8541078f0 0x7fe8541a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8537fe640 1 -- 172.21.15.78:0/289353873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8541a0d30 con 0x7fe854102bf0 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe853fff640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8541019b0 0x7fe8540747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8537fe640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe854102bf0 0x7fe85419dfc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00d950 tx=0x7fe83c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8517fa640 1 -- 172.21.15.78:0/289353873 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c017030 con 0x7fe854102bf0 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8517fa640 1 -- 172.21.15.78:0/289353873 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe83c004020 con 0x7fe854102bf0 2024-05-21T15:15:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8541a0fc0 con 0x7fe854102bf0 2024-05-21T15:15:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe8517fa640 1 -- 172.21.15.78:0/289353873 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0047a0 con 0x7fe854102bf0 2024-05-21T15:15:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.776+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8541a14b0 con 0x7fe854102bf0 2024-05-21T15:15:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.777+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe854105ff0 con 0x7fe854102bf0 2024-05-21T15:15:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.779+0000 7fe8517fa640 1 -- 172.21.15.78:0/289353873 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe83c0041d0 con 0x7fe854102bf0 2024-05-21T15:15:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.779+0000 7fe8517fa640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8400a2b90 0x7fe8400a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.779+0000 7fe853fff640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8400a2b90 0x7fe8400a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.779+0000 7fe8517fa640 1 -- 172.21.15.78:0/289353873 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe83c0cf1a0 con 0x7fe854102bf0 2024-05-21T15:15:57.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.780+0000 7fe853fff640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8400a2b90 0x7fe8400a50a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8440058c0 tx=0x7fe844005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:57.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.783+0000 7fe8517fa640 1 -- 172.21.15.78:0/289353873 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe83c09c0e0 con 0x7fe854102bf0 2024-05-21T15:15:57.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.994+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe85402d2a0 con 0x7fe854102bf0 2024-05-21T15:15:57.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.995+0000 7fe8517fa640 1 -- 172.21.15.78:0/289353873 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11925 (secure 0 0 0) 0x7fe83c09b9e0 con 0x7fe854102bf0 2024-05-21T15:15:57.997 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:57.998 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:14:18.291660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:15:57.998 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-05-21T15:15:58.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8400a2b90 msgr2=0x7fe8400a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8400a2b90 0x7fe8400a50a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8440058c0 tx=0x7fe844005d50 comp rx=0 tx=0).stop 2024-05-21T15:15:58.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe854102bf0 msgr2=0x7fe85419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe854102bf0 0x7fe85419dfc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00d950 tx=0x7fe83c00de10 comp rx=0 tx=0).stop 2024-05-21T15:15:58.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 shutdown_connections 2024-05-21T15:15:58.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fe8400a2b90 0x7fe8400a50a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8541078f0 0x7fe8541a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe854102bf0 0x7fe85419dfc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 --2- 172.21.15.78:0/289353873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8541019b0 0x7fe8540747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 >> 172.21.15.78:0/289353873 conn(0x7fe8540fcba0 msgr2=0x7fe8540ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:58.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 shutdown_connections 2024-05-21T15:15:58.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:57.997+0000 7fe85a27a640 1 -- 172.21.15.78:0/289353873 wait complete. 2024-05-21T15:15:58.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:15:58.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 -- 172.21.15.78:0/3451276577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 msgr2=0x7f4880102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 --2- 172.21.15.78:0/3451276577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 0x7f4880102fe0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4874009900 tx=0x7f487402a410 comp rx=0 tx=0).stop 2024-05-21T15:15:58.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 -- 172.21.15.78:0/3451276577 shutdown_connections 2024-05-21T15:15:58.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 --2- 172.21.15.78:0/3451276577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f48801078d0 0x7f4880109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 --2- 172.21.15.78:0/3451276577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 0x7f4880102fe0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 --2- 172.21.15.78:0/3451276577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4880101990 0x7f4880102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 -- 172.21.15.78:0/3451276577 >> 172.21.15.78:0/3451276577 conn(0x7f48800fcba0 msgr2=0x7f48800ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:58.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 -- 172.21.15.78:0/3451276577 shutdown_connections 2024-05-21T15:15:58.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.121+0000 7f4887a73640 1 -- 172.21.15.78:0/3451276577 wait complete. 2024-05-21T15:15:58.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4887a73640 1 Processor -- start 2024-05-21T15:15:58.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4887a73640 1 -- start start 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4887a73640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880101990 0x7f4880199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4887a73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 0x7f48801998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4887a73640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48801078d0 0x7f48801a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4887a73640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f488010e7b0 con 0x7f4880102bd0 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4887a73640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f488010e630 con 0x7f4880101990 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4887a73640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f488010e930 con 0x7f48801078d0 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 0x7f48801998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 0x7f48801998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44430/0 (socket says 172.21.15.78:44430) 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 -- 172.21.15.78:0/3466417790 learned_addr learned my addr 172.21.15.78:0/3466417790 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488580f640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880101990 0x7f4880199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f4886010640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48801078d0 0x7f48801a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 -- 172.21.15.78:0/3466417790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48801078d0 msgr2=0x7f48801a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48801078d0 0x7f48801a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 -- 172.21.15.78:0/3466417790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880101990 msgr2=0x7f4880199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880101990 0x7f4880199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 -- 172.21.15.78:0/3466417790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48801a0ec0 con 0x7f4880102bd0 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.122+0000 7f488500e640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 0x7f48801998c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4874005ab0 tx=0x7f4874032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.123+0000 7f487effd640 1 -- 172.21.15.78:0/3466417790 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487403f030 con 0x7f4880102bd0 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.123+0000 7f487effd640 1 -- 172.21.15.78:0/3466417790 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4874003ce0 con 0x7f4880102bd0 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.123+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48801a1150 con 0x7f4880102bd0 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.123+0000 7f487effd640 1 -- 172.21.15.78:0/3466417790 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874035070 con 0x7f4880102bd0 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.123+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801a1710 con 0x7f4880102bd0 2024-05-21T15:15:58.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.124+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48801066e0 con 0x7f4880102bd0 2024-05-21T15:15:58.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.125+0000 7f487effd640 1 -- 172.21.15.78:0/3466417790 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4874042030 con 0x7f4880102bd0 2024-05-21T15:15:58.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.125+0000 7f487effd640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f48680a2c10 0x7f48680a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.126+0000 7f488580f640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f48680a2c10 0x7f48680a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.126+0000 7f487effd640 1 -- 172.21.15.78:0/3466417790 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f48740ef900 con 0x7f4880102bd0 2024-05-21T15:15:58.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.126+0000 7f488580f640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f48680a2c10 0x7f48680a5120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4880102590 tx=0x7f4878009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:58.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.129+0000 7f487effd640 1 -- 172.21.15.78:0/3466417790 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f48740bc840 con 0x7f4880102bd0 2024-05-21T15:15:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.308+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4880062750 con 0x7f4880102bd0 2024-05-21T15:15:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.310+0000 7f487effd640 1 -- 172.21.15.78:0/3466417790 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22616 (secure 0 0 0) 0x7f48740bc140 con 0x7f4880102bd0 2024-05-21T15:15:58.313 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:58.313 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":87,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:15:16.363058+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:15:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f48680a2c10 msgr2=0x7f48680a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f48680a2c10 0x7f48680a5120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4880102590 tx=0x7f4878009110 comp rx=0 tx=0).stop 2024-05-21T15:15:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 msgr2=0x7f48801998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 0x7f48801998c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4874005ab0 tx=0x7f4874032ce0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 shutdown_connections 2024-05-21T15:15:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f48680a2c10 0x7f48680a5120 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48801078d0 0x7f48801a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4880102bd0 0x7f48801998c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 --2- 172.21.15.78:0/3466417790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4880101990 0x7f4880199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 >> 172.21.15.78:0/3466417790 conn(0x7f48800fcba0 msgr2=0x7f48801038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 shutdown_connections 2024-05-21T15:15:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.312+0000 7f4887a73640 1 -- 172.21.15.78:0/3466417790 wait complete. 2024-05-21T15:15:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:15:58.327 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:15:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 -- 172.21.15.78:0/483135681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c108710 msgr2=0x7fc57c10ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 --2- 172.21.15.78:0/483135681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c108710 0x7fc57c10ab30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc574009ca0 tx=0x7fc57402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 -- 172.21.15.78:0/483135681 shutdown_connections 2024-05-21T15:15:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 --2- 172.21.15.78:0/483135681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c105db0 0x7fc57c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 --2- 172.21.15.78:0/483135681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c108710 0x7fc57c10ab30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 --2- 172.21.15.78:0/483135681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57c103450 0x7fc57c105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 -- 172.21.15.78:0/483135681 >> 172.21.15.78:0/483135681 conn(0x7fc57c0fca50 msgr2=0x7fc57c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 -- 172.21.15.78:0/483135681 shutdown_connections 2024-05-21T15:15:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.438+0000 7fc582b9d640 1 -- 172.21.15.78:0/483135681 wait complete. 2024-05-21T15:15:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc582b9d640 1 Processor -- start 2024-05-21T15:15:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc582b9d640 1 -- start start 2024-05-21T15:15:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc582b9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c103450 0x7fc57c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc582b9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57c105db0 0x7fc57c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc582b9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c108710 0x7fc57c1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc582b9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c10e570 con 0x7fc57c103450 2024-05-21T15:15:58.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc582b9d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c10e3f0 con 0x7fc57c108710 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc582b9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c10e6f0 con 0x7fc57c105db0 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc581b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c103450 0x7fc57c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc581b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c103450 0x7fc57c1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44450/0 (socket says 172.21.15.78:44450) 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.439+0000 7fc581b9b640 1 -- 172.21.15.78:0/2470285824 learned_addr learned my addr 172.21.15.78:0/2470285824 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc58239c640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c108710 0x7fc57c1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc58139a640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57c105db0 0x7fc57c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc581b9b640 1 -- 172.21.15.78:0/2470285824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57c105db0 msgr2=0x7fc57c199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc581b9b640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57c105db0 0x7fc57c199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc581b9b640 1 -- 172.21.15.78:0/2470285824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c108710 msgr2=0x7fc57c1a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc581b9b640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c108710 0x7fc57c1a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc581b9b640 1 -- 172.21.15.78:0/2470285824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57c1a0b70 con 0x7fc57c103450 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc581b9b640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c103450 0x7fc57c1990e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c970 tx=0x7fc57000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc56effd640 1 -- 172.21.15.78:0/2470285824 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570007bb0 con 0x7fc57c103450 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc56effd640 1 -- 172.21.15.78:0/2470285824 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc570007d50 con 0x7fc57c103450 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57c1a0e60 con 0x7fc57c103450 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc56effd640 1 -- 172.21.15.78:0/2470285824 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5700187a0 con 0x7fc57c103450 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.440+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc57c1a12c0 con 0x7fc57c103450 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.441+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57c0fe570 con 0x7fc57c103450 2024-05-21T15:15:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.442+0000 7fc56effd640 1 -- 172.21.15.78:0/2470285824 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc57000f450 con 0x7fc57c103450 2024-05-21T15:15:58.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.443+0000 7fc56effd640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5680a2b80 0x7fc5680a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.443+0000 7fc56effd640 1 -- 172.21.15.78:0/2470285824 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc5700cf140 con 0x7fc57c103450 2024-05-21T15:15:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.443+0000 7fc58139a640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5680a2b80 0x7fc5680a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.443+0000 7fc58139a640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5680a2b80 0x7fc5680a5090 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc578005fd0 tx=0x7fc578005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.447+0000 7fc56effd640 1 -- 172.21.15.78:0/2470285824 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc570014030 con 0x7fc57c103450 2024-05-21T15:15:58.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:58 smithi078 ceph-mon[31997]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:15:58.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:58 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/547885738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:58.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:58 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/289353873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:58.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:58 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3466417790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.623+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc57c02d2a0 con 0x7fc57c103450 2024-05-21T15:15:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.623+0000 7fc56effd640 1 -- 172.21.15.78:0/2470285824 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fc57009ba30 con 0x7fc57c103450 2024-05-21T15:15:58.626 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:15:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5680a2b80 msgr2=0x7fc5680a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5680a2b80 0x7fc5680a5090 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc578005fd0 tx=0x7fc578005ce0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c103450 msgr2=0x7fc57c1990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c103450 0x7fc57c1990e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc57000c970 tx=0x7fc57000ce30 comp rx=0 tx=0).stop 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 shutdown_connections 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc5680a2b80 0x7fc5680a5090 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c108710 0x7fc57c1a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc57c105db0 0x7fc57c199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 --2- 172.21.15.78:0/2470285824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc57c103450 0x7fc57c1990e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 >> 172.21.15.78:0/2470285824 conn(0x7fc57c0fca50 msgr2=0x7fc57c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 shutdown_connections 2024-05-21T15:15:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.625+0000 7fc582b9d640 1 -- 172.21.15.78:0/2470285824 wait complete. 2024-05-21T15:15:58.640 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:15:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:58 smithi084 ceph-mon[36334]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:15:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:58 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/547885738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:58 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/289353873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:58 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3466417790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:58.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:58 smithi172 ceph-mon[36402]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-21T15:15:58.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:58 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/547885738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:58.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:58 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/289353873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:15:58.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:58 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3466417790' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:15:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 -- 172.21.15.78:0/3663826399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420102aa0 msgr2=0x7f54201037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 --2- 172.21.15.78:0/3663826399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420102aa0 0x7f54201037a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5414009900 tx=0x7f541402a410 comp rx=0 tx=0).stop 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 -- 172.21.15.78:0/3663826399 shutdown_connections 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 --2- 172.21.15.78:0/3663826399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54201089e0 0x7f542010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 --2- 172.21.15.78:0/3663826399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5420103ce0 0x7f54201040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 --2- 172.21.15.78:0/3663826399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420102aa0 0x7f54201037a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 -- 172.21.15.78:0/3663826399 >> 172.21.15.78:0/3663826399 conn(0x7f54200fdc90 msgr2=0x7f5420100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 -- 172.21.15.78:0/3663826399 shutdown_connections 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.751+0000 7f5427806640 1 -- 172.21.15.78:0/3663826399 wait complete. 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5427806640 1 Processor -- start 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5427806640 1 -- start start 2024-05-21T15:15:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5427806640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420102aa0 0x7f54201992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5427806640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420103ce0 0x7f5420199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5427806640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54201089e0 0x7f54201a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5427806640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f542010e6f0 con 0x7f5420103ce0 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5427806640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f542010e570 con 0x7f5420102aa0 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5427806640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f542010e870 con 0x7f54201089e0 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5424da1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420103ce0 0x7f5420199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5424da1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420103ce0 0x7f5420199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49254/0 (socket says 172.21.15.78:49254) 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5424da1640 1 -- 172.21.15.78:0/272752209 learned_addr learned my addr 172.21.15.78:0/272752209 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f54255a2640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420102aa0 0x7f54201992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5425da3640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54201089e0 0x7f54201a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5424da1640 1 -- 172.21.15.78:0/272752209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54201089e0 msgr2=0x7f54201a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5424da1640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54201089e0 0x7f54201a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5424da1640 1 -- 172.21.15.78:0/272752209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420102aa0 msgr2=0x7f54201992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5424da1640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420102aa0 0x7f54201992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5424da1640 1 -- 172.21.15.78:0/272752209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54201a0e10 con 0x7f5420103ce0 2024-05-21T15:15:58.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f5425da3640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54201089e0 0x7f54201a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:58.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.752+0000 7f54255a2640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420102aa0 0x7f54201992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:15:58.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.753+0000 7f5424da1640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420103ce0 0x7f5420199810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c970 tx=0x7f541c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:58.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.753+0000 7f54127fc640 1 -- 172.21.15.78:0/272752209 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c007bb0 con 0x7f5420103ce0 2024-05-21T15:15:58.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.753+0000 7f54127fc640 1 -- 172.21.15.78:0/272752209 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541c007d50 con 0x7f5420103ce0 2024-05-21T15:15:58.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.753+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54201a1100 con 0x7f5420103ce0 2024-05-21T15:15:58.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.753+0000 7f54127fc640 1 -- 172.21.15.78:0/272752209 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c00f450 con 0x7f5420103ce0 2024-05-21T15:15:58.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.753+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54201a1600 con 0x7f5420103ce0 2024-05-21T15:15:58.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.754+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54201077f0 con 0x7f5420103ce0 2024-05-21T15:15:58.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.755+0000 7f54127fc640 1 -- 172.21.15.78:0/272752209 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f541c004fe0 con 0x7f5420103ce0 2024-05-21T15:15:58.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.756+0000 7f54127fc640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f540c0a2b90 0x7f540c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:58.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.756+0000 7f54255a2640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f540c0a2b90 0x7f540c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:58.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.756+0000 7f54127fc640 1 -- 172.21.15.78:0/272752209 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f541c0cf2e0 con 0x7f5420103ce0 2024-05-21T15:15:58.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.756+0000 7f54255a2640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f540c0a2b90 0x7f540c0a50a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5414000c00 tx=0x7f5414031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:58.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.760+0000 7f54127fc640 1 -- 172.21.15.78:0/272752209 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f541c014030 con 0x7f5420103ce0 2024-05-21T15:15:58.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.934+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5420062750 con 0x7f5420103ce0 2024-05-21T15:15:58.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.935+0000 7f54127fc640 1 -- 172.21.15.78:0/272752209 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f541c09c220 con 0x7f5420103ce0 2024-05-21T15:15:58.938 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f540c0a2b90 msgr2=0x7f540c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f540c0a2b90 0x7f540c0a50a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5414000c00 tx=0x7f5414031400 comp rx=0 tx=0).stop 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420103ce0 msgr2=0x7f5420199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420103ce0 0x7f5420199810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c970 tx=0x7f541c00ce30 comp rx=0 tx=0).stop 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 shutdown_connections 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f540c0a2b90 0x7f540c0a50a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54201089e0 0x7f54201a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5420103ce0 0x7f5420199810 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.936+0000 7f5427806640 1 --2- 172.21.15.78:0/272752209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5420102aa0 0x7f54201992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.937+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 >> 172.21.15.78:0/272752209 conn(0x7f54200fdc90 msgr2=0x7f54201049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.937+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 shutdown_connections 2024-05-21T15:15:58.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:58.937+0000 7f5427806640 1 -- 172.21.15.78:0/272752209 wait complete. 2024-05-21T15:15:58.951 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:15:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 -- 172.21.15.78:0/4192730921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50107080 msgr2=0x7f1c501094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 --2- 172.21.15.78:0/4192730921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50107080 0x7f1c501094a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800adb0 tx=0x7f1c4802b7a0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 -- 172.21.15.78:0/4192730921 shutdown_connections 2024-05-21T15:15:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 --2- 172.21.15.78:0/4192730921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c50102380 0x7f1c50102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 --2- 172.21.15.78:0/4192730921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50107080 0x7f1c501094a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 --2- 172.21.15.78:0/4192730921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c50101140 0x7f1c50101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 -- 172.21.15.78:0/4192730921 >> 172.21.15.78:0/4192730921 conn(0x7f1c500fc330 msgr2=0x7f1c500fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:59.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 -- 172.21.15.78:0/4192730921 shutdown_connections 2024-05-21T15:15:59.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.062+0000 7f1c55145640 1 -- 172.21.15.78:0/4192730921 wait complete. 2024-05-21T15:15:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c55145640 1 Processor -- start 2024-05-21T15:15:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c55145640 1 -- start start 2024-05-21T15:15:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c55145640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c50101140 0x7f1c50199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c55145640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50102380 0x7f1c50199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c55145640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c50107080 0x7f1c501a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c55145640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5010c520 con 0x7f1c50102380 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c55145640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5010c3a0 con 0x7f1c50107080 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c55145640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5010c6a0 con 0x7f1c50101140 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c4e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50102380 0x7f1c50199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c4e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50102380 0x7f1c50199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49272/0 (socket says 172.21.15.78:49272) 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.063+0000 7f1c4e59c640 1 -- 172.21.15.78:0/2469705309 learned_addr learned my addr 172.21.15.78:0/2469705309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4f59e640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c50107080 0x7f1c501a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4ed9d640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c50101140 0x7f1c50199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4e59c640 1 -- 172.21.15.78:0/2469705309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c50101140 msgr2=0x7f1c50199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4e59c640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c50101140 0x7f1c50199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4e59c640 1 -- 172.21.15.78:0/2469705309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c50107080 msgr2=0x7f1c501a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4e59c640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c50107080 0x7f1c501a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4e59c640 1 -- 172.21.15.78:0/2469705309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c501a0d60 con 0x7f1c50102380 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4ed9d640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c50101140 0x7f1c50199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c4e59c640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50102380 0x7f1c50199760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400c890 tx=0x7f1c4400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c37fff640 1 -- 172.21.15.78:0/2469705309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44004220 con 0x7f1c50102380 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c37fff640 1 -- 172.21.15.78:0/2469705309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c440043c0 con 0x7f1c50102380 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c501a0ff0 con 0x7f1c50102380 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c37fff640 1 -- 172.21.15.78:0/2469705309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440049e0 con 0x7f1c50102380 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.064+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c501a14e0 con 0x7f1c50102380 2024-05-21T15:15:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.065+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c50105e90 con 0x7f1c50102380 2024-05-21T15:15:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.066+0000 7f1c37fff640 1 -- 172.21.15.78:0/2469705309 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c440079b0 con 0x7f1c50102380 2024-05-21T15:15:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.067+0000 7f1c37fff640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1c380a2b90 0x7f1c380a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.067+0000 7f1c4ed9d640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1c380a2b90 0x7f1c380a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.068+0000 7f1c37fff640 1 -- 172.21.15.78:0/2469705309 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f1c440cf110 con 0x7f1c50102380 2024-05-21T15:15:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.068+0000 7f1c4ed9d640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1c380a2b90 0x7f1c380a50a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005d00 tx=0x7f1c40005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.072+0000 7f1c37fff640 1 -- 172.21.15.78:0/2469705309 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1c44014030 con 0x7f1c50102380 2024-05-21T15:15:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.246+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c50062750 con 0x7f1c50102380 2024-05-21T15:15:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.246+0000 7f1c37fff640 1 -- 172.21.15.78:0/2469705309 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f1c4409c050 con 0x7f1c50102380 2024-05-21T15:15:59.249 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:15:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1c380a2b90 msgr2=0x7f1c380a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1c380a2b90 0x7f1c380a50a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005d00 tx=0x7f1c40005c90 comp rx=0 tx=0).stop 2024-05-21T15:15:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50102380 msgr2=0x7f1c50199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50102380 0x7f1c50199760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400c890 tx=0x7f1c4400cd50 comp rx=0 tx=0).stop 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 shutdown_connections 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1c380a2b90 0x7f1c380a50a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c50107080 0x7f1c501a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c50102380 0x7f1c50199760 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 --2- 172.21.15.78:0/2469705309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c50101140 0x7f1c50199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 >> 172.21.15.78:0/2469705309 conn(0x7f1c500fc330 msgr2=0x7f1c50103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 shutdown_connections 2024-05-21T15:15:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.248+0000 7f1c55145640 1 -- 172.21.15.78:0/2469705309 wait complete. 2024-05-21T15:15:59.263 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:15:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.371+0000 7f4e3ee32640 1 -- 172.21.15.78:0/644263269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 msgr2=0x7f4e381055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.371+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/644263269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 0x7f4e381055e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009900 tx=0x7f4e3402a410 comp rx=0 tx=0).stop 2024-05-21T15:15:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 -- 172.21.15.78:0/644263269 shutdown_connections 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/644263269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e38108740 0x7f4e3810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/644263269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e38105bd0 0x7f4e381080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/644263269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 0x7f4e381055e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 -- 172.21.15.78:0/644263269 >> 172.21.15.78:0/644263269 conn(0x7f4e380fcba0 msgr2=0x7f4e380ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 -- 172.21.15.78:0/644263269 shutdown_connections 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 -- 172.21.15.78:0/644263269 wait complete. 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 Processor -- start 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 -- start start 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 0x7f4e381992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e38105bd0 0x7f4e381997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e38108740 0x7f4e381a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3810e6d0 con 0x7f4e381031c0 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.372+0000 7f4e3ee32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3810e550 con 0x7f4e38108740 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3ee32640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3810e850 con 0x7f4e38105bd0 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 0x7f4e381992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 0x7f4e381992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49290/0 (socket says 172.21.15.78:49290) 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 -- 172.21.15.78:0/1469639888 learned_addr learned my addr 172.21.15.78:0/1469639888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e2ffff640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e38105bd0 0x7f4e381997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3d3cf640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e38108740 0x7f4e381a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 -- 172.21.15.78:0/1469639888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e38105bd0 msgr2=0x7f4e381997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e38105bd0 0x7f4e381997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 -- 172.21.15.78:0/1469639888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e38108740 msgr2=0x7f4e381a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e38108740 0x7f4e381a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 -- 172.21.15.78:0/1469639888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e381a0de0 con 0x7f4e381031c0 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3d3cf640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e38108740 0x7f4e381a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3cbce640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 0x7f4e381992a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4e34006010 tx=0x7f4e34031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1469639888 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3403f030 con 0x7f4e381031c0 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1469639888 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e34003ad0 con 0x7f4e381031c0 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e381a1070 con 0x7f4e381031c0 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.373+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1469639888 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34035070 con 0x7f4e381031c0 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.374+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e381a1560 con 0x7f4e381031c0 2024-05-21T15:15:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.374+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e380fe6c0 con 0x7f4e381031c0 2024-05-21T15:15:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.376+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1469639888 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e34042030 con 0x7f4e381031c0 2024-05-21T15:15:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.376+0000 7f4e2dffb640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4e200a2b90 0x7f4e200a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.376+0000 7f4e2ffff640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4e200a2b90 0x7f4e200a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.377+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1469639888 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4e340ef8b0 con 0x7f4e381031c0 2024-05-21T15:15:59.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.377+0000 7f4e2ffff640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4e200a2b90 0x7f4e200a50a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4e3819a340 tx=0x7f4e2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:59.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.380+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1469639888 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e340bc7f0 con 0x7f4e381031c0 2024-05-21T15:15:59.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.593+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e3802d2a0 con 0x7f4e381031c0 2024-05-21T15:15:59.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.594+0000 7f4e2dffb640 1 -- 172.21.15.78:0/1469639888 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11925 (secure 0 0 0) 0x7f4e340bc0f0 con 0x7f4e381031c0 2024-05-21T15:15:59.597 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:59.597 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:14:18.291660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:15:59.597 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-05-21T15:15:59.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:59 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2470285824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:59 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/272752209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:15:59 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2469705309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4e200a2b90 msgr2=0x7f4e200a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4e200a2b90 0x7f4e200a50a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4e3819a340 tx=0x7f4e2800b410 comp rx=0 tx=0).stop 2024-05-21T15:15:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 msgr2=0x7f4e381992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 0x7f4e381992a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4e34006010 tx=0x7f4e34031b60 comp rx=0 tx=0).stop 2024-05-21T15:15:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 shutdown_connections 2024-05-21T15:15:59.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f4e200a2b90 0x7f4e200a50a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e38108740 0x7f4e381a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e38105bd0 0x7f4e381997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 --2- 172.21.15.78:0/1469639888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e381031c0 0x7f4e381992a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 >> 172.21.15.78:0/1469639888 conn(0x7f4e380fcba0 msgr2=0x7f4e38103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 shutdown_connections 2024-05-21T15:15:59.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.596+0000 7f4e3ee32640 1 -- 172.21.15.78:0/1469639888 wait complete. 2024-05-21T15:15:59.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:15:59.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:59 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2470285824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:59 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/272752209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:15:59 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2469705309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:59 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2470285824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:59 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/272752209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:15:59 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2469705309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:15:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 -- 172.21.15.78:0/2438484331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 msgr2=0x7f0694109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 --2- 172.21.15.78:0/2438484331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 0x7f0694109cd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f068c009ca0 tx=0x7f068c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 -- 172.21.15.78:0/2438484331 shutdown_connections 2024-05-21T15:15:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 --2- 172.21.15.78:0/2438484331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0694102bb0 0x7f0694102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 --2- 172.21.15.78:0/2438484331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 0x7f0694109cd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 --2- 172.21.15.78:0/2438484331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694101970 0x7f0694102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 -- 172.21.15.78:0/2438484331 >> 172.21.15.78:0/2438484331 conn(0x7f06940fcb80 msgr2=0x7f06940ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:59.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 -- 172.21.15.78:0/2438484331 shutdown_connections 2024-05-21T15:15:59.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.719+0000 7f0698eb8640 1 -- 172.21.15.78:0/2438484331 wait complete. 2024-05-21T15:15:59.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0698eb8640 1 Processor -- start 2024-05-21T15:15:59.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0698eb8640 1 -- start start 2024-05-21T15:15:59.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0698eb8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0694101970 0x7f06940698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0698eb8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694102bb0 0x7f069419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0698eb8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 0x7f06941a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0698eb8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f069410d5f0 con 0x7f06941078b0 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0698eb8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f069410d470 con 0x7f0694101970 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0698eb8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f069410d770 con 0x7f0694102bb0 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 0x7f06941a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 0x7f06941a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49308/0 (socket says 172.21.15.78:49308) 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 -- 172.21.15.78:0/175201191 learned_addr learned my addr 172.21.15.78:0/175201191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0691d9b640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694102bb0 0x7f069419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f069259c640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0694101970 0x7f06940698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 -- 172.21.15.78:0/175201191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694102bb0 msgr2=0x7f069419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694102bb0 0x7f069419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 -- 172.21.15.78:0/175201191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0694101970 msgr2=0x7f06940698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0694101970 0x7f06940698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 -- 172.21.15.78:0/175201191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06941a0cd0 con 0x7f06941078b0 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f069259c640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0694101970 0x7f06940698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.720+0000 7f0692d9d640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 0x7f06941a0540 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f068c009430 tx=0x7f068c02f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.721+0000 7f06837fe640 1 -- 172.21.15.78:0/175201191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c003b70 con 0x7f06941078b0 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.721+0000 7f06837fe640 1 -- 172.21.15.78:0/175201191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c003d10 con 0x7f06941078b0 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.721+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06941a0f60 con 0x7f06941078b0 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.721+0000 7f06837fe640 1 -- 172.21.15.78:0/175201191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c0047d0 con 0x7f06941078b0 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.721+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06941a13c0 con 0x7f06941078b0 2024-05-21T15:15:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.722+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0694069f50 con 0x7f06941078b0 2024-05-21T15:15:59.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.723+0000 7f06837fe640 1 -- 172.21.15.78:0/175201191 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f068c02f800 con 0x7f06941078b0 2024-05-21T15:15:59.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.724+0000 7f06837fe640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f067c0a2b90 0x7f067c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:15:59.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.724+0000 7f069259c640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f067c0a2b90 0x7f067c0a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:15:59.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.724+0000 7f06837fe640 1 -- 172.21.15.78:0/175201191 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f068c0f0620 con 0x7f06941078b0 2024-05-21T15:15:59.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.725+0000 7f069259c640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f067c0a2b90 0x7f067c0a50a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06840058c0 tx=0x7f0684005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:15:59.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.728+0000 7f06837fe640 1 -- 172.21.15.78:0/175201191 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f068c0bd560 con 0x7f06941078b0 2024-05-21T15:15:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.906+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f069402d360 con 0x7f06941078b0 2024-05-21T15:15:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.908+0000 7f06837fe640 1 -- 172.21.15.78:0/175201191 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22616 (secure 0 0 0) 0x7f068c0bce60 con 0x7f06941078b0 2024-05-21T15:15:59.911 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:15:59.911 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":87,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:15:16.363058+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f067c0a2b90 msgr2=0x7f067c0a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f067c0a2b90 0x7f067c0a50a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06840058c0 tx=0x7f0684005ec0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 msgr2=0x7f06941a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 0x7f06941a0540 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f068c009430 tx=0x7f068c02f610 comp rx=0 tx=0).stop 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 shutdown_connections 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f067c0a2b90 0x7f067c0a50a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06941078b0 0x7f06941a0540 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0694102bb0 0x7f069419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 --2- 172.21.15.78:0/175201191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0694101970 0x7f06940698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 >> 172.21.15.78:0/175201191 conn(0x7f06940fcb80 msgr2=0x7f0694103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 shutdown_connections 2024-05-21T15:15:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:15:59.910+0000 7f0698eb8640 1 -- 172.21.15.78:0/175201191 wait complete. 2024-05-21T15:15:59.926 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T15:14:11.918954+0000', 'modified': '2024-05-21T15:14:18.291660+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24526, 'mds_2': 24502, 'mds_3': 24448, 'mds_4': 24419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14667': {'gid': 14667, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/3881579182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 3881579182}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 3881579182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14679': {'gid': 14679, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6841/2631303889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 2631303889}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 2631303889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'k', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.84:6833/502215426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6832', 'nonce': 502215426}, {'type': 'v1', 'addr': '172.21.15.84:6833', 'nonce': 502215426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.172:6833/1961121180', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 1961121180}, {'type': 'v1', 'addr': '172.21.15.172:6833', 'nonce': 1961121180}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6835/4071213416', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6834', 'nonce': 4071213416}, {'type': 'v1', 'addr': '172.21.15.84:6835', 'nonce': 4071213416}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6837/3570369413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6836', 'nonce': 3570369413}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 3570369413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/204442136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 204442136}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 204442136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.172:6837/3093965276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 3093965276}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 3093965276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.172:6839/3459432480', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 3459432480}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 3459432480}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6839/652490614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6838', 'nonce': 652490614}, {'type': 'v1', 'addr': '172.21.15.84:6839', 'nonce': 652490614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24526, 24527, 24502, 24448, 24419]} 2024-05-21T15:15:59.926 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T15:15:59.926 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-05-21T15:16:00.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.033+0000 7ff0ad475640 1 -- 172.21.15.78:0/3367470401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a81019b0 msgr2=0x7ff0a81026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.033+0000 7ff0ad475640 1 --2- 172.21.15.78:0/3367470401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a81019b0 0x7ff0a81026b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff098009900 tx=0x7ff09802a3f0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 -- 172.21.15.78:0/3367470401 shutdown_connections 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 --2- 172.21.15.78:0/3367470401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a81078f0 0x7ff0a8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 --2- 172.21.15.78:0/3367470401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0a8102bf0 0x7ff0a8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 --2- 172.21.15.78:0/3367470401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a81019b0 0x7ff0a81026b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 -- 172.21.15.78:0/3367470401 >> 172.21.15.78:0/3367470401 conn(0x7ff0a80fcba0 msgr2=0x7ff0a80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 -- 172.21.15.78:0/3367470401 shutdown_connections 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 -- 172.21.15.78:0/3367470401 wait complete. 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 Processor -- start 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.034+0000 7ff0ad475640 1 -- start start 2024-05-21T15:16:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0ad475640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a81019b0 0x7ff0a80747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0ad475640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a8102bf0 0x7ff0a819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0ad475640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0a81078f0 0x7ff0a81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0ad475640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a810d650 con 0x7ff0a8102bf0 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0ad475640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a810d4d0 con 0x7ff0a81019b0 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0ad475640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a810d7d0 con 0x7ff0a81078f0 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a8102bf0 0x7ff0a819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a8102bf0 0x7ff0a819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49326/0 (socket says 172.21.15.78:49326) 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 -- 172.21.15.78:0/2083192437 learned_addr learned my addr 172.21.15.78:0/2083192437 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0a81078f0 0x7ff0a81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a81019b0 0x7ff0a80747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 -- 172.21.15.78:0/2083192437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0a81078f0 msgr2=0x7ff0a81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0a81078f0 0x7ff0a81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 -- 172.21.15.78:0/2083192437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a81019b0 msgr2=0x7ff0a80747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a81019b0 0x7ff0a80747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 -- 172.21.15.78:0/2083192437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a81a0e20 con 0x7ff0a8102bf0 2024-05-21T15:16:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0a81078f0 0x7ff0a81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a8102bf0 0x7ff0a819e0b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00d770 tx=0x7ff09c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff08ffff640 1 -- 172.21.15.78:0/2083192437 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c017070 con 0x7ff0a8102bf0 2024-05-21T15:16:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0ad475640 1 -- 172.21.15.78:0/2083192437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a81a1110 con 0x7ff0a8102bf0 2024-05-21T15:16:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff08ffff640 1 -- 172.21.15.78:0/2083192437 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09c005500 con 0x7ff0a8102bf0 2024-05-21T15:16:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff08ffff640 1 -- 172.21.15.78:0/2083192437 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c004e00 con 0x7ff0a8102bf0 2024-05-21T15:16:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.035+0000 7ff0ad475640 1 -- 172.21.15.78:0/2083192437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a81a1660 con 0x7ff0a8102bf0 2024-05-21T15:16:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0b43f0740 1 Processor -- start 2024-05-21T15:16:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0b43f0740 1 -- start start 2024-05-21T15:16:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff08ffff640 1 -- 172.21.15.78:0/2083192437 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff09c010460 con 0x7ff0a8102bf0 2024-05-21T15:16:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda71880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6a43b0 0x5600fda71e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6ac590 0x5600fda78d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600fd9d14d0 con 0x5600fd6cd270 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5600fd9793e0 con 0x5600fd6ac590 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600fd9cd490 con 0x5600fd6a43b0 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda71880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda71880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49330/0 (socket says 172.21.15.78:49330) 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0a77fe640 1 -- 172.21.15.78:0/2924472895 learned_addr learned my addr 172.21.15.78:0/2924472895 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6ac590 0x5600fda78d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.038+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6a43b0 0x5600fda71e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff0a77fe640 1 -- 172.21.15.78:0/2924472895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6a43b0 msgr2=0x5600fda71e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6a43b0 0x5600fda71e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff0a77fe640 1 -- 172.21.15.78:0/2924472895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6ac590 msgr2=0x5600fda78d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6ac590 0x5600fda78d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff0a77fe640 1 -- 172.21.15.78:0/2924472895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fd6e52d0 con 0x5600fd6cd270 2024-05-21T15:16:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6ac590 0x5600fda78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda71880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000f410 tx=0x7ff0a000f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caffb98089dbc15c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff08ffff640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0900a2b90 0x7ff0900a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff08d7fa640 1 -- 172.21.15.78:0/2924472895 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0015070 con 0x5600fd6cd270 2024-05-21T15:16:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0900a2b90 0x7ff0900a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff08d7fa640 1 -- 172.21.15.78:0/2924472895 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0011be0 con 0x5600fd6cd270 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff08d7fa640 1 -- 172.21.15.78:0/2924472895 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0010d10 con 0x5600fd6cd270 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.039+0000 7ff08ffff640 1 -- 172.21.15.78:0/2083192437 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff09c0ceb10 con 0x7ff0a8102bf0 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2924472895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 msgr2=0x5600fda71880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0900a2b90 0x7ff0900a50a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff09802a9e0 tx=0x7ff0980315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda71880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000f410 tx=0x7ff0a000f8d0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2924472895 shutdown_connections 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6ac590 0x5600fda78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6a43b0 0x5600fda71e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2924472895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda71880 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2924472895 >> 172.21.15.78:0/2924472895 conn(0x5600fd6c13c0 msgr2=0x5600fda68e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2924472895 shutdown_connections 2024-05-21T15:16:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2924472895 wait complete. 2024-05-21T15:16:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 -- start start 2024-05-21T15:16:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.040+0000 7ff0b43f0740 1 Processor -- start 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6a43b0 0x5600fda7b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600fda79750 con 0x5600fd6cd270 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5600fd98c8e0 con 0x5600fd6a43b0 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600fda798d0 con 0x5600fd6ac590 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0b43f0740 10 client.0 _ll_register_callbacks cb 0x5600fd9599c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda82c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49340/0 (socket says 172.21.15.78:49340) 2024-05-21T15:16:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 -- 172.21.15.78:0/3791292265 learned_addr learned my addr 172.21.15.78:0/3791292265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6a43b0 0x5600fda7b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 -- 172.21.15.78:0/3791292265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 msgr2=0x5600fda7bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 -- 172.21.15.78:0/3791292265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6a43b0 msgr2=0x5600fda7b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6a43b0 0x5600fda7b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 -- 172.21.15.78:0/3791292265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fd7af6b0 con 0x5600fd6cd270 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda82c40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009d90 tx=0x7ff0a0010510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b586ed568480a63 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff08aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff08aff5640 1 -- 172.21.15.78:0/3791292265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0015050 con 0x5600fd6cd270 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff08aff5640 1 -- 172.21.15.78:0/3791292265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff0a0010ca0 con 0x5600fd6cd270 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.041+0000 7ff0b43f0740 1 -- 172.21.15.78:0/3791292265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a0011be0 con 0x5600fd6cd270 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff08aff5640 1 -- 172.21.15.78:0/3791292265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0a0004a40 con 0x5600fd6cd270 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0b43f0740 10 client.15297 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0b43f0740 1 -- 172.21.15.78:0/3791292265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600fda85ae0 con 0x5600fd6cd270 2024-05-21T15:16:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff08aff5640 1 -- 172.21.15.78:0/3791292265 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0011be0 con 0x5600fd6cd270 2024-05-21T15:16:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff08aff5640 1 -- 172.21.15.78:0/3791292265 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0a0005220 con 0x5600fd6cd270 2024-05-21T15:16:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0b43f0740 10 client.15297 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0b43f0740 10 client.15297 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-21T15:16:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0b43f0740 20 client.15297 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x5600fd68be40 0x5600fda891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0b43f0740 4 client.15297 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0b43f0740 1 -- 172.21.15.78:0/3791292265 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600fd9dec00 con 0x5600fd68be40 2024-05-21T15:16:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.042+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x5600fd68be40 0x5600fda891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.043+0000 7ff08aff5640 10 client.15297 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:16:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.043+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x5600fd68be40 0x5600fda891a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=81ff202ca1386340 server_cookie=1e40b9f7dde00a5f in_seq=0 out_seq=0 2024-05-21T15:16:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.043+0000 7ff08aff5640 1 -- 172.21.15.78:0/3791292265 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600fd9dec00 con 0x5600fd68be40 2024-05-21T15:16:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.043+0000 7ff08aff5640 10 client.15297 handle_command_reply: tid=0 2024-05-21T15:16:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.043+0000 7ff0b43f0740 1 client.15297 shutdown 2024-05-21T15:16:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 -- 172.21.15.78:0/3791292265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 msgr2=0x5600fda82c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda82c40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009d90 tx=0x7ff0a0010510 comp rx=0 tx=0).stop 2024-05-21T15:16:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 -- 172.21.15.78:0/3791292265 shutdown_connections 2024-05-21T15:16:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x5600fd68be40 0x5600fda891a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6cd270 0x5600fda82c40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/3791292265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6a43b0 0x5600fda7b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff08aff5640 0 client.15297 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:16:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 -- 172.21.15.78:0/3791292265 >> 172.21.15.78:0/3791292265 conn(0x5600fd6c13c0 msgr2=0x5600fda67ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 -- 172.21.15.78:0/3791292265 shutdown_connections 2024-05-21T15:16:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 1 -- 172.21.15.78:0/3791292265 wait complete. 2024-05-21T15:16:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.044+0000 7ff0b43f0740 20 client.15297 trim_cache size 0 max 16384 2024-05-21T15:16:00.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0b43f0740 1 Processor -- start 2024-05-21T15:16:00.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0b43f0740 1 -- start start 2024-05-21T15:16:00.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda86f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda75ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 0x5600fda760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600fda874e0 con 0x5600fd6a43b0 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5600fda87bb0 con 0x5600fd6cd270 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600fda88280 con 0x5600fd6ac590 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda86f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda86f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49352/0 (socket says 172.21.15.78:49352) 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.062+0000 7ff0a6ffd640 1 -- 172.21.15.78:0/2403471764 learned_addr learned my addr 172.21.15.78:0/2403471764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff0a6ffd640 1 -- 172.21.15.78:0/2403471764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 msgr2=0x5600fda75ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2403471764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda75ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff0a6ffd640 1 -- 172.21.15.78:0/2403471764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 msgr2=0x5600fda760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2403471764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 0x5600fda760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff0a6ffd640 1 -- 172.21.15.78:0/2403471764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fda7a820 con 0x5600fd6a43b0 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2403471764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda86f40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff098004490 tx=0x7ff098040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e50caaebb139b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff08cff9640 1 -- 172.21.15.78:0/2403471764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098055070 con 0x5600fd6a43b0 2024-05-21T15:16:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff08cff9640 1 -- 172.21.15.78:0/2403471764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff098004c00 con 0x5600fd6a43b0 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.063+0000 7ff08cff9640 1 -- 172.21.15.78:0/2403471764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098035040 con 0x5600fd6a43b0 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2403471764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 msgr2=0x5600fda86f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2403471764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda86f40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff098004490 tx=0x7ff098040000 comp rx=0 tx=0).stop 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2403471764 shutdown_connections 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2403471764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 0x5600fda760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2403471764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda75ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2403471764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda86f40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2403471764 >> 172.21.15.78:0/2403471764 conn(0x5600fd6c13c0 msgr2=0x5600fda74e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2403471764 shutdown_connections 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2403471764 wait complete. 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- start start 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 Processor -- start 2024-05-21T15:16:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda7d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 0x5600fdaf0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600fda89020 con 0x5600fd6a43b0 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5600fda88950 con 0x5600fd6cd270 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.064+0000 7ff0b43f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600fda896f0 con 0x5600fd6ac590 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0b43f0740 10 client.0 _ll_register_callbacks cb 0x7ff0a0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda7d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda7d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49356/0 (socket says 172.21.15.78:49356) 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 -- 172.21.15.78:0/2293150994 learned_addr learned my addr 172.21.15.78:0/2293150994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 -- 172.21.15.78:0/2293150994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 msgr2=0x5600fda7d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a77fe640 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 0x5600fdaf0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 -- 172.21.15.78:0/2293150994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 msgr2=0x5600fdaf0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 0x5600fdaf0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 -- 172.21.15.78:0/2293150994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fdaf05d0 con 0x5600fd6a43b0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0a6ffd640 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda7d300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff098006010 tx=0x7ff098004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f832e179cae49b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff08aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff08aff5640 1 -- 172.21.15.78:0/2293150994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09805e060 con 0x5600fd6a43b0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff08aff5640 1 -- 172.21.15.78:0/2293150994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff098032df0 con 0x5600fd6a43b0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2293150994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff098004c00 con 0x5600fd6a43b0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff08aff5640 1 -- 172.21.15.78:0/2293150994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff098035070 con 0x5600fd6a43b0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0b43f0740 10 client.15303 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2293150994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600fdaf1810 con 0x5600fd6a43b0 2024-05-21T15:16:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.065+0000 7ff08aff5640 1 -- 172.21.15.78:0/2293150994 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff098004c00 con 0x5600fd6a43b0 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff08aff5640 1 -- 172.21.15.78:0/2293150994 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff098050ce0 con 0x5600fd6a43b0 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff0b43f0740 10 client.15303 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff0b43f0740 10 client.15303 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff0b43f0740 20 client.15303 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x5600fd661280 0x5600fdaf4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff0b43f0740 4 client.15303 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2293150994 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "status"}) -- 0x5600fd9d8f50 con 0x5600fd661280 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x5600fd661280 0x5600fdaf4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff0a67fc640 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x5600fd661280 0x5600fdaf4c00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=818584fd43a83888 server_cookie=d9500e7c489154e3 in_seq=0 out_seq=0 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.066+0000 7ff08aff5640 10 client.15303 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff08aff5640 1 -- 172.21.15.78:0/2293150994 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5600fd9d8f50 con 0x5600fd661280 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff08aff5640 10 client.15303 handle_command_reply: tid=0 2024-05-21T15:16:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 client.15303 shutdown 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2293150994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 msgr2=0x5600fda7d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda7d300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff098006010 tx=0x7ff098004e50 comp rx=0 tx=0).stop 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2293150994 shutdown_connections 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x5600fd661280 0x5600fdaf4c00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5600fd6cd270 0x5600fdaf0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600fd6ac590 0x5600fda7d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff08aff5640 0 client.15303 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 --2- 172.21.15.78:0/2293150994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5600fd6a43b0 0x5600fda7d300 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2293150994 >> 172.21.15.78:0/2293150994 conn(0x5600fd6c13c0 msgr2=0x5600fda75ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2293150994 shutdown_connections 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 1 -- 172.21.15.78:0/2293150994 wait complete. 2024-05-21T15:16:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0b43f0740 20 client.15303 trim_cache size 0 max 16384 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24419, 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 102.793040898, 2024-05-21T15:16:00.078 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:16:00.079 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:16:00.079 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:16:00.079 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 141.23588313600001 2024-05-21T15:16:00.079 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0ad475640 1 -- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0900a2b90 msgr2=0x7ff0900a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0ad475640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0900a2b90 0x7ff0900a50a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff09802a9e0 tx=0x7ff0980315b0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0ad475640 1 -- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a8102bf0 msgr2=0x7ff0a819e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0ad475640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a8102bf0 0x7ff0a819e0b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00d770 tx=0x7ff09c00dc30 comp rx=0 tx=0).stop 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0ad475640 1 -- 172.21.15.78:0/2083192437 shutdown_connections 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0ad475640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0900a2b90 0x7ff0900a50a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.067+0000 7ff0ad475640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0a81078f0 0x7ff0a81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.068+0000 7ff0ad475640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a8102bf0 0x7ff0a819e0b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.068+0000 7ff0ad475640 1 --2- 172.21.15.78:0/2083192437 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0a81019b0 0x7ff0a80747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.068+0000 7ff0ad475640 1 -- 172.21.15.78:0/2083192437 >> 172.21.15.78:0/2083192437 conn(0x7ff0a80fcba0 msgr2=0x7ff0a81038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.068+0000 7ff0ad475640 1 -- 172.21.15.78:0/2083192437 shutdown_connections 2024-05-21T15:16:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.068+0000 7ff0ad475640 1 -- 172.21.15.78:0/2083192437 wait complete. 2024-05-21T15:16:00.081 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-05-21T15:16:00.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 -- 172.21.15.78:0/73156974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 msgr2=0x7fc9581055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/73156974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 0x7fc9581055e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc948009900 tx=0x7fc94802a410 comp rx=0 tx=0).stop 2024-05-21T15:16:00.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 -- 172.21.15.78:0/73156974 shutdown_connections 2024-05-21T15:16:00.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/73156974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc958108720 0x7fc95810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/73156974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc958105bb0 0x7fc958108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/73156974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 0x7fc9581055e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 -- 172.21.15.78:0/73156974 >> 172.21.15.78:0/73156974 conn(0x7fc9580fcb80 msgr2=0x7fc9580ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 -- 172.21.15.78:0/73156974 shutdown_connections 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 -- 172.21.15.78:0/73156974 wait complete. 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 Processor -- start 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.191+0000 7fc95d6cc640 1 -- start start 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc95d6cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 0x7fc9581996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc95d6cc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc958105bb0 0x7fc958199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc95d6cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc958108720 0x7fc9581a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc95d6cc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810e7b0 con 0x7fc9581031c0 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc95d6cc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810e630 con 0x7fc958105bb0 2024-05-21T15:16:00.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc95d6cc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc95810e930 con 0x7fc958108720 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 0x7fc9581996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 0x7fc9581996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49380/0 (socket says 172.21.15.78:49380) 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 -- 172.21.15.78:0/3171130196 learned_addr learned my addr 172.21.15.78:0/3171130196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc9567fc640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc958105bb0 0x7fc958199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc958108720 0x7fc9581a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 -- 172.21.15.78:0/3171130196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc958108720 msgr2=0x7fc9581a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc958108720 0x7fc9581a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 -- 172.21.15.78:0/3171130196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc958105bb0 msgr2=0x7fc958199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc958105bb0 0x7fc958199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 -- 172.21.15.78:0/3171130196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9581a1220 con 0x7fc9581031c0 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc958108720 0x7fc9581a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc9567fc640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc958105bb0 0x7fc958199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 0x7fc9581996e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc948006010 tx=0x7fc9480043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc93ffff640 1 -- 172.21.15.78:0/3171130196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94803f030 con 0x7fc9581031c0 2024-05-21T15:16:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc95d6cc640 1 -- 172.21.15.78:0/3171130196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9581a14b0 con 0x7fc9581031c0 2024-05-21T15:16:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc93ffff640 1 -- 172.21.15.78:0/3171130196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc948003ce0 con 0x7fc9581031c0 2024-05-21T15:16:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.192+0000 7fc93ffff640 1 -- 172.21.15.78:0/3171130196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948035070 con 0x7fc9581031c0 2024-05-21T15:16:00.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.193+0000 7fc95d6cc640 1 -- 172.21.15.78:0/3171130196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9581a1a20 con 0x7fc9581031c0 2024-05-21T15:16:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.195+0000 7fc964568740 1 Processor -- start 2024-05-21T15:16:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc964568740 1 -- start start 2024-05-21T15:16:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc93ffff640 1 -- 172.21.15.78:0/3171130196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc948042030 con 0x7fc9581031c0 2024-05-21T15:16:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc964568740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94c0270 0x55caf9860470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc964568740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94973b0 0x55caf98609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc964568740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf949f590 0x55caf98678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc964568740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55caf97c44d0 con 0x55caf94973b0 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc964568740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55caf976c3e0 con 0x55caf949f590 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc964568740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55caf97c0490 con 0x55caf94c0270 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94973b0 0x55caf98609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94973b0 0x55caf98609f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49394/0 (socket says 172.21.15.78:49394) 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 -- 172.21.15.78:0/1157852100 learned_addr learned my addr 172.21.15.78:0/1157852100 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc9567fc640 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf949f590 0x55caf98678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc9577fe640 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94c0270 0x55caf9860470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 -- 172.21.15.78:0/1157852100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94c0270 msgr2=0x55caf9860470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94c0270 0x55caf9860470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 -- 172.21.15.78:0/1157852100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf949f590 msgr2=0x55caf98678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf949f590 0x55caf98678f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 -- 172.21.15.78:0/1157852100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf94d82d0 con 0x55caf94973b0 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc9567fc640 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf949f590 0x55caf98678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc956ffd640 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94973b0 0x55caf98609f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc9480f5c10 tx=0x7fc9480f5d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf430456149a55e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc93ffff640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc9400a2b90 0x7fc9400a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.196+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc9400a2b90 0x7fc9400a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc93d7fa640 1 -- 172.21.15.78:0/1157852100 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480fe070 con 0x55caf94973b0 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc93d7fa640 1 -- 172.21.15.78:0/1157852100 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9480bc730 con 0x55caf94973b0 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc93d7fa640 1 -- 172.21.15.78:0/1157852100 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9480b82d0 con 0x55caf94973b0 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc93ffff640 1 -- 172.21.15.78:0/3171130196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc9480ef7f0 con 0x7fc9581031c0 2024-05-21T15:16:00.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc964568740 1 -- 172.21.15.78:0/1157852100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94973b0 msgr2=0x55caf98609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc9400a2b90 0x7fc9400a50a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55caf9861640 tx=0x7fc950007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc964568740 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94973b0 0x55caf98609f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc9480f5c10 tx=0x7fc9480f5d00 comp rx=0 tx=0).stop 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc964568740 1 -- 172.21.15.78:0/1157852100 shutdown_connections 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.197+0000 7fc964568740 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf949f590 0x55caf98678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94973b0 0x55caf98609f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 --2- 172.21.15.78:0/1157852100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94c0270 0x55caf9860470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 -- 172.21.15.78:0/1157852100 >> 172.21.15.78:0/1157852100 conn(0x55caf94b43c0 msgr2=0x55caf985be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 -- 172.21.15.78:0/1157852100 shutdown_connections 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 -- 172.21.15.78:0/1157852100 wait complete. 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 -- start start 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 Processor -- start 2024-05-21T15:16:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9869f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 0x55caf986a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9871420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55caf9868100 con 0x55caf94c0270 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55caf977f8e0 con 0x55caf94973b0 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55caf9868280 con 0x55caf949f590 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc964568740 10 client.0 _ll_register_callbacks cb 0x55caf974c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc9577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9871420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc9577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9871420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49400/0 (socket says 172.21.15.78:49400) 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.198+0000 7fc9577fe640 1 -- 172.21.15.78:0/3261260184 learned_addr learned my addr 172.21.15.78:0/3261260184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9869f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc9567fc640 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 0x55caf986a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc9577fe640 1 -- 172.21.15.78:0/3261260184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 msgr2=0x55caf986a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 0x55caf986a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc9577fe640 1 -- 172.21.15.78:0/3261260184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 msgr2=0x55caf9869f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9869f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc9577fe640 1 -- 172.21.15.78:0/3261260184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf95a26b0 con 0x55caf94c0270 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9869f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9871420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc950010df0 tx=0x7fc950004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26e4ed5063d9361 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc93aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc93aff5640 1 -- 172.21.15.78:0/3261260184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950021070 con 0x55caf94c0270 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc964568740 1 -- 172.21.15.78:0/3261260184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9480bc730 con 0x55caf94c0270 2024-05-21T15:16:00.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc93aff5640 1 -- 172.21.15.78:0/3261260184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc95001d650 con 0x55caf94c0270 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc93aff5640 1 -- 172.21.15.78:0/3261260184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc95002d050 con 0x55caf94c0270 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc964568740 10 client.15315 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc964568740 1 -- 172.21.15.78:0/3261260184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55caf9874470 con 0x55caf94c0270 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc93aff5640 1 -- 172.21.15.78:0/3261260184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9480bc730 con 0x55caf94c0270 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.199+0000 7fc93aff5640 1 -- 172.21.15.78:0/3261260184 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc9500257d0 con 0x55caf94c0270 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc964568740 10 client.15315 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc964568740 10 client.15315 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc964568740 20 client.15315 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc964568740 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55caf947ee40 0x55caf9877f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc964568740 4 client.15315 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc964568740 1 -- 172.21.15.78:0/3261260184 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf97d1c00 con 0x55caf947ee40 2024-05-21T15:16:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55caf947ee40 0x55caf9877f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc93aff5640 10 client.15315 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.200+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55caf947ee40 0x55caf9877f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83f1b60dd409db49 server_cookie=6174ae7ae104cb1e in_seq=0 out_seq=0 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc93aff5640 1 -- 172.21.15.78:0/3261260184 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55caf97d1c00 con 0x55caf947ee40 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc93aff5640 10 client.15315 handle_command_reply: tid=0 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc964568740 1 client.15315 shutdown 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc964568740 1 -- 172.21.15.78:0/3261260184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 msgr2=0x55caf9871420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc964568740 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9871420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc950010df0 tx=0x7fc950004a40 comp rx=0 tx=0).stop 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc964568740 1 -- 172.21.15.78:0/3261260184 shutdown_connections 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc964568740 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55caf947ee40 0x55caf9877f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc964568740 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9871420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc964568740 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 0x55caf986a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc93aff5640 0 client.15315 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:16:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.201+0000 7fc964568740 1 --2- 172.21.15.78:0/3261260184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9869f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.202+0000 7fc964568740 1 -- 172.21.15.78:0/3261260184 >> 172.21.15.78:0/3261260184 conn(0x55caf94b43c0 msgr2=0x55caf985aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.202+0000 7fc964568740 1 -- 172.21.15.78:0/3261260184 shutdown_connections 2024-05-21T15:16:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.202+0000 7fc964568740 1 -- 172.21.15.78:0/3261260184 wait complete. 2024-05-21T15:16:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.202+0000 7fc964568740 20 client.15315 trim_cache size 0 max 16384 2024-05-21T15:16:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc964568740 1 Processor -- start 2024-05-21T15:16:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc964568740 1 -- start start 2024-05-21T15:16:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc964568740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9868570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc964568740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 0x55caf9871340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc964568740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9868ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc964568740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55caf9876240 con 0x55caf94c0270 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc964568740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55caf9876910 con 0x55caf94973b0 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc964568740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55caf9876fe0 con 0x55caf949f590 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9868ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9868ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49404/0 (socket says 172.21.15.78:49404) 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 -- 172.21.15.78:0/3252748036 learned_addr learned my addr 172.21.15.78:0/3252748036 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9868570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 -- 172.21.15.78:0/3252748036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 msgr2=0x55caf9871340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 0x55caf9871340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 -- 172.21.15.78:0/3252748036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 msgr2=0x55caf9868570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9868570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 -- 172.21.15.78:0/3252748036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf9861670 con 0x55caf94c0270 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc956ffd640 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9868570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.220+0000 7fc9577fe640 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9868ab0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc950010840 tx=0x7fc9500076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4987c87c3cd8b9dd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc93cff9640 1 -- 172.21.15.78:0/3252748036 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950035070 con 0x55caf94c0270 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc93cff9640 1 -- 172.21.15.78:0/3252748036 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc950010e20 con 0x55caf94c0270 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc93cff9640 1 -- 172.21.15.78:0/3252748036 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950021070 con 0x55caf94c0270 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 -- 172.21.15.78:0/3252748036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 msgr2=0x55caf9868ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9868ab0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc950010840 tx=0x7fc9500076f0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 -- 172.21.15.78:0/3252748036 shutdown_connections 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf94c0270 0x55caf9868ab0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf949f590 0x55caf9871340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 --2- 172.21.15.78:0/3252748036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94973b0 0x55caf9868570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 -- 172.21.15.78:0/3252748036 >> 172.21.15.78:0/3252748036 conn(0x55caf94b43c0 msgr2=0x55caf9864690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 -- 172.21.15.78:0/3252748036 shutdown_connections 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.221+0000 7fc964568740 1 -- 172.21.15.78:0/3252748036 wait complete. 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 1 -- start start 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 1 Processor -- start 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94973b0 0x55caf986c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf949f590 0x55caf986cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94c0270 0x55caf986fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55caf9877d80 con 0x55caf949f590 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55caf98776b0 con 0x55caf94c0270 2024-05-21T15:16:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55caf9878450 con 0x55caf94973b0 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc964568740 10 client.0 _ll_register_callbacks cb 0x7fc950014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf949f590 0x55caf986cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf949f590 0x55caf986cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49408/0 (socket says 172.21.15.78:49408) 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 -- 172.21.15.78:0/488358416 learned_addr learned my addr 172.21.15.78:0/488358416 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9577fe640 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94c0270 0x55caf986fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 -- 172.21.15.78:0/488358416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94973b0 msgr2=0x55caf986c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94973b0 0x55caf986c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 -- 172.21.15.78:0/488358416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94c0270 msgr2=0x55caf986fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94c0270 0x55caf986fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 -- 172.21.15.78:0/488358416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf98700e0 con 0x55caf949f590 2024-05-21T15:16:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc9567fc640 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf949f590 0x55caf986cc30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55caf986aff0 tx=0x7fc94c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9fb85ead84a7e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.222+0000 7fc93aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc93aff5640 1 -- 172.21.15.78:0/488358416 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c018070 con 0x55caf949f590 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc93aff5640 1 -- 172.21.15.78:0/488358416 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc94c0145f0 con 0x55caf949f590 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 1 -- 172.21.15.78:0/488358416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc950010e20 con 0x55caf949f590 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc93aff5640 1 -- 172.21.15.78:0/488358416 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc94c024050 con 0x55caf949f590 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 10 client.15321 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 1 -- 172.21.15.78:0/488358416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55caf9871460 con 0x55caf949f590 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc93aff5640 1 -- 172.21.15.78:0/488358416 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc950010e20 con 0x55caf949f590 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc93aff5640 1 -- 172.21.15.78:0/488358416 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc94c01c950 con 0x55caf949f590 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 10 client.15321 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 10 client.15321 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-21T15:16:00.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 20 client.15321 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55caf9454280 0x55caf98e19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 4 client.15321 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc964568740 1 -- 172.21.15.78:0/488358416 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "status"}) -- 0x55caf97cbf50 con 0x55caf9454280 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.223+0000 7fc956ffd640 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55caf9454280 0x55caf98e19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc93aff5640 10 client.15321 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc956ffd640 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55caf9454280 0x55caf98e19f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68105eb1f5322871 server_cookie=e6223f378c436abd in_seq=0 out_seq=0 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc93aff5640 1 -- 172.21.15.78:0/488358416 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55caf97cbf50 con 0x55caf9454280 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc93aff5640 10 client.15321 handle_command_reply: tid=0 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 client.15321 shutdown 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 -- 172.21.15.78:0/488358416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf949f590 msgr2=0x55caf986cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf949f590 0x55caf986cc30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55caf986aff0 tx=0x7fc94c007f90 comp rx=0 tx=0).stop 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 -- 172.21.15.78:0/488358416 shutdown_connections 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55caf9454280 0x55caf98e19f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55caf94c0270 0x55caf986fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc93aff5640 0 client.15321 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55caf949f590 0x55caf986cc30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 --2- 172.21.15.78:0/488358416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf94973b0 0x55caf986c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 -- 172.21.15.78:0/488358416 >> 172.21.15.78:0/488358416 conn(0x55caf94b43c0 msgr2=0x55caf9868570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 -- 172.21.15.78:0/488358416 shutdown_connections 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 1 -- 172.21.15.78:0/488358416 wait complete. 2024-05-21T15:16:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.224+0000 7fc964568740 20 client.15321 trim_cache size 0 max 16384 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24448, 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 103.960325927, 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:16:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:16:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 144.23910240399999 2024-05-21T15:16:00.237 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 -- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc9400a2b90 msgr2=0x7fc9400a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc9400a2b90 0x7fc9400a50a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55caf9861640 tx=0x7fc950007520 comp rx=0 tx=0).stop 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 -- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 msgr2=0x7fc9581996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 0x7fc9581996e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc948006010 tx=0x7fc9480043b0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 -- 172.21.15.78:0/3171130196 shutdown_connections 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc9400a2b90 0x7fc9400a50a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc958105bb0 0x7fc958199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc958108720 0x7fc9581a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 --2- 172.21.15.78:0/3171130196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9581031c0 0x7fc9581996e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 -- 172.21.15.78:0/3171130196 >> 172.21.15.78:0/3171130196 conn(0x7fc9580fcb80 msgr2=0x7fc958104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 -- 172.21.15.78:0/3171130196 shutdown_connections 2024-05-21T15:16:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.225+0000 7fc95d6cc640 1 -- 172.21.15.78:0/3171130196 wait complete. 2024-05-21T15:16:00.240 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-21T15:16:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/387490760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 msgr2=0x7ff0b8100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/387490760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 0x7ff0b8100b60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009900 tx=0x7ff0a802a410 comp rx=0 tx=0).stop 2024-05-21T15:16:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/387490760 shutdown_connections 2024-05-21T15:16:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/387490760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b81011f0 0x7ff0b810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/387490760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 0x7ff0b8100b60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/387490760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b8104040 0x7ff0b8104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/387490760 >> 172.21.15.78:0/387490760 conn(0x7ff0b80fc2f0 msgr2=0x7ff0b80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/387490760 shutdown_connections 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/387490760 wait complete. 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 Processor -- start 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.350+0000 7ff0bfa2b640 1 -- start start 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bfa2b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 0x7ff0b81031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bfa2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b81011f0 0x7ff0b8101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bfa2b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b8104040 0x7ff0b8101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bfa2b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b810e720 con 0x7ff0b81006f0 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bfa2b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b810e5a0 con 0x7ff0b81011f0 2024-05-21T15:16:00.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bfa2b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b810e8a0 con 0x7ff0b8104040 2024-05-21T15:16:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 0x7ff0b81031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 0x7ff0b81031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49418/0 (socket says 172.21.15.78:49418) 2024-05-21T15:16:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 -- 172.21.15.78:0/2198521528 learned_addr learned my addr 172.21.15.78:0/2198521528 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b81011f0 0x7ff0b8101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 -- 172.21.15.78:0/2198521528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b8104040 msgr2=0x7ff0b8101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b8104040 0x7ff0b8101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 -- 172.21.15.78:0/2198521528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b81011f0 msgr2=0x7ff0b8101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b81011f0 0x7ff0b8101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 -- 172.21.15.78:0/2198521528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b8102500 con 0x7ff0b81006f0 2024-05-21T15:16:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b81011f0 0x7ff0b8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bd7c7640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 0x7ff0b81031f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4005d00 tx=0x7ff0b400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0ae7fc640 1 -- 172.21.15.78:0/2198521528 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4017070 con 0x7ff0b81006f0 2024-05-21T15:16:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0ae7fc640 1 -- 172.21.15.78:0/2198521528 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b4009c30 con 0x7ff0b81006f0 2024-05-21T15:16:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/2198521528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b81a33b0 con 0x7ff0b81006f0 2024-05-21T15:16:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0ae7fc640 1 -- 172.21.15.78:0/2198521528 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400f620 con 0x7ff0b81006f0 2024-05-21T15:16:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.351+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/2198521528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b81a3820 con 0x7ff0b81006f0 2024-05-21T15:16:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0ae7fc640 1 -- 172.21.15.78:0/2198521528 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff0b401a070 con 0x7ff0b81006f0 2024-05-21T15:16:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0c69ae740 1 Processor -- start 2024-05-21T15:16:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0c69ae740 1 -- start start 2024-05-21T15:16:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0444d0 con 0x555e1bd40270 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555e1bfec3e0 con 0x555e1bd1f590 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.354+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e1c040490 con 0x555e1bd173b0 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0e0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49420/0 (socket says 172.21.15.78:49420) 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/437311077 learned_addr learned my addr 172.21.15.78:0/437311077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bd7c7640 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/437311077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 msgr2=0x555e1c0e09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0e09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/437311077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 msgr2=0x555e1c0e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/437311077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1bd582d0 con 0x555e1bd40270 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bd7c7640 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0e09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0ae7fc640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0a40a2b90 0x7ff0a40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bd7c7640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0a40a2b90 0x7ff0a40a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0e0470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009ea0 tx=0x7ff0b0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bad1f401713688 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0a37fe640 1 -- 172.21.15.78:0/437311077 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0004d20 con 0x555e1bd40270 2024-05-21T15:16:00.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0ae7fc640 1 -- 172.21.15.78:0/2198521528 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff0b40ceb70 con 0x7ff0b81006f0 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0a37fe640 1 -- 172.21.15.78:0/437311077 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b000ee60 con 0x555e1bd40270 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.355+0000 7ff0a37fe640 1 -- 172.21.15.78:0/437311077 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00108a0 con 0x555e1bd40270 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 -- 172.21.15.78:0/437311077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 msgr2=0x555e1c0e0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0bd7c7640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0a40a2b90 0x7ff0a40a50a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x555e1c0e1850 tx=0x7ff0b40d2160 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0e0470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009ea0 tx=0x7ff0b0009f90 comp rx=0 tx=0).stop 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 -- 172.21.15.78:0/437311077 shutdown_connections 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0e09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/437311077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0e0470 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 -- 172.21.15.78:0/437311077 >> 172.21.15.78:0/437311077 conn(0x555e1bd343c0 msgr2=0x555e1c0dbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 -- 172.21.15.78:0/437311077 shutdown_connections 2024-05-21T15:16:00.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.356+0000 7ff0c69ae740 1 -- 172.21.15.78:0/437311077 wait complete. 2024-05-21T15:16:00.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 1 -- start start 2024-05-21T15:16:00.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 1 Processor -- start 2024-05-21T15:16:00.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd1f590 0x555e1c0ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd40270 0x555e1c0f1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0e8340 con 0x555e1bd173b0 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555e1bfff8e0 con 0x555e1bd40270 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0e84c0 con 0x555e1bd1f590 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0c69ae740 10 client.0 _ll_register_callbacks cb 0x555e1bfcc9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bdfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bdfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0ea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49424/0 (socket says 172.21.15.78:49424) 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/3034765485 learned_addr learned my addr 172.21.15.78:0/3034765485 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd40270 0x555e1c0f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd1f590 0x555e1c0ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/3034765485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd1f590 msgr2=0x555e1c0ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd1f590 0x555e1c0ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/3034765485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd40270 msgr2=0x555e1c0f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.357+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd40270 0x555e1c0f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/3034765485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1be226b0 con 0x555e1bd173b0 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0ea2d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0000f80 tx=0x7ff0b0009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bf8c91dc4eb1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/3034765485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b002c060 con 0x555e1bd173b0 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/3034765485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff0b0010df0 con 0x555e1bd173b0 2024-05-21T15:16:00.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3034765485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b000ee60 con 0x555e1bd173b0 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/3034765485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0b001d040 con 0x555e1bd173b0 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 10 client.15333 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3034765485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e1c0f46a0 con 0x555e1bd173b0 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/3034765485 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b000ee60 con 0x555e1bd173b0 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/3034765485 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0b0018830 con 0x555e1bd173b0 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 10 client.15333 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 10 client.15333 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 20 client.15333 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x555e1bcfee40 0x555e1c0f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 4 client.15333 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:00.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.358+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3034765485 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e1c051c00 con 0x555e1bcfee40 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.359+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x555e1bcfee40 0x555e1c0f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.359+0000 7ff0a0ff9640 10 client.15333 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.359+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x555e1bcfee40 0x555e1c0f7d20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27faca20e3f110b server_cookie=230c027146bc42c in_seq=0 out_seq=0 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/3034765485 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e1c051c00 con 0x555e1bcfee40 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0a0ff9640 10 client.15333 handle_command_reply: tid=0 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 client.15333 shutdown 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3034765485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 msgr2=0x555e1c0ea2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0ea2d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0000f80 tx=0x7ff0b0009be0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3034765485 shutdown_connections 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x555e1bcfee40 0x555e1c0f7d20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd40270 0x555e1c0f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd1f590 0x555e1c0ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3034765485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0ea2d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0a0ff9640 0 client.15333 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:16:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3034765485 >> 172.21.15.78:0/3034765485 conn(0x555e1bd343c0 msgr2=0x555e1c0daea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3034765485 shutdown_connections 2024-05-21T15:16:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3034765485 wait complete. 2024-05-21T15:16:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.360+0000 7ff0c69ae740 20 client.15333 trim_cache size 0 max 16384 2024-05-21T15:16:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.379+0000 7ff0c69ae740 1 Processor -- start 2024-05-21T15:16:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.379+0000 7ff0c69ae740 1 -- start start 2024-05-21T15:16:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.379+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0dbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.379+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.379+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd40270 0x555e1c0f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.379+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0f6060 con 0x555e1bd173b0 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.379+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0f6730 con 0x555e1bd1f590 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.379+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0f6e00 con 0x555e1bd40270 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0dbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0dbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49436/0 (socket says 172.21.15.78:49436) 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/3748886780 learned_addr learned my addr 172.21.15.78:0/3748886780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/3748886780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd40270 msgr2=0x555e1c0f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/3748886780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd40270 0x555e1c0f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/3748886780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 msgr2=0x555e1c0f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/3748886780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/3748886780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1c0e0020 con 0x555e1bd173b0 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/3748886780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0dbf00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0005fb0 tx=0x7ff0b0025400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8933de46b3beac28 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0a2ffd640 1 -- 172.21.15.78:0/3748886780 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b003f070 con 0x555e1bd173b0 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0a2ffd640 1 -- 172.21.15.78:0/3748886780 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b000ee60 con 0x555e1bd173b0 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.380+0000 7ff0a2ffd640 1 -- 172.21.15.78:0/3748886780 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b002c030 con 0x555e1bd173b0 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3748886780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 msgr2=0x555e1c0dbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3748886780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0dbf00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0005fb0 tx=0x7ff0b0025400 comp rx=0 tx=0).stop 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3748886780 shutdown_connections 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3748886780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3748886780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd40270 0x555e1c0f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/3748886780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd173b0 0x555e1c0dbf00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3748886780 >> 172.21.15.78:0/3748886780 conn(0x555e1bd343c0 msgr2=0x555e1c0db7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3748886780 shutdown_connections 2024-05-21T15:16:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 -- 172.21.15.78:0/3748886780 wait complete. 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 -- start start 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.381+0000 7ff0c69ae740 1 Processor -- start 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0ed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0c69ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0f7ba0 con 0x555e1bd40270 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0f74d0 con 0x555e1bd1f590 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0c69ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e1c0f8270 con 0x555e1bd173b0 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0c69ae740 10 client.0 _ll_register_callbacks cb 0x7ff0b0009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0f00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49452/0 (socket says 172.21.15.78:49452) 2024-05-21T15:16:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/16717487 learned_addr learned my addr 172.21.15.78:0/16717487 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/16717487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 msgr2=0x555e1c0ecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0ecc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/16717487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 msgr2=0x555e1c0ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0ed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 -- 172.21.15.78:0/16717487 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1c0f0670 con 0x555e1bd40270 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0bdfc8640 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0f00b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0018b80 tx=0x7ff0b0009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859d8243b932869c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.382+0000 7ff0a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/16717487 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b003f040 con 0x555e1bd40270 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/16717487 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff0b0025720 con 0x555e1bd40270 2024-05-21T15:16:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 1 -- 172.21.15.78:0/16717487 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b000ee60 con 0x555e1bd40270 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/16717487 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0b002c040 con 0x555e1bd40270 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 10 client.15339 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 1 -- 172.21.15.78:0/16717487 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e1c0f18f0 con 0x555e1bd40270 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/16717487 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b000ee60 con 0x555e1bd40270 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/16717487 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0b003aa90 con 0x555e1bd40270 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 10 client.15339 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 10 client.15339 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 20 client.15339 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x555e1bcd4280 0x555e1c1618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 4 client.15339 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0c69ae740 1 -- 172.21.15.78:0/16717487 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "status"}) -- 0x555e1c04bf50 con 0x555e1bcd4280 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.383+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x555e1bcd4280 0x555e1c1618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0bcfc6640 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x555e1bcd4280 0x555e1c1618c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fad202eff841f33 server_cookie=5197ebac5384bfa8 in_seq=0 out_seq=0 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0a0ff9640 10 client.15339 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0a0ff9640 1 -- 172.21.15.78:0/16717487 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x555e1c04bf50 con 0x555e1bcd4280 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0a0ff9640 10 client.15339 handle_command_reply: tid=0 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 client.15339 shutdown 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 -- 172.21.15.78:0/16717487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 msgr2=0x555e1c0f00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0f00b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0018b80 tx=0x7ff0b0009b90 comp rx=0 tx=0).stop 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 -- 172.21.15.78:0/16717487 shutdown_connections 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x555e1bcd4280 0x555e1c1618c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e1bd1f590 0x555e1c0ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0a0ff9640 0 client.15339 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e1bd40270 0x555e1c0f00b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 --2- 172.21.15.78:0/16717487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e1bd173b0 0x555e1c0ecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 -- 172.21.15.78:0/16717487 >> 172.21.15.78:0/16717487 conn(0x555e1bd343c0 msgr2=0x555e1c0db7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 -- 172.21.15.78:0/16717487 shutdown_connections 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 1 -- 172.21.15.78:0/16717487 wait complete. 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.384+0000 7ff0c69ae740 20 client.15339 trim_cache size 0 max 16384 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24502, 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 104.807297064, 2024-05-21T15:16:00.395 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:16:00.396 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:16:00.396 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:16:00.396 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 127.64378991 2024-05-21T15:16:00.396 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0a40a2b90 msgr2=0x7ff0a40a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0a40a2b90 0x7ff0a40a50a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x555e1c0e1850 tx=0x7ff0b40d2160 comp rx=0 tx=0).stop 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 msgr2=0x7ff0b81031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 0x7ff0b81031f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4005d00 tx=0x7ff0b400dde0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/2198521528 shutdown_connections 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7ff0a40a2b90 0x7ff0a40a50a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0b81011f0 0x7ff0b8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b8104040 0x7ff0b8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 --2- 172.21.15.78:0/2198521528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0b81006f0 0x7ff0b81031f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/2198521528 >> 172.21.15.78:0/2198521528 conn(0x7ff0b80fc2f0 msgr2=0x7ff0b8104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/2198521528 shutdown_connections 2024-05-21T15:16:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.385+0000 7ff0bfa2b640 1 -- 172.21.15.78:0/2198521528 wait complete. 2024-05-21T15:16:00.399 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-21T15:16:00.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 -- 172.21.15.78:0/461819552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 msgr2=0x7f1bc41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 --2- 172.21.15.78:0/461819552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 0x7f1bc41037a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009900 tx=0x7f1bbc02a410 comp rx=0 tx=0).stop 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 -- 172.21.15.78:0/461819552 shutdown_connections 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 --2- 172.21.15.78:0/461819552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bc41089e0 0x7f1bc410ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 --2- 172.21.15.78:0/461819552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bc4103ce0 0x7f1bc41040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 --2- 172.21.15.78:0/461819552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 0x7f1bc41037a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 -- 172.21.15.78:0/461819552 >> 172.21.15.78:0/461819552 conn(0x7f1bc40fdc90 msgr2=0x7f1bc4100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 -- 172.21.15.78:0/461819552 shutdown_connections 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.508+0000 7f1bcb909640 1 -- 172.21.15.78:0/461819552 wait complete. 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bcb909640 1 Processor -- start 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bcb909640 1 -- start start 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bcb909640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 0x7f1bc40768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bcb909640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bc4103ce0 0x7f1bc4074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bcb909640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bc41089e0 0x7f1bc40754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bcb909640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc410e6f0 con 0x7f1bc41089e0 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bcb909640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc410e570 con 0x7f1bc4102aa0 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bcb909640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc410e870 con 0x7f1bc4103ce0 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bc9ea6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bc41089e0 0x7f1bc40754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bc9ea6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bc41089e0 0x7f1bc40754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49468/0 (socket says 172.21.15.78:49468) 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bc9ea6640 1 -- 172.21.15.78:0/1207007603 learned_addr learned my addr 172.21.15.78:0/1207007603 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 0x7f1bc40768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.509+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bc4103ce0 0x7f1bc4074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bc9ea6640 1 -- 172.21.15.78:0/1207007603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bc4103ce0 msgr2=0x7f1bc4074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bc4103ce0 0x7f1bc4074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bc9ea6640 1 -- 172.21.15.78:0/1207007603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 msgr2=0x7f1bc40768a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 0x7f1bc40768a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bc9ea6640 1 -- 172.21.15.78:0/1207007603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc4075c20 con 0x7f1bc41089e0 2024-05-21T15:16:00.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bc4103ce0 0x7f1bc4074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 0x7f1bc40768a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bc41089e0 0x7f1bc40754f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000cad0 tx=0x7f1bc000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bba7fc640 1 -- 172.21.15.78:0/1207007603 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000ae20 con 0x7f1bc41089e0 2024-05-21T15:16:00.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bcb909640 1 -- 172.21.15.78:0/1207007603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc41a3410 con 0x7f1bc41089e0 2024-05-21T15:16:00.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bba7fc640 1 -- 172.21.15.78:0/1207007603 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc0005020 con 0x7f1bc41089e0 2024-05-21T15:16:00.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bba7fc640 1 -- 172.21.15.78:0/1207007603 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000f6b0 con 0x7f1bc41089e0 2024-05-21T15:16:00.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.510+0000 7f1bcb909640 1 -- 172.21.15.78:0/1207007603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc41a3870 con 0x7f1bc41089e0 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.512+0000 7f1bba7fc640 1 -- 172.21.15.78:0/1207007603 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1bc00051d0 con 0x7f1bc41089e0 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bd27b5740 1 Processor -- start 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bd27b5740 1 -- start start 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bba7fc640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bb00a2b90 0x7f1bb00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b977470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5ae3b0 0x55585b9779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b97e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55585b8db4d0 con 0x55585b5d7270 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55585b8833e0 con 0x55585b5ae3b0 2024-05-21T15:16:00.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55585b8d7490 con 0x55585b5b6590 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bb00a2b90 0x7f1bb00a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bba7fc640 1 -- 172.21.15.78:0/1207007603 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f1bc00cf390 con 0x7f1bc41089e0 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc96a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b977470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc96a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b977470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49482/0 (socket says 172.21.15.78:49482) 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc96a5640 1 -- 172.21.15.78:0/2406302959 learned_addr learned my addr 172.21.15.78:0/2406302959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5ae3b0 0x55585b9779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b97e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc96a5640 1 -- 172.21.15.78:0/2406302959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 msgr2=0x55585b97e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b97e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc96a5640 1 -- 172.21.15.78:0/2406302959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5ae3b0 msgr2=0x55585b9779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5ae3b0 0x55585b9779f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc96a5640 1 -- 172.21.15.78:0/2406302959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585b5ef2d0 con 0x55585b5d7270 2024-05-21T15:16:00.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.513+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b97e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.514+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bb00a2b90 0x7f1bb00a50a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40762b0 tx=0x7f1bb4009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.514+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b977470 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc031550 tx=0x7f1bbc032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53480b427097a4bf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.514+0000 7f1baf7fe640 1 -- 172.21.15.78:0/2406302959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc035030 con 0x55585b5d7270 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.514+0000 7f1baf7fe640 1 -- 172.21.15.78:0/2406302959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0318b0 con 0x55585b5d7270 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.514+0000 7f1baf7fe640 1 -- 172.21.15.78:0/2406302959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0046c0 con 0x55585b5d7270 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2406302959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 msgr2=0x55585b977470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b977470 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc031550 tx=0x7f1bbc032cb0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2406302959 shutdown_connections 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5ae3b0 0x55585b9779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b97e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2406302959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b977470 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2406302959 >> 172.21.15.78:0/2406302959 conn(0x55585b5cb3c0 msgr2=0x55585b972e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2406302959 shutdown_connections 2024-05-21T15:16:00.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2406302959 wait complete. 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 -- start start 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.515+0000 7f1bd27b5740 1 Processor -- start 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5ae3b0 0x55585b981200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 0x55585b981740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b9886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55585b97f2e0 con 0x55585b5d7270 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55585b8968e0 con 0x55585b5b6590 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55585b97f460 con 0x55585b5ae3b0 2024-05-21T15:16:00.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bd27b5740 10 client.0 _ll_register_callbacks cb 0x55585b8639c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b9886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b9886a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49494/0 (socket says 172.21.15.78:49494) 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 -- 172.21.15.78:0/644195696 learned_addr learned my addr 172.21.15.78:0/644195696 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 0x55585b981740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5ae3b0 0x55585b981200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 -- 172.21.15.78:0/644195696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5ae3b0 msgr2=0x55585b981200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5ae3b0 0x55585b981200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 -- 172.21.15.78:0/644195696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 msgr2=0x55585b981740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 0x55585b981740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 -- 172.21.15.78:0/644195696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585b6b96b0 con 0x55585b5d7270 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b9886a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00cce70 tx=0x7f1bc009c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62de592a5c11be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bacff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bacff9640 1 -- 172.21.15.78:0/644195696 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc001d070 con 0x55585b5d7270 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bacff9640 1 -- 172.21.15.78:0/644195696 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f1bc009be30 con 0x55585b5d7270 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bd27b5740 1 -- 172.21.15.78:0/644195696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bbc0318b0 con 0x55585b5d7270 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.516+0000 7f1bacff9640 1 -- 172.21.15.78:0/644195696 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bc00a4840 con 0x55585b5d7270 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bd27b5740 10 client.15351 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bd27b5740 1 -- 172.21.15.78:0/644195696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55585b98b610 con 0x55585b5d7270 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bacff9640 1 -- 172.21.15.78:0/644195696 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0318b0 con 0x55585b5d7270 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bacff9640 1 -- 172.21.15.78:0/644195696 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1bc009b100 con 0x55585b5d7270 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bd27b5740 10 client.15351 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bd27b5740 10 client.15351 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bd27b5740 20 client.15351 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55585b595e40 0x55585b98f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bd27b5740 4 client.15351 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bd27b5740 1 -- 172.21.15.78:0/644195696 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55585b8e8c00 con 0x55585b595e40 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.517+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55585b595e40 0x55585b98f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.518+0000 7f1bacff9640 10 client.15351 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.518+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55585b595e40 0x55585b98f060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d11ecf3936f57878 server_cookie=51924b55f0d82321 in_seq=0 out_seq=0 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.518+0000 7f1bacff9640 1 -- 172.21.15.78:0/644195696 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55585b8e8c00 con 0x55585b595e40 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.518+0000 7f1bacff9640 10 client.15351 handle_command_reply: tid=0 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.518+0000 7f1bd27b5740 1 client.15351 shutdown 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 -- 172.21.15.78:0/644195696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 msgr2=0x55585b9886a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b9886a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00cce70 tx=0x7f1bc009c390 comp rx=0 tx=0).stop 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 -- 172.21.15.78:0/644195696 shutdown_connections 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55585b595e40 0x55585b98f060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 0x55585b981740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bacff9640 0 client.15351 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5d7270 0x55585b9886a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/644195696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5ae3b0 0x55585b981200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 -- 172.21.15.78:0/644195696 >> 172.21.15.78:0/644195696 conn(0x55585b5cb3c0 msgr2=0x55585b971ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 -- 172.21.15.78:0/644195696 shutdown_connections 2024-05-21T15:16:00.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 1 -- 172.21.15.78:0/644195696 wait complete. 2024-05-21T15:16:00.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.519+0000 7f1bd27b5740 20 client.15351 trim_cache size 0 max 16384 2024-05-21T15:16:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.537+0000 7f1bd27b5740 1 Processor -- start 2024-05-21T15:16:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.537+0000 7f1bd27b5740 1 -- start start 2024-05-21T15:16:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b595e40 0x55585b972f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b9885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b988b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55585b98d3a0 con 0x55585b5ae3b0 2024-05-21T15:16:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55585b98da70 con 0x55585b595e40 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55585b98e140 con 0x55585b5b6590 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b9885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b9885c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49502/0 (socket says 172.21.15.78:49502) 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 -- 172.21.15.78:0/1454122083 learned_addr learned my addr 172.21.15.78:0/1454122083 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b988b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b595e40 0x55585b972f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 -- 172.21.15.78:0/1454122083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 msgr2=0x55585b988b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b988b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 -- 172.21.15.78:0/1454122083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b595e40 msgr2=0x55585b972f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b595e40 0x55585b972f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 -- 172.21.15.78:0/1454122083 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585b977020 con 0x55585b5ae3b0 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b9885c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40151a0 tx=0x7f1bb4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbe723f4385f12e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1baeffd640 1 -- 172.21.15.78:0/1454122083 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4005020 con 0x55585b5ae3b0 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1baeffd640 1 -- 172.21.15.78:0/1454122083 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb40056a0 con 0x55585b5ae3b0 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.538+0000 7f1baeffd640 1 -- 172.21.15.78:0/1454122083 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40216d0 con 0x55585b5ae3b0 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 -- 172.21.15.78:0/1454122083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 msgr2=0x55585b9885c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b9885c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40151a0 tx=0x7f1bb4018000 comp rx=0 tx=0).stop 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 -- 172.21.15.78:0/1454122083 shutdown_connections 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b5b6590 0x55585b988b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b9885c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/1454122083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b595e40 0x55585b972f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 -- 172.21.15.78:0/1454122083 >> 172.21.15.78:0/1454122083 conn(0x55585b5cb3c0 msgr2=0x55585b9727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 -- 172.21.15.78:0/1454122083 shutdown_connections 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 -- 172.21.15.78:0/1454122083 wait complete. 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.539+0000 7f1bd27b5740 1 -- start start 2024-05-21T15:16:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bd27b5740 1 Processor -- start 2024-05-21T15:16:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b595e40 0x55585b983d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b984290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bd27b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 0x55585b9f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55585b98eee0 con 0x55585b5ae3b0 2024-05-21T15:16:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55585b98e810 con 0x55585b5b6590 2024-05-21T15:16:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bd27b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55585b98f5b0 con 0x55585b595e40 2024-05-21T15:16:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bd27b5740 10 client.0 _ll_register_callbacks cb 0x7f1bc000a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc8ea4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b984290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc8ea4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b984290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49510/0 (socket says 172.21.15.78:49510) 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc8ea4640 1 -- 172.21.15.78:0/2605453143 learned_addr learned my addr 172.21.15.78:0/2605453143 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b595e40 0x55585b983d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc9ea6640 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 0x55585b9f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc8ea4640 1 -- 172.21.15.78:0/2605453143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b595e40 msgr2=0x55585b983d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b595e40 0x55585b983d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc8ea4640 1 -- 172.21.15.78:0/2605453143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 msgr2=0x55585b9f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 0x55585b9f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.540+0000 7f1bc8ea4640 1 -- 172.21.15.78:0/2605453143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585b9f6410 con 0x55585b5ae3b0 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bc8ea4640 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b984290 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4004da0 tx=0x7f1bb4004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897f51a465971fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bacff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bacff9640 1 -- 172.21.15.78:0/2605453143 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb402f070 con 0x55585b5ae3b0 2024-05-21T15:16:00.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bacff9640 1 -- 172.21.15.78:0/2605453143 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f1bb40051a0 con 0x55585b5ae3b0 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2605453143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb40056a0 con 0x55585b5ae3b0 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bacff9640 1 -- 172.21.15.78:0/2605453143 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1bb403a070 con 0x55585b5ae3b0 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 10 client.15357 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2605453143 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55585b9f7570 con 0x55585b5ae3b0 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bacff9640 1 -- 172.21.15.78:0/2605453143 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb40056a0 con 0x55585b5ae3b0 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bacff9640 1 -- 172.21.15.78:0/2605453143 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1bb401f040 con 0x55585b5ae3b0 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 10 client.15357 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 10 client.15357 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 20 client.15357 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55585b5d7270 0x55585b9fab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 4 client.15357 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.541+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2605453143 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "status"}) -- 0x55585b8e2f50 con 0x55585b5d7270 2024-05-21T15:16:00.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55585b5d7270 0x55585b9fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bacff9640 10 client.15357 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bc96a5640 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55585b5d7270 0x55585b9fab80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2dcaa4a56ed4fa9c server_cookie=777391a62af21067 in_seq=0 out_seq=0 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bacff9640 1 -- 172.21.15.78:0/2605453143 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55585b8e2f50 con 0x55585b5d7270 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bacff9640 10 client.15357 handle_command_reply: tid=0 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 client.15357 shutdown 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2605453143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 msgr2=0x55585b984290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b984290 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4004da0 tx=0x7f1bb4004c30 comp rx=0 tx=0).stop 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2605453143 shutdown_connections 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55585b5d7270 0x55585b9fab80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55585b5b6590 0x55585b9f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bacff9640 0 client.15357 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:16:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55585b5ae3b0 0x55585b984290 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 --2- 172.21.15.78:0/2605453143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55585b595e40 0x55585b983d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2605453143 >> 172.21.15.78:0/2605453143 conn(0x55585b5cb3c0 msgr2=0x55585b97f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2605453143 shutdown_connections 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.542+0000 7f1bd27b5740 1 -- 172.21.15.78:0/2605453143 wait complete. 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bd27b5740 20 client.15357 trim_cache size 0 max 16384 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24526, 2024-05-21T15:16:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:16:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:16:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:16:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 105.97984738, 2024-05-21T15:16:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:16:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:16:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:16:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 122.20667950399999 2024-05-21T15:16:00.553 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:00.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 -- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bb00a2b90 msgr2=0x7f1bb00a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bb00a2b90 0x7f1bb00a50a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40762b0 tx=0x7f1bb4009f90 comp rx=0 tx=0).stop 2024-05-21T15:16:00.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 -- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bc41089e0 msgr2=0x7f1bc40754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bc41089e0 0x7f1bc40754f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000cad0 tx=0x7f1bc000cf90 comp rx=0 tx=0).stop 2024-05-21T15:16:00.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 -- 172.21.15.78:0/1207007603 shutdown_connections 2024-05-21T15:16:00.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f1bb00a2b90 0x7f1bb00a50a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bc41089e0 0x7f1bc40754f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bc4103ce0 0x7f1bc4074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 --2- 172.21.15.78:0/1207007603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1bc4102aa0 0x7f1bc40768a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 -- 172.21.15.78:0/1207007603 >> 172.21.15.78:0/1207007603 conn(0x7f1bc40fdc90 msgr2=0x7f1bc41049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 -- 172.21.15.78:0/1207007603 shutdown_connections 2024-05-21T15:16:00.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.543+0000 7f1bcb909640 1 -- 172.21.15.78:0/1207007603 wait complete. 2024-05-21T15:16:00.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:00 smithi078 ceph-mon[31997]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:16:00.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:00 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1469639888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:00.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:00 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/175201191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:00.557 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-21T15:16:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:00 smithi084 ceph-mon[36334]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:16:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:00 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1469639888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:00 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/175201191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 -- 172.21.15.78:0/1924255575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0108750 msgr2=0x7fc4a010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/1924255575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0108750 0x7fc4a010ac60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc498009ca0 tx=0x7fc49802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 -- 172.21.15.78:0/1924255575 shutdown_connections 2024-05-21T15:16:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/1924255575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a0105be0 0x7fc4a01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/1924255575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0108750 0x7fc4a010ac60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/1924255575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4a0100e00 0x7fc4a0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 -- 172.21.15.78:0/1924255575 >> 172.21.15.78:0/1924255575 conn(0x7fc4a00fc330 msgr2=0x7fc4a00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 -- 172.21.15.78:0/1924255575 shutdown_connections 2024-05-21T15:16:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.666+0000 7fc4a66ca640 1 -- 172.21.15.78:0/1924255575 wait complete. 2024-05-21T15:16:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc4a66ca640 1 Processor -- start 2024-05-21T15:16:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc4a66ca640 1 -- start start 2024-05-21T15:16:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc4a66ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0100e00 0x7fc4a019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc4a66ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4a0105be0 0x7fc4a019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc4a66ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a0108750 0x7fc4a01a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc4a66ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a010e730 con 0x7fc4a0100e00 2024-05-21T15:16:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc4a66ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a010e5b0 con 0x7fc4a0108750 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc4a66ca640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a010e8b0 con 0x7fc4a0105be0 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc49ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0100e00 0x7fc4a019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc49ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0100e00 0x7fc4a019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49524/0 (socket says 172.21.15.78:49524) 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.667+0000 7fc49ffff640 1 -- 172.21.15.78:0/2089093700 learned_addr learned my addr 172.21.15.78:0/2089093700 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4a0105be0 0x7fc4a019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a0108750 0x7fc4a01a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49ffff640 1 -- 172.21.15.78:0/2089093700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4a0105be0 msgr2=0x7fc4a019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49ffff640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4a0105be0 0x7fc4a019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49ffff640 1 -- 172.21.15.78:0/2089093700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a0108750 msgr2=0x7fc4a01a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49ffff640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a0108750 0x7fc4a01a2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49ffff640 1 -- 172.21.15.78:0/2089093700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a01a2fa0 con 0x7fc4a0100e00 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a0108750 0x7fc4a01a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49ffff640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0100e00 0x7fc4a019b460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc49000d740 tx=0x7fc49000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49d7fa640 1 -- 172.21.15.78:0/2089093700 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490017070 con 0x7fc4a0100e00 2024-05-21T15:16:00.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc4a66ca640 1 -- 172.21.15.78:0/2089093700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a01a3290 con 0x7fc4a0100e00 2024-05-21T15:16:00.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49d7fa640 1 -- 172.21.15.78:0/2089093700 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49000de50 con 0x7fc4a0100e00 2024-05-21T15:16:00.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc49d7fa640 1 -- 172.21.15.78:0/2089093700 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490004e00 con 0x7fc4a0100e00 2024-05-21T15:16:00.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.668+0000 7fc4a66ca640 1 -- 172.21.15.78:0/2089093700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a01a3790 con 0x7fc4a0100e00 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc4ad572740 1 Processor -- start 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc4ad572740 1 -- start start 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc49d7fa640 1 -- 172.21.15.78:0/2089093700 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc490004fa0 con 0x7fc4a0100e00 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5c2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c201590 0x55fb3c5c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc4ad572740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5264d0 con 0x55fb3c1f93b0 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc4ad572740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c4ce3e0 con 0x55fb3c222270 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc4ad572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c522490 con 0x55fb3c201590 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.671+0000 7fc49ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49534/0 (socket says 172.21.15.78:49534) 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49ffff640 1 -- 172.21.15.78:0/3907436617 learned_addr learned my addr 172.21.15.78:0/3907436617 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c201590 0x55fb3c5c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49ffff640 1 -- 172.21.15.78:0/3907436617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c201590 msgr2=0x55fb3c5c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49ffff640 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c201590 0x55fb3c5c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49ffff640 1 -- 172.21.15.78:0/3907436617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 msgr2=0x55fb3c5c2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49ffff640 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5c2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49ffff640 1 -- 172.21.15.78:0/3907436617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3c23a2d0 con 0x55fb3c1f93b0 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c201590 0x55fb3c5c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49ffff640 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5c29f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc4900d2ca0 tx=0x7fc4900d75d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517b7b10bdaca018 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49d7fa640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc4880a2b90 0x7fc4880a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc4867fc640 1 -- 172.21.15.78:0/3907436617 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4900975e0 con 0x55fb3c1f93b0 2024-05-21T15:16:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc4880a2b90 0x7fc4880a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc49d7fa640 1 -- 172.21.15.78:0/2089093700 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc4900ceb00 con 0x7fc4a0100e00 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc4867fc640 1 -- 172.21.15.78:0/3907436617 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4900e0070 con 0x55fb3c1f93b0 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.672+0000 7fc4867fc640 1 -- 172.21.15.78:0/3907436617 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49009a790 con 0x55fb3c1f93b0 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 -- 172.21.15.78:0/3907436617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 msgr2=0x55fb3c5c29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc4880a2b90 0x7fc4880a50a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc498006010 tx=0x7fc498009420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5c29f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc4900d2ca0 tx=0x7fc4900d75d0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 -- 172.21.15.78:0/3907436617 shutdown_connections 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c201590 0x55fb3c5c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5c29f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 --2- 172.21.15.78:0/3907436617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5c2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 -- 172.21.15.78:0/3907436617 >> 172.21.15.78:0/3907436617 conn(0x55fb3c2163c0 msgr2=0x55fb3c5bde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 -- 172.21.15.78:0/3907436617 shutdown_connections 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.673+0000 7fc4ad572740 1 -- 172.21.15.78:0/3907436617 wait complete. 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 1 -- start start 2024-05-21T15:16:00.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 1 Processor -- start 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c201590 0x55fb3c5cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5ca2e0 con 0x55fb3c201590 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c4e18e0 con 0x55fb3c222270 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5ca460 con 0x55fb3c1f93b0 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc4ad572740 10 client.0 _ll_register_callbacks cb 0x55fb3c4ae9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c201590 0x55fb3c5cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c201590 0x55fb3c5cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49542/0 (socket says 172.21.15.78:49542) 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.674+0000 7fc49f7fe640 1 -- 172.21.15.78:0/2348763943 learned_addr learned my addr 172.21.15.78:0/2348763943 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc49f7fe640 1 -- 172.21.15.78:0/2348763943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1f93b0 msgr2=0x55fb3c5cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc49ffff640 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc49f7fe640 1 -- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 msgr2=0x55fb3c5d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc49f7fe640 1 -- 172.21.15.78:0/2348763943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb3c3046b0 con 0x55fb3c201590 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c201590 0x55fb3c5cc850 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc49400bf20 tx=0x7fc494011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f90f91a6f614540 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc47f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc47f7fe640 1 -- 172.21.15.78:0/2348763943 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494012710 con 0x55fb3c201590 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc47f7fe640 1 -- 172.21.15.78:0/2348763943 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc4940128b0 con 0x55fb3c201590 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc4ad572740 1 -- 172.21.15.78:0/2348763943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4900e0070 con 0x55fb3c201590 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc47f7fe640 1 -- 172.21.15.78:0/2348763943 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc494012ab0 con 0x55fb3c201590 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc4ad572740 10 client.15369 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc4ad572740 1 -- 172.21.15.78:0/2348763943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb3c5d66b0 con 0x55fb3c201590 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc47f7fe640 1 -- 172.21.15.78:0/2348763943 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4900e0070 con 0x55fb3c201590 2024-05-21T15:16:00.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.675+0000 7fc47f7fe640 1 -- 172.21.15.78:0/2348763943 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc494025420 con 0x55fb3c201590 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc4ad572740 10 client.15369 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc4ad572740 10 client.15369 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc4ad572740 20 client.15369 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55fb3c1e0e40 0x55fb3c5da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc4ad572740 4 client.15369 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc4ad572740 1 -- 172.21.15.78:0/2348763943 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb3c533c00 con 0x55fb3c1e0e40 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc49ffff640 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55fb3c1e0e40 0x55fb3c5da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc47f7fe640 10 client.15369 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.676+0000 7fc49ffff640 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55fb3c1e0e40 0x55fb3c5da1d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20829ffabda0128e server_cookie=33bc65e55c9a9d35 in_seq=0 out_seq=0 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc47f7fe640 1 -- 172.21.15.78:0/2348763943 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb3c533c00 con 0x55fb3c1e0e40 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc47f7fe640 10 client.15369 handle_command_reply: tid=0 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 client.15369 shutdown 2024-05-21T15:16:00.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 -- 172.21.15.78:0/2348763943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c201590 msgr2=0x55fb3c5cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c201590 0x55fb3c5cc850 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc49400bf20 tx=0x7fc494011870 comp rx=0 tx=0).stop 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 -- 172.21.15.78:0/2348763943 shutdown_connections 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55fb3c1e0e40 0x55fb3c5da1d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c222270 0x55fb3c5d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc47f7fe640 0 client.15369 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c201590 0x55fb3c5cc850 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2348763943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.677+0000 7fc4ad572740 1 -- 172.21.15.78:0/2348763943 >> 172.21.15.78:0/2348763943 conn(0x55fb3c2163c0 msgr2=0x55fb3c5bcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.678+0000 7fc4ad572740 1 -- 172.21.15.78:0/2348763943 shutdown_connections 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.678+0000 7fc4ad572740 1 -- 172.21.15.78:0/2348763943 wait complete. 2024-05-21T15:16:00.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.678+0000 7fc4ad572740 20 client.15369 trim_cache size 0 max 16384 2024-05-21T15:16:00.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:00 smithi172 ceph-mon[36402]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-21T15:16:00.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:00 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1469639888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:00.688 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:00 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/175201191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc4ad572740 1 Processor -- start 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc4ad572740 1 -- start start 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 0x55fb3c5cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 0x55fb3c5d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc4ad572740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5d8510 con 0x55fb3c1f93b0 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc4ad572740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5d8be0 con 0x55fb3c201590 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc4ad572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5d92b0 con 0x55fb3c1e0e40 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5d3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49554/0 (socket says 172.21.15.78:49554) 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc49f7fe640 1 -- 172.21.15.78:0/2252324385 learned_addr learned my addr 172.21.15.78:0/2252324385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc49f7fe640 1 -- 172.21.15.78:0/2252324385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 msgr2=0x55fb3c5cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/2252324385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 0x55fb3c5cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc49f7fe640 1 -- 172.21.15.78:0/2252324385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 msgr2=0x55fb3c5d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.696+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/2252324385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 0x55fb3c5d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.697+0000 7fc49f7fe640 1 -- 172.21.15.78:0/2252324385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3c5c2020 con 0x55fb3c1f93b0 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.697+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/2252324385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5d3660 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc494025a30 tx=0x7fc49400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80734b0ea0be6cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.697+0000 7fc485ffb640 1 -- 172.21.15.78:0/2252324385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49402b070 con 0x55fb3c1f93b0 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.697+0000 7fc485ffb640 1 -- 172.21.15.78:0/2252324385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc494012c50 con 0x55fb3c1f93b0 2024-05-21T15:16:00.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.697+0000 7fc485ffb640 1 -- 172.21.15.78:0/2252324385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940055a0 con 0x55fb3c1f93b0 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 -- 172.21.15.78:0/2252324385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 msgr2=0x55fb3c5d3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2252324385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5d3660 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc494025a30 tx=0x7fc49400b6a0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 -- 172.21.15.78:0/2252324385 shutdown_connections 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2252324385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 0x55fb3c5d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2252324385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5d3660 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 --2- 172.21.15.78:0/2252324385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 0x55fb3c5cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 -- 172.21.15.78:0/2252324385 >> 172.21.15.78:0/2252324385 conn(0x55fb3c2163c0 msgr2=0x55fb3c5ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 -- 172.21.15.78:0/2252324385 shutdown_connections 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 -- 172.21.15.78:0/2252324385 wait complete. 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 -- start start 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 Processor -- start 2024-05-21T15:16:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 0x55fb3c5cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 0x55fb3c5d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.698+0000 7fc4ad572740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5da050 con 0x55fb3c1f93b0 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc4ad572740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5d9980 con 0x55fb3c201590 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc4ad572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fb3c5da720 con 0x55fb3c1e0e40 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc4ad572740 10 client.0 _ll_register_callbacks cb 0x7fc49400b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49564/0 (socket says 172.21.15.78:49564) 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 -- 172.21.15.78:0/176143344 learned_addr learned my addr 172.21.15.78:0/176143344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49ffff640 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 0x55fb3c5cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 0x55fb3c5d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 -- 172.21.15.78:0/176143344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 msgr2=0x55fb3c5cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 0x55fb3c5cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 -- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 msgr2=0x55fb3c5d2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:00.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 0x55fb3c5d2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 -- 172.21.15.78:0/176143344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb3c5d2c30 con 0x55fb3c1f93b0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc4a4c67640 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 0x55fb3c5d2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc49f7fe640 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cf730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc4940259b0 tx=0x7fc49400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd71a437a9e593d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc47f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc47f7fe640 1 -- 172.21.15.78:0/176143344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49402b040 con 0x55fb3c1f93b0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc4ad572740 1 -- 172.21.15.78:0/176143344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc494012c50 con 0x55fb3c1f93b0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc47f7fe640 1 -- 172.21.15.78:0/176143344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc494005320 con 0x55fb3c1f93b0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc47f7fe640 1 -- 172.21.15.78:0/176143344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc494005520 con 0x55fb3c1f93b0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc4ad572740 10 client.15375 fetch_fsmap learned FSMap version 25 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.699+0000 7fc4ad572740 1 -- 172.21.15.78:0/176143344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb3c5d3eb0 con 0x55fb3c1f93b0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc47f7fe640 1 -- 172.21.15.78:0/176143344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc494012c50 con 0x55fb3c1f93b0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc47f7fe640 1 -- 172.21.15.78:0/176143344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc494003680 con 0x55fb3c1f93b0 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc4ad572740 10 client.15375 fetch_fsmap finished waiting for FSMap version 25 2024-05-21T15:16:00.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc4ad572740 10 client.15375 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc4ad572740 20 client.15375 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc4ad572740 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55fb3c222270 0x55fb3c644000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc4ad572740 4 client.15375 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc4ad572740 1 -- 172.21.15.78:0/176143344 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "status"}) -- 0x55fb3c5b8ee0 con 0x55fb3c222270 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc49ffff640 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55fb3c222270 0x55fb3c644000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc47f7fe640 10 client.15375 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.700+0000 7fc49ffff640 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55fb3c222270 0x55fb3c644000 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78fb4ccd91297dc server_cookie=1e06480564ac028a in_seq=0 out_seq=0 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc47f7fe640 1 -- 172.21.15.78:0/176143344 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55fb3c5b8ee0 con 0x55fb3c222270 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc47f7fe640 10 client.15375 handle_command_reply: tid=0 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 client.15375 shutdown 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 -- 172.21.15.78:0/176143344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 msgr2=0x55fb3c5cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cf730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc4940259b0 tx=0x7fc49400bbb0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 -- 172.21.15.78:0/176143344 shutdown_connections 2024-05-21T15:16:00.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55fb3c222270 0x55fb3c644000 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fb3c201590 0x55fb3c5d2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc47f7fe640 0 client.15375 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb3c1f93b0 0x55fb3c5cf730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 --2- 172.21.15.78:0/176143344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb3c1e0e40 0x55fb3c5cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 -- 172.21.15.78:0/176143344 >> 172.21.15.78:0/176143344 conn(0x55fb3c2163c0 msgr2=0x55fb3c5ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 -- 172.21.15.78:0/176143344 shutdown_connections 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 1 -- 172.21.15.78:0/176143344 wait complete. 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.701+0000 7fc4ad572740 20 client.15375 trim_cache size 0 max 16384 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:00.710 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "852c9762-1783-11ef-bc9a-c7b262605968", 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24527, 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 108.148294005, 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 87, 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 85, 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 113.69957445 2024-05-21T15:16:00.711 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 -- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc4880a2b90 msgr2=0x7fc4880a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc4880a2b90 0x7fc4880a50a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc498006010 tx=0x7fc498009420 comp rx=0 tx=0).stop 2024-05-21T15:16:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 -- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0100e00 msgr2=0x7fc4a019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0100e00 0x7fc4a019b460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc49000d740 tx=0x7fc49000dc00 comp rx=0 tx=0).stop 2024-05-21T15:16:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 -- 172.21.15.78:0/2089093700 shutdown_connections 2024-05-21T15:16:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fc4880a2b90 0x7fc4880a50a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4a0108750 0x7fc4a01a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4a0105be0 0x7fc4a019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 --2- 172.21.15.78:0/2089093700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4a0100e00 0x7fc4a019b460 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 -- 172.21.15.78:0/2089093700 >> 172.21.15.78:0/2089093700 conn(0x7fc4a00fc330 msgr2=0x7fc4a0102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 -- 172.21.15.78:0/2089093700 shutdown_connections 2024-05-21T15:16:00.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.702+0000 7fc4a66ca640 1 -- 172.21.15.78:0/2089093700 wait complete. 2024-05-21T15:16:00.716 INFO:teuthology.run_tasks:Running task sequential... 2024-05-21T15:16:00.725 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-21T15:16:00.725 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-21T15:16:00.726 INFO:teuthology.task.print:Enabling mgr modules 2024-05-21T15:16:00.726 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-21T15:16:00.727 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T15:16:00.727 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:16:00.727 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-21T15:16:00.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 -- 172.21.15.78:0/1466043085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a8102c10 msgr2=0x7f36a8103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 --2- 172.21.15.78:0/1466043085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a8102c10 0x7f36a8103020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f369c009900 tx=0x7f369c02a410 comp rx=0 tx=0).stop 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 -- 172.21.15.78:0/1466043085 shutdown_connections 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 --2- 172.21.15.78:0/1466043085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36a8107910 0x7f36a8109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 --2- 172.21.15.78:0/1466043085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a8102c10 0x7f36a8103020 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 --2- 172.21.15.78:0/1466043085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a81019d0 0x7f36a81026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 -- 172.21.15.78:0/1466043085 >> 172.21.15.78:0/1466043085 conn(0x7f36a80fcbc0 msgr2=0x7f36a80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 -- 172.21.15.78:0/1466043085 shutdown_connections 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.835+0000 7f36ad521640 1 -- 172.21.15.78:0/1466043085 wait complete. 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36ad521640 1 Processor -- start 2024-05-21T15:16:00.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36ad521640 1 -- start start 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36ad521640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a81019d0 0x7f36a8069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36ad521640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a8102c10 0x7f36a819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36ad521640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36a8107910 0x7f36a81a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36ad521640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36a810d620 con 0x7f36a81019d0 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36ad521640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36a810d4a0 con 0x7f36a8102c10 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36ad521640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36a810d7a0 con 0x7f36a8107910 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a81019d0 0x7f36a8069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a81019d0 0x7f36a8069e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49590/0 (socket says 172.21.15.78:49590) 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.836+0000 7f36a6ffd640 1 -- 172.21.15.78:0/2323609181 learned_addr learned my addr 172.21.15.78:0/2323609181 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36a67fc640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a8102c10 0x7f36a819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36a6ffd640 1 -- 172.21.15.78:0/2323609181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36a8107910 msgr2=0x7f36a81a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36a6ffd640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36a8107910 0x7f36a81a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36a6ffd640 1 -- 172.21.15.78:0/2323609181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a8102c10 msgr2=0x7f36a819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36a6ffd640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a8102c10 0x7f36a819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36a6ffd640 1 -- 172.21.15.78:0/2323609181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a81a0de0 con 0x7f36a81019d0 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36a67fc640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a8102c10 0x7f36a819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36a6ffd640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a81019d0 0x7f36a8069e00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f369800e9f0 tx=0x7f369800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f368ffff640 1 -- 172.21.15.78:0/2323609181 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369800ccc0 con 0x7f36a81019d0 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f368ffff640 1 -- 172.21.15.78:0/2323609181 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36980035f0 con 0x7f36a81019d0 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a81a10d0 con 0x7f36a81019d0 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f368ffff640 1 -- 172.21.15.78:0/2323609181 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369800ce60 con 0x7f36a81019d0 2024-05-21T15:16:00.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.837+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a81a1620 con 0x7f36a81019d0 2024-05-21T15:16:00.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.838+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a8106720 con 0x7f36a81019d0 2024-05-21T15:16:00.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.839+0000 7f368ffff640 1 -- 172.21.15.78:0/2323609181 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36980037a0 con 0x7f36a81019d0 2024-05-21T15:16:00.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.840+0000 7f368ffff640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f36900a2b80 0x7f36900a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:00.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.840+0000 7f36a67fc640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f36900a2b80 0x7f36900a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:00.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.840+0000 7f368ffff640 1 -- 172.21.15.78:0/2323609181 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f36980cf5c0 con 0x7f36a81019d0 2024-05-21T15:16:00.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.840+0000 7f36a67fc640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f36900a2b80 0x7f36900a5090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f369c000c00 tx=0x7f369c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:00.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:00.843+0000 7f368ffff640 1 -- 172.21.15.78:0/2323609181 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f369809c500 con 0x7f36a81019d0 2024-05-21T15:16:01.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.067+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f36a802d2a0 con 0x7f36a81019d0 2024-05-21T15:16:01.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.375+0000 7f368ffff640 1 -- 172.21.15.78:0/2323609181 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f369809be00 con 0x7f36a81019d0 2024-05-21T15:16:01.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.378+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f36900a2b80 msgr2=0x7f36900a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:01.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.378+0000 7f36ad521640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f36900a2b80 0x7f36900a5090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f369c000c00 tx=0x7f369c031400 comp rx=0 tx=0).stop 2024-05-21T15:16:01.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.378+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a81019d0 msgr2=0x7f36a8069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:01.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.378+0000 7f36ad521640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a81019d0 0x7f36a8069e00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f369800e9f0 tx=0x7f369800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:16:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.379+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 shutdown_connections 2024-05-21T15:16:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.379+0000 7f36ad521640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f36900a2b80 0x7f36900a5090 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.379+0000 7f36ad521640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a8102c10 0x7f36a819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.379+0000 7f36ad521640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36a8107910 0x7f36a81a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.379+0000 7f36ad521640 1 --2- 172.21.15.78:0/2323609181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a81019d0 0x7f36a8069e00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:01.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.379+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 >> 172.21.15.78:0/2323609181 conn(0x7f36a80fcbc0 msgr2=0x7f36a81038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.380+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 shutdown_connections 2024-05-21T15:16:01.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:01.380+0000 7f36ad521640 1 -- 172.21.15.78:0/2323609181 wait complete. 2024-05-21T15:16:01.399 INFO:teuthology.run_tasks:Running task kclient... 2024-05-21T15:16:01.424 INFO:tasks.kclient:Mounting kernel clients... 2024-05-21T15:16:01.424 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-21T15:16:01.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:01.425 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-05-21T15:16:01.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:01.441 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-05-21T15:16:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:Cannot find device "ceph-brx" 2024-05-21T15:16:01.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:16:01.511 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-21T15:16:01.511 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-21T15:16:01.511 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-21T15:16:01.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-21T15:16:01.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:01 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2323609181' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T15:16:01.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:01 smithi084 ceph-mon[36334]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-21T15:16:01.646 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:01 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: ignoring --setuser ceph since I am not root 2024-05-21T15:16:01.646 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:01 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: ignoring --setgroup ceph since I am not root 2024-05-21T15:16:01.646 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:01 smithi084 ceph-mgr[37721]: -- 172.21.15.84:0/419393719 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7fd3b25a0 con 0x55e7fd4f0900 2024-05-21T15:16:01.646 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:01 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:01.582+0000 7f12b5eef200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:16:01.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:01 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2323609181' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T15:16:01.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:01 smithi172 ceph-mon[36402]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-21T15:16:01.684 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:01 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: ignoring --setuser ceph since I am not root 2024-05-21T15:16:01.684 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:01 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: ignoring --setgroup ceph since I am not root 2024-05-21T15:16:01.684 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:01 smithi172 ceph-mgr[38259]: -- 172.21.15.172:0/415802498 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626f004c5a0 con 0x5626f018a480 2024-05-21T15:16:01.684 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:01 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:01.579+0000 7f6214cc9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:16:01.796 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:01 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2323609181' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T15:16:01.796 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:01 smithi078 ceph-mon[31997]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-21T15:16:01.796 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:01 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: ignoring --setuser ceph since I am not root 2024-05-21T15:16:01.796 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:01 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: ignoring --setgroup ceph since I am not root 2024-05-21T15:16:01.796 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:01 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:01.591+0000 7f7d894a2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T15:16:01.797 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:16:02.049 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:01 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:01.792+0000 7f7d894a2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:16:02.050 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:01 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:01.901+0000 7f7d894a2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:16:02.145 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:01 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:01.770+0000 7f12b5eef200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:16:02.146 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:01 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:01.872+0000 7f12b5eef200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:16:02.182 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:01 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:01.767+0000 7f6214cc9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:16:02.182 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:01 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:01.870+0000 7f6214cc9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T15:16:02.311 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:02 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:02.220+0000 7f7d894a2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:16:02.311 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:02 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:02.308+0000 7f7d894a2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:16:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:02 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2323609181' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T15:16:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:02 smithi172 ceph-mon[36402]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-05-21T15:16:02.434 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:02 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:02.179+0000 7f6214cc9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:16:02.434 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:02 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:02.265+0000 7f6214cc9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:16:02.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:02 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2323609181' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T15:16:02.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:02 smithi078 ceph-mon[31997]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-05-21T15:16:02.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:02 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2323609181' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T15:16:02.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:02 smithi084 ceph-mon[36334]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-05-21T15:16:02.646 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:02 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:02.180+0000 7f12b5eef200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T15:16:02.646 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:02 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:02.267+0000 7f12b5eef200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 -- 172.21.15.78:0/3487460666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 msgr2=0x7fb5081100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 --2- 172.21.15.78:0/3487460666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 0x7fb5081100c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b0d0 tx=0x7fb50000b3e0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 -- 172.21.15.78:0/3487460666 shutdown_connections 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 --2- 172.21.15.78:0/3487460666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 0x7fb5081100c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 --2- 172.21.15.78:0/3487460666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5081019b0 0x7fb508101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 --2- 172.21.15.78:0/3487460666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb508107dd0 0x7fb5081081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 -- 172.21.15.78:0/3487460666 >> 172.21.15.78:0/3487460666 conn(0x7fb5080fd570 msgr2=0x7fb5080ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 -- 172.21.15.78:0/3487460666 shutdown_connections 2024-05-21T15:16:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.438+0000 7fb50de53700 1 -- 172.21.15.78:0/3487460666 wait complete. 2024-05-21T15:16:03.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.439+0000 7fb50de53700 1 Processor -- start 2024-05-21T15:16:03.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.439+0000 7fb50de53700 1 -- start start 2024-05-21T15:16:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.439+0000 7fb50de53700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5081019b0 0x7fb508102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.439+0000 7fb50de53700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb508107dd0 0x7fb508102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.439+0000 7fb50de53700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 0x7fb508103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.439+0000 7fb50de53700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081134e0 con 0x7fb508108720 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.439+0000 7fb50de53700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb508113360 con 0x7fb508107dd0 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.439+0000 7fb50de53700 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb508113660 con 0x7fb5081019b0 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 0x7fb508103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb5077fe700 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5081019b0 0x7fb508102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 0x7fb508103380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49638/0 (socket says 172.21.15.78:49638) 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb506ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb508107dd0 0x7fb508102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 -- 172.21.15.78:0/4098728514 learned_addr learned my addr 172.21.15.78:0/4098728514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 -- 172.21.15.78:0/4098728514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5081019b0 msgr2=0x7fb508102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5081019b0 0x7fb508102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 -- 172.21.15.78:0/4098728514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb508107dd0 msgr2=0x7fb508102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb508107dd0 0x7fb508102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 -- 172.21.15.78:0/4098728514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5081a80f0 con 0x7fb508108720 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb507fff700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 0x7fb508103380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb500006070 tx=0x7fb500004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb504ff9700 1 -- 172.21.15.78:0/4098728514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500021070 con 0x7fb508108720 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb504ff9700 1 -- 172.21.15.78:0/4098728514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb500007b80 con 0x7fb508108720 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb504ff9700 1 -- 172.21.15.78:0/4098728514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5000052a0 con 0x7fb508108720 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.440+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5081a8380 con 0x7fb508108720 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.441+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5081a8830 con 0x7fb508108720 2024-05-21T15:16:03.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.441+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb508050170 con 0x7fb508108720 2024-05-21T15:16:03.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.443+0000 7fb504ff9700 1 -- 172.21.15.78:0/4098728514 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fb500024070 con 0x7fb508108720 2024-05-21T15:16:03.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.444+0000 7fb504ff9700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb4f00a2c70 0x7fb4f00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:03.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.444+0000 7fb5077fe700 1 -- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb4f00a2c70 msgr2=0x7fb4f00a5180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:03.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.444+0000 7fb504ff9700 1 -- 172.21.15.78:0/4098728514 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb50000f020 con 0x7fb508108720 2024-05-21T15:16:03.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.444+0000 7fb5077fe700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb4f00a2c70 0x7fb4f00a5180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:16:03.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.452+0000 7fb504ff9700 1 -- 172.21.15.78:0/4098728514 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb50002b020 con 0x7fb508108720 2024-05-21T15:16:03.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.645+0000 7fb5077fe700 1 -- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb4f00a2c70 msgr2=0x7fb4f00a5180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:03.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.645+0000 7fb5077fe700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb4f00a2c70 0x7fb4f00a5180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:16:03.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.698+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb50802dc60 con 0x7fb508108720 2024-05-21T15:16:03.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.698+0000 7fb504ff9700 1 -- 172.21.15.78:0/4098728514 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fb50009daa0 con 0x7fb508108720 2024-05-21T15:16:03.701 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-21T15:16:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb4f00a2c70 msgr2=0x7fb4f00a5180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:16:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb4f00a2c70 0x7fb4f00a5180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 msgr2=0x7fb508103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 0x7fb508103380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb500006070 tx=0x7fb500004d00 comp rx=0 tx=0).stop 2024-05-21T15:16:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 shutdown_connections 2024-05-21T15:16:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fb4f00a2c70 0x7fb4f00a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb508108720 0x7fb508103380 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5081019b0 0x7fb508102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 --2- 172.21.15.78:0/4098728514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb508107dd0 0x7fb508102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:03.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 >> 172.21.15.78:0/4098728514 conn(0x7fb5080fd570 msgr2=0x7fb508107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:03.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 shutdown_connections 2024-05-21T15:16:03.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:03.701+0000 7fb50de53700 1 -- 172.21.15.78:0/4098728514 wait complete. 2024-05-21T15:16:03.753 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:03 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:03.448+0000 7f12b5eef200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:16:03.753 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:03 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:03.620+0000 7f12b5eef200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:16:03.753 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:03 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:03.449+0000 7f6214cc9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:16:03.753 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:03 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:03.622+0000 7f6214cc9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:16:03.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:03 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4098728514' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T15:16:03.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:03 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:03.535+0000 7f7d894a2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T15:16:03.848 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:03 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:03.715+0000 7f7d894a2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T15:16:04.086 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:03 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4098728514' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T15:16:04.086 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:03 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:03.792+0000 7f12b5eef200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:16:04.087 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:03 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4098728514' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T15:16:04.088 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:03 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:03.796+0000 7f6214cc9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:16:04.192 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:03 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:03.891+0000 7f7d894a2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T15:16:04.383 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:04 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:04.082+0000 7f12b5eef200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:16:04.383 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:04 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:04.177+0000 7f12b5eef200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:16:04.383 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:04 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:04.380+0000 7f12b5eef200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:16:04.388 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:04 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:04.084+0000 7f6214cc9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:16:04.388 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:04 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:04.179+0000 7f6214cc9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:16:04.388 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:04 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:04.384+0000 7f6214cc9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:16:04.463 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:04 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:04.189+0000 7f7d894a2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T15:16:04.464 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:04 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:04.284+0000 7f7d894a2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T15:16:04.499 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-21T15:16:04.499 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-21T15:16:04.499 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-05-21T15:16:04.499 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-21T15:16:04.499 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-21T15:16:04.499 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-21T15:16:04.499 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-21T15:16:04.499 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-21T15:16:04.500 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-21T15:16:04.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:04.500 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-05-21T15:16:04.516 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-21T15:16:04.516 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-21T15:16:04.516 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-05-21T15:16:04.516 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-05-21T15:16:04.516 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-05-21T15:16:04.516 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-05-21T15:16:04.516 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42101sec preferred_lft 42101sec 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-05-21T15:16:04.517 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-05-21T15:16:04.518 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-21T15:16:04.518 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:16:04.518 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-05-21T15:16:04.519 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-05-21T15:16:04.519 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-05-21T15:16:04.519 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-05-21T15:16:04.519 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-21T15:16:04.519 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-05-21T15:16:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:16:04.645 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:04 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:04.577+0000 7f12b5eef200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:16:04.683 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:04 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:04.581+0000 7f6214cc9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:16:04.714 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:04 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:04.502+0000 7f7d894a2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T15:16:04.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:04.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:04.734 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-21T15:16:04.761 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-05-21T15:16:04.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:04.763 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-05-21T15:16:04.836 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-05-21T15:16:04.836 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-21T15:16:04.836 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-21T15:16:04.836 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-21T15:16:04.836 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-21T15:16:04.837 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:16:04.837 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-05-21T15:16:04.837 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-21T15:16:04.837 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-21T15:16:04.837 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-21T15:16:04.837 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-05-21T15:16:04.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:16:05.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:04 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:04.714+0000 7f7d894a2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T15:16:05.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:05.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:05.164 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-05-21T15:16:05.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:05.180 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-05-21T15:16:05.237 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:16:05.237 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-05-21T15:16:05.237 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-21T15:16:05.237 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-21T15:16:05.237 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-05-21T15:16:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:16:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:05.357 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-21T15:16:05.358 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:16:05.358 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-05-21T15:16:05.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-21T15:16:05.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-21T15:16:05.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-21T15:16:05.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-21T15:16:05.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-21T15:16:05.359 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-05-21T15:16:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:16:05.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:05.499 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T15:16:05.499 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-05-21T15:16:05.499 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-05-21T15:16:05.499 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-05-21T15:16:05.500 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-05-21T15:16:05.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:16:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:05.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-21T15:16:05.584 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-21T15:16:05.598 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:05 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:05.359+0000 7f7d894a2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:16:05.609 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:05.183+0000 7f6214cc9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:16:05.610 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-21T15:16:05.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-21T15:16:05.611 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-21T15:16:05.611 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:05.176+0000 7f12b5eef200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T15:16:05.665 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-21T15:16:05.666 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-21T15:16:05.666 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/305cad38-0d45-4543-b894-976a9e5440da 2024-05-21T15:16:05.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:05.666 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/305cad38-0d45-4543-b894-976a9e5440da /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-21T15:16:05.774 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-21T15:16:05.774 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-21T15:16:05.774 INFO:teuthology.orchestra.run.smithi078.stdout:invalid new device string format 2024-05-21T15:16:05.895 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:05.608+0000 7f12b5eef200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:16:05.896 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:05.706+0000 7f12b5eef200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:16:05.898 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-21T15:16:05.898 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-21T15:16:05.898 INFO:teuthology.orchestra.run.smithi078.stdout:invalid new device string format 2024-05-21T15:16:05.898 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.84:3300,172.21.15.172:3300" 2024-05-21T15:16:05.898 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.78:3300,172.21.15.84:3300,172.21.15.172:3300:/volumes/qa/sv_1/305cad38-0d45-4543-b894-976a9e5440da 2024-05-21T15:16:05.898 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=852c9762-1783-11ef-bc9a-c7b262605968" will pass to kernel 2024-05-21T15:16:05.899 INFO:teuthology.orchestra.run.smithi078.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-21T15:16:05.899 INFO:teuthology.orchestra.run.smithi078.stdout: You just mounted a file system that supports labels which does not 2024-05-21T15:16:05.899 INFO:teuthology.orchestra.run.smithi078.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-21T15:16:05.899 INFO:teuthology.orchestra.run.smithi078.stdout: applications will generate AVC messages and not be allowed access to 2024-05-21T15:16:05.899 INFO:teuthology.orchestra.run.smithi078.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-21T15:16:05.901 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-21T15:16:05.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:05.902 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-21T15:16:05.908 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:05.606+0000 7f6214cc9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:16:05.909 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:05.704+0000 7f6214cc9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:16:05.909 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:05.905+0000 7f6214cc9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:16:05.931 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T15:16:05.981 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:05 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:05.869+0000 7f7d894a2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T15:16:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T15:16:06.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:06.030 INFO:teuthology.orchestra.run.smithi078.stdout:client15408 2024-05-21T15:16:06.030 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:16:06.030 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T15:16:06.088 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T15:16:06.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T15:16:06.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:06.190 INFO:teuthology.orchestra.run.smithi078.stdout:client15408 2024-05-21T15:16:06.190 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T15:16:06.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T15:16:06.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:06.248 INFO:teuthology.orchestra.run.smithi078.stdout:client15408 2024-05-21T15:16:06.249 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/852c9762-1783-11ef-bc9a-c7b262605968.client15408/status') 2024-05-21T15:16:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T15:16:06.307 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:05 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:05.978+0000 7f7d894a2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T15:16:06.307 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:06 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:06.201+0000 7f7d894a2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:16:06.309 INFO:teuthology.orchestra.run.smithi078.stdout:instance: client.15408 (3)192.168.0.1:0/1896686553 2024-05-21T15:16:06.309 INFO:teuthology.orchestra.run.smithi078.stdout:blocklisted: false 2024-05-21T15:16:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records in 2024-05-21T15:16:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records out 2024-05-21T15:16:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:70 bytes copied, 2.327e-05 s, 3.0 MB/s 2024-05-21T15:16:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T15:16:06.317 INFO:teuthology.run_tasks:Running task exec... 2024-05-21T15:16:06.326 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T15:16:06.326 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:16:06.326 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-21T15:16:06.395 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:05.910+0000 7f12b5eef200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T15:16:06.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.452+0000 7f4696ad7640 1 -- 172.21.15.78:0/2031167108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690101f40 msgr2=0x7f4690102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.452+0000 7f4696ad7640 1 --2- 172.21.15.78:0/2031167108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690101f40 0x7f4690102350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4684009900 tx=0x7f468402a410 comp rx=0 tx=0).stop 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 -- 172.21.15.78:0/2031167108 shutdown_connections 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 --2- 172.21.15.78:0/2031167108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4690102920 0x7f4690102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 --2- 172.21.15.78:0/2031167108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4690103460 0x7f4690109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 --2- 172.21.15.78:0/2031167108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690101f40 0x7f4690102350 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 -- 172.21.15.78:0/2031167108 >> 172.21.15.78:0/2031167108 conn(0x7f46900fc340 msgr2=0x7f46900fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 -- 172.21.15.78:0/2031167108 shutdown_connections 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 -- 172.21.15.78:0/2031167108 wait complete. 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 Processor -- start 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 -- start start 2024-05-21T15:16:06.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4690101f40 0x7f4690198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690102920 0x7f46901987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4690103460 0x7f469019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f469010d690 con 0x7f4690102920 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f469010d510 con 0x7f4690101f40 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.453+0000 7f4696ad7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f469010d810 con 0x7f4690103460 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690102920 0x7f46901987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690102920 0x7f46901987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49654/0 (socket says 172.21.15.78:49654) 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 -- 172.21.15.78:0/1204917028 learned_addr learned my addr 172.21.15.78:0/1204917028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f4694873640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4690101f40 0x7f4690198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f4695074640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4690103460 0x7f469019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 -- 172.21.15.78:0/1204917028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4690103460 msgr2=0x7f469019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4690103460 0x7f469019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 -- 172.21.15.78:0/1204917028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4690101f40 msgr2=0x7f4690198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:06.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4690101f40 0x7f4690198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 -- 172.21.15.78:0/1204917028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469019fd50 con 0x7f4690102920 2024-05-21T15:16:06.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f4694873640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4690101f40 0x7f4690198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:06.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468ffff640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690102920 0x7f46901987a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e9f0 tx=0x7f467c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:06.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468dffb640 1 -- 172.21.15.78:0/1204917028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c00cd80 con 0x7f4690102920 2024-05-21T15:16:06.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469019ffe0 con 0x7f4690102920 2024-05-21T15:16:06.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468dffb640 1 -- 172.21.15.78:0/1204917028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467c010910 con 0x7f4690102920 2024-05-21T15:16:06.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f468dffb640 1 -- 172.21.15.78:0/1204917028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c0188f0 con 0x7f4690102920 2024-05-21T15:16:06.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.454+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a0520 con 0x7f4690102920 2024-05-21T15:16:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.456+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469004f720 con 0x7f4690102920 2024-05-21T15:16:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.457+0000 7f468dffb640 1 -- 172.21.15.78:0/1204917028 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f467c018b90 con 0x7f4690102920 2024-05-21T15:16:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.457+0000 7f468dffb640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f46800a2bd0 0x7f46800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.457+0000 7f4694873640 1 -- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f46800a2bd0 msgr2=0x7f46800a50e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.457+0000 7f468dffb640 1 -- 172.21.15.78:0/1204917028 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f467c0cf450 con 0x7f4690102920 2024-05-21T15:16:06.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.457+0000 7f4694873640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f46800a2bd0 0x7f46800a50e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:16:06.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.462+0000 7f468dffb640 1 -- 172.21.15.78:0/1204917028 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f467c09bd40 con 0x7f4690102920 2024-05-21T15:16:06.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.658+0000 7f4694873640 1 -- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f46800a2bd0 msgr2=0x7f46800a50e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:06.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.658+0000 7f4694873640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f46800a2bd0 0x7f46800a50e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T15:16:06.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.683+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f4690062750 con 0x7f4690102920 2024-05-21T15:16:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.735+0000 7f468dffb640 1 -- 172.21.15.78:0/1204917028 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f467c0a0040 con 0x7f4690102920 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f46800a2bd0 msgr2=0x7f46800a50e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f46800a2bd0 0x7f46800a50e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690102920 msgr2=0x7f46901987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690102920 0x7f46901987a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e9f0 tx=0x7f467c00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 shutdown_connections 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f46800a2bd0 0x7f46800a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4690103460 0x7f469019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4690102920 0x7f46901987a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 --2- 172.21.15.78:0/1204917028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4690101f40 0x7f4690198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 >> 172.21.15.78:0/1204917028 conn(0x7f46900fc340 msgr2=0x7f4690103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 shutdown_connections 2024-05-21T15:16:06.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.738+0000 7f4696ad7640 1 -- 172.21.15.78:0/1204917028 wait complete. 2024-05-21T15:16:06.745 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:06.434+0000 7f12b5eef200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:16:06.745 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:06.528+0000 7f12b5eef200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:16:06.745 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:06 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:06.430+0000 7f6214cc9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:16:06.745 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:06 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:06.525+0000 7f6214cc9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:16:06.752 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-21T15:16:06.765 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-21T15:16:06.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 -- 172.21.15.78:0/79421464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 msgr2=0x7fa5b4104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 --2- 172.21.15.78:0/79421464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 0x7fa5b4104460 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009900 tx=0x7fa5a802a410 comp rx=0 tx=0).stop 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 -- 172.21.15.78:0/79421464 shutdown_connections 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 --2- 172.21.15.78:0/79421464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5b4100700 0x7fa5b4100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 --2- 172.21.15.78:0/79421464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5b4101200 0x7fa5b410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 --2- 172.21.15.78:0/79421464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 0x7fa5b4104460 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 -- 172.21.15.78:0/79421464 >> 172.21.15.78:0/79421464 conn(0x7fa5b40fc320 msgr2=0x7fa5b40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 -- 172.21.15.78:0/79421464 shutdown_connections 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.901+0000 7fa5b9984640 1 -- 172.21.15.78:0/79421464 wait complete. 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.902+0000 7fa5b9984640 1 Processor -- start 2024-05-21T15:16:06.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.902+0000 7fa5b9984640 1 -- start start 2024-05-21T15:16:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.902+0000 7fa5b9984640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5b4100700 0x7fa5b4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.902+0000 7fa5b9984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5b4101200 0x7fa5b41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.902+0000 7fa5b9984640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 0x7fa5b41a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.902+0000 7fa5b9984640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b410e6f0 con 0x7fa5b4104050 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.902+0000 7fa5b9984640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b410e570 con 0x7fa5b4100700 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.902+0000 7fa5b9984640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b410e870 con 0x7fa5b4101200 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 0x7fa5b41a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5b4100700 0x7fa5b4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5b4101200 0x7fa5b41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5b4100700 0x7fa5b4199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:37920/0 (socket says 172.21.15.78:37920) 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 0x7fa5b41a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49672/0 (socket says 172.21.15.78:49672) 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 -- 172.21.15.78:0/2923346516 learned_addr learned my addr 172.21.15.78:0/2923346516 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 -- 172.21.15.78:0/2923346516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5b4101200 msgr2=0x7fa5b41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5b4101200 0x7fa5b41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 -- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 msgr2=0x7fa5b41a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 0x7fa5b41a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 -- 172.21.15.78:0/2923346516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b41a0db0 con 0x7fa5b4100700 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b2ffd640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5b4100700 0x7fa5b4199270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8006010 tx=0x7fa5a8031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b8982640 1 -- 172.21.15.78:0/2923346516 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a803f030 con 0x7fa5b4100700 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b8982640 1 -- 172.21.15.78:0/2923346516 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a8003ad0 con 0x7fa5b4100700 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b41a1040 con 0x7fa5b4100700 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b8982640 1 -- 172.21.15.78:0/2923346516 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8035070 con 0x7fa5b4100700 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.903+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b41a1530 con 0x7fa5b4100700 2024-05-21T15:16:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.904+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b41079e0 con 0x7fa5b4100700 2024-05-21T15:16:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.906+0000 7fa5b8982640 1 -- 172.21.15.78:0/2923346516 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fa5a8042030 con 0x7fa5b4100700 2024-05-21T15:16:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.906+0000 7fa5b8982640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa59c0a2be0 0x7fa59c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.906+0000 7fa5b27fc640 1 -- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa59c0a2be0 msgr2=0x7fa59c0a50f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.906+0000 7fa5b8982640 1 -- 172.21.15.78:0/2923346516 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa5a80ef8e0 con 0x7fa5b4100700 2024-05-21T15:16:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.906+0000 7fa5b27fc640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa59c0a2be0 0x7fa59c0a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:16:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:06.910+0000 7fa5b8982640 1 -- 172.21.15.78:0/2923346516 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5a80bc170 con 0x7fa5b4100700 2024-05-21T15:16:07.005 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:06 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1204917028' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T15:16:07.006 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:06.773+0000 7f12b5eef200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:16:07.006 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:06.905+0000 7f12b5eef200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:16:07.007 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:06 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1204917028' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T15:16:07.007 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:06 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:06.774+0000 7f6214cc9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:16:07.007 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:06 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:06.908+0000 7f6214cc9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:16:07.007 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:07 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:07.004+0000 7f6214cc9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:16:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.090+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5b4062750 con 0x7fa5b4100700 2024-05-21T15:16:07.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.092+0000 7fa5b8982640 1 -- 172.21.15.78:0/2923346516 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22616 (secure 0 0 0) 0x7fa5a80c0470 con 0x7fa5b4100700 2024-05-21T15:16:07.095 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:07.095 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":87,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:15:16.363058+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa59c0a2be0 msgr2=0x7fa59c0a50f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa59c0a2be0 0x7fa59c0a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5b4100700 msgr2=0x7fa5b4199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5b4100700 0x7fa5b4199270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8006010 tx=0x7fa5a8031b60 comp rx=0 tx=0).stop 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 shutdown_connections 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa59c0a2be0 0x7fa59c0a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5b4104050 0x7fa5b41a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5b4101200 0x7fa5b41997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 --2- 172.21.15.78:0/2923346516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5b4100700 0x7fa5b4199270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 >> 172.21.15.78:0/2923346516 conn(0x7fa5b40fc320 msgr2=0x7fa5b4104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:07.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 shutdown_connections 2024-05-21T15:16:07.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.094+0000 7fa5b9984640 1 -- 172.21.15.78:0/2923346516 wait complete. 2024-05-21T15:16:07.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:06 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1204917028' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T15:16:07.098 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:06 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:06.756+0000 7f7d894a2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T15:16:07.098 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:06 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:06.856+0000 7f7d894a2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T15:16:07.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:16:07.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:16:07.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 -- 172.21.15.78:0/82001773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41078d0 msgr2=0x7f8aa4109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 --2- 172.21.15.78:0/82001773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41078d0 0x7f8aa4109cf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009ca0 tx=0x7f8a9c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 -- 172.21.15.78:0/82001773 shutdown_connections 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 --2- 172.21.15.78:0/82001773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa41078d0 0x7f8aa4109cf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 --2- 172.21.15.78:0/82001773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4102bd0 0x7f8aa4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 --2- 172.21.15.78:0/82001773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa4101990 0x7f8aa4102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 -- 172.21.15.78:0/82001773 >> 172.21.15.78:0/82001773 conn(0x7f8aa40fcb80 msgr2=0x7f8aa40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 -- 172.21.15.78:0/82001773 shutdown_connections 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.223+0000 7f8aaa061640 1 -- 172.21.15.78:0/82001773 wait complete. 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aaa061640 1 Processor -- start 2024-05-21T15:16:07.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aaa061640 1 -- start start 2024-05-21T15:16:07.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aaa061640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa4101990 0x7f8aa40fea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aaa061640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4102bd0 0x7f8aa40fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aaa061640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa41078d0 0x7f8aa40fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aaa061640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa410d610 con 0x7f8aa4101990 2024-05-21T15:16:07.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aaa061640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa410d490 con 0x7f8aa41078d0 2024-05-21T15:16:07.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aaa061640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa410d790 con 0x7f8aa4102bd0 2024-05-21T15:16:07.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aa37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa4101990 0x7f8aa40fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aa37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa4101990 0x7f8aa40fea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49688/0 (socket says 172.21.15.78:49688) 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aa37fe640 1 -- 172.21.15.78:0/1137639063 learned_addr learned my addr 172.21.15.78:0/1137639063 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aa2ffd640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4102bd0 0x7f8aa40fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.224+0000 7f8aa3fff640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa41078d0 0x7f8aa40fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa37fe640 1 -- 172.21.15.78:0/1137639063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4102bd0 msgr2=0x7f8aa40fd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa37fe640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4102bd0 0x7f8aa40fd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa37fe640 1 -- 172.21.15.78:0/1137639063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa41078d0 msgr2=0x7f8aa40fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa37fe640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa41078d0 0x7f8aa40fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa37fe640 1 -- 172.21.15.78:0/1137639063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa40fdd20 con 0x7f8aa4101990 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa2ffd640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4102bd0 0x7f8aa40fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa3fff640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa41078d0 0x7f8aa40fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa37fe640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa4101990 0x7f8aa40fea10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e9f0 tx=0x7f8a9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:07.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa0ff9640 1 -- 172.21.15.78:0/1137639063 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400ccc0 con 0x7f8aa4101990 2024-05-21T15:16:07.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa0ff9640 1 -- 172.21.15.78:0/1137639063 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a940035f0 con 0x7f8aa4101990 2024-05-21T15:16:07.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa41a3600 con 0x7f8aa4101990 2024-05-21T15:16:07.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aa0ff9640 1 -- 172.21.15.78:0/1137639063 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400ce60 con 0x7f8aa4101990 2024-05-21T15:16:07.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.225+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa41a3ac0 con 0x7f8aa4101990 2024-05-21T15:16:07.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.226+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa406b250 con 0x7f8aa4101990 2024-05-21T15:16:07.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.228+0000 7f8aa0ff9640 1 -- 172.21.15.78:0/1137639063 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f8a940037a0 con 0x7f8aa4101990 2024-05-21T15:16:07.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.229+0000 7f8aa0ff9640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8a900a2be0 0x7f8a900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.229+0000 7f8aa2ffd640 1 -- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8a900a2be0 msgr2=0x7f8a900a50f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:07.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.229+0000 7f8aa0ff9640 1 -- 172.21.15.78:0/1137639063 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8a940cf620 con 0x7f8aa4101990 2024-05-21T15:16:07.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.229+0000 7f8aa2ffd640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8a900a2be0 0x7f8a900a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:16:07.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.233+0000 7f8aa0ff9640 1 -- 172.21.15.78:0/1137639063 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a9409c560 con 0x7f8aa4101990 2024-05-21T15:16:07.373 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:07.121+0000 7f7d894a2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T15:16:07.374 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:07.268+0000 7f7d894a2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T15:16:07.395 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:07 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:07.001+0000 7f12b5eef200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:16:07.396 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:07 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:07.120+0000 7f12b5eef200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:16:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.411+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8aa41a3d80 con 0x7f8aa4101990 2024-05-21T15:16:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.412+0000 7f8aa0ff9640 1 -- 172.21.15.78:0/1137639063 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f8a9409be60 con 0x7f8aa4101990 2024-05-21T15:16:07.415 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.413+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8a900a2be0 msgr2=0x7f8a900a50f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.413+0000 7f8aaa061640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8a900a2be0 0x7f8a900a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa4101990 msgr2=0x7f8aa40fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa4101990 0x7f8aa40fea10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e9f0 tx=0x7f8a9400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 shutdown_connections 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f8a900a2be0 0x7f8a900a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8aa41078d0 0x7f8aa40fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4102bd0 0x7f8aa40fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 --2- 172.21.15.78:0/1137639063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8aa4101990 0x7f8aa40fea10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 >> 172.21.15.78:0/1137639063 conn(0x7f8aa40fcb80 msgr2=0x7f8aa4108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 shutdown_connections 2024-05-21T15:16:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.414+0000 7f8aaa061640 1 -- 172.21.15.78:0/1137639063 wait complete. 2024-05-21T15:16:07.428 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:16:07.431 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:07 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:07.122+0000 7f6214cc9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:16:07.431 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:07 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:07.427+0000 7f6214cc9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:16:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 -- 172.21.15.78:0/2359050321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c01078d0 msgr2=0x7f84c0109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2359050321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c01078d0 0x7f84c0109cf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009ca0 tx=0x7f84b802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 -- 172.21.15.78:0/2359050321 shutdown_connections 2024-05-21T15:16:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2359050321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84c0102bd0 0x7f84c0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2359050321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c01078d0 0x7f84c0109cf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2359050321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c0101990 0x7f84c0102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 -- 172.21.15.78:0/2359050321 >> 172.21.15.78:0/2359050321 conn(0x7f84c00fcb80 msgr2=0x7f84c00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 -- 172.21.15.78:0/2359050321 shutdown_connections 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.541+0000 7f84c6baa640 1 -- 172.21.15.78:0/2359050321 wait complete. 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c6baa640 1 Processor -- start 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c6baa640 1 -- start start 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c6baa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c0101990 0x7f84c0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c6baa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84c0102bd0 0x7f84c01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c6baa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c01078d0 0x7f84c019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c6baa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010d610 con 0x7f84c0101990 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c6baa640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010d490 con 0x7f84c0102bd0 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c6baa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010d790 con 0x7f84c01078d0 2024-05-21T15:16:07.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c4946640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c0101990 0x7f84c0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c4946640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c0101990 0x7f84c0198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49698/0 (socket says 172.21.15.78:49698) 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.542+0000 7f84c4946640 1 -- 172.21.15.78:0/2877348117 learned_addr learned my addr 172.21.15.78:0/2877348117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c5147640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c01078d0 0x7f84c019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84bffff640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84c0102bd0 0x7f84c01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c4946640 1 -- 172.21.15.78:0/2877348117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c01078d0 msgr2=0x7f84c019f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c4946640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c01078d0 0x7f84c019f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c4946640 1 -- 172.21.15.78:0/2877348117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84c0102bd0 msgr2=0x7f84c01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c4946640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84c0102bd0 0x7f84c01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c4946640 1 -- 172.21.15.78:0/2877348117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c019ffc0 con 0x7f84c0101990 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c4946640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c0101990 0x7f84c0198480 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f84b400e9f0 tx=0x7f84b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84bdffb640 1 -- 172.21.15.78:0/2877348117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400ccc0 con 0x7f84c0101990 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c01a02b0 con 0x7f84c0101990 2024-05-21T15:16:07.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84bdffb640 1 -- 172.21.15.78:0/2877348117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b40035f0 con 0x7f84c0101990 2024-05-21T15:16:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84bdffb640 1 -- 172.21.15.78:0/2877348117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400ce60 con 0x7f84c0101990 2024-05-21T15:16:07.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.543+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a07b0 con 0x7f84c0101990 2024-05-21T15:16:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.544+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c01066e0 con 0x7f84c0101990 2024-05-21T15:16:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.546+0000 7f84bdffb640 1 -- 172.21.15.78:0/2877348117 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f84b40037a0 con 0x7f84c0101990 2024-05-21T15:16:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.546+0000 7f84bdffb640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84b00a2be0 0x7f84b00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.546+0000 7f84bffff640 1 -- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84b00a2be0 msgr2=0x7f84b00a50f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.546+0000 7f84bdffb640 1 -- 172.21.15.78:0/2877348117 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f84b40cf5c0 con 0x7f84c0101990 2024-05-21T15:16:07.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.546+0000 7f84bffff640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84b00a2be0 0x7f84b00a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:16:07.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.551+0000 7f84bdffb640 1 -- 172.21.15.78:0/2877348117 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f84b409c500 con 0x7f84c0101990 2024-05-21T15:16:07.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.730+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f84c002d480 con 0x7f84c0101990 2024-05-21T15:16:07.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.730+0000 7f84bdffb640 1 -- 172.21.15.78:0/2877348117 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f84b409be00 con 0x7f84c0101990 2024-05-21T15:16:07.733 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:16:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84b00a2be0 msgr2=0x7f84b00a50f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:16:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84b00a2be0 0x7f84b00a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c0101990 msgr2=0x7f84c0198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c0101990 0x7f84c0198480 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f84b400e9f0 tx=0x7f84b400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 shutdown_connections 2024-05-21T15:16:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f84b00a2be0 0x7f84b00a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84c0102bd0 0x7f84c01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c01078d0 0x7f84c019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 --2- 172.21.15.78:0/2877348117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84c0101990 0x7f84c0198480 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 >> 172.21.15.78:0/2877348117 conn(0x7f84c00fcb80 msgr2=0x7f84c01038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 shutdown_connections 2024-05-21T15:16:07.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.732+0000 7f84c6baa640 1 -- 172.21.15.78:0/2877348117 wait complete. 2024-05-21T15:16:07.746 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:07.370+0000 7f7d894a2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T15:16:07.746 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:07.498+0000 7f7d894a2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T15:16:07.746 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:07 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:07.421+0000 7f12b5eef200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:16:07.747 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:16:07.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.860+0000 7fd26ff85640 1 -- 172.21.15.78:0/1099917344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268107940 msgr2=0x7fd268109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.860+0000 7fd26ff85640 1 --2- 172.21.15.78:0/1099917344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268107940 0x7fd268109d60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd260009ca0 tx=0x7fd26002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.860+0000 7fd26ff85640 1 -- 172.21.15.78:0/1099917344 shutdown_connections 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.860+0000 7fd26ff85640 1 --2- 172.21.15.78:0/1099917344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268102c40 0x7fd268103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.860+0000 7fd26ff85640 1 --2- 172.21.15.78:0/1099917344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268107940 0x7fd268109d60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.860+0000 7fd26ff85640 1 --2- 172.21.15.78:0/1099917344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268101a00 0x7fd268102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.860+0000 7fd26ff85640 1 -- 172.21.15.78:0/1099917344 >> 172.21.15.78:0/1099917344 conn(0x7fd2680fcbf0 msgr2=0x7fd2680ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 -- 172.21.15.78:0/1099917344 shutdown_connections 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 -- 172.21.15.78:0/1099917344 wait complete. 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 Processor -- start 2024-05-21T15:16:07.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 -- start start 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268101a00 0x7fd2681981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102c40 0x7fd2681986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268107940 0x7fd26819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810d640 con 0x7fd268102c40 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810d4c0 con 0x7fd268101a00 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.861+0000 7fd26ff85640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810d7c0 con 0x7fd268107940 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102c40 0x7fd2681986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102c40 0x7fd2681986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49722/0 (socket says 172.21.15.78:49722) 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 -- 172.21.15.78:0/2889267196 learned_addr learned my addr 172.21.15.78:0/2889267196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26dd21640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268101a00 0x7fd2681981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26e522640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268107940 0x7fd26819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 -- 172.21.15.78:0/2889267196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268107940 msgr2=0x7fd26819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268107940 0x7fd26819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 -- 172.21.15.78:0/2889267196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268101a00 msgr2=0x7fd2681981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268101a00 0x7fd2681981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 -- 172.21.15.78:0/2889267196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26819fc80 con 0x7fd268102c40 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26e522640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268107940 0x7fd26819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26d520640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102c40 0x7fd2681986f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd25800d950 tx=0x7fd25800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd25effd640 1 -- 172.21.15.78:0/2889267196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258017070 con 0x7fd268102c40 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd25effd640 1 -- 172.21.15.78:0/2889267196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd258009c30 con 0x7fd268102c40 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26819ff70 con 0x7fd268102c40 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd25effd640 1 -- 172.21.15.78:0/2889267196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25800f6b0 con 0x7fd268102c40 2024-05-21T15:16:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.862+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd268073d80 con 0x7fd268102c40 2024-05-21T15:16:07.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.863+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2680747a0 con 0x7fd268102c40 2024-05-21T15:16:07.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.865+0000 7fd25effd640 1 -- 172.21.15.78:0/2889267196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fd2580109d0 con 0x7fd268102c40 2024-05-21T15:16:07.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.865+0000 7fd25effd640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2540a4df0 0x7fd2540a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:07.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.865+0000 7fd26dd21640 1 -- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2540a4df0 msgr2=0x7fd2540a7300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:07.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.865+0000 7fd25effd640 1 -- 172.21.15.78:0/2889267196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd2580ced00 con 0x7fd268102c40 2024-05-21T15:16:07.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.865+0000 7fd26dd21640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2540a4df0 0x7fd2540a7300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:16:07.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:07.873+0000 7fd25effd640 1 -- 172.21.15.78:0/2889267196 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd258009de0 con 0x7fd268102c40 2024-05-21T15:16:08.023 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:07 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1204917028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T15:16:08.024 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:07 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:16:08.024 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:07 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2923346516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:08.024 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:07 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1137639063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:08.024 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:07 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2877348117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:08.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:07 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1204917028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T15:16:08.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:07 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:16:08.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:07 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2923346516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:08.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:07 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1137639063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:08.035 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:07 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2877348117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:08.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.052+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd268106260 con 0x7fd268102c40 2024-05-21T15:16:08.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.052+0000 7fd25effd640 1 -- 172.21.15.78:0/2889267196 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fd25809bc40 con 0x7fd268102c40 2024-05-21T15:16:08.055 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:16:08.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2540a4df0 msgr2=0x7fd2540a7300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:16:08.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2540a4df0 0x7fd2540a7300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102c40 msgr2=0x7fd2681986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102c40 0x7fd2681986f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd25800d950 tx=0x7fd25800de10 comp rx=0 tx=0).stop 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 shutdown_connections 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fd2540a4df0 0x7fd2540a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268107940 0x7fd26819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102c40 0x7fd2681986f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 --2- 172.21.15.78:0/2889267196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd268101a00 0x7fd2681981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.054+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 >> 172.21.15.78:0/2889267196 conn(0x7fd2680fcbf0 msgr2=0x7fd268103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.055+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 shutdown_connections 2024-05-21T15:16:08.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.055+0000 7fd26ff85640 1 -- 172.21.15.78:0/2889267196 wait complete. 2024-05-21T15:16:08.067 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:07.808+0000 7f7d894a2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T15:16:08.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:07 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1204917028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T15:16:08.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:07 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:16:08.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:07 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2923346516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:08.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:07 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1137639063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:08.068 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:07 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2877348117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:08.069 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2044544880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10103cf0 msgr2=0x7f4a10104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2044544880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10103cf0 0x7f4a10104100 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009900 tx=0x7f4a0002a410 comp rx=0 tx=0).stop 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2044544880 shutdown_connections 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2044544880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a101089f0 0x7f4a1010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2044544880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10103cf0 0x7f4a10104100 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2044544880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a10102ab0 0x7f4a101037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2044544880 >> 172.21.15.78:0/2044544880 conn(0x7f4a100fdca0 msgr2=0x7f4a10100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2044544880 shutdown_connections 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2044544880 wait complete. 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 Processor -- start 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.178+0000 7f4a17e5f640 1 -- start start 2024-05-21T15:16:08.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a17e5f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10102ab0 0x7f4a10075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a17e5f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a10103cf0 0x7f4a10073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a17e5f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a101089f0 0x7f4a100743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a17e5f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010e770 con 0x7f4a10102ab0 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a17e5f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010e5f0 con 0x7f4a101089f0 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a17e5f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010e8f0 con 0x7f4a10103cf0 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10102ab0 0x7f4a10075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10102ab0 0x7f4a10075800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49736/0 (socket says 172.21.15.78:49736) 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 -- 172.21.15.78:0/2756069571 learned_addr learned my addr 172.21.15.78:0/2756069571 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a163fc640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a101089f0 0x7f4a100743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a153fa640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a10103cf0 0x7f4a10073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 -- 172.21.15.78:0/2756069571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a10103cf0 msgr2=0x7f4a10073ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a10103cf0 0x7f4a10073ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 -- 172.21.15.78:0/2756069571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a101089f0 msgr2=0x7f4a100743e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a101089f0 0x7f4a100743e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 -- 172.21.15.78:0/2756069571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a10074b10 con 0x7f4a10102ab0 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a153fa640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a10103cf0 0x7f4a10073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a15bfb640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10102ab0 0x7f4a10075800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00e9f0 tx=0x7f4a0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a06ffd640 1 -- 172.21.15.78:0/2756069571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00ccc0 con 0x7f4a10102ab0 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a101a3370 con 0x7f4a10102ab0 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a06ffd640 1 -- 172.21.15.78:0/2756069571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0c0035f0 con 0x7f4a10102ab0 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a06ffd640 1 -- 172.21.15.78:0/2756069571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00ce60 con 0x7f4a10102ab0 2024-05-21T15:16:08.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.179+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a101a3830 con 0x7f4a10102ab0 2024-05-21T15:16:08.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.180+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a10107800 con 0x7f4a10102ab0 2024-05-21T15:16:08.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.182+0000 7f4a06ffd640 1 -- 172.21.15.78:0/2756069571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f4a0c0037a0 con 0x7f4a10102ab0 2024-05-21T15:16:08.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.182+0000 7f4a06ffd640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f49fc0a2be0 0x7f49fc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.182+0000 7f4a153fa640 1 -- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f49fc0a2be0 msgr2=0x7f49fc0a50f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:08.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.183+0000 7f4a06ffd640 1 -- 172.21.15.78:0/2756069571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4a0c0cf780 con 0x7f4a10102ab0 2024-05-21T15:16:08.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.183+0000 7f4a153fa640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f49fc0a2be0 0x7f49fc0a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:16:08.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.190+0000 7f4a06ffd640 1 -- 172.21.15.78:0/2756069571 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4a0c09c6c0 con 0x7f4a10102ab0 2024-05-21T15:16:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.373+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a1002d470 con 0x7f4a10102ab0 2024-05-21T15:16:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.375+0000 7f4a06ffd640 1 -- 172.21.15.78:0/2756069571 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22616 (secure 0 0 0) 0x7f4a0c09bfc0 con 0x7f4a10102ab0 2024-05-21T15:16:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:08.379 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":87,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:15:16.363058+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:16:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f49fc0a2be0 msgr2=0x7f49fc0a50f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:16:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f49fc0a2be0 0x7f49fc0a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10102ab0 msgr2=0x7f4a10075800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10102ab0 0x7f4a10075800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00e9f0 tx=0x7f4a0c00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 shutdown_connections 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7f49fc0a2be0 0x7f49fc0a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a101089f0 0x7f4a100743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a10103cf0 0x7f4a10073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 --2- 172.21.15.78:0/2756069571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a10102ab0 0x7f4a10075800 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 >> 172.21.15.78:0/2756069571 conn(0x7f4a100fdca0 msgr2=0x7f4a101049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 shutdown_connections 2024-05-21T15:16:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.377+0000 7f4a17e5f640 1 -- 172.21.15.78:0/2756069571 wait complete. 2024-05-21T15:16:08.392 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:16:08.392 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:16:08.395 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:08 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:08.020+0000 7f12b5eef200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:16:08.396 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:16:08 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y[37717]: 2024-05-21T15:16:08.236+0000 7f12b5eef200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:16:08.433 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:08 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:08.032+0000 7f6214cc9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:16:08.433 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:16:08 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z[38255]: 2024-05-21T15:16:08.251+0000 7f6214cc9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:16:08.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 -- 172.21.15.78:0/3810863567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 msgr2=0x7fa1a0102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/3810863567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 0x7fa1a0102700 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa190009900 tx=0x7fa19002a410 comp rx=0 tx=0).stop 2024-05-21T15:16:08.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 -- 172.21.15.78:0/3810863567 shutdown_connections 2024-05-21T15:16:08.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/3810863567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0102c40 0x7fa1a0103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/3810863567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a0107940 0x7fa1a0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/3810863567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 0x7fa1a0102700 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 -- 172.21.15.78:0/3810863567 >> 172.21.15.78:0/3810863567 conn(0x7fa1a00fcbf0 msgr2=0x7fa1a00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:08.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 -- 172.21.15.78:0/3810863567 shutdown_connections 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 -- 172.21.15.78:0/3810863567 wait complete. 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 Processor -- start 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.505+0000 7fa1a74ca640 1 -- start start 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a74ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 0x7fa1a01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a74ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0102c40 0x7fa1a0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a74ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a0107940 0x7fa1a019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a74ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a010d5c0 con 0x7fa1a0101a00 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a74ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a010d440 con 0x7fa1a0102c40 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a74ca640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a010d740 con 0x7fa1a0107940 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 0x7fa1a01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 0x7fa1a01981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49752/0 (socket says 172.21.15.78:49752) 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 -- 172.21.15.78:0/2828417929 learned_addr learned my addr 172.21.15.78:0/2828417929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:08.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a4a65640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0102c40 0x7fa1a0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5a67640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a0107940 0x7fa1a019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 -- 172.21.15.78:0/2828417929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a0107940 msgr2=0x7fa1a019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a0107940 0x7fa1a019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 -- 172.21.15.78:0/2828417929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0102c40 msgr2=0x7fa1a0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0102c40 0x7fa1a0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 -- 172.21.15.78:0/2828417929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a019fd30 con 0x7fa1a0101a00 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a5266640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 0x7fa1a01981f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa190006010 tx=0x7fa1900043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1967fc640 1 -- 172.21.15.78:0/2828417929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19003f030 con 0x7fa1a0101a00 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1967fc640 1 -- 172.21.15.78:0/2828417929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa190003ce0 con 0x7fa1a0101a00 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a019ffc0 con 0x7fa1a0101a00 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.506+0000 7fa1967fc640 1 -- 172.21.15.78:0/2828417929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190035070 con 0x7fa1a0101a00 2024-05-21T15:16:08.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.507+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a01a03c0 con 0x7fa1a0101a00 2024-05-21T15:16:08.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.507+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a0106750 con 0x7fa1a0101a00 2024-05-21T15:16:08.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.509+0000 7fa1967fc640 1 -- 172.21.15.78:0/2828417929 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fa190042030 con 0x7fa1a0101a00 2024-05-21T15:16:08.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.509+0000 7fa1967fc640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa18c0a2be0 0x7fa18c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.510+0000 7fa1a4a65640 1 -- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa18c0a2be0 msgr2=0x7fa18c0a50f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2228258510 2024-05-21T15:16:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.510+0000 7fa1967fc640 1 -- 172.21.15.78:0/2828417929 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa1900ef7b0 con 0x7fa1a0101a00 2024-05-21T15:16:08.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.510+0000 7fa1a4a65640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa18c0a2be0 0x7fa18c0a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T15:16:08.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.517+0000 7fa1967fc640 1 -- 172.21.15.78:0/2828417929 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa1900bc0a0 con 0x7fa1a0101a00 2024-05-21T15:16:08.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.696+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1a0062780 con 0x7fa1a0101a00 2024-05-21T15:16:08.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.696+0000 7fa1967fc640 1 -- 172.21.15.78:0/2828417929 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fa1900c03a0 con 0x7fa1a0101a00 2024-05-21T15:16:08.699 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:16:08.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.456+0000 7f7d894a2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T15:16:08.700 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.696+0000 7f7d894a2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.698+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa18c0a2be0 msgr2=0x7fa18c0a50f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa18c0a2be0 0x7fa18c0a50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 msgr2=0x7fa1a01981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 0x7fa1a01981f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa190006010 tx=0x7fa1900043b0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 shutdown_connections 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:6800/2228258510,v1:172.21.15.78:6801/2228258510] conn(0x7fa18c0a2be0 0x7fa18c0a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1a0102c40 0x7fa1a0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a0107940 0x7fa1a019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 --2- 172.21.15.78:0/2828417929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1a0101a00 0x7fa1a01981f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 >> 172.21.15.78:0/2828417929 conn(0x7fa1a00fcbf0 msgr2=0x7fa1a0103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:08.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 shutdown_connections 2024-05-21T15:16:08.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.699+0000 7fa1a74ca640 1 -- 172.21.15.78:0/2828417929 wait complete. 2024-05-21T15:16:08.717 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:16:08.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 -- 172.21.15.78:0/2612248218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 msgr2=0x7f69bc102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 --2- 172.21.15.78:0/2612248218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 0x7f69bc102690 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009900 tx=0x7f69b002a410 comp rx=0 tx=0).stop 2024-05-21T15:16:08.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 -- 172.21.15.78:0/2612248218 shutdown_connections 2024-05-21T15:16:08.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 --2- 172.21.15.78:0/2612248218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc102bd0 0x7f69bc102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 --2- 172.21.15.78:0/2612248218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69bc1078d0 0x7f69bc109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 --2- 172.21.15.78:0/2612248218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 0x7f69bc102690 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 -- 172.21.15.78:0/2612248218 >> 172.21.15.78:0/2612248218 conn(0x7f69bc0fcb80 msgr2=0x7f69bc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:08.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 -- 172.21.15.78:0/2612248218 shutdown_connections 2024-05-21T15:16:08.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.849+0000 7f69c2d10640 1 -- 172.21.15.78:0/2612248218 wait complete. 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.850+0000 7f69c2d10640 1 Processor -- start 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.850+0000 7f69c2d10640 1 -- start start 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.850+0000 7f69c2d10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 0x7f69bc19b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.850+0000 7f69c2d10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc102bd0 0x7f69bc19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.850+0000 7f69c2d10640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69bc1078d0 0x7f69bc1a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c0aac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 0x7f69bc19b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c0aac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 0x7f69bc19b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46758/0 (socket says 172.21.15.78:46758) 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc102bd0 0x7f69bc19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc102bd0 0x7f69bc19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:40940/0 (socket says 172.21.15.78:40940) 2024-05-21T15:16:08.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c0aac640 1 -- 172.21.15.78:0/2321884157 learned_addr learned my addr 172.21.15.78:0/2321884157 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc10d630 con 0x7f69bc101990 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc10d4b0 con 0x7f69bc102bd0 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc10d7b0 con 0x7f69bc1078d0 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c12ad640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69bc1078d0 0x7f69bc1a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c0aac640 1 -- 172.21.15.78:0/2321884157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69bc1078d0 msgr2=0x7f69bc1a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c0aac640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69bc1078d0 0x7f69bc1a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c0aac640 1 -- 172.21.15.78:0/2321884157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc102bd0 msgr2=0x7f69bc19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c0aac640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc102bd0 0x7f69bc19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c0aac640 1 -- 172.21.15.78:0/2321884157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc1a3150 con 0x7f69bc101990 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.851+0000 7f69c12ad640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69bc1078d0 0x7f69bc1a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.852+0000 7f69c0aac640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 0x7f69bc19b610 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009850 tx=0x7f69b0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:08.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.852+0000 7f69b9ffb640 1 -- 172.21.15.78:0/2321884157 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b003f070 con 0x7f69bc101990 2024-05-21T15:16:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.852+0000 7f69b9ffb640 1 -- 172.21.15.78:0/2321884157 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b003b460 con 0x7f69bc101990 2024-05-21T15:16:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.852+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc1a33e0 con 0x7f69bc101990 2024-05-21T15:16:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.853+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc1a3920 con 0x7f69bc101990 2024-05-21T15:16:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.853+0000 7f69b9ffb640 1 -- 172.21.15.78:0/2321884157 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0035030 con 0x7f69bc101990 2024-05-21T15:16:08.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.855+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc069f50 con 0x7f69bc101990 2024-05-21T15:16:08.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.856+0000 7f69b9ffb640 1 -- 172.21.15.78:0/2321884157 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f69b0042070 con 0x7f69bc101990 2024-05-21T15:16:08.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.857+0000 7f69b9ffb640 1 -- 172.21.15.78:0/2321884157 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f69b00ef240 con 0x7f69bc101990 2024-05-21T15:16:08.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:08.866+0000 7f69b9ffb640 1 -- 172.21.15.78:0/2321884157 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f69b00bbe30 con 0x7f69bc101990 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2889267196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: Standby manager daemon y restarted 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: Standby manager daemon y started 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: Standby manager daemon z restarted 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: Standby manager daemon z started 2024-05-21T15:16:08.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2756069571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2828417929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: Active manager daemon x restarted 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: Activating manager daemon x 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: mgrmap e20: x(active, starting, since 0.0127087s), standbys: z, y 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:16:08.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:16:08.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:16:08.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:16:08.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:16:08.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:16:08.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:16:08.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:16:08.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:16:08.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:08 smithi078 ceph-mon[31997]: Manager daemon x is now available 2024-05-21T15:16:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.125+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69bc02d340 con 0x7f69bc101990 2024-05-21T15:16:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.126+0000 7f69b9ffb640 1 -- 172.21.15.78:0/2321884157 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f69b00c0130 con 0x7f69bc101990 2024-05-21T15:16:09.129 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 msgr2=0x7f69bc19b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 0x7f69bc19b610 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009850 tx=0x7f69b0003f80 comp rx=0 tx=0).stop 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 shutdown_connections 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69bc102bd0 0x7f69bc19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69bc1078d0 0x7f69bc1a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 --2- 172.21.15.78:0/2321884157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc101990 0x7f69bc19b610 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 >> 172.21.15.78:0/2321884157 conn(0x7f69bc0fcb80 msgr2=0x7f69bc0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 shutdown_connections 2024-05-21T15:16:09.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.128+0000 7f69c2d10640 1 -- 172.21.15.78:0/2321884157 wait complete. 2024-05-21T15:16:09.142 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:16:09.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2889267196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:09.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: Standby manager daemon y restarted 2024-05-21T15:16:09.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: Standby manager daemon y started 2024-05-21T15:16:09.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:16:09.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: Standby manager daemon z restarted 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: Standby manager daemon z started 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2756069571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2828417929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: Active manager daemon x restarted 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: Activating manager daemon x 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T15:16:09.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: mgrmap e20: x(active, starting, since 0.0127087s), standbys: z, y 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:16:09.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:16:09.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:16:09.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:08 smithi084 ceph-mon[36334]: Manager daemon x is now available 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2889267196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: Standby manager daemon y restarted 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: Standby manager daemon y started 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.84:0/893273545' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: Standby manager daemon z restarted 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: Standby manager daemon z started 2024-05-21T15:16:09.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.? 172.21.15.172:0/4208163893' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2756069571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2828417929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: Active manager daemon x restarted 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: Activating manager daemon x 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: mgrmap e20: x(active, starting, since 0.0127087s), standbys: z, y 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T15:16:09.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T15:16:09.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T15:16:09.187 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T15:16:09.188 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T15:16:09.188 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T15:16:09.188 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:08 smithi172 ceph-mon[36402]: Manager daemon x is now available 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.256+0000 7f0024d21640 1 -- 172.21.15.78:0/1519693687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 msgr2=0x7f002010ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.256+0000 7f0024d21640 1 --2- 172.21.15.78:0/1519693687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 0x7f002010ac30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0018009ca0 tx=0x7f001802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 -- 172.21.15.78:0/1519693687 shutdown_connections 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 --2- 172.21.15.78:0/1519693687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 0x7f002010ac30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 --2- 172.21.15.78:0/1519693687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020105bb0 0x7f0020108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 --2- 172.21.15.78:0/1519693687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020100dd0 0x7f00201011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 -- 172.21.15.78:0/1519693687 >> 172.21.15.78:0/1519693687 conn(0x7f00200fc300 msgr2=0x7f00200fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 -- 172.21.15.78:0/1519693687 shutdown_connections 2024-05-21T15:16:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 -- 172.21.15.78:0/1519693687 wait complete. 2024-05-21T15:16:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 Processor -- start 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.257+0000 7f0024d21640 1 -- start start 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f0024d21640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020100dd0 0x7f00201032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f0024d21640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020105bb0 0x7f0020101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f0024d21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 0x7f0020101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f0024d21640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f002010e700 con 0x7f0020108720 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f0024d21640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f002010e580 con 0x7f0020100dd0 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f0024d21640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f002010e880 con 0x7f0020105bb0 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020105bb0 0x7f0020101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 0x7f0020101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020100dd0 0x7f00201032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020105bb0 0x7f0020101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:59024/0 (socket says 172.21.15.78:59024) 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 -- 172.21.15.78:0/1710716904 learned_addr learned my addr 172.21.15.78:0/1710716904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 0x7f0020101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46804/0 (socket says 172.21.15.78:46804) 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 -- 172.21.15.78:0/1710716904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020100dd0 msgr2=0x7f00201032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 --2- 172.21.15.78:0/1710716904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020100dd0 0x7f00201032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 -- 172.21.15.78:0/1710716904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 msgr2=0x7f0020101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 --2- 172.21.15.78:0/1710716904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 0x7f0020101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 -- 172.21.15.78:0/1710716904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00201025b0 con 0x7f0020105bb0 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001e59c640 1 --2- 172.21.15.78:0/1710716904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020100dd0 0x7f00201032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.258+0000 7f001dd9b640 1 --2- 172.21.15.78:0/1710716904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020105bb0 0x7f0020101940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f001400e9f0 tx=0x7f001400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.259+0000 7f000f7fe640 1 -- 172.21.15.78:0/1710716904 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001400cd80 con 0x7f0020105bb0 2024-05-21T15:16:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.259+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00201a32b0 con 0x7f0020105bb0 2024-05-21T15:16:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.259+0000 7f000f7fe640 1 -- 172.21.15.78:0/1710716904 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0014005500 con 0x7f0020105bb0 2024-05-21T15:16:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.259+0000 7f000f7fe640 1 -- 172.21.15.78:0/1710716904 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014010690 con 0x7f0020105bb0 2024-05-21T15:16:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.259+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00201a3770 con 0x7f0020105bb0 2024-05-21T15:16:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.260+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00200fde20 con 0x7f0020105bb0 2024-05-21T15:16:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.261+0000 7f000f7fe640 1 -- 172.21.15.78:0/1710716904 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f00140046c0 con 0x7f0020105bb0 2024-05-21T15:16:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.262+0000 7f000f7fe640 1 -- 172.21.15.78:0/1710716904 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f00140cf140 con 0x7f0020105bb0 2024-05-21T15:16:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.265+0000 7f000f7fe640 1 -- 172.21.15.78:0/1710716904 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0014053850 con 0x7f0020105bb0 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.968+0000 7f7cea360700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.968+0000 7f7cea360700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.968+0000 7f7cea360700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.968+0000 7f7cea360700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.968+0000 7f7cea360700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.969+0000 7f7ce7b5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.969+0000 7f7ce7b5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.969+0000 7f7ce7b5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.969+0000 7f7ce7b5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.348 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:16:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:16:08.969+0000 7f7ce7b5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:16:09.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.447+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f002002d430 con 0x7f0020105bb0 2024-05-21T15:16:09.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.447+0000 7f000f7fe640 1 -- 172.21.15.78:0/1710716904 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f001409c3a0 con 0x7f0020105bb0 2024-05-21T15:16:09.450 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:16:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020105bb0 msgr2=0x7f0020101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 --2- 172.21.15.78:0/1710716904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020105bb0 0x7f0020101940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f001400e9f0 tx=0x7f001400edf0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 shutdown_connections 2024-05-21T15:16:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 --2- 172.21.15.78:0/1710716904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0020108720 0x7f0020101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 --2- 172.21.15.78:0/1710716904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0020105bb0 0x7f0020101940 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 --2- 172.21.15.78:0/1710716904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0020100dd0 0x7f00201032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 >> 172.21.15.78:0/1710716904 conn(0x7f00200fc300 msgr2=0x7f0020106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:09.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 shutdown_connections 2024-05-21T15:16:09.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.450+0000 7f0024d21640 1 -- 172.21.15.78:0/1710716904 wait complete. 2024-05-21T15:16:09.468 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-21T15:16:09.468 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:16:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2460667466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc102350 msgr2=0x7fa9bc102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2460667466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc102350 0x7fa9bc102760 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009900 tx=0x7fa9b002a410 comp rx=0 tx=0).stop 2024-05-21T15:16:09.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2460667466 shutdown_connections 2024-05-21T15:16:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2460667466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9bc107050 0x7fa9bc109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2460667466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc102350 0x7fa9bc102760 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2460667466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9bc101110 0x7fa9bc101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2460667466 >> 172.21.15.78:0/2460667466 conn(0x7fa9bc0fc2e0 msgr2=0x7fa9bc0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2460667466 shutdown_connections 2024-05-21T15:16:09.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.578+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2460667466 wait complete. 2024-05-21T15:16:09.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9c0dce640 1 Processor -- start 2024-05-21T15:16:09.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9c0dce640 1 -- start start 2024-05-21T15:16:09.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9c0dce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc101110 0x7fa9bc1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9c0dce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9bc102350 0x7fa9bc1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9c0dce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9bc107050 0x7fa9bc1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9c0dce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10c540 con 0x7fa9bc101110 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9c0dce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10c3c0 con 0x7fa9bc102350 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9c0dce640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10c6c0 con 0x7fa9bc107050 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9ba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc101110 0x7fa9bc1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9b9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9bc102350 0x7fa9bc1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9ba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc101110 0x7fa9bc1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46824/0 (socket says 172.21.15.78:46824) 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.579+0000 7fa9ba59c640 1 -- 172.21.15.78:0/2566135600 learned_addr learned my addr 172.21.15.78:0/2566135600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9bad9d640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9bc107050 0x7fa9bc1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ba59c640 1 -- 172.21.15.78:0/2566135600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9bc107050 msgr2=0x7fa9bc1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ba59c640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9bc107050 0x7fa9bc1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ba59c640 1 -- 172.21.15.78:0/2566135600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9bc102350 msgr2=0x7fa9bc1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ba59c640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9bc102350 0x7fa9bc1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ba59c640 1 -- 172.21.15.78:0/2566135600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc1a0df0 con 0x7fa9bc101110 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9b9d9b640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9bc102350 0x7fa9bc1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ba59c640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc101110 0x7fa9bc1992b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00ea90 tx=0x7fa9ac00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ab7fe640 1 -- 172.21.15.78:0/2566135600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac01b070 con 0x7fa9bc101110 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ab7fe640 1 -- 172.21.15.78:0/2566135600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac00cda0 con 0x7fa9bc101110 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc1a10e0 con 0x7fa9bc101110 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9ab7fe640 1 -- 172.21.15.78:0/2566135600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00b7c0 con 0x7fa9bc101110 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.580+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc1a1600 con 0x7fa9bc101110 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.581+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc1a18c0 con 0x7fa9bc101110 2024-05-21T15:16:09.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.584+0000 7fa9ab7fe640 1 -- 172.21.15.78:0/2566135600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7fa9ac01e070 con 0x7fa9bc101110 2024-05-21T15:16:09.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.585+0000 7fa9ab7fe640 1 -- 172.21.15.78:0/2566135600 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa9ac0cedc0 con 0x7fa9bc101110 2024-05-21T15:16:09.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.588+0000 7fa9ab7fe640 1 -- 172.21.15.78:0/2566135600 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa9ac025070 con 0x7fa9bc101110 2024-05-21T15:16:09.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.716+0000 7fa9ab7fe640 1 -- 172.21.15.78:0/2566135600 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fa9ac00a420 con 0x7fa9bc101110 2024-05-21T15:16:09.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.717+0000 7fa9ab7fe640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa9a40a99d0 0x7fa9a40abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.723+0000 7fa9b9d9b640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa9a40a99d0 0x7fa9a40abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.726+0000 7fa9b9d9b640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa9a40a99d0 0x7fa9a40abdf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005ab0 tx=0x7fa9b00315b0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.835+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9bc1a1a90 con 0x7fa9bc101110 2024-05-21T15:16:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.836+0000 7fa9ab7fe640 1 -- 172.21.15.78:0/2566135600 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11925 (secure 0 0 0) 0x7fa9ac092ce0 con 0x7fa9bc101110 2024-05-21T15:16:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:09.839 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:16:06.684612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24419,24448,24502,24526,24527]},"id":1}]} 2024-05-21T15:16:09.839 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa9a40a99d0 msgr2=0x7fa9a40abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa9a40a99d0 0x7fa9a40abdf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005ab0 tx=0x7fa9b00315b0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc101110 msgr2=0x7fa9bc1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc101110 0x7fa9bc1992b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00ea90 tx=0x7fa9ac00ef50 comp rx=0 tx=0).stop 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 shutdown_connections 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9bc102350 0x7fa9bc1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa9a40a99d0 0x7fa9a40abdf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9bc107050 0x7fa9bc1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 --2- 172.21.15.78:0/2566135600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9bc101110 0x7fa9bc1992b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 >> 172.21.15.78:0/2566135600 conn(0x7fa9bc0fc2e0 msgr2=0x7fa9bc105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 shutdown_connections 2024-05-21T15:16:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.838+0000 7fa9c0dce640 1 -- 172.21.15.78:0/2566135600 wait complete. 2024-05-21T15:16:09.853 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-21T15:16:09.853 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-21T15:16:09.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.961+0000 7fdc8668b640 1 -- 172.21.15.78:0/1312902727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80108a00 msgr2=0x7fdc8010ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.961+0000 7fdc8668b640 1 --2- 172.21.15.78:0/1312902727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80108a00 0x7fdc8010ae20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009ca0 tx=0x7fdc7802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.961+0000 7fdc8668b640 1 -- 172.21.15.78:0/1312902727 shutdown_connections 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.961+0000 7fdc8668b640 1 --2- 172.21.15.78:0/1312902727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80108a00 0x7fdc8010ae20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.961+0000 7fdc8668b640 1 --2- 172.21.15.78:0/1312902727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc80103d00 0x7fdc80104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.961+0000 7fdc8668b640 1 --2- 172.21.15.78:0/1312902727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc80102ac0 0x7fdc801037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.961+0000 7fdc8668b640 1 -- 172.21.15.78:0/1312902727 >> 172.21.15.78:0/1312902727 conn(0x7fdc800fdcf0 msgr2=0x7fdc80100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.961+0000 7fdc8668b640 1 -- 172.21.15.78:0/1312902727 shutdown_connections 2024-05-21T15:16:09.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.962+0000 7fdc8668b640 1 -- 172.21.15.78:0/1312902727 wait complete. 2024-05-21T15:16:09.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.962+0000 7fdc8668b640 1 Processor -- start 2024-05-21T15:16:09.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.962+0000 7fdc8668b640 1 -- start start 2024-05-21T15:16:09.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.962+0000 7fdc8668b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80102ac0 0x7fdc80199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.962+0000 7fdc8668b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc80103d00 0x7fdc801997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc8668b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc80108a00 0x7fdc801a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc8668b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8010e760 con 0x7fdc80102ac0 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc8668b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8010e5e0 con 0x7fdc80103d00 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc8668b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8010e8e0 con 0x7fdc80108a00 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80102ac0 0x7fdc80199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80102ac0 0x7fdc80199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46846/0 (socket says 172.21.15.78:46846) 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 -- 172.21.15.78:0/3246868182 learned_addr learned my addr 172.21.15.78:0/3246868182 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7f7fe640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc80103d00 0x7fdc801997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc84c28640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc80108a00 0x7fdc801a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 -- 172.21.15.78:0/3246868182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc80108a00 msgr2=0x7fdc801a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc80108a00 0x7fdc801a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 -- 172.21.15.78:0/3246868182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc80103d00 msgr2=0x7fdc801997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc80103d00 0x7fdc801997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 -- 172.21.15.78:0/3246868182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc801a0db0 con 0x7fdc80102ac0 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7ffff640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80102ac0 0x7fdc80199270 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000e9f0 tx=0x7fdc7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7d7fa640 1 -- 172.21.15.78:0/3246868182 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000ccc0 con 0x7fdc80102ac0 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc801a10a0 con 0x7fdc80102ac0 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7d7fa640 1 -- 172.21.15.78:0/3246868182 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc700035f0 con 0x7fdc80102ac0 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc7d7fa640 1 -- 172.21.15.78:0/3246868182 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000ce60 con 0x7fdc80102ac0 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.963+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc801a15a0 con 0x7fdc80102ac0 2024-05-21T15:16:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.964+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc80107810 con 0x7fdc80102ac0 2024-05-21T15:16:09.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.966+0000 7fdc7d7fa640 1 -- 172.21.15.78:0/3246868182 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fdc700037a0 con 0x7fdc80102ac0 2024-05-21T15:16:09.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.967+0000 7fdc7d7fa640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc680a2990 0x7fdc680a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:09.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.967+0000 7fdc7f7fe640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc680a2990 0x7fdc680a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:09.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.967+0000 7fdc7d7fa640 1 -- 172.21.15.78:0/3246868182 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fdc700cf370 con 0x7fdc80102ac0 2024-05-21T15:16:09.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.967+0000 7fdc7f7fe640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc680a2990 0x7fdc680a4ea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc8019a310 tx=0x7fdc74005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:09.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:09.971+0000 7fdc7d7fa640 1 -- 172.21.15.78:0/3246868182 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdc7009c4a0 con 0x7fdc80102ac0 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: [21/May/2024:15:16:09] ENGINE Bus STARTING 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2321884157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: [21/May/2024:15:16:09] ENGINE Serving on http://172.21.15.78:8765 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: [21/May/2024:15:16:09] ENGINE Serving on https://172.21.15.78:7150 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: [21/May/2024:15:16:09] ENGINE Bus STARTED 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1710716904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:10.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:09 smithi078 ceph-mon[31997]: mgrmap e21: x(active, since 1.0207s), standbys: z, y 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: [21/May/2024:15:16:09] ENGINE Bus STARTING 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2321884157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: [21/May/2024:15:16:09] ENGINE Serving on http://172.21.15.78:8765 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: [21/May/2024:15:16:09] ENGINE Serving on https://172.21.15.78:7150 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: [21/May/2024:15:16:09] ENGINE Bus STARTED 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1710716904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:10.074 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:09 smithi172 ceph-mon[36402]: mgrmap e21: x(active, since 1.0207s), standbys: z, y 2024-05-21T15:16:10.098 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:10.098 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T15:16:10.099 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T15:16:10.099 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: [21/May/2024:15:16:09] ENGINE Bus STARTING 2024-05-21T15:16:10.099 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2321884157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:10.099 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: [21/May/2024:15:16:09] ENGINE Serving on http://172.21.15.78:8765 2024-05-21T15:16:10.099 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: [21/May/2024:15:16:09] ENGINE Serving on https://172.21.15.78:7150 2024-05-21T15:16:10.099 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: [21/May/2024:15:16:09] ENGINE Bus STARTED 2024-05-21T15:16:10.099 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1710716904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:10.099 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:09 smithi084 ceph-mon[36334]: mgrmap e21: x(active, since 1.0207s), standbys: z, y 2024-05-21T15:16:10.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.153+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc80061f90 con 0x7fdc680a2990 2024-05-21T15:16:10.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.161+0000 7fdc7d7fa640 1 -- 172.21.15.78:0/3246868182 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257782 (secure 0 0 0) 0x7fdc80061f90 con 0x7fdc680a2990 2024-05-21T15:16:10.166 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:10.168 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-05-21T15:16:10.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc680a2990 msgr2=0x7fdc680a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc680a2990 0x7fdc680a4ea0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdc8019a310 tx=0x7fdc74005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80102ac0 msgr2=0x7fdc80199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80102ac0 0x7fdc80199270 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000e9f0 tx=0x7fdc7000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 shutdown_connections 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc80103d00 0x7fdc801997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc680a2990 0x7fdc680a4ea0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc80108a00 0x7fdc801a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 --2- 172.21.15.78:0/3246868182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80102ac0 0x7fdc80199270 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 >> 172.21.15.78:0/3246868182 conn(0x7fdc800fdcf0 msgr2=0x7fdc801049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 shutdown_connections 2024-05-21T15:16:10.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.165+0000 7fdc8668b640 1 -- 172.21.15.78:0/3246868182 wait complete. 2024-05-21T15:16:10.180 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":3,"stamp":"2024-05-21T15:16:09.730596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190702,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1392,"num_read_kb":855,"num_write":559,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181408,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359672,"kb_used_data":14592,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582152,"statfs":{"total":1151940427776,"available":1151572123648,"internally_reserved":0,"allocated":14942208,"data_stored":8595277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705906+0000","last_change":"2024-05-21T15:14:12.254239+0000","last_active":"2024-05-21T15:16:08.705906+0000","last_peered":"2024-05-21T15:16:08.705906+0000","last_clean":"2024-05-21T15:16:08.705906+0000","last_became_active":"2024-05-21T15:14:12.253885+0000","last_became_peered":"2024-05-21T15:14:12.253885+0000","last_unstale":"2024-05-21T15:16:08.705906+0000","last_undegraded":"2024-05-21T15:16:08.705906+0000","last_fullsized":"2024-05-21T15:16:08.705906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:35.126836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983550+0000","last_change":"2024-05-21T15:14:12.262984+0000","last_active":"2024-05-21T15:16:08.983550+0000","last_peered":"2024-05-21T15:16:08.983550+0000","last_clean":"2024-05-21T15:16:08.983550+0000","last_became_active":"2024-05-21T15:14:12.262667+0000","last_became_peered":"2024-05-21T15:14:12.262667+0000","last_unstale":"2024-05-21T15:16:08.983550+0000","last_undegraded":"2024-05-21T15:16:08.983550+0000","last_fullsized":"2024-05-21T15:16:08.983550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:47:47.434314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983456+0000","last_change":"2024-05-21T15:14:12.260573+0000","last_active":"2024-05-21T15:16:08.983456+0000","last_peered":"2024-05-21T15:16:08.983456+0000","last_clean":"2024-05-21T15:16:08.983456+0000","last_became_active":"2024-05-21T15:14:12.259073+0000","last_became_peered":"2024-05-21T15:14:12.259073+0000","last_unstale":"2024-05-21T15:16:08.983456+0000","last_undegraded":"2024-05-21T15:16:08.983456+0000","last_fullsized":"2024-05-21T15:16:08.983456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:15:01.919464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972349+0000","last_change":"2024-05-21T15:14:12.256992+0000","last_active":"2024-05-21T15:16:08.972349+0000","last_peered":"2024-05-21T15:16:08.972349+0000","last_clean":"2024-05-21T15:16:08.972349+0000","last_became_active":"2024-05-21T15:14:12.256448+0000","last_became_peered":"2024-05-21T15:14:12.256448+0000","last_unstale":"2024-05-21T15:16:08.972349+0000","last_undegraded":"2024-05-21T15:16:08.972349+0000","last_fullsized":"2024-05-21T15:16:08.972349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:40.569263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905144+0000","last_change":"2024-05-21T15:14:12.251533+0000","last_active":"2024-05-21T15:16:08.905144+0000","last_peered":"2024-05-21T15:16:08.905144+0000","last_clean":"2024-05-21T15:16:08.905144+0000","last_became_active":"2024-05-21T15:14:12.251210+0000","last_became_peered":"2024-05-21T15:14:12.251210+0000","last_unstale":"2024-05-21T15:16:08.905144+0000","last_undegraded":"2024-05-21T15:16:08.905144+0000","last_fullsized":"2024-05-21T15:16:08.905144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:43:34.109179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705582+0000","last_change":"2024-05-21T15:14:12.251767+0000","last_active":"2024-05-21T15:16:08.705582+0000","last_peered":"2024-05-21T15:16:08.705582+0000","last_clean":"2024-05-21T15:16:08.705582+0000","last_became_active":"2024-05-21T15:14:12.251427+0000","last_became_peered":"2024-05-21T15:14:12.251427+0000","last_unstale":"2024-05-21T15:16:08.705582+0000","last_undegraded":"2024-05-21T15:16:08.705582+0000","last_fullsized":"2024-05-21T15:16:08.705582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:48:41.804074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975895+0000","last_change":"2024-05-21T15:14:12.253472+0000","last_active":"2024-05-21T15:16:08.975895+0000","last_peered":"2024-05-21T15:16:08.975895+0000","last_clean":"2024-05-21T15:16:08.975895+0000","last_became_active":"2024-05-21T15:14:12.252762+0000","last_became_peered":"2024-05-21T15:14:12.252762+0000","last_unstale":"2024-05-21T15:16:08.975895+0000","last_undegraded":"2024-05-21T15:16:08.975895+0000","last_fullsized":"2024-05-21T15:16:08.975895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:11:44.497404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807382+0000","last_change":"2024-05-21T15:14:12.258627+0000","last_active":"2024-05-21T15:16:08.807382+0000","last_peered":"2024-05-21T15:16:08.807382+0000","last_clean":"2024-05-21T15:16:08.807382+0000","last_became_active":"2024-05-21T15:14:12.258118+0000","last_became_peered":"2024-05-21T15:14:12.258118+0000","last_unstale":"2024-05-21T15:16:08.807382+0000","last_undegraded":"2024-05-21T15:16:08.807382+0000","last_fullsized":"2024-05-21T15:16:08.807382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:56.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979322+0000","last_change":"2024-05-21T15:14:12.255998+0000","last_active":"2024-05-21T15:16:08.979322+0000","last_peered":"2024-05-21T15:16:08.979322+0000","last_clean":"2024-05-21T15:16:08.979322+0000","last_became_active":"2024-05-21T15:14:12.255658+0000","last_became_peered":"2024-05-21T15:14:12.255658+0000","last_unstale":"2024-05-21T15:16:08.979322+0000","last_undegraded":"2024-05-21T15:16:08.979322+0000","last_fullsized":"2024-05-21T15:16:08.979322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:44:42.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.706089+0000","last_change":"2024-05-21T15:14:12.257617+0000","last_active":"2024-05-21T15:16:08.706089+0000","last_peered":"2024-05-21T15:16:08.706089+0000","last_clean":"2024-05-21T15:16:08.706089+0000","last_became_active":"2024-05-21T15:14:12.256947+0000","last_became_peered":"2024-05-21T15:14:12.256947+0000","last_unstale":"2024-05-21T15:16:08.706089+0000","last_undegraded":"2024-05-21T15:16:08.706089+0000","last_fullsized":"2024-05-21T15:16:08.706089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:45:58.174954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976285+0000","last_change":"2024-05-21T15:14:12.260454+0000","last_active":"2024-05-21T15:16:08.976285+0000","last_peered":"2024-05-21T15:16:08.976285+0000","last_clean":"2024-05-21T15:16:08.976285+0000","last_became_active":"2024-05-21T15:14:12.260126+0000","last_became_peered":"2024-05-21T15:14:12.260126+0000","last_unstale":"2024-05-21T15:16:08.976285+0000","last_undegraded":"2024-05-21T15:16:08.976285+0000","last_fullsized":"2024-05-21T15:16:08.976285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:49:45.212050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972342+0000","last_change":"2024-05-21T15:14:12.252071+0000","last_active":"2024-05-21T15:16:08.972342+0000","last_peered":"2024-05-21T15:16:08.972342+0000","last_clean":"2024-05-21T15:16:08.972342+0000","last_became_active":"2024-05-21T15:14:12.251706+0000","last_became_peered":"2024-05-21T15:14:12.251706+0000","last_unstale":"2024-05-21T15:16:08.972342+0000","last_undegraded":"2024-05-21T15:16:08.972342+0000","last_fullsized":"2024-05-21T15:16:08.972342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:02:44.349368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979900+0000","last_change":"2024-05-21T15:14:12.239877+0000","last_active":"2024-05-21T15:16:08.979900+0000","last_peered":"2024-05-21T15:16:08.979900+0000","last_clean":"2024-05-21T15:16:08.979900+0000","last_became_active":"2024-05-21T15:14:12.239615+0000","last_became_peered":"2024-05-21T15:14:12.239615+0000","last_unstale":"2024-05-21T15:16:08.979900+0000","last_undegraded":"2024-05-21T15:16:08.979900+0000","last_fullsized":"2024-05-21T15:16:08.979900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:23.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983191+0000","last_change":"2024-05-21T15:14:12.260349+0000","last_active":"2024-05-21T15:16:08.983191+0000","last_peered":"2024-05-21T15:16:08.983191+0000","last_clean":"2024-05-21T15:16:08.983191+0000","last_became_active":"2024-05-21T15:14:12.259658+0000","last_became_peered":"2024-05-21T15:14:12.259658+0000","last_unstale":"2024-05-21T15:16:08.983191+0000","last_undegraded":"2024-05-21T15:16:08.983191+0000","last_fullsized":"2024-05-21T15:16:08.983191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:59:32.326571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983078+0000","last_change":"2024-05-21T15:15:17.015535+0000","last_active":"2024-05-21T15:16:08.983078+0000","last_peered":"2024-05-21T15:16:08.983078+0000","last_clean":"2024-05-21T15:16:08.983078+0000","last_became_active":"2024-05-21T15:15:17.015057+0000","last_became_peered":"2024-05-21T15:15:17.015057+0000","last_unstale":"2024-05-21T15:16:08.983078+0000","last_undegraded":"2024-05-21T15:16:08.983078+0000","last_fullsized":"2024-05-21T15:16:08.983078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:48:14.652870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.978891+0000","last_change":"2024-05-21T15:14:12.264667+0000","last_active":"2024-05-21T15:16:08.978891+0000","last_peered":"2024-05-21T15:16:08.978891+0000","last_clean":"2024-05-21T15:16:08.978891+0000","last_became_active":"2024-05-21T15:14:12.264284+0000","last_became_peered":"2024-05-21T15:14:12.264284+0000","last_unstale":"2024-05-21T15:16:08.978891+0000","last_undegraded":"2024-05-21T15:16:08.978891+0000","last_fullsized":"2024-05-21T15:16:08.978891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:24.861321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705966+0000","last_change":"2024-05-21T15:14:12.259912+0000","last_active":"2024-05-21T15:16:08.705966+0000","last_peered":"2024-05-21T15:16:08.705966+0000","last_clean":"2024-05-21T15:16:08.705966+0000","last_became_active":"2024-05-21T15:14:12.259578+0000","last_became_peered":"2024-05-21T15:14:12.259578+0000","last_unstale":"2024-05-21T15:16:08.705966+0000","last_undegraded":"2024-05-21T15:16:08.705966+0000","last_fullsized":"2024-05-21T15:16:08.705966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:05:33.372481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704201+0000","last_change":"2024-05-21T15:14:12.258355+0000","last_active":"2024-05-21T15:16:08.704201+0000","last_peered":"2024-05-21T15:16:08.704201+0000","last_clean":"2024-05-21T15:16:08.704201+0000","last_became_active":"2024-05-21T15:14:12.257991+0000","last_became_peered":"2024-05-21T15:14:12.257991+0000","last_unstale":"2024-05-21T15:16:08.704201+0000","last_undegraded":"2024-05-21T15:16:08.704201+0000","last_fullsized":"2024-05-21T15:16:08.704201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:54.014553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976238+0000","last_change":"2024-05-21T15:14:12.258873+0000","last_active":"2024-05-21T15:16:08.976238+0000","last_peered":"2024-05-21T15:16:08.976238+0000","last_clean":"2024-05-21T15:16:08.976238+0000","last_became_active":"2024-05-21T15:14:12.258548+0000","last_became_peered":"2024-05-21T15:14:12.258548+0000","last_unstale":"2024-05-21T15:16:08.976238+0000","last_undegraded":"2024-05-21T15:16:08.976238+0000","last_fullsized":"2024-05-21T15:16:08.976238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:00.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":167,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704409+0000","last_change":"2024-05-21T15:15:16.626507+0000","last_active":"2024-05-21T15:16:08.704409+0000","last_peered":"2024-05-21T15:16:08.704409+0000","last_clean":"2024-05-21T15:16:08.704409+0000","last_became_active":"2024-05-21T15:15:16.615945+0000","last_became_peered":"2024-05-21T15:15:16.615945+0000","last_unstale":"2024-05-21T15:16:08.704409+0000","last_undegraded":"2024-05-21T15:16:08.704409+0000","last_fullsized":"2024-05-21T15:16:08.704409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:16:01.229449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808883+0000","last_change":"2024-05-21T15:14:12.261632+0000","last_active":"2024-05-21T15:16:08.808883+0000","last_peered":"2024-05-21T15:16:08.808883+0000","last_clean":"2024-05-21T15:16:08.808883+0000","last_became_active":"2024-05-21T15:14:12.261149+0000","last_became_peered":"2024-05-21T15:14:12.261149+0000","last_unstale":"2024-05-21T15:16:08.808883+0000","last_undegraded":"2024-05-21T15:16:08.808883+0000","last_fullsized":"2024-05-21T15:16:08.808883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:11.349395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809030+0000","last_change":"2024-05-21T15:14:11.567859+0000","last_active":"2024-05-21T15:16:08.809030+0000","last_peered":"2024-05-21T15:16:08.809030+0000","last_clean":"2024-05-21T15:16:08.809030+0000","last_became_active":"2024-05-21T15:14:11.566855+0000","last_became_peered":"2024-05-21T15:14:11.566855+0000","last_unstale":"2024-05-21T15:16:08.809030+0000","last_undegraded":"2024-05-21T15:16:08.809030+0000","last_fullsized":"2024-05-21T15:16:08.809030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:21.568436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983569+0000","last_change":"2024-05-21T15:14:12.262153+0000","last_active":"2024-05-21T15:16:08.983569+0000","last_peered":"2024-05-21T15:16:08.983569+0000","last_clean":"2024-05-21T15:16:08.983569+0000","last_became_active":"2024-05-21T15:14:12.261780+0000","last_became_peered":"2024-05-21T15:14:12.261780+0000","last_unstale":"2024-05-21T15:16:08.983569+0000","last_undegraded":"2024-05-21T15:16:08.983569+0000","last_fullsized":"2024-05-21T15:16:08.983569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:21:25.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'14","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905483+0000","last_change":"2024-05-21T15:14:11.570316+0000","last_active":"2024-05-21T15:16:08.905483+0000","last_peered":"2024-05-21T15:16:08.905483+0000","last_clean":"2024-05-21T15:16:08.905483+0000","last_became_active":"2024-05-21T15:14:11.569967+0000","last_became_peered":"2024-05-21T15:14:11.569967+0000","last_unstale":"2024-05-21T15:16:08.905483+0000","last_undegraded":"2024-05-21T15:16:08.905483+0000","last_fullsized":"2024-05-21T15:16:08.905483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:30:27.364742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":20,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979138+0000","last_change":"2024-05-21T15:14:12.261716+0000","last_active":"2024-05-21T15:16:08.979138+0000","last_peered":"2024-05-21T15:16:08.979138+0000","last_clean":"2024-05-21T15:16:08.979138+0000","last_became_active":"2024-05-21T15:14:12.261341+0000","last_became_peered":"2024-05-21T15:14:12.261341+0000","last_unstale":"2024-05-21T15:16:08.979138+0000","last_undegraded":"2024-05-21T15:16:08.979138+0000","last_fullsized":"2024-05-21T15:16:08.979138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:40:10.464974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905502+0000","last_change":"2024-05-21T15:14:11.573305+0000","last_active":"2024-05-21T15:16:08.905502+0000","last_peered":"2024-05-21T15:16:08.905502+0000","last_clean":"2024-05-21T15:16:08.905502+0000","last_became_active":"2024-05-21T15:14:11.572655+0000","last_became_peered":"2024-05-21T15:14:11.572655+0000","last_unstale":"2024-05-21T15:16:08.905502+0000","last_undegraded":"2024-05-21T15:16:08.905502+0000","last_fullsized":"2024-05-21T15:16:08.905502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:37.268440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815691+0000","last_change":"2024-05-21T15:14:12.253788+0000","last_active":"2024-05-21T15:16:08.815691+0000","last_peered":"2024-05-21T15:16:08.815691+0000","last_clean":"2024-05-21T15:16:08.815691+0000","last_became_active":"2024-05-21T15:14:12.253450+0000","last_became_peered":"2024-05-21T15:14:12.253450+0000","last_unstale":"2024-05-21T15:16:08.815691+0000","last_undegraded":"2024-05-21T15:16:08.815691+0000","last_fullsized":"2024-05-21T15:16:08.815691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:57.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":149,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430893+0000","last_change":"2024-05-21T15:14:11.567400+0000","last_active":"2024-05-21T15:16:09.430893+0000","last_peered":"2024-05-21T15:16:09.430893+0000","last_clean":"2024-05-21T15:16:09.430893+0000","last_became_active":"2024-05-21T15:14:11.565995+0000","last_became_peered":"2024-05-21T15:14:11.565995+0000","last_unstale":"2024-05-21T15:16:09.430893+0000","last_undegraded":"2024-05-21T15:16:09.430893+0000","last_fullsized":"2024-05-21T15:16:09.430893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:50.463339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905055+0000","last_change":"2024-05-21T15:14:12.251565+0000","last_active":"2024-05-21T15:16:08.905055+0000","last_peered":"2024-05-21T15:16:08.905055+0000","last_clean":"2024-05-21T15:16:08.905055+0000","last_became_active":"2024-05-21T15:14:12.251289+0000","last_became_peered":"2024-05-21T15:14:12.251289+0000","last_unstale":"2024-05-21T15:16:08.905055+0000","last_undegraded":"2024-05-21T15:16:08.905055+0000","last_fullsized":"2024-05-21T15:16:08.905055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:44.619800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705549+0000","last_change":"2024-05-21T15:14:11.575193+0000","last_active":"2024-05-21T15:16:08.705549+0000","last_peered":"2024-05-21T15:16:08.705549+0000","last_clean":"2024-05-21T15:16:08.705549+0000","last_became_active":"2024-05-21T15:14:11.574861+0000","last_became_peered":"2024-05-21T15:14:11.574861+0000","last_unstale":"2024-05-21T15:16:08.705549+0000","last_undegraded":"2024-05-21T15:16:08.705549+0000","last_fullsized":"2024-05-21T15:16:08.705549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:54.548408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815610+0000","last_change":"2024-05-21T15:15:17.015620+0000","last_active":"2024-05-21T15:16:08.815610+0000","last_peered":"2024-05-21T15:16:08.815610+0000","last_clean":"2024-05-21T15:16:08.815610+0000","last_became_active":"2024-05-21T15:15:17.015086+0000","last_became_peered":"2024-05-21T15:15:17.015086+0000","last_unstale":"2024-05-21T15:16:08.815610+0000","last_undegraded":"2024-05-21T15:16:08.815610+0000","last_fullsized":"2024-05-21T15:16:08.815610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:10.856246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979503+0000","last_change":"2024-05-21T15:14:11.572087+0000","last_active":"2024-05-21T15:16:08.979503+0000","last_peered":"2024-05-21T15:16:08.979503+0000","last_clean":"2024-05-21T15:16:08.979503+0000","last_became_active":"2024-05-21T15:14:11.571716+0000","last_became_peered":"2024-05-21T15:14:11.571716+0000","last_unstale":"2024-05-21T15:16:08.979503+0000","last_undegraded":"2024-05-21T15:16:08.979503+0000","last_fullsized":"2024-05-21T15:16:08.979503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:07.082766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807382+0000","last_change":"2024-05-21T15:14:12.252233+0000","last_active":"2024-05-21T15:16:08.807382+0000","last_peered":"2024-05-21T15:16:08.807382+0000","last_clean":"2024-05-21T15:16:08.807382+0000","last_became_active":"2024-05-21T15:14:12.251901+0000","last_became_peered":"2024-05-21T15:14:12.251901+0000","last_unstale":"2024-05-21T15:16:08.807382+0000","last_undegraded":"2024-05-21T15:16:08.807382+0000","last_fullsized":"2024-05-21T15:16:08.807382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:05.107436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815255+0000","last_change":"2024-05-21T15:14:11.565946+0000","last_active":"2024-05-21T15:16:08.815255+0000","last_peered":"2024-05-21T15:16:08.815255+0000","last_clean":"2024-05-21T15:16:08.815255+0000","last_became_active":"2024-05-21T15:14:11.565601+0000","last_became_peered":"2024-05-21T15:14:11.565601+0000","last_unstale":"2024-05-21T15:16:08.815255+0000","last_undegraded":"2024-05-21T15:16:08.815255+0000","last_fullsized":"2024-05-21T15:16:08.815255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:41:55.243473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704086+0000","last_change":"2024-05-21T15:14:12.255188+0000","last_active":"2024-05-21T15:16:08.704086+0000","last_peered":"2024-05-21T15:16:08.704086+0000","last_clean":"2024-05-21T15:16:08.704086+0000","last_became_active":"2024-05-21T15:14:12.254503+0000","last_became_peered":"2024-05-21T15:14:12.254503+0000","last_unstale":"2024-05-21T15:16:08.704086+0000","last_undegraded":"2024-05-21T15:16:08.704086+0000","last_fullsized":"2024-05-21T15:16:08.704086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:22:16.336740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":254,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430238+0000","last_change":"2024-05-21T15:14:11.569882+0000","last_active":"2024-05-21T15:16:09.430238+0000","last_peered":"2024-05-21T15:16:09.430238+0000","last_clean":"2024-05-21T15:16:09.430238+0000","last_became_active":"2024-05-21T15:14:11.569532+0000","last_became_peered":"2024-05-21T15:14:11.569532+0000","last_unstale":"2024-05-21T15:16:09.430238+0000","last_undegraded":"2024-05-21T15:16:09.430238+0000","last_fullsized":"2024-05-21T15:16:09.430238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:58:50.081709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979327+0000","last_change":"2024-05-21T15:14:12.262691+0000","last_active":"2024-05-21T15:16:08.979327+0000","last_peered":"2024-05-21T15:16:08.979327+0000","last_clean":"2024-05-21T15:16:08.979327+0000","last_became_active":"2024-05-21T15:14:12.261492+0000","last_became_peered":"2024-05-21T15:14:12.261492+0000","last_unstale":"2024-05-21T15:16:08.979327+0000","last_undegraded":"2024-05-21T15:16:08.979327+0000","last_fullsized":"2024-05-21T15:16:08.979327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:35:00.121256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976229+0000","last_change":"2024-05-21T15:14:11.569231+0000","last_active":"2024-05-21T15:16:08.976229+0000","last_peered":"2024-05-21T15:16:08.976229+0000","last_clean":"2024-05-21T15:16:08.976229+0000","last_became_active":"2024-05-21T15:14:11.568916+0000","last_became_peered":"2024-05-21T15:14:11.568916+0000","last_unstale":"2024-05-21T15:16:08.976229+0000","last_undegraded":"2024-05-21T15:16:08.976229+0000","last_fullsized":"2024-05-21T15:16:08.976229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:26:13.411854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983497+0000","last_change":"2024-05-21T15:14:12.256307+0000","last_active":"2024-05-21T15:16:08.983497+0000","last_peered":"2024-05-21T15:16:08.983497+0000","last_clean":"2024-05-21T15:16:08.983497+0000","last_became_active":"2024-05-21T15:14:12.255828+0000","last_became_peered":"2024-05-21T15:14:12.255828+0000","last_unstale":"2024-05-21T15:16:08.983497+0000","last_undegraded":"2024-05-21T15:16:08.983497+0000","last_fullsized":"2024-05-21T15:16:08.983497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:11:40.068779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815696+0000","last_change":"2024-05-21T15:14:11.568239+0000","last_active":"2024-05-21T15:16:08.815696+0000","last_peered":"2024-05-21T15:16:08.815696+0000","last_clean":"2024-05-21T15:16:08.815696+0000","last_became_active":"2024-05-21T15:14:11.567836+0000","last_became_peered":"2024-05-21T15:14:11.567836+0000","last_unstale":"2024-05-21T15:16:08.815696+0000","last_undegraded":"2024-05-21T15:16:08.815696+0000","last_fullsized":"2024-05-21T15:16:08.815696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:10.244172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979346+0000","last_change":"2024-05-21T15:14:12.257762+0000","last_active":"2024-05-21T15:16:08.979346+0000","last_peered":"2024-05-21T15:16:08.979346+0000","last_clean":"2024-05-21T15:16:08.979346+0000","last_became_active":"2024-05-21T15:14:12.256335+0000","last_became_peered":"2024-05-21T15:14:12.256335+0000","last_unstale":"2024-05-21T15:16:08.979346+0000","last_undegraded":"2024-05-21T15:16:08.979346+0000","last_fullsized":"2024-05-21T15:16:08.979346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:35.219673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":300,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.460777+0000","last_change":"2024-05-21T15:14:11.564407+0000","last_active":"2024-05-21T15:16:09.460777+0000","last_peered":"2024-05-21T15:16:09.460777+0000","last_clean":"2024-05-21T15:16:09.460777+0000","last_became_active":"2024-05-21T15:14:11.564012+0000","last_became_peered":"2024-05-21T15:14:11.564012+0000","last_unstale":"2024-05-21T15:16:09.460777+0000","last_undegraded":"2024-05-21T15:16:09.460777+0000","last_fullsized":"2024-05-21T15:16:09.460777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:34:54.177844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65676,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":184,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905245+0000","last_change":"2024-05-21T15:14:12.253820+0000","last_active":"2024-05-21T15:16:08.905245+0000","last_peered":"2024-05-21T15:16:08.905245+0000","last_clean":"2024-05-21T15:16:08.905245+0000","last_became_active":"2024-05-21T15:14:12.253360+0000","last_became_peered":"2024-05-21T15:14:12.253360+0000","last_unstale":"2024-05-21T15:16:08.905245+0000","last_undegraded":"2024-05-21T15:16:08.905245+0000","last_fullsized":"2024-05-21T15:16:08.905245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:38:19.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976498+0000","last_change":"2024-05-21T15:14:11.571161+0000","last_active":"2024-05-21T15:16:08.976498+0000","last_peered":"2024-05-21T15:16:08.976498+0000","last_clean":"2024-05-21T15:16:08.976498+0000","last_became_active":"2024-05-21T15:14:11.570837+0000","last_became_peered":"2024-05-21T15:14:11.570837+0000","last_unstale":"2024-05-21T15:16:08.976498+0000","last_undegraded":"2024-05-21T15:16:08.976498+0000","last_fullsized":"2024-05-21T15:16:08.976498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:24:55.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979900+0000","last_change":"2024-05-21T15:14:12.255713+0000","last_active":"2024-05-21T15:16:08.979900+0000","last_peered":"2024-05-21T15:16:08.979900+0000","last_clean":"2024-05-21T15:16:08.979900+0000","last_became_active":"2024-05-21T15:14:12.254458+0000","last_became_peered":"2024-05-21T15:14:12.254458+0000","last_unstale":"2024-05-21T15:16:08.979900+0000","last_undegraded":"2024-05-21T15:16:08.979900+0000","last_fullsized":"2024-05-21T15:16:08.979900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:55:24.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815443+0000","last_change":"2024-05-21T15:14:11.565916+0000","last_active":"2024-05-21T15:16:08.815443+0000","last_peered":"2024-05-21T15:16:08.815443+0000","last_clean":"2024-05-21T15:16:08.815443+0000","last_became_active":"2024-05-21T15:14:11.565460+0000","last_became_peered":"2024-05-21T15:14:11.565460+0000","last_unstale":"2024-05-21T15:16:08.815443+0000","last_undegraded":"2024-05-21T15:16:08.815443+0000","last_fullsized":"2024-05-21T15:16:08.815443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:09.982208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976147+0000","last_change":"2024-05-21T15:14:12.251856+0000","last_active":"2024-05-21T15:16:08.976147+0000","last_peered":"2024-05-21T15:16:08.976147+0000","last_clean":"2024-05-21T15:16:08.976147+0000","last_became_active":"2024-05-21T15:14:12.251496+0000","last_became_peered":"2024-05-21T15:14:12.251496+0000","last_unstale":"2024-05-21T15:16:08.976147+0000","last_undegraded":"2024-05-21T15:16:08.976147+0000","last_fullsized":"2024-05-21T15:16:08.976147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:17.566276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430144+0000","last_change":"2024-05-21T15:14:11.565485+0000","last_active":"2024-05-21T15:16:09.430144+0000","last_peered":"2024-05-21T15:16:09.430144+0000","last_clean":"2024-05-21T15:16:09.430144+0000","last_became_active":"2024-05-21T15:14:11.565092+0000","last_became_peered":"2024-05-21T15:14:11.565092+0000","last_unstale":"2024-05-21T15:16:09.430144+0000","last_undegraded":"2024-05-21T15:16:09.430144+0000","last_fullsized":"2024-05-21T15:16:09.430144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:01:16.101793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975843+0000","last_change":"2024-05-21T15:14:12.262283+0000","last_active":"2024-05-21T15:16:08.975843+0000","last_peered":"2024-05-21T15:16:08.975843+0000","last_clean":"2024-05-21T15:16:08.975843+0000","last_became_active":"2024-05-21T15:14:12.261610+0000","last_became_peered":"2024-05-21T15:14:12.261610+0000","last_unstale":"2024-05-21T15:16:08.975843+0000","last_undegraded":"2024-05-21T15:16:08.975843+0000","last_fullsized":"2024-05-21T15:16:08.975843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:44:33.087115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.904945+0000","last_change":"2024-05-21T15:14:11.569905+0000","last_active":"2024-05-21T15:16:08.904945+0000","last_peered":"2024-05-21T15:16:08.904945+0000","last_clean":"2024-05-21T15:16:08.904945+0000","last_became_active":"2024-05-21T15:14:11.569221+0000","last_became_peered":"2024-05-21T15:14:11.569221+0000","last_unstale":"2024-05-21T15:16:08.904945+0000","last_undegraded":"2024-05-21T15:16:08.904945+0000","last_fullsized":"2024-05-21T15:16:08.904945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:11:04.875890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815306+0000","last_change":"2024-05-21T15:15:16.386384+0000","last_active":"2024-05-21T15:16:08.815306+0000","last_peered":"2024-05-21T15:16:08.815306+0000","last_clean":"2024-05-21T15:16:08.815306+0000","last_became_active":"2024-05-21T15:15:16.385810+0000","last_became_peered":"2024-05-21T15:15:16.385810+0000","last_unstale":"2024-05-21T15:16:08.815306+0000","last_undegraded":"2024-05-21T15:16:08.815306+0000","last_fullsized":"2024-05-21T15:16:08.815306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:49:45.031393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905410+0000","last_change":"2024-05-21T15:14:11.569670+0000","last_active":"2024-05-21T15:16:08.905410+0000","last_peered":"2024-05-21T15:16:08.905410+0000","last_clean":"2024-05-21T15:16:08.905410+0000","last_became_active":"2024-05-21T15:14:11.569257+0000","last_became_peered":"2024-05-21T15:14:11.569257+0000","last_unstale":"2024-05-21T15:16:08.905410+0000","last_undegraded":"2024-05-21T15:16:08.905410+0000","last_fullsized":"2024-05-21T15:16:08.905410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:18.584920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705771+0000","last_change":"2024-05-21T15:15:16.374845+0000","last_active":"2024-05-21T15:16:08.705771+0000","last_peered":"2024-05-21T15:16:08.705771+0000","last_clean":"2024-05-21T15:16:08.705771+0000","last_became_active":"2024-05-21T15:15:16.374393+0000","last_became_peered":"2024-05-21T15:15:16.374393+0000","last_unstale":"2024-05-21T15:16:08.705771+0000","last_undegraded":"2024-05-21T15:16:08.705771+0000","last_fullsized":"2024-05-21T15:16:08.705771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:38:21.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704254+0000","last_change":"2024-05-21T15:14:11.567849+0000","last_active":"2024-05-21T15:16:08.704254+0000","last_peered":"2024-05-21T15:16:08.704254+0000","last_clean":"2024-05-21T15:16:08.704254+0000","last_became_active":"2024-05-21T15:14:11.567476+0000","last_became_peered":"2024-05-21T15:14:11.567476+0000","last_unstale":"2024-05-21T15:16:08.704254+0000","last_undegraded":"2024-05-21T15:16:08.704254+0000","last_fullsized":"2024-05-21T15:16:08.704254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:13.516765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705800+0000","last_change":"2024-05-21T15:14:12.255165+0000","last_active":"2024-05-21T15:16:08.705800+0000","last_peered":"2024-05-21T15:16:08.705800+0000","last_clean":"2024-05-21T15:16:08.705800+0000","last_became_active":"2024-05-21T15:14:12.254743+0000","last_became_peered":"2024-05-21T15:14:12.254743+0000","last_unstale":"2024-05-21T15:16:08.705800+0000","last_undegraded":"2024-05-21T15:16:08.705800+0000","last_fullsized":"2024-05-21T15:16:08.705800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:53:11.059597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'8","reported_seq":143,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.690138+0000","last_change":"2024-05-21T15:14:11.569745+0000","last_active":"2024-05-21T15:16:09.690138+0000","last_peered":"2024-05-21T15:16:09.690138+0000","last_clean":"2024-05-21T15:16:09.690138+0000","last_became_active":"2024-05-21T15:14:11.569409+0000","last_became_peered":"2024-05-21T15:14:11.569409+0000","last_unstale":"2024-05-21T15:16:09.690138+0000","last_undegraded":"2024-05-21T15:16:09.690138+0000","last_fullsized":"2024-05-21T15:16:09.690138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:00:37.963797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1678,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705802+0000","last_change":"2024-05-21T15:14:12.254631+0000","last_active":"2024-05-21T15:16:08.705802+0000","last_peered":"2024-05-21T15:16:08.705802+0000","last_clean":"2024-05-21T15:16:08.705802+0000","last_became_active":"2024-05-21T15:14:12.254239+0000","last_became_peered":"2024-05-21T15:14:12.254239+0000","last_unstale":"2024-05-21T15:16:08.705802+0000","last_undegraded":"2024-05-21T15:16:08.705802+0000","last_fullsized":"2024-05-21T15:16:08.705802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:55:17.135550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.703987+0000","last_change":"2024-05-21T15:15:16.615801+0000","last_active":"2024-05-21T15:16:08.703987+0000","last_peered":"2024-05-21T15:16:08.703987+0000","last_clean":"2024-05-21T15:16:08.703987+0000","last_became_active":"2024-05-21T15:15:16.615323+0000","last_became_peered":"2024-05-21T15:15:16.615323+0000","last_unstale":"2024-05-21T15:16:08.703987+0000","last_undegraded":"2024-05-21T15:16:08.703987+0000","last_fullsized":"2024-05-21T15:16:08.703987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:15.233980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.982372+0000","last_change":"2024-05-21T15:14:11.574352+0000","last_active":"2024-05-21T15:16:08.982372+0000","last_peered":"2024-05-21T15:16:08.982372+0000","last_clean":"2024-05-21T15:16:08.982372+0000","last_became_active":"2024-05-21T15:14:11.574041+0000","last_became_peered":"2024-05-21T15:14:11.574041+0000","last_unstale":"2024-05-21T15:16:08.982372+0000","last_undegraded":"2024-05-21T15:16:08.982372+0000","last_fullsized":"2024-05-21T15:16:08.982372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.563218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808128+0000","last_change":"2024-05-21T15:14:12.257886+0000","last_active":"2024-05-21T15:16:08.808128+0000","last_peered":"2024-05-21T15:16:08.808128+0000","last_clean":"2024-05-21T15:16:08.808128+0000","last_became_active":"2024-05-21T15:14:12.257546+0000","last_became_peered":"2024-05-21T15:14:12.257546+0000","last_unstale":"2024-05-21T15:16:08.808128+0000","last_undegraded":"2024-05-21T15:16:08.808128+0000","last_fullsized":"2024-05-21T15:16:08.808128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:20.953421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972256+0000","last_change":"2024-05-21T15:14:11.568524+0000","last_active":"2024-05-21T15:16:08.972256+0000","last_peered":"2024-05-21T15:16:08.972256+0000","last_clean":"2024-05-21T15:16:08.972256+0000","last_became_active":"2024-05-21T15:14:11.567981+0000","last_became_peered":"2024-05-21T15:14:11.567981+0000","last_unstale":"2024-05-21T15:16:08.972256+0000","last_undegraded":"2024-05-21T15:16:08.972256+0000","last_fullsized":"2024-05-21T15:16:08.972256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975897+0000","last_change":"2024-05-21T15:13:02.249260+0000","last_active":"2024-05-21T15:16:08.975897+0000","last_peered":"2024-05-21T15:16:08.975897+0000","last_clean":"2024-05-21T15:16:08.975897+0000","last_became_active":"2024-05-21T15:13:02.248637+0000","last_became_peered":"2024-05-21T15:13:02.248637+0000","last_unstale":"2024-05-21T15:16:08.975897+0000","last_undegraded":"2024-05-21T15:16:08.975897+0000","last_fullsized":"2024-05-21T15:16:08.975897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_clean_scrub_stamp":"2024-05-21T15:09:41.506463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:26.250910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979324+0000","last_change":"2024-05-21T15:14:12.259861+0000","last_active":"2024-05-21T15:16:08.979324+0000","last_peered":"2024-05-21T15:16:08.979324+0000","last_clean":"2024-05-21T15:16:08.979324+0000","last_became_active":"2024-05-21T15:14:12.259510+0000","last_became_peered":"2024-05-21T15:14:12.259510+0000","last_unstale":"2024-05-21T15:16:08.979324+0000","last_undegraded":"2024-05-21T15:16:08.979324+0000","last_fullsized":"2024-05-21T15:16:08.979324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:49.548366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976144+0000","last_change":"2024-05-21T15:14:11.570705+0000","last_active":"2024-05-21T15:16:08.976144+0000","last_peered":"2024-05-21T15:16:08.976144+0000","last_clean":"2024-05-21T15:16:08.976144+0000","last_became_active":"2024-05-21T15:14:11.570403+0000","last_became_peered":"2024-05-21T15:14:11.570403+0000","last_unstale":"2024-05-21T15:16:08.976144+0000","last_undegraded":"2024-05-21T15:16:08.976144+0000","last_fullsized":"2024-05-21T15:16:08.976144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:02:34.156258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905269+0000","last_change":"2024-05-21T15:14:11.572311+0000","last_active":"2024-05-21T15:16:08.905269+0000","last_peered":"2024-05-21T15:16:08.905269+0000","last_clean":"2024-05-21T15:16:08.905269+0000","last_became_active":"2024-05-21T15:14:11.571949+0000","last_became_peered":"2024-05-21T15:14:11.571949+0000","last_unstale":"2024-05-21T15:16:08.905269+0000","last_undegraded":"2024-05-21T15:16:08.905269+0000","last_fullsized":"2024-05-21T15:16:08.905269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:57:59.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979695+0000","last_change":"2024-05-21T15:14:12.260717+0000","last_active":"2024-05-21T15:16:08.979695+0000","last_peered":"2024-05-21T15:16:08.979695+0000","last_clean":"2024-05-21T15:16:08.979695+0000","last_became_active":"2024-05-21T15:14:12.259911+0000","last_became_peered":"2024-05-21T15:14:12.259911+0000","last_unstale":"2024-05-21T15:16:08.979695+0000","last_undegraded":"2024-05-21T15:16:08.979695+0000","last_fullsized":"2024-05-21T15:16:08.979695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:02.609299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.933483+0000","last_change":"2024-05-21T15:15:16.381952+0000","last_active":"2024-05-21T15:16:08.933483+0000","last_peered":"2024-05-21T15:16:08.933483+0000","last_clean":"2024-05-21T15:16:08.933483+0000","last_became_active":"2024-05-21T15:15:16.373966+0000","last_became_peered":"2024-05-21T15:15:16.373966+0000","last_unstale":"2024-05-21T15:16:08.933483+0000","last_undegraded":"2024-05-21T15:16:08.933483+0000","last_fullsized":"2024-05-21T15:16:08.933483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:49:05.368066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809256+0000","last_change":"2024-05-21T15:14:12.253634+0000","last_active":"2024-05-21T15:16:08.809256+0000","last_peered":"2024-05-21T15:16:08.809256+0000","last_clean":"2024-05-21T15:16:08.809256+0000","last_became_active":"2024-05-21T15:14:12.253284+0000","last_became_peered":"2024-05-21T15:14:12.253284+0000","last_unstale":"2024-05-21T15:16:08.809256+0000","last_undegraded":"2024-05-21T15:16:08.809256+0000","last_fullsized":"2024-05-21T15:16:08.809256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:28:07.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972367+0000","last_change":"2024-05-21T15:14:11.574147+0000","last_active":"2024-05-21T15:16:08.972367+0000","last_peered":"2024-05-21T15:16:08.972367+0000","last_clean":"2024-05-21T15:16:08.972367+0000","last_became_active":"2024-05-21T15:14:11.573721+0000","last_became_peered":"2024-05-21T15:14:11.573721+0000","last_unstale":"2024-05-21T15:16:08.972367+0000","last_undegraded":"2024-05-21T15:16:08.972367+0000","last_fullsized":"2024-05-21T15:16:08.972367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:10.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983413+0000","last_change":"2024-05-21T15:14:12.258632+0000","last_active":"2024-05-21T15:16:08.983413+0000","last_peered":"2024-05-21T15:16:08.983413+0000","last_clean":"2024-05-21T15:16:08.983413+0000","last_became_active":"2024-05-21T15:14:12.257908+0000","last_became_peered":"2024-05-21T15:14:12.257908+0000","last_unstale":"2024-05-21T15:16:08.983413+0000","last_undegraded":"2024-05-21T15:16:08.983413+0000","last_fullsized":"2024-05-21T15:16:08.983413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:12.062387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.978955+0000","last_change":"2024-05-21T15:14:11.562592+0000","last_active":"2024-05-21T15:16:08.978955+0000","last_peered":"2024-05-21T15:16:08.978955+0000","last_clean":"2024-05-21T15:16:08.978955+0000","last_became_active":"2024-05-21T15:14:11.562265+0000","last_became_peered":"2024-05-21T15:14:11.562265+0000","last_unstale":"2024-05-21T15:16:08.978955+0000","last_undegraded":"2024-05-21T15:16:08.978955+0000","last_fullsized":"2024-05-21T15:16:08.978955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:24.662576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905370+0000","last_change":"2024-05-21T15:14:12.249207+0000","last_active":"2024-05-21T15:16:08.905370+0000","last_peered":"2024-05-21T15:16:08.905370+0000","last_clean":"2024-05-21T15:16:08.905370+0000","last_became_active":"2024-05-21T15:14:12.248819+0000","last_became_peered":"2024-05-21T15:14:12.248819+0000","last_unstale":"2024-05-21T15:16:08.905370+0000","last_undegraded":"2024-05-21T15:16:08.905370+0000","last_fullsized":"2024-05-21T15:16:08.905370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:38.208636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807711+0000","last_change":"2024-05-21T15:14:11.572044+0000","last_active":"2024-05-21T15:16:08.807711+0000","last_peered":"2024-05-21T15:16:08.807711+0000","last_clean":"2024-05-21T15:16:08.807711+0000","last_became_active":"2024-05-21T15:14:11.570721+0000","last_became_peered":"2024-05-21T15:14:11.570721+0000","last_unstale":"2024-05-21T15:16:08.807711+0000","last_undegraded":"2024-05-21T15:16:08.807711+0000","last_fullsized":"2024-05-21T15:16:08.807711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:34:08.413781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705475+0000","last_change":"2024-05-21T15:14:12.256327+0000","last_active":"2024-05-21T15:16:08.705475+0000","last_peered":"2024-05-21T15:16:08.705475+0000","last_clean":"2024-05-21T15:16:08.705475+0000","last_became_active":"2024-05-21T15:14:12.255962+0000","last_became_peered":"2024-05-21T15:14:12.255962+0000","last_unstale":"2024-05-21T15:16:08.705475+0000","last_undegraded":"2024-05-21T15:16:08.705475+0000","last_fullsized":"2024-05-21T15:16:08.705475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:05.076323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705571+0000","last_change":"2024-05-21T15:15:16.385658+0000","last_active":"2024-05-21T15:16:08.705571+0000","last_peered":"2024-05-21T15:16:08.705571+0000","last_clean":"2024-05-21T15:16:08.705571+0000","last_became_active":"2024-05-21T15:15:16.375122+0000","last_became_peered":"2024-05-21T15:15:16.375122+0000","last_unstale":"2024-05-21T15:16:08.705571+0000","last_undegraded":"2024-05-21T15:16:08.705571+0000","last_fullsized":"2024-05-21T15:16:08.705571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:57:59.599504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808366+0000","last_change":"2024-05-21T15:14:12.256993+0000","last_active":"2024-05-21T15:16:08.808366+0000","last_peered":"2024-05-21T15:16:08.808366+0000","last_clean":"2024-05-21T15:16:08.808366+0000","last_became_active":"2024-05-21T15:14:12.256179+0000","last_became_peered":"2024-05-21T15:14:12.256179+0000","last_unstale":"2024-05-21T15:16:08.808366+0000","last_undegraded":"2024-05-21T15:16:08.808366+0000","last_fullsized":"2024-05-21T15:16:08.808366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:01:09.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905244+0000","last_change":"2024-05-21T15:15:16.391272+0000","last_active":"2024-05-21T15:16:08.905244+0000","last_peered":"2024-05-21T15:16:08.905244+0000","last_clean":"2024-05-21T15:16:08.905244+0000","last_became_active":"2024-05-21T15:15:16.386921+0000","last_became_peered":"2024-05-21T15:15:16.386921+0000","last_unstale":"2024-05-21T15:16:08.905244+0000","last_undegraded":"2024-05-21T15:16:08.905244+0000","last_fullsized":"2024-05-21T15:16:08.905244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:01:55.466989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976147+0000","last_change":"2024-05-21T15:14:12.260506+0000","last_active":"2024-05-21T15:16:08.976147+0000","last_peered":"2024-05-21T15:16:08.976147+0000","last_clean":"2024-05-21T15:16:08.976147+0000","last_became_active":"2024-05-21T15:14:12.260168+0000","last_became_peered":"2024-05-21T15:14:12.260168+0000","last_unstale":"2024-05-21T15:16:08.976147+0000","last_undegraded":"2024-05-21T15:16:08.976147+0000","last_fullsized":"2024-05-21T15:16:08.976147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:11.321622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.479294+0000","last_change":"2024-05-21T15:14:11.568630+0000","last_active":"2024-05-21T15:16:09.479294+0000","last_peered":"2024-05-21T15:16:09.479294+0000","last_clean":"2024-05-21T15:16:09.479294+0000","last_became_active":"2024-05-21T15:14:11.568229+0000","last_became_peered":"2024-05-21T15:14:11.568229+0000","last_unstale":"2024-05-21T15:16:09.479294+0000","last_undegraded":"2024-05-21T15:16:09.479294+0000","last_fullsized":"2024-05-21T15:16:09.479294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:21.372395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976294+0000","last_change":"2024-05-21T15:14:12.259408+0000","last_active":"2024-05-21T15:16:08.976294+0000","last_peered":"2024-05-21T15:16:08.976294+0000","last_clean":"2024-05-21T15:16:08.976294+0000","last_became_active":"2024-05-21T15:14:12.259082+0000","last_became_peered":"2024-05-21T15:14:12.259082+0000","last_unstale":"2024-05-21T15:16:08.976294+0000","last_undegraded":"2024-05-21T15:16:08.976294+0000","last_fullsized":"2024-05-21T15:16:08.976294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:32:09.059307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.930782+0000","last_change":"2024-05-21T15:14:11.571077+0000","last_active":"2024-05-21T15:16:08.930782+0000","last_peered":"2024-05-21T15:16:08.930782+0000","last_clean":"2024-05-21T15:16:08.930782+0000","last_became_active":"2024-05-21T15:14:11.570664+0000","last_became_peered":"2024-05-21T15:14:11.570664+0000","last_unstale":"2024-05-21T15:16:08.930782+0000","last_undegraded":"2024-05-21T15:16:08.930782+0000","last_fullsized":"2024-05-21T15:16:08.930782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:37:16.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976028+0000","last_change":"2024-05-21T15:14:12.252972+0000","last_active":"2024-05-21T15:16:08.976028+0000","last_peered":"2024-05-21T15:16:08.976028+0000","last_clean":"2024-05-21T15:16:08.976028+0000","last_became_active":"2024-05-21T15:14:12.252591+0000","last_became_peered":"2024-05-21T15:14:12.252591+0000","last_unstale":"2024-05-21T15:16:08.976028+0000","last_undegraded":"2024-05-21T15:16:08.976028+0000","last_fullsized":"2024-05-21T15:16:08.976028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:14:16.430371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979634+0000","last_change":"2024-05-21T15:14:11.577138+0000","last_active":"2024-05-21T15:16:08.979634+0000","last_peered":"2024-05-21T15:16:08.979634+0000","last_clean":"2024-05-21T15:16:08.979634+0000","last_became_active":"2024-05-21T15:14:11.576629+0000","last_became_peered":"2024-05-21T15:14:11.576629+0000","last_unstale":"2024-05-21T15:16:08.979634+0000","last_undegraded":"2024-05-21T15:16:08.979634+0000","last_fullsized":"2024-05-21T15:16:08.979634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:20:20.134764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976368+0000","last_change":"2024-05-21T15:14:12.256174+0000","last_active":"2024-05-21T15:16:08.976368+0000","last_peered":"2024-05-21T15:16:08.976368+0000","last_clean":"2024-05-21T15:16:08.976368+0000","last_became_active":"2024-05-21T15:14:12.255839+0000","last_became_peered":"2024-05-21T15:14:12.255839+0000","last_unstale":"2024-05-21T15:16:08.976368+0000","last_undegraded":"2024-05-21T15:16:08.976368+0000","last_fullsized":"2024-05-21T15:16:08.976368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:12:40.743118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705604+0000","last_change":"2024-05-21T15:14:12.257028+0000","last_active":"2024-05-21T15:16:08.705604+0000","last_peered":"2024-05-21T15:16:08.705604+0000","last_clean":"2024-05-21T15:16:08.705604+0000","last_became_active":"2024-05-21T15:14:12.256611+0000","last_became_peered":"2024-05-21T15:14:12.256611+0000","last_unstale":"2024-05-21T15:16:08.705604+0000","last_undegraded":"2024-05-21T15:16:08.705604+0000","last_fullsized":"2024-05-21T15:16:08.705604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:08:43.881073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976078+0000","last_change":"2024-05-21T15:14:12.255724+0000","last_active":"2024-05-21T15:16:08.976078+0000","last_peered":"2024-05-21T15:16:08.976078+0000","last_clean":"2024-05-21T15:16:08.976078+0000","last_became_active":"2024-05-21T15:14:12.255272+0000","last_became_peered":"2024-05-21T15:14:12.255272+0000","last_unstale":"2024-05-21T15:16:08.976078+0000","last_undegraded":"2024-05-21T15:16:08.976078+0000","last_fullsized":"2024-05-21T15:16:08.976078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:07:07.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976048+0000","last_change":"2024-05-21T15:14:12.262508+0000","last_active":"2024-05-21T15:16:08.976048+0000","last_peered":"2024-05-21T15:16:08.976048+0000","last_clean":"2024-05-21T15:16:08.976048+0000","last_became_active":"2024-05-21T15:14:12.262155+0000","last_became_peered":"2024-05-21T15:14:12.262155+0000","last_unstale":"2024-05-21T15:16:08.976048+0000","last_undegraded":"2024-05-21T15:16:08.976048+0000","last_fullsized":"2024-05-21T15:16:08.976048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:54:19.982044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983316+0000","last_change":"2024-05-21T15:14:12.251715+0000","last_active":"2024-05-21T15:16:08.983316+0000","last_peered":"2024-05-21T15:16:08.983316+0000","last_clean":"2024-05-21T15:16:08.983316+0000","last_became_active":"2024-05-21T15:14:12.251337+0000","last_became_peered":"2024-05-21T15:14:12.251337+0000","last_unstale":"2024-05-21T15:16:08.983316+0000","last_undegraded":"2024-05-21T15:16:08.983316+0000","last_fullsized":"2024-05-21T15:16:08.983316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:03.562179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979661+0000","last_change":"2024-05-21T15:14:12.253166+0000","last_active":"2024-05-21T15:16:08.979661+0000","last_peered":"2024-05-21T15:16:08.979661+0000","last_clean":"2024-05-21T15:16:08.979661+0000","last_became_active":"2024-05-21T15:14:12.251830+0000","last_became_peered":"2024-05-21T15:14:12.251830+0000","last_unstale":"2024-05-21T15:16:08.979661+0000","last_undegraded":"2024-05-21T15:16:08.979661+0000","last_fullsized":"2024-05-21T15:16:08.979661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:11:58.679568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705713+0000","last_change":"2024-05-21T15:14:12.256131+0000","last_active":"2024-05-21T15:16:08.705713+0000","last_peered":"2024-05-21T15:16:08.705713+0000","last_clean":"2024-05-21T15:16:08.705713+0000","last_became_active":"2024-05-21T15:14:12.255772+0000","last_became_peered":"2024-05-21T15:14:12.255772+0000","last_unstale":"2024-05-21T15:16:08.705713+0000","last_undegraded":"2024-05-21T15:16:08.705713+0000","last_fullsized":"2024-05-21T15:16:08.705713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:26:48.047278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976512+0000","last_change":"2024-05-21T15:14:12.260829+0000","last_active":"2024-05-21T15:16:08.976512+0000","last_peered":"2024-05-21T15:16:08.976512+0000","last_clean":"2024-05-21T15:16:08.976512+0000","last_became_active":"2024-05-21T15:14:12.259686+0000","last_became_peered":"2024-05-21T15:14:12.259686+0000","last_unstale":"2024-05-21T15:16:08.976512+0000","last_undegraded":"2024-05-21T15:16:08.976512+0000","last_fullsized":"2024-05-21T15:16:08.976512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:48.480814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809168+0000","last_change":"2024-05-21T15:14:12.262892+0000","last_active":"2024-05-21T15:16:08.809168+0000","last_peered":"2024-05-21T15:16:08.809168+0000","last_clean":"2024-05-21T15:16:08.809168+0000","last_became_active":"2024-05-21T15:14:12.262532+0000","last_became_peered":"2024-05-21T15:14:12.262532+0000","last_unstale":"2024-05-21T15:16:08.809168+0000","last_undegraded":"2024-05-21T15:16:08.809168+0000","last_fullsized":"2024-05-21T15:16:08.809168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:22.424783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.904921+0000","last_change":"2024-05-21T15:15:16.375027+0000","last_active":"2024-05-21T15:16:08.904921+0000","last_peered":"2024-05-21T15:16:08.904921+0000","last_clean":"2024-05-21T15:16:08.904921+0000","last_became_active":"2024-05-21T15:15:16.374542+0000","last_became_peered":"2024-05-21T15:15:16.374542+0000","last_unstale":"2024-05-21T15:16:08.904921+0000","last_undegraded":"2024-05-21T15:16:08.904921+0000","last_fullsized":"2024-05-21T15:16:08.904921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:55:14.399237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807833+0000","last_change":"2024-05-21T15:14:12.259845+0000","last_active":"2024-05-21T15:16:08.807833+0000","last_peered":"2024-05-21T15:16:08.807833+0000","last_clean":"2024-05-21T15:16:08.807833+0000","last_became_active":"2024-05-21T15:14:12.259455+0000","last_became_peered":"2024-05-21T15:14:12.259455+0000","last_unstale":"2024-05-21T15:16:08.807833+0000","last_undegraded":"2024-05-21T15:16:08.807833+0000","last_fullsized":"2024-05-21T15:16:08.807833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:24:59.655178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979108+0000","last_change":"2024-05-21T15:14:12.259713+0000","last_active":"2024-05-21T15:16:08.979108+0000","last_peered":"2024-05-21T15:16:08.979108+0000","last_clean":"2024-05-21T15:16:08.979108+0000","last_became_active":"2024-05-21T15:14:12.259057+0000","last_became_peered":"2024-05-21T15:14:12.259057+0000","last_unstale":"2024-05-21T15:16:08.979108+0000","last_undegraded":"2024-05-21T15:16:08.979108+0000","last_fullsized":"2024-05-21T15:16:08.979108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:46:22.377652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.980055+0000","last_change":"2024-05-21T15:14:12.254389+0000","last_active":"2024-05-21T15:16:08.980055+0000","last_peered":"2024-05-21T15:16:08.980055+0000","last_clean":"2024-05-21T15:16:08.980055+0000","last_became_active":"2024-05-21T15:14:12.253846+0000","last_became_peered":"2024-05-21T15:14:12.253846+0000","last_unstale":"2024-05-21T15:16:08.980055+0000","last_undegraded":"2024-05-21T15:16:08.980055+0000","last_fullsized":"2024-05-21T15:16:08.980055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:32.932316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705619+0000","last_change":"2024-05-21T15:14:12.252292+0000","last_active":"2024-05-21T15:16:08.705619+0000","last_peered":"2024-05-21T15:16:08.705619+0000","last_clean":"2024-05-21T15:16:08.705619+0000","last_became_active":"2024-05-21T15:14:12.251816+0000","last_became_peered":"2024-05-21T15:14:12.251816+0000","last_unstale":"2024-05-21T15:16:08.705619+0000","last_undegraded":"2024-05-21T15:16:08.705619+0000","last_fullsized":"2024-05-21T15:16:08.705619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:59:14.332010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600100,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1256,"num_read_kb":736,"num_write":376,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":672,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1800300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383716,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28356,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716796,"statfs":{"total":95995035648,"available":95965999104,"internally_reserved":0,"allocated":1368064,"data_stored":859573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28788,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716364,"statfs":{"total":95995035648,"available":95965556736,"internally_reserved":0,"allocated":1814528,"data_stored":1254662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1392640,"data_stored":859112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":465602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1576960,"data_stored":1050256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":466703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":533292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":457967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":534127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":456453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":466537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32936,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712216,"statfs":{"total":95995035648,"available":95961309184,"internally_reserved":0,"allocated":1740800,"data_stored":1190993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":199403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":394799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:16:10.185 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:16:10.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2868530546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 msgr2=0x7ff0ec109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2868530546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 0x7ff0ec109cf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009ca0 tx=0x7ff0e402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2868530546 shutdown_connections 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2868530546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 0x7ff0ec109cf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2868530546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ec102bd0 0x7ff0ec102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2868530546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0ec101990 0x7ff0ec102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2868530546 >> 172.21.15.78:0/2868530546 conn(0x7ff0ec0fcb80 msgr2=0x7ff0ec0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2868530546 shutdown_connections 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2868530546 wait complete. 2024-05-21T15:16:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 Processor -- start 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.331+0000 7ff0f26bc640 1 -- start start 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f26bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ec101990 0x7ff0ec0747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f26bc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0ec102bd0 0x7ff0ec19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f26bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 0x7ff0ec1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f26bc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec10d610 con 0x7ff0ec1078d0 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f26bc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec10d490 con 0x7ff0ec102bd0 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f26bc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec10d790 con 0x7ff0ec101990 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 0x7ff0ec1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 0x7ff0ec1a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46862/0 (socket says 172.21.15.78:46862) 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 -- 172.21.15.78:0/2415846176 learned_addr learned my addr 172.21.15.78:0/2415846176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0e3fff640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0ec102bd0 0x7ff0ec19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0ebfff640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ec101990 0x7ff0ec0747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 -- 172.21.15.78:0/2415846176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ec101990 msgr2=0x7ff0ec0747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ec101990 0x7ff0ec0747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 -- 172.21.15.78:0/2415846176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0ec102bd0 msgr2=0x7ff0ec19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0ec102bd0 0x7ff0ec19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 -- 172.21.15.78:0/2415846176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec1a0dd0 con 0x7ff0ec1078d0 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0e3fff640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0ec102bd0 0x7ff0ec19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f0c59640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 0x7ff0ec1a05b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005e30 tx=0x7ff0e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0e9ffb640 1 -- 172.21.15.78:0/2415846176 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4003b70 con 0x7ff0ec1078d0 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0e9ffb640 1 -- 172.21.15.78:0/2415846176 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e4003d10 con 0x7ff0ec1078d0 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0e9ffb640 1 -- 172.21.15.78:0/2415846176 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e402fad0 con 0x7ff0ec1078d0 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.332+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec1a1060 con 0x7ff0ec1078d0 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.333+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec1a15b0 con 0x7ff0ec1078d0 2024-05-21T15:16:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.334+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec1066e0 con 0x7ff0ec1078d0 2024-05-21T15:16:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.335+0000 7ff0e9ffb640 1 -- 172.21.15.78:0/2415846176 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7ff0e402fc70 con 0x7ff0ec1078d0 2024-05-21T15:16:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.335+0000 7ff0e9ffb640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0d80a2990 0x7ff0d80a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.336+0000 7ff0ebfff640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0d80a2990 0x7ff0d80a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.336+0000 7ff0e9ffb640 1 -- 172.21.15.78:0/2415846176 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff0e40eff70 con 0x7ff0ec1078d0 2024-05-21T15:16:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.336+0000 7ff0ebfff640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0d80a2990 0x7ff0d80a4ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec102590 tx=0x7ff0dc005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.339+0000 7ff0e9ffb640 1 -- 172.21.15.78:0/2415846176 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff0e40bd0a0 con 0x7ff0ec1078d0 2024-05-21T15:16:10.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.514+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0ec061f90 con 0x7ff0d80a2990 2024-05-21T15:16:10.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.521+0000 7ff0e9ffb640 1 -- 172.21.15.78:0/2415846176 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257782 (secure 0 0 0) 0x7ff0ec061f90 con 0x7ff0d80a2990 2024-05-21T15:16:10.526 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0d80a2990 msgr2=0x7ff0d80a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0d80a2990 0x7ff0d80a4ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec102590 tx=0x7ff0dc005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 msgr2=0x7ff0ec1a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 0x7ff0ec1a05b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005e30 tx=0x7ff0e4004ab0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 shutdown_connections 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0ec102bd0 0x7ff0ec19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0d80a2990 0x7ff0d80a4ea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0ec1078d0 0x7ff0ec1a05b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 --2- 172.21.15.78:0/2415846176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ec101990 0x7ff0ec0747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 >> 172.21.15.78:0/2415846176 conn(0x7ff0ec0fcb80 msgr2=0x7ff0ec1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 shutdown_connections 2024-05-21T15:16:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.524+0000 7ff0f26bc640 1 -- 172.21.15.78:0/2415846176 wait complete. 2024-05-21T15:16:10.539 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":3,"stamp":"2024-05-21T15:16:09.730596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190702,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1392,"num_read_kb":855,"num_write":559,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181408,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359672,"kb_used_data":14592,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582152,"statfs":{"total":1151940427776,"available":1151572123648,"internally_reserved":0,"allocated":14942208,"data_stored":8595277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705906+0000","last_change":"2024-05-21T15:14:12.254239+0000","last_active":"2024-05-21T15:16:08.705906+0000","last_peered":"2024-05-21T15:16:08.705906+0000","last_clean":"2024-05-21T15:16:08.705906+0000","last_became_active":"2024-05-21T15:14:12.253885+0000","last_became_peered":"2024-05-21T15:14:12.253885+0000","last_unstale":"2024-05-21T15:16:08.705906+0000","last_undegraded":"2024-05-21T15:16:08.705906+0000","last_fullsized":"2024-05-21T15:16:08.705906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:35.126836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983550+0000","last_change":"2024-05-21T15:14:12.262984+0000","last_active":"2024-05-21T15:16:08.983550+0000","last_peered":"2024-05-21T15:16:08.983550+0000","last_clean":"2024-05-21T15:16:08.983550+0000","last_became_active":"2024-05-21T15:14:12.262667+0000","last_became_peered":"2024-05-21T15:14:12.262667+0000","last_unstale":"2024-05-21T15:16:08.983550+0000","last_undegraded":"2024-05-21T15:16:08.983550+0000","last_fullsized":"2024-05-21T15:16:08.983550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:47:47.434314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983456+0000","last_change":"2024-05-21T15:14:12.260573+0000","last_active":"2024-05-21T15:16:08.983456+0000","last_peered":"2024-05-21T15:16:08.983456+0000","last_clean":"2024-05-21T15:16:08.983456+0000","last_became_active":"2024-05-21T15:14:12.259073+0000","last_became_peered":"2024-05-21T15:14:12.259073+0000","last_unstale":"2024-05-21T15:16:08.983456+0000","last_undegraded":"2024-05-21T15:16:08.983456+0000","last_fullsized":"2024-05-21T15:16:08.983456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:15:01.919464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972349+0000","last_change":"2024-05-21T15:14:12.256992+0000","last_active":"2024-05-21T15:16:08.972349+0000","last_peered":"2024-05-21T15:16:08.972349+0000","last_clean":"2024-05-21T15:16:08.972349+0000","last_became_active":"2024-05-21T15:14:12.256448+0000","last_became_peered":"2024-05-21T15:14:12.256448+0000","last_unstale":"2024-05-21T15:16:08.972349+0000","last_undegraded":"2024-05-21T15:16:08.972349+0000","last_fullsized":"2024-05-21T15:16:08.972349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:40.569263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905144+0000","last_change":"2024-05-21T15:14:12.251533+0000","last_active":"2024-05-21T15:16:08.905144+0000","last_peered":"2024-05-21T15:16:08.905144+0000","last_clean":"2024-05-21T15:16:08.905144+0000","last_became_active":"2024-05-21T15:14:12.251210+0000","last_became_peered":"2024-05-21T15:14:12.251210+0000","last_unstale":"2024-05-21T15:16:08.905144+0000","last_undegraded":"2024-05-21T15:16:08.905144+0000","last_fullsized":"2024-05-21T15:16:08.905144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:43:34.109179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705582+0000","last_change":"2024-05-21T15:14:12.251767+0000","last_active":"2024-05-21T15:16:08.705582+0000","last_peered":"2024-05-21T15:16:08.705582+0000","last_clean":"2024-05-21T15:16:08.705582+0000","last_became_active":"2024-05-21T15:14:12.251427+0000","last_became_peered":"2024-05-21T15:14:12.251427+0000","last_unstale":"2024-05-21T15:16:08.705582+0000","last_undegraded":"2024-05-21T15:16:08.705582+0000","last_fullsized":"2024-05-21T15:16:08.705582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:48:41.804074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975895+0000","last_change":"2024-05-21T15:14:12.253472+0000","last_active":"2024-05-21T15:16:08.975895+0000","last_peered":"2024-05-21T15:16:08.975895+0000","last_clean":"2024-05-21T15:16:08.975895+0000","last_became_active":"2024-05-21T15:14:12.252762+0000","last_became_peered":"2024-05-21T15:14:12.252762+0000","last_unstale":"2024-05-21T15:16:08.975895+0000","last_undegraded":"2024-05-21T15:16:08.975895+0000","last_fullsized":"2024-05-21T15:16:08.975895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:11:44.497404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807382+0000","last_change":"2024-05-21T15:14:12.258627+0000","last_active":"2024-05-21T15:16:08.807382+0000","last_peered":"2024-05-21T15:16:08.807382+0000","last_clean":"2024-05-21T15:16:08.807382+0000","last_became_active":"2024-05-21T15:14:12.258118+0000","last_became_peered":"2024-05-21T15:14:12.258118+0000","last_unstale":"2024-05-21T15:16:08.807382+0000","last_undegraded":"2024-05-21T15:16:08.807382+0000","last_fullsized":"2024-05-21T15:16:08.807382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:56.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979322+0000","last_change":"2024-05-21T15:14:12.255998+0000","last_active":"2024-05-21T15:16:08.979322+0000","last_peered":"2024-05-21T15:16:08.979322+0000","last_clean":"2024-05-21T15:16:08.979322+0000","last_became_active":"2024-05-21T15:14:12.255658+0000","last_became_peered":"2024-05-21T15:14:12.255658+0000","last_unstale":"2024-05-21T15:16:08.979322+0000","last_undegraded":"2024-05-21T15:16:08.979322+0000","last_fullsized":"2024-05-21T15:16:08.979322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:44:42.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.706089+0000","last_change":"2024-05-21T15:14:12.257617+0000","last_active":"2024-05-21T15:16:08.706089+0000","last_peered":"2024-05-21T15:16:08.706089+0000","last_clean":"2024-05-21T15:16:08.706089+0000","last_became_active":"2024-05-21T15:14:12.256947+0000","last_became_peered":"2024-05-21T15:14:12.256947+0000","last_unstale":"2024-05-21T15:16:08.706089+0000","last_undegraded":"2024-05-21T15:16:08.706089+0000","last_fullsized":"2024-05-21T15:16:08.706089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:45:58.174954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976285+0000","last_change":"2024-05-21T15:14:12.260454+0000","last_active":"2024-05-21T15:16:08.976285+0000","last_peered":"2024-05-21T15:16:08.976285+0000","last_clean":"2024-05-21T15:16:08.976285+0000","last_became_active":"2024-05-21T15:14:12.260126+0000","last_became_peered":"2024-05-21T15:14:12.260126+0000","last_unstale":"2024-05-21T15:16:08.976285+0000","last_undegraded":"2024-05-21T15:16:08.976285+0000","last_fullsized":"2024-05-21T15:16:08.976285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:49:45.212050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972342+0000","last_change":"2024-05-21T15:14:12.252071+0000","last_active":"2024-05-21T15:16:08.972342+0000","last_peered":"2024-05-21T15:16:08.972342+0000","last_clean":"2024-05-21T15:16:08.972342+0000","last_became_active":"2024-05-21T15:14:12.251706+0000","last_became_peered":"2024-05-21T15:14:12.251706+0000","last_unstale":"2024-05-21T15:16:08.972342+0000","last_undegraded":"2024-05-21T15:16:08.972342+0000","last_fullsized":"2024-05-21T15:16:08.972342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:02:44.349368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979900+0000","last_change":"2024-05-21T15:14:12.239877+0000","last_active":"2024-05-21T15:16:08.979900+0000","last_peered":"2024-05-21T15:16:08.979900+0000","last_clean":"2024-05-21T15:16:08.979900+0000","last_became_active":"2024-05-21T15:14:12.239615+0000","last_became_peered":"2024-05-21T15:14:12.239615+0000","last_unstale":"2024-05-21T15:16:08.979900+0000","last_undegraded":"2024-05-21T15:16:08.979900+0000","last_fullsized":"2024-05-21T15:16:08.979900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:23.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983191+0000","last_change":"2024-05-21T15:14:12.260349+0000","last_active":"2024-05-21T15:16:08.983191+0000","last_peered":"2024-05-21T15:16:08.983191+0000","last_clean":"2024-05-21T15:16:08.983191+0000","last_became_active":"2024-05-21T15:14:12.259658+0000","last_became_peered":"2024-05-21T15:14:12.259658+0000","last_unstale":"2024-05-21T15:16:08.983191+0000","last_undegraded":"2024-05-21T15:16:08.983191+0000","last_fullsized":"2024-05-21T15:16:08.983191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:59:32.326571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983078+0000","last_change":"2024-05-21T15:15:17.015535+0000","last_active":"2024-05-21T15:16:08.983078+0000","last_peered":"2024-05-21T15:16:08.983078+0000","last_clean":"2024-05-21T15:16:08.983078+0000","last_became_active":"2024-05-21T15:15:17.015057+0000","last_became_peered":"2024-05-21T15:15:17.015057+0000","last_unstale":"2024-05-21T15:16:08.983078+0000","last_undegraded":"2024-05-21T15:16:08.983078+0000","last_fullsized":"2024-05-21T15:16:08.983078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:48:14.652870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.978891+0000","last_change":"2024-05-21T15:14:12.264667+0000","last_active":"2024-05-21T15:16:08.978891+0000","last_peered":"2024-05-21T15:16:08.978891+0000","last_clean":"2024-05-21T15:16:08.978891+0000","last_became_active":"2024-05-21T15:14:12.264284+0000","last_became_peered":"2024-05-21T15:14:12.264284+0000","last_unstale":"2024-05-21T15:16:08.978891+0000","last_undegraded":"2024-05-21T15:16:08.978891+0000","last_fullsized":"2024-05-21T15:16:08.978891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:24.861321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705966+0000","last_change":"2024-05-21T15:14:12.259912+0000","last_active":"2024-05-21T15:16:08.705966+0000","last_peered":"2024-05-21T15:16:08.705966+0000","last_clean":"2024-05-21T15:16:08.705966+0000","last_became_active":"2024-05-21T15:14:12.259578+0000","last_became_peered":"2024-05-21T15:14:12.259578+0000","last_unstale":"2024-05-21T15:16:08.705966+0000","last_undegraded":"2024-05-21T15:16:08.705966+0000","last_fullsized":"2024-05-21T15:16:08.705966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:05:33.372481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704201+0000","last_change":"2024-05-21T15:14:12.258355+0000","last_active":"2024-05-21T15:16:08.704201+0000","last_peered":"2024-05-21T15:16:08.704201+0000","last_clean":"2024-05-21T15:16:08.704201+0000","last_became_active":"2024-05-21T15:14:12.257991+0000","last_became_peered":"2024-05-21T15:14:12.257991+0000","last_unstale":"2024-05-21T15:16:08.704201+0000","last_undegraded":"2024-05-21T15:16:08.704201+0000","last_fullsized":"2024-05-21T15:16:08.704201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:54.014553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976238+0000","last_change":"2024-05-21T15:14:12.258873+0000","last_active":"2024-05-21T15:16:08.976238+0000","last_peered":"2024-05-21T15:16:08.976238+0000","last_clean":"2024-05-21T15:16:08.976238+0000","last_became_active":"2024-05-21T15:14:12.258548+0000","last_became_peered":"2024-05-21T15:14:12.258548+0000","last_unstale":"2024-05-21T15:16:08.976238+0000","last_undegraded":"2024-05-21T15:16:08.976238+0000","last_fullsized":"2024-05-21T15:16:08.976238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:00.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":167,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704409+0000","last_change":"2024-05-21T15:15:16.626507+0000","last_active":"2024-05-21T15:16:08.704409+0000","last_peered":"2024-05-21T15:16:08.704409+0000","last_clean":"2024-05-21T15:16:08.704409+0000","last_became_active":"2024-05-21T15:15:16.615945+0000","last_became_peered":"2024-05-21T15:15:16.615945+0000","last_unstale":"2024-05-21T15:16:08.704409+0000","last_undegraded":"2024-05-21T15:16:08.704409+0000","last_fullsized":"2024-05-21T15:16:08.704409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:16:01.229449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808883+0000","last_change":"2024-05-21T15:14:12.261632+0000","last_active":"2024-05-21T15:16:08.808883+0000","last_peered":"2024-05-21T15:16:08.808883+0000","last_clean":"2024-05-21T15:16:08.808883+0000","last_became_active":"2024-05-21T15:14:12.261149+0000","last_became_peered":"2024-05-21T15:14:12.261149+0000","last_unstale":"2024-05-21T15:16:08.808883+0000","last_undegraded":"2024-05-21T15:16:08.808883+0000","last_fullsized":"2024-05-21T15:16:08.808883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:11.349395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809030+0000","last_change":"2024-05-21T15:14:11.567859+0000","last_active":"2024-05-21T15:16:08.809030+0000","last_peered":"2024-05-21T15:16:08.809030+0000","last_clean":"2024-05-21T15:16:08.809030+0000","last_became_active":"2024-05-21T15:14:11.566855+0000","last_became_peered":"2024-05-21T15:14:11.566855+0000","last_unstale":"2024-05-21T15:16:08.809030+0000","last_undegraded":"2024-05-21T15:16:08.809030+0000","last_fullsized":"2024-05-21T15:16:08.809030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:21.568436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983569+0000","last_change":"2024-05-21T15:14:12.262153+0000","last_active":"2024-05-21T15:16:08.983569+0000","last_peered":"2024-05-21T15:16:08.983569+0000","last_clean":"2024-05-21T15:16:08.983569+0000","last_became_active":"2024-05-21T15:14:12.261780+0000","last_became_peered":"2024-05-21T15:14:12.261780+0000","last_unstale":"2024-05-21T15:16:08.983569+0000","last_undegraded":"2024-05-21T15:16:08.983569+0000","last_fullsized":"2024-05-21T15:16:08.983569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:21:25.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'14","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905483+0000","last_change":"2024-05-21T15:14:11.570316+0000","last_active":"2024-05-21T15:16:08.905483+0000","last_peered":"2024-05-21T15:16:08.905483+0000","last_clean":"2024-05-21T15:16:08.905483+0000","last_became_active":"2024-05-21T15:14:11.569967+0000","last_became_peered":"2024-05-21T15:14:11.569967+0000","last_unstale":"2024-05-21T15:16:08.905483+0000","last_undegraded":"2024-05-21T15:16:08.905483+0000","last_fullsized":"2024-05-21T15:16:08.905483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:30:27.364742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":20,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979138+0000","last_change":"2024-05-21T15:14:12.261716+0000","last_active":"2024-05-21T15:16:08.979138+0000","last_peered":"2024-05-21T15:16:08.979138+0000","last_clean":"2024-05-21T15:16:08.979138+0000","last_became_active":"2024-05-21T15:14:12.261341+0000","last_became_peered":"2024-05-21T15:14:12.261341+0000","last_unstale":"2024-05-21T15:16:08.979138+0000","last_undegraded":"2024-05-21T15:16:08.979138+0000","last_fullsized":"2024-05-21T15:16:08.979138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:40:10.464974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905502+0000","last_change":"2024-05-21T15:14:11.573305+0000","last_active":"2024-05-21T15:16:08.905502+0000","last_peered":"2024-05-21T15:16:08.905502+0000","last_clean":"2024-05-21T15:16:08.905502+0000","last_became_active":"2024-05-21T15:14:11.572655+0000","last_became_peered":"2024-05-21T15:14:11.572655+0000","last_unstale":"2024-05-21T15:16:08.905502+0000","last_undegraded":"2024-05-21T15:16:08.905502+0000","last_fullsized":"2024-05-21T15:16:08.905502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:37.268440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815691+0000","last_change":"2024-05-21T15:14:12.253788+0000","last_active":"2024-05-21T15:16:08.815691+0000","last_peered":"2024-05-21T15:16:08.815691+0000","last_clean":"2024-05-21T15:16:08.815691+0000","last_became_active":"2024-05-21T15:14:12.253450+0000","last_became_peered":"2024-05-21T15:14:12.253450+0000","last_unstale":"2024-05-21T15:16:08.815691+0000","last_undegraded":"2024-05-21T15:16:08.815691+0000","last_fullsized":"2024-05-21T15:16:08.815691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:57.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":149,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430893+0000","last_change":"2024-05-21T15:14:11.567400+0000","last_active":"2024-05-21T15:16:09.430893+0000","last_peered":"2024-05-21T15:16:09.430893+0000","last_clean":"2024-05-21T15:16:09.430893+0000","last_became_active":"2024-05-21T15:14:11.565995+0000","last_became_peered":"2024-05-21T15:14:11.565995+0000","last_unstale":"2024-05-21T15:16:09.430893+0000","last_undegraded":"2024-05-21T15:16:09.430893+0000","last_fullsized":"2024-05-21T15:16:09.430893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:50.463339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905055+0000","last_change":"2024-05-21T15:14:12.251565+0000","last_active":"2024-05-21T15:16:08.905055+0000","last_peered":"2024-05-21T15:16:08.905055+0000","last_clean":"2024-05-21T15:16:08.905055+0000","last_became_active":"2024-05-21T15:14:12.251289+0000","last_became_peered":"2024-05-21T15:14:12.251289+0000","last_unstale":"2024-05-21T15:16:08.905055+0000","last_undegraded":"2024-05-21T15:16:08.905055+0000","last_fullsized":"2024-05-21T15:16:08.905055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:44.619800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705549+0000","last_change":"2024-05-21T15:14:11.575193+0000","last_active":"2024-05-21T15:16:08.705549+0000","last_peered":"2024-05-21T15:16:08.705549+0000","last_clean":"2024-05-21T15:16:08.705549+0000","last_became_active":"2024-05-21T15:14:11.574861+0000","last_became_peered":"2024-05-21T15:14:11.574861+0000","last_unstale":"2024-05-21T15:16:08.705549+0000","last_undegraded":"2024-05-21T15:16:08.705549+0000","last_fullsized":"2024-05-21T15:16:08.705549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:54.548408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815610+0000","last_change":"2024-05-21T15:15:17.015620+0000","last_active":"2024-05-21T15:16:08.815610+0000","last_peered":"2024-05-21T15:16:08.815610+0000","last_clean":"2024-05-21T15:16:08.815610+0000","last_became_active":"2024-05-21T15:15:17.015086+0000","last_became_peered":"2024-05-21T15:15:17.015086+0000","last_unstale":"2024-05-21T15:16:08.815610+0000","last_undegraded":"2024-05-21T15:16:08.815610+0000","last_fullsized":"2024-05-21T15:16:08.815610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:10.856246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979503+0000","last_change":"2024-05-21T15:14:11.572087+0000","last_active":"2024-05-21T15:16:08.979503+0000","last_peered":"2024-05-21T15:16:08.979503+0000","last_clean":"2024-05-21T15:16:08.979503+0000","last_became_active":"2024-05-21T15:14:11.571716+0000","last_became_peered":"2024-05-21T15:14:11.571716+0000","last_unstale":"2024-05-21T15:16:08.979503+0000","last_undegraded":"2024-05-21T15:16:08.979503+0000","last_fullsized":"2024-05-21T15:16:08.979503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:07.082766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807382+0000","last_change":"2024-05-21T15:14:12.252233+0000","last_active":"2024-05-21T15:16:08.807382+0000","last_peered":"2024-05-21T15:16:08.807382+0000","last_clean":"2024-05-21T15:16:08.807382+0000","last_became_active":"2024-05-21T15:14:12.251901+0000","last_became_peered":"2024-05-21T15:14:12.251901+0000","last_unstale":"2024-05-21T15:16:08.807382+0000","last_undegraded":"2024-05-21T15:16:08.807382+0000","last_fullsized":"2024-05-21T15:16:08.807382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:05.107436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815255+0000","last_change":"2024-05-21T15:14:11.565946+0000","last_active":"2024-05-21T15:16:08.815255+0000","last_peered":"2024-05-21T15:16:08.815255+0000","last_clean":"2024-05-21T15:16:08.815255+0000","last_became_active":"2024-05-21T15:14:11.565601+0000","last_became_peered":"2024-05-21T15:14:11.565601+0000","last_unstale":"2024-05-21T15:16:08.815255+0000","last_undegraded":"2024-05-21T15:16:08.815255+0000","last_fullsized":"2024-05-21T15:16:08.815255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:41:55.243473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704086+0000","last_change":"2024-05-21T15:14:12.255188+0000","last_active":"2024-05-21T15:16:08.704086+0000","last_peered":"2024-05-21T15:16:08.704086+0000","last_clean":"2024-05-21T15:16:08.704086+0000","last_became_active":"2024-05-21T15:14:12.254503+0000","last_became_peered":"2024-05-21T15:14:12.254503+0000","last_unstale":"2024-05-21T15:16:08.704086+0000","last_undegraded":"2024-05-21T15:16:08.704086+0000","last_fullsized":"2024-05-21T15:16:08.704086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:22:16.336740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":254,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430238+0000","last_change":"2024-05-21T15:14:11.569882+0000","last_active":"2024-05-21T15:16:09.430238+0000","last_peered":"2024-05-21T15:16:09.430238+0000","last_clean":"2024-05-21T15:16:09.430238+0000","last_became_active":"2024-05-21T15:14:11.569532+0000","last_became_peered":"2024-05-21T15:14:11.569532+0000","last_unstale":"2024-05-21T15:16:09.430238+0000","last_undegraded":"2024-05-21T15:16:09.430238+0000","last_fullsized":"2024-05-21T15:16:09.430238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:58:50.081709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979327+0000","last_change":"2024-05-21T15:14:12.262691+0000","last_active":"2024-05-21T15:16:08.979327+0000","last_peered":"2024-05-21T15:16:08.979327+0000","last_clean":"2024-05-21T15:16:08.979327+0000","last_became_active":"2024-05-21T15:14:12.261492+0000","last_became_peered":"2024-05-21T15:14:12.261492+0000","last_unstale":"2024-05-21T15:16:08.979327+0000","last_undegraded":"2024-05-21T15:16:08.979327+0000","last_fullsized":"2024-05-21T15:16:08.979327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:35:00.121256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976229+0000","last_change":"2024-05-21T15:14:11.569231+0000","last_active":"2024-05-21T15:16:08.976229+0000","last_peered":"2024-05-21T15:16:08.976229+0000","last_clean":"2024-05-21T15:16:08.976229+0000","last_became_active":"2024-05-21T15:14:11.568916+0000","last_became_peered":"2024-05-21T15:14:11.568916+0000","last_unstale":"2024-05-21T15:16:08.976229+0000","last_undegraded":"2024-05-21T15:16:08.976229+0000","last_fullsized":"2024-05-21T15:16:08.976229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:26:13.411854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983497+0000","last_change":"2024-05-21T15:14:12.256307+0000","last_active":"2024-05-21T15:16:08.983497+0000","last_peered":"2024-05-21T15:16:08.983497+0000","last_clean":"2024-05-21T15:16:08.983497+0000","last_became_active":"2024-05-21T15:14:12.255828+0000","last_became_peered":"2024-05-21T15:14:12.255828+0000","last_unstale":"2024-05-21T15:16:08.983497+0000","last_undegraded":"2024-05-21T15:16:08.983497+0000","last_fullsized":"2024-05-21T15:16:08.983497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:11:40.068779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815696+0000","last_change":"2024-05-21T15:14:11.568239+0000","last_active":"2024-05-21T15:16:08.815696+0000","last_peered":"2024-05-21T15:16:08.815696+0000","last_clean":"2024-05-21T15:16:08.815696+0000","last_became_active":"2024-05-21T15:14:11.567836+0000","last_became_peered":"2024-05-21T15:14:11.567836+0000","last_unstale":"2024-05-21T15:16:08.815696+0000","last_undegraded":"2024-05-21T15:16:08.815696+0000","last_fullsized":"2024-05-21T15:16:08.815696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:10.244172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979346+0000","last_change":"2024-05-21T15:14:12.257762+0000","last_active":"2024-05-21T15:16:08.979346+0000","last_peered":"2024-05-21T15:16:08.979346+0000","last_clean":"2024-05-21T15:16:08.979346+0000","last_became_active":"2024-05-21T15:14:12.256335+0000","last_became_peered":"2024-05-21T15:14:12.256335+0000","last_unstale":"2024-05-21T15:16:08.979346+0000","last_undegraded":"2024-05-21T15:16:08.979346+0000","last_fullsized":"2024-05-21T15:16:08.979346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:35.219673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":300,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.460777+0000","last_change":"2024-05-21T15:14:11.564407+0000","last_active":"2024-05-21T15:16:09.460777+0000","last_peered":"2024-05-21T15:16:09.460777+0000","last_clean":"2024-05-21T15:16:09.460777+0000","last_became_active":"2024-05-21T15:14:11.564012+0000","last_became_peered":"2024-05-21T15:14:11.564012+0000","last_unstale":"2024-05-21T15:16:09.460777+0000","last_undegraded":"2024-05-21T15:16:09.460777+0000","last_fullsized":"2024-05-21T15:16:09.460777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:34:54.177844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65676,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":184,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905245+0000","last_change":"2024-05-21T15:14:12.253820+0000","last_active":"2024-05-21T15:16:08.905245+0000","last_peered":"2024-05-21T15:16:08.905245+0000","last_clean":"2024-05-21T15:16:08.905245+0000","last_became_active":"2024-05-21T15:14:12.253360+0000","last_became_peered":"2024-05-21T15:14:12.253360+0000","last_unstale":"2024-05-21T15:16:08.905245+0000","last_undegraded":"2024-05-21T15:16:08.905245+0000","last_fullsized":"2024-05-21T15:16:08.905245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:38:19.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976498+0000","last_change":"2024-05-21T15:14:11.571161+0000","last_active":"2024-05-21T15:16:08.976498+0000","last_peered":"2024-05-21T15:16:08.976498+0000","last_clean":"2024-05-21T15:16:08.976498+0000","last_became_active":"2024-05-21T15:14:11.570837+0000","last_became_peered":"2024-05-21T15:14:11.570837+0000","last_unstale":"2024-05-21T15:16:08.976498+0000","last_undegraded":"2024-05-21T15:16:08.976498+0000","last_fullsized":"2024-05-21T15:16:08.976498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:24:55.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979900+0000","last_change":"2024-05-21T15:14:12.255713+0000","last_active":"2024-05-21T15:16:08.979900+0000","last_peered":"2024-05-21T15:16:08.979900+0000","last_clean":"2024-05-21T15:16:08.979900+0000","last_became_active":"2024-05-21T15:14:12.254458+0000","last_became_peered":"2024-05-21T15:14:12.254458+0000","last_unstale":"2024-05-21T15:16:08.979900+0000","last_undegraded":"2024-05-21T15:16:08.979900+0000","last_fullsized":"2024-05-21T15:16:08.979900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:55:24.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815443+0000","last_change":"2024-05-21T15:14:11.565916+0000","last_active":"2024-05-21T15:16:08.815443+0000","last_peered":"2024-05-21T15:16:08.815443+0000","last_clean":"2024-05-21T15:16:08.815443+0000","last_became_active":"2024-05-21T15:14:11.565460+0000","last_became_peered":"2024-05-21T15:14:11.565460+0000","last_unstale":"2024-05-21T15:16:08.815443+0000","last_undegraded":"2024-05-21T15:16:08.815443+0000","last_fullsized":"2024-05-21T15:16:08.815443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:09.982208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976147+0000","last_change":"2024-05-21T15:14:12.251856+0000","last_active":"2024-05-21T15:16:08.976147+0000","last_peered":"2024-05-21T15:16:08.976147+0000","last_clean":"2024-05-21T15:16:08.976147+0000","last_became_active":"2024-05-21T15:14:12.251496+0000","last_became_peered":"2024-05-21T15:14:12.251496+0000","last_unstale":"2024-05-21T15:16:08.976147+0000","last_undegraded":"2024-05-21T15:16:08.976147+0000","last_fullsized":"2024-05-21T15:16:08.976147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:17.566276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430144+0000","last_change":"2024-05-21T15:14:11.565485+0000","last_active":"2024-05-21T15:16:09.430144+0000","last_peered":"2024-05-21T15:16:09.430144+0000","last_clean":"2024-05-21T15:16:09.430144+0000","last_became_active":"2024-05-21T15:14:11.565092+0000","last_became_peered":"2024-05-21T15:14:11.565092+0000","last_unstale":"2024-05-21T15:16:09.430144+0000","last_undegraded":"2024-05-21T15:16:09.430144+0000","last_fullsized":"2024-05-21T15:16:09.430144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:01:16.101793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975843+0000","last_change":"2024-05-21T15:14:12.262283+0000","last_active":"2024-05-21T15:16:08.975843+0000","last_peered":"2024-05-21T15:16:08.975843+0000","last_clean":"2024-05-21T15:16:08.975843+0000","last_became_active":"2024-05-21T15:14:12.261610+0000","last_became_peered":"2024-05-21T15:14:12.261610+0000","last_unstale":"2024-05-21T15:16:08.975843+0000","last_undegraded":"2024-05-21T15:16:08.975843+0000","last_fullsized":"2024-05-21T15:16:08.975843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:44:33.087115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.904945+0000","last_change":"2024-05-21T15:14:11.569905+0000","last_active":"2024-05-21T15:16:08.904945+0000","last_peered":"2024-05-21T15:16:08.904945+0000","last_clean":"2024-05-21T15:16:08.904945+0000","last_became_active":"2024-05-21T15:14:11.569221+0000","last_became_peered":"2024-05-21T15:14:11.569221+0000","last_unstale":"2024-05-21T15:16:08.904945+0000","last_undegraded":"2024-05-21T15:16:08.904945+0000","last_fullsized":"2024-05-21T15:16:08.904945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:11:04.875890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815306+0000","last_change":"2024-05-21T15:15:16.386384+0000","last_active":"2024-05-21T15:16:08.815306+0000","last_peered":"2024-05-21T15:16:08.815306+0000","last_clean":"2024-05-21T15:16:08.815306+0000","last_became_active":"2024-05-21T15:15:16.385810+0000","last_became_peered":"2024-05-21T15:15:16.385810+0000","last_unstale":"2024-05-21T15:16:08.815306+0000","last_undegraded":"2024-05-21T15:16:08.815306+0000","last_fullsized":"2024-05-21T15:16:08.815306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:49:45.031393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905410+0000","last_change":"2024-05-21T15:14:11.569670+0000","last_active":"2024-05-21T15:16:08.905410+0000","last_peered":"2024-05-21T15:16:08.905410+0000","last_clean":"2024-05-21T15:16:08.905410+0000","last_became_active":"2024-05-21T15:14:11.569257+0000","last_became_peered":"2024-05-21T15:14:11.569257+0000","last_unstale":"2024-05-21T15:16:08.905410+0000","last_undegraded":"2024-05-21T15:16:08.905410+0000","last_fullsized":"2024-05-21T15:16:08.905410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:18.584920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705771+0000","last_change":"2024-05-21T15:15:16.374845+0000","last_active":"2024-05-21T15:16:08.705771+0000","last_peered":"2024-05-21T15:16:08.705771+0000","last_clean":"2024-05-21T15:16:08.705771+0000","last_became_active":"2024-05-21T15:15:16.374393+0000","last_became_peered":"2024-05-21T15:15:16.374393+0000","last_unstale":"2024-05-21T15:16:08.705771+0000","last_undegraded":"2024-05-21T15:16:08.705771+0000","last_fullsized":"2024-05-21T15:16:08.705771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:38:21.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704254+0000","last_change":"2024-05-21T15:14:11.567849+0000","last_active":"2024-05-21T15:16:08.704254+0000","last_peered":"2024-05-21T15:16:08.704254+0000","last_clean":"2024-05-21T15:16:08.704254+0000","last_became_active":"2024-05-21T15:14:11.567476+0000","last_became_peered":"2024-05-21T15:14:11.567476+0000","last_unstale":"2024-05-21T15:16:08.704254+0000","last_undegraded":"2024-05-21T15:16:08.704254+0000","last_fullsized":"2024-05-21T15:16:08.704254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:13.516765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705800+0000","last_change":"2024-05-21T15:14:12.255165+0000","last_active":"2024-05-21T15:16:08.705800+0000","last_peered":"2024-05-21T15:16:08.705800+0000","last_clean":"2024-05-21T15:16:08.705800+0000","last_became_active":"2024-05-21T15:14:12.254743+0000","last_became_peered":"2024-05-21T15:14:12.254743+0000","last_unstale":"2024-05-21T15:16:08.705800+0000","last_undegraded":"2024-05-21T15:16:08.705800+0000","last_fullsized":"2024-05-21T15:16:08.705800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:53:11.059597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'8","reported_seq":143,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.690138+0000","last_change":"2024-05-21T15:14:11.569745+0000","last_active":"2024-05-21T15:16:09.690138+0000","last_peered":"2024-05-21T15:16:09.690138+0000","last_clean":"2024-05-21T15:16:09.690138+0000","last_became_active":"2024-05-21T15:14:11.569409+0000","last_became_peered":"2024-05-21T15:14:11.569409+0000","last_unstale":"2024-05-21T15:16:09.690138+0000","last_undegraded":"2024-05-21T15:16:09.690138+0000","last_fullsized":"2024-05-21T15:16:09.690138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:00:37.963797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1678,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705802+0000","last_change":"2024-05-21T15:14:12.254631+0000","last_active":"2024-05-21T15:16:08.705802+0000","last_peered":"2024-05-21T15:16:08.705802+0000","last_clean":"2024-05-21T15:16:08.705802+0000","last_became_active":"2024-05-21T15:14:12.254239+0000","last_became_peered":"2024-05-21T15:14:12.254239+0000","last_unstale":"2024-05-21T15:16:08.705802+0000","last_undegraded":"2024-05-21T15:16:08.705802+0000","last_fullsized":"2024-05-21T15:16:08.705802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:55:17.135550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.703987+0000","last_change":"2024-05-21T15:15:16.615801+0000","last_active":"2024-05-21T15:16:08.703987+0000","last_peered":"2024-05-21T15:16:08.703987+0000","last_clean":"2024-05-21T15:16:08.703987+0000","last_became_active":"2024-05-21T15:15:16.615323+0000","last_became_peered":"2024-05-21T15:15:16.615323+0000","last_unstale":"2024-05-21T15:16:08.703987+0000","last_undegraded":"2024-05-21T15:16:08.703987+0000","last_fullsized":"2024-05-21T15:16:08.703987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:15.233980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.982372+0000","last_change":"2024-05-21T15:14:11.574352+0000","last_active":"2024-05-21T15:16:08.982372+0000","last_peered":"2024-05-21T15:16:08.982372+0000","last_clean":"2024-05-21T15:16:08.982372+0000","last_became_active":"2024-05-21T15:14:11.574041+0000","last_became_peered":"2024-05-21T15:14:11.574041+0000","last_unstale":"2024-05-21T15:16:08.982372+0000","last_undegraded":"2024-05-21T15:16:08.982372+0000","last_fullsized":"2024-05-21T15:16:08.982372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.563218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808128+0000","last_change":"2024-05-21T15:14:12.257886+0000","last_active":"2024-05-21T15:16:08.808128+0000","last_peered":"2024-05-21T15:16:08.808128+0000","last_clean":"2024-05-21T15:16:08.808128+0000","last_became_active":"2024-05-21T15:14:12.257546+0000","last_became_peered":"2024-05-21T15:14:12.257546+0000","last_unstale":"2024-05-21T15:16:08.808128+0000","last_undegraded":"2024-05-21T15:16:08.808128+0000","last_fullsized":"2024-05-21T15:16:08.808128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:20.953421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972256+0000","last_change":"2024-05-21T15:14:11.568524+0000","last_active":"2024-05-21T15:16:08.972256+0000","last_peered":"2024-05-21T15:16:08.972256+0000","last_clean":"2024-05-21T15:16:08.972256+0000","last_became_active":"2024-05-21T15:14:11.567981+0000","last_became_peered":"2024-05-21T15:14:11.567981+0000","last_unstale":"2024-05-21T15:16:08.972256+0000","last_undegraded":"2024-05-21T15:16:08.972256+0000","last_fullsized":"2024-05-21T15:16:08.972256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975897+0000","last_change":"2024-05-21T15:13:02.249260+0000","last_active":"2024-05-21T15:16:08.975897+0000","last_peered":"2024-05-21T15:16:08.975897+0000","last_clean":"2024-05-21T15:16:08.975897+0000","last_became_active":"2024-05-21T15:13:02.248637+0000","last_became_peered":"2024-05-21T15:13:02.248637+0000","last_unstale":"2024-05-21T15:16:08.975897+0000","last_undegraded":"2024-05-21T15:16:08.975897+0000","last_fullsized":"2024-05-21T15:16:08.975897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_clean_scrub_stamp":"2024-05-21T15:09:41.506463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:26.250910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979324+0000","last_change":"2024-05-21T15:14:12.259861+0000","last_active":"2024-05-21T15:16:08.979324+0000","last_peered":"2024-05-21T15:16:08.979324+0000","last_clean":"2024-05-21T15:16:08.979324+0000","last_became_active":"2024-05-21T15:14:12.259510+0000","last_became_peered":"2024-05-21T15:14:12.259510+0000","last_unstale":"2024-05-21T15:16:08.979324+0000","last_undegraded":"2024-05-21T15:16:08.979324+0000","last_fullsized":"2024-05-21T15:16:08.979324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:49.548366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976144+0000","last_change":"2024-05-21T15:14:11.570705+0000","last_active":"2024-05-21T15:16:08.976144+0000","last_peered":"2024-05-21T15:16:08.976144+0000","last_clean":"2024-05-21T15:16:08.976144+0000","last_became_active":"2024-05-21T15:14:11.570403+0000","last_became_peered":"2024-05-21T15:14:11.570403+0000","last_unstale":"2024-05-21T15:16:08.976144+0000","last_undegraded":"2024-05-21T15:16:08.976144+0000","last_fullsized":"2024-05-21T15:16:08.976144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:02:34.156258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905269+0000","last_change":"2024-05-21T15:14:11.572311+0000","last_active":"2024-05-21T15:16:08.905269+0000","last_peered":"2024-05-21T15:16:08.905269+0000","last_clean":"2024-05-21T15:16:08.905269+0000","last_became_active":"2024-05-21T15:14:11.571949+0000","last_became_peered":"2024-05-21T15:14:11.571949+0000","last_unstale":"2024-05-21T15:16:08.905269+0000","last_undegraded":"2024-05-21T15:16:08.905269+0000","last_fullsized":"2024-05-21T15:16:08.905269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:57:59.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979695+0000","last_change":"2024-05-21T15:14:12.260717+0000","last_active":"2024-05-21T15:16:08.979695+0000","last_peered":"2024-05-21T15:16:08.979695+0000","last_clean":"2024-05-21T15:16:08.979695+0000","last_became_active":"2024-05-21T15:14:12.259911+0000","last_became_peered":"2024-05-21T15:14:12.259911+0000","last_unstale":"2024-05-21T15:16:08.979695+0000","last_undegraded":"2024-05-21T15:16:08.979695+0000","last_fullsized":"2024-05-21T15:16:08.979695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:02.609299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.933483+0000","last_change":"2024-05-21T15:15:16.381952+0000","last_active":"2024-05-21T15:16:08.933483+0000","last_peered":"2024-05-21T15:16:08.933483+0000","last_clean":"2024-05-21T15:16:08.933483+0000","last_became_active":"2024-05-21T15:15:16.373966+0000","last_became_peered":"2024-05-21T15:15:16.373966+0000","last_unstale":"2024-05-21T15:16:08.933483+0000","last_undegraded":"2024-05-21T15:16:08.933483+0000","last_fullsized":"2024-05-21T15:16:08.933483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:49:05.368066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809256+0000","last_change":"2024-05-21T15:14:12.253634+0000","last_active":"2024-05-21T15:16:08.809256+0000","last_peered":"2024-05-21T15:16:08.809256+0000","last_clean":"2024-05-21T15:16:08.809256+0000","last_became_active":"2024-05-21T15:14:12.253284+0000","last_became_peered":"2024-05-21T15:14:12.253284+0000","last_unstale":"2024-05-21T15:16:08.809256+0000","last_undegraded":"2024-05-21T15:16:08.809256+0000","last_fullsized":"2024-05-21T15:16:08.809256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:28:07.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972367+0000","last_change":"2024-05-21T15:14:11.574147+0000","last_active":"2024-05-21T15:16:08.972367+0000","last_peered":"2024-05-21T15:16:08.972367+0000","last_clean":"2024-05-21T15:16:08.972367+0000","last_became_active":"2024-05-21T15:14:11.573721+0000","last_became_peered":"2024-05-21T15:14:11.573721+0000","last_unstale":"2024-05-21T15:16:08.972367+0000","last_undegraded":"2024-05-21T15:16:08.972367+0000","last_fullsized":"2024-05-21T15:16:08.972367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:10.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983413+0000","last_change":"2024-05-21T15:14:12.258632+0000","last_active":"2024-05-21T15:16:08.983413+0000","last_peered":"2024-05-21T15:16:08.983413+0000","last_clean":"2024-05-21T15:16:08.983413+0000","last_became_active":"2024-05-21T15:14:12.257908+0000","last_became_peered":"2024-05-21T15:14:12.257908+0000","last_unstale":"2024-05-21T15:16:08.983413+0000","last_undegraded":"2024-05-21T15:16:08.983413+0000","last_fullsized":"2024-05-21T15:16:08.983413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:12.062387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.978955+0000","last_change":"2024-05-21T15:14:11.562592+0000","last_active":"2024-05-21T15:16:08.978955+0000","last_peered":"2024-05-21T15:16:08.978955+0000","last_clean":"2024-05-21T15:16:08.978955+0000","last_became_active":"2024-05-21T15:14:11.562265+0000","last_became_peered":"2024-05-21T15:14:11.562265+0000","last_unstale":"2024-05-21T15:16:08.978955+0000","last_undegraded":"2024-05-21T15:16:08.978955+0000","last_fullsized":"2024-05-21T15:16:08.978955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:24.662576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905370+0000","last_change":"2024-05-21T15:14:12.249207+0000","last_active":"2024-05-21T15:16:08.905370+0000","last_peered":"2024-05-21T15:16:08.905370+0000","last_clean":"2024-05-21T15:16:08.905370+0000","last_became_active":"2024-05-21T15:14:12.248819+0000","last_became_peered":"2024-05-21T15:14:12.248819+0000","last_unstale":"2024-05-21T15:16:08.905370+0000","last_undegraded":"2024-05-21T15:16:08.905370+0000","last_fullsized":"2024-05-21T15:16:08.905370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:38.208636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807711+0000","last_change":"2024-05-21T15:14:11.572044+0000","last_active":"2024-05-21T15:16:08.807711+0000","last_peered":"2024-05-21T15:16:08.807711+0000","last_clean":"2024-05-21T15:16:08.807711+0000","last_became_active":"2024-05-21T15:14:11.570721+0000","last_became_peered":"2024-05-21T15:14:11.570721+0000","last_unstale":"2024-05-21T15:16:08.807711+0000","last_undegraded":"2024-05-21T15:16:08.807711+0000","last_fullsized":"2024-05-21T15:16:08.807711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:34:08.413781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705475+0000","last_change":"2024-05-21T15:14:12.256327+0000","last_active":"2024-05-21T15:16:08.705475+0000","last_peered":"2024-05-21T15:16:08.705475+0000","last_clean":"2024-05-21T15:16:08.705475+0000","last_became_active":"2024-05-21T15:14:12.255962+0000","last_became_peered":"2024-05-21T15:14:12.255962+0000","last_unstale":"2024-05-21T15:16:08.705475+0000","last_undegraded":"2024-05-21T15:16:08.705475+0000","last_fullsized":"2024-05-21T15:16:08.705475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:05.076323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705571+0000","last_change":"2024-05-21T15:15:16.385658+0000","last_active":"2024-05-21T15:16:08.705571+0000","last_peered":"2024-05-21T15:16:08.705571+0000","last_clean":"2024-05-21T15:16:08.705571+0000","last_became_active":"2024-05-21T15:15:16.375122+0000","last_became_peered":"2024-05-21T15:15:16.375122+0000","last_unstale":"2024-05-21T15:16:08.705571+0000","last_undegraded":"2024-05-21T15:16:08.705571+0000","last_fullsized":"2024-05-21T15:16:08.705571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:57:59.599504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808366+0000","last_change":"2024-05-21T15:14:12.256993+0000","last_active":"2024-05-21T15:16:08.808366+0000","last_peered":"2024-05-21T15:16:08.808366+0000","last_clean":"2024-05-21T15:16:08.808366+0000","last_became_active":"2024-05-21T15:14:12.256179+0000","last_became_peered":"2024-05-21T15:14:12.256179+0000","last_unstale":"2024-05-21T15:16:08.808366+0000","last_undegraded":"2024-05-21T15:16:08.808366+0000","last_fullsized":"2024-05-21T15:16:08.808366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:01:09.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905244+0000","last_change":"2024-05-21T15:15:16.391272+0000","last_active":"2024-05-21T15:16:08.905244+0000","last_peered":"2024-05-21T15:16:08.905244+0000","last_clean":"2024-05-21T15:16:08.905244+0000","last_became_active":"2024-05-21T15:15:16.386921+0000","last_became_peered":"2024-05-21T15:15:16.386921+0000","last_unstale":"2024-05-21T15:16:08.905244+0000","last_undegraded":"2024-05-21T15:16:08.905244+0000","last_fullsized":"2024-05-21T15:16:08.905244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:01:55.466989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976147+0000","last_change":"2024-05-21T15:14:12.260506+0000","last_active":"2024-05-21T15:16:08.976147+0000","last_peered":"2024-05-21T15:16:08.976147+0000","last_clean":"2024-05-21T15:16:08.976147+0000","last_became_active":"2024-05-21T15:14:12.260168+0000","last_became_peered":"2024-05-21T15:14:12.260168+0000","last_unstale":"2024-05-21T15:16:08.976147+0000","last_undegraded":"2024-05-21T15:16:08.976147+0000","last_fullsized":"2024-05-21T15:16:08.976147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:11.321622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.479294+0000","last_change":"2024-05-21T15:14:11.568630+0000","last_active":"2024-05-21T15:16:09.479294+0000","last_peered":"2024-05-21T15:16:09.479294+0000","last_clean":"2024-05-21T15:16:09.479294+0000","last_became_active":"2024-05-21T15:14:11.568229+0000","last_became_peered":"2024-05-21T15:14:11.568229+0000","last_unstale":"2024-05-21T15:16:09.479294+0000","last_undegraded":"2024-05-21T15:16:09.479294+0000","last_fullsized":"2024-05-21T15:16:09.479294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:21.372395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976294+0000","last_change":"2024-05-21T15:14:12.259408+0000","last_active":"2024-05-21T15:16:08.976294+0000","last_peered":"2024-05-21T15:16:08.976294+0000","last_clean":"2024-05-21T15:16:08.976294+0000","last_became_active":"2024-05-21T15:14:12.259082+0000","last_became_peered":"2024-05-21T15:14:12.259082+0000","last_unstale":"2024-05-21T15:16:08.976294+0000","last_undegraded":"2024-05-21T15:16:08.976294+0000","last_fullsized":"2024-05-21T15:16:08.976294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:32:09.059307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.930782+0000","last_change":"2024-05-21T15:14:11.571077+0000","last_active":"2024-05-21T15:16:08.930782+0000","last_peered":"2024-05-21T15:16:08.930782+0000","last_clean":"2024-05-21T15:16:08.930782+0000","last_became_active":"2024-05-21T15:14:11.570664+0000","last_became_peered":"2024-05-21T15:14:11.570664+0000","last_unstale":"2024-05-21T15:16:08.930782+0000","last_undegraded":"2024-05-21T15:16:08.930782+0000","last_fullsized":"2024-05-21T15:16:08.930782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:37:16.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976028+0000","last_change":"2024-05-21T15:14:12.252972+0000","last_active":"2024-05-21T15:16:08.976028+0000","last_peered":"2024-05-21T15:16:08.976028+0000","last_clean":"2024-05-21T15:16:08.976028+0000","last_became_active":"2024-05-21T15:14:12.252591+0000","last_became_peered":"2024-05-21T15:14:12.252591+0000","last_unstale":"2024-05-21T15:16:08.976028+0000","last_undegraded":"2024-05-21T15:16:08.976028+0000","last_fullsized":"2024-05-21T15:16:08.976028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:14:16.430371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979634+0000","last_change":"2024-05-21T15:14:11.577138+0000","last_active":"2024-05-21T15:16:08.979634+0000","last_peered":"2024-05-21T15:16:08.979634+0000","last_clean":"2024-05-21T15:16:08.979634+0000","last_became_active":"2024-05-21T15:14:11.576629+0000","last_became_peered":"2024-05-21T15:14:11.576629+0000","last_unstale":"2024-05-21T15:16:08.979634+0000","last_undegraded":"2024-05-21T15:16:08.979634+0000","last_fullsized":"2024-05-21T15:16:08.979634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:20:20.134764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976368+0000","last_change":"2024-05-21T15:14:12.256174+0000","last_active":"2024-05-21T15:16:08.976368+0000","last_peered":"2024-05-21T15:16:08.976368+0000","last_clean":"2024-05-21T15:16:08.976368+0000","last_became_active":"2024-05-21T15:14:12.255839+0000","last_became_peered":"2024-05-21T15:14:12.255839+0000","last_unstale":"2024-05-21T15:16:08.976368+0000","last_undegraded":"2024-05-21T15:16:08.976368+0000","last_fullsized":"2024-05-21T15:16:08.976368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:12:40.743118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705604+0000","last_change":"2024-05-21T15:14:12.257028+0000","last_active":"2024-05-21T15:16:08.705604+0000","last_peered":"2024-05-21T15:16:08.705604+0000","last_clean":"2024-05-21T15:16:08.705604+0000","last_became_active":"2024-05-21T15:14:12.256611+0000","last_became_peered":"2024-05-21T15:14:12.256611+0000","last_unstale":"2024-05-21T15:16:08.705604+0000","last_undegraded":"2024-05-21T15:16:08.705604+0000","last_fullsized":"2024-05-21T15:16:08.705604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:08:43.881073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976078+0000","last_change":"2024-05-21T15:14:12.255724+0000","last_active":"2024-05-21T15:16:08.976078+0000","last_peered":"2024-05-21T15:16:08.976078+0000","last_clean":"2024-05-21T15:16:08.976078+0000","last_became_active":"2024-05-21T15:14:12.255272+0000","last_became_peered":"2024-05-21T15:14:12.255272+0000","last_unstale":"2024-05-21T15:16:08.976078+0000","last_undegraded":"2024-05-21T15:16:08.976078+0000","last_fullsized":"2024-05-21T15:16:08.976078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:07:07.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976048+0000","last_change":"2024-05-21T15:14:12.262508+0000","last_active":"2024-05-21T15:16:08.976048+0000","last_peered":"2024-05-21T15:16:08.976048+0000","last_clean":"2024-05-21T15:16:08.976048+0000","last_became_active":"2024-05-21T15:14:12.262155+0000","last_became_peered":"2024-05-21T15:14:12.262155+0000","last_unstale":"2024-05-21T15:16:08.976048+0000","last_undegraded":"2024-05-21T15:16:08.976048+0000","last_fullsized":"2024-05-21T15:16:08.976048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:54:19.982044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983316+0000","last_change":"2024-05-21T15:14:12.251715+0000","last_active":"2024-05-21T15:16:08.983316+0000","last_peered":"2024-05-21T15:16:08.983316+0000","last_clean":"2024-05-21T15:16:08.983316+0000","last_became_active":"2024-05-21T15:14:12.251337+0000","last_became_peered":"2024-05-21T15:14:12.251337+0000","last_unstale":"2024-05-21T15:16:08.983316+0000","last_undegraded":"2024-05-21T15:16:08.983316+0000","last_fullsized":"2024-05-21T15:16:08.983316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:03.562179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979661+0000","last_change":"2024-05-21T15:14:12.253166+0000","last_active":"2024-05-21T15:16:08.979661+0000","last_peered":"2024-05-21T15:16:08.979661+0000","last_clean":"2024-05-21T15:16:08.979661+0000","last_became_active":"2024-05-21T15:14:12.251830+0000","last_became_peered":"2024-05-21T15:14:12.251830+0000","last_unstale":"2024-05-21T15:16:08.979661+0000","last_undegraded":"2024-05-21T15:16:08.979661+0000","last_fullsized":"2024-05-21T15:16:08.979661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:11:58.679568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705713+0000","last_change":"2024-05-21T15:14:12.256131+0000","last_active":"2024-05-21T15:16:08.705713+0000","last_peered":"2024-05-21T15:16:08.705713+0000","last_clean":"2024-05-21T15:16:08.705713+0000","last_became_active":"2024-05-21T15:14:12.255772+0000","last_became_peered":"2024-05-21T15:14:12.255772+0000","last_unstale":"2024-05-21T15:16:08.705713+0000","last_undegraded":"2024-05-21T15:16:08.705713+0000","last_fullsized":"2024-05-21T15:16:08.705713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:26:48.047278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976512+0000","last_change":"2024-05-21T15:14:12.260829+0000","last_active":"2024-05-21T15:16:08.976512+0000","last_peered":"2024-05-21T15:16:08.976512+0000","last_clean":"2024-05-21T15:16:08.976512+0000","last_became_active":"2024-05-21T15:14:12.259686+0000","last_became_peered":"2024-05-21T15:14:12.259686+0000","last_unstale":"2024-05-21T15:16:08.976512+0000","last_undegraded":"2024-05-21T15:16:08.976512+0000","last_fullsized":"2024-05-21T15:16:08.976512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:48.480814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809168+0000","last_change":"2024-05-21T15:14:12.262892+0000","last_active":"2024-05-21T15:16:08.809168+0000","last_peered":"2024-05-21T15:16:08.809168+0000","last_clean":"2024-05-21T15:16:08.809168+0000","last_became_active":"2024-05-21T15:14:12.262532+0000","last_became_peered":"2024-05-21T15:14:12.262532+0000","last_unstale":"2024-05-21T15:16:08.809168+0000","last_undegraded":"2024-05-21T15:16:08.809168+0000","last_fullsized":"2024-05-21T15:16:08.809168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:22.424783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.904921+0000","last_change":"2024-05-21T15:15:16.375027+0000","last_active":"2024-05-21T15:16:08.904921+0000","last_peered":"2024-05-21T15:16:08.904921+0000","last_clean":"2024-05-21T15:16:08.904921+0000","last_became_active":"2024-05-21T15:15:16.374542+0000","last_became_peered":"2024-05-21T15:15:16.374542+0000","last_unstale":"2024-05-21T15:16:08.904921+0000","last_undegraded":"2024-05-21T15:16:08.904921+0000","last_fullsized":"2024-05-21T15:16:08.904921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:55:14.399237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807833+0000","last_change":"2024-05-21T15:14:12.259845+0000","last_active":"2024-05-21T15:16:08.807833+0000","last_peered":"2024-05-21T15:16:08.807833+0000","last_clean":"2024-05-21T15:16:08.807833+0000","last_became_active":"2024-05-21T15:14:12.259455+0000","last_became_peered":"2024-05-21T15:14:12.259455+0000","last_unstale":"2024-05-21T15:16:08.807833+0000","last_undegraded":"2024-05-21T15:16:08.807833+0000","last_fullsized":"2024-05-21T15:16:08.807833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:24:59.655178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979108+0000","last_change":"2024-05-21T15:14:12.259713+0000","last_active":"2024-05-21T15:16:08.979108+0000","last_peered":"2024-05-21T15:16:08.979108+0000","last_clean":"2024-05-21T15:16:08.979108+0000","last_became_active":"2024-05-21T15:14:12.259057+0000","last_became_peered":"2024-05-21T15:14:12.259057+0000","last_unstale":"2024-05-21T15:16:08.979108+0000","last_undegraded":"2024-05-21T15:16:08.979108+0000","last_fullsized":"2024-05-21T15:16:08.979108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:46:22.377652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.980055+0000","last_change":"2024-05-21T15:14:12.254389+0000","last_active":"2024-05-21T15:16:08.980055+0000","last_peered":"2024-05-21T15:16:08.980055+0000","last_clean":"2024-05-21T15:16:08.980055+0000","last_became_active":"2024-05-21T15:14:12.253846+0000","last_became_peered":"2024-05-21T15:14:12.253846+0000","last_unstale":"2024-05-21T15:16:08.980055+0000","last_undegraded":"2024-05-21T15:16:08.980055+0000","last_fullsized":"2024-05-21T15:16:08.980055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:32.932316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705619+0000","last_change":"2024-05-21T15:14:12.252292+0000","last_active":"2024-05-21T15:16:08.705619+0000","last_peered":"2024-05-21T15:16:08.705619+0000","last_clean":"2024-05-21T15:16:08.705619+0000","last_became_active":"2024-05-21T15:14:12.251816+0000","last_became_peered":"2024-05-21T15:14:12.251816+0000","last_unstale":"2024-05-21T15:16:08.705619+0000","last_undegraded":"2024-05-21T15:16:08.705619+0000","last_fullsized":"2024-05-21T15:16:08.705619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:59:14.332010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600100,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1256,"num_read_kb":736,"num_write":376,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":672,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1800300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383716,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28356,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716796,"statfs":{"total":95995035648,"available":95965999104,"internally_reserved":0,"allocated":1368064,"data_stored":859573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28788,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716364,"statfs":{"total":95995035648,"available":95965556736,"internally_reserved":0,"allocated":1814528,"data_stored":1254662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1392640,"data_stored":859112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":465602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1576960,"data_stored":1050256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":466703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":533292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":457967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":534127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":456453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":466537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32936,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712216,"statfs":{"total":95995035648,"available":95961309184,"internally_reserved":0,"allocated":1740800,"data_stored":1190993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":199403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":394799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:16:10.543 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-21T15:16:10.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T15:16:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 -- 172.21.15.78:0/3588928409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1006e0 msgr2=0x7f5b6c100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 --2- 172.21.15.78:0/3588928409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1006e0 0x7f5b6c100b50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009900 tx=0x7f5b5802a410 comp rx=0 tx=0).stop 2024-05-21T15:16:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 -- 172.21.15.78:0/3588928409 shutdown_connections 2024-05-21T15:16:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 --2- 172.21.15.78:0/3588928409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6c1011e0 0x7f5b6c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 --2- 172.21.15.78:0/3588928409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1006e0 0x7f5b6c100b50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 --2- 172.21.15.78:0/3588928409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c104030 0x7f5b6c104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 -- 172.21.15.78:0/3588928409 >> 172.21.15.78:0/3588928409 conn(0x7f5b6c0fc2e0 msgr2=0x7f5b6c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 -- 172.21.15.78:0/3588928409 shutdown_connections 2024-05-21T15:16:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.710+0000 7f5b72a73640 1 -- 172.21.15.78:0/3588928409 wait complete. 2024-05-21T15:16:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.711+0000 7f5b72a73640 1 Processor -- start 2024-05-21T15:16:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.711+0000 7f5b72a73640 1 -- start start 2024-05-21T15:16:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.711+0000 7f5b72a73640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c1006e0 0x7f5b6c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.711+0000 7f5b72a73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1011e0 0x7f5b6c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b72a73640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6c104030 0x7f5b6c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b72a73640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10e760 con 0x7f5b6c1011e0 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b72a73640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10e5e0 con 0x7f5b6c1006e0 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b72a73640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10e8e0 con 0x7f5b6c104030 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1011e0 0x7f5b6c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1011e0 0x7f5b6c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46894/0 (socket says 172.21.15.78:46894) 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 -- 172.21.15.78:0/4116730560 learned_addr learned my addr 172.21.15.78:0/4116730560 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b7080f640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c1006e0 0x7f5b6c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b71010640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6c104030 0x7f5b6c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 -- 172.21.15.78:0/4116730560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6c104030 msgr2=0x7f5b6c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6c104030 0x7f5b6c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 -- 172.21.15.78:0/4116730560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c1006e0 msgr2=0x7f5b6c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c1006e0 0x7f5b6c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 -- 172.21.15.78:0/4116730560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c1a0dc0 con 0x7f5b6c1011e0 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b6bfff640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1011e0 0x7f5b6c199810 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5b58031440 tx=0x7f5b58032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b69ffb640 1 -- 172.21.15.78:0/4116730560 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5803f030 con 0x7f5b6c1011e0 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b69ffb640 1 -- 172.21.15.78:0/4116730560 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b58003ce0 con 0x7f5b6c1011e0 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c1a1050 con 0x7f5b6c1011e0 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.712+0000 7f5b69ffb640 1 -- 172.21.15.78:0/4116730560 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58035070 con 0x7f5b6c1011e0 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.713+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a1590 con 0x7f5b6c1011e0 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.714+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c1079c0 con 0x7f5b6c1011e0 2024-05-21T15:16:10.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.715+0000 7f5b69ffb640 1 -- 172.21.15.78:0/4116730560 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f5b58042030 con 0x7f5b6c1011e0 2024-05-21T15:16:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.716+0000 7f5b69ffb640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b5c0a2a10 0x7f5b5c0a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.716+0000 7f5b7080f640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b5c0a2a10 0x7f5b5c0a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.716+0000 7f5b69ffb640 1 -- 172.21.15.78:0/4116730560 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5b580ef4c0 con 0x7f5b6c1011e0 2024-05-21T15:16:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.717+0000 7f5b7080f640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b5c0a2a10 0x7f5b5c0a4f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b600095f0 tx=0x7f5b60005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:10.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.719+0000 7f5b69ffb640 1 -- 172.21.15.78:0/4116730560 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b580bbf60 con 0x7f5b6c1011e0 2024-05-21T15:16:10.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.769+0000 7f5b69ffb640 1 -- 172.21.15.78:0/4116730560 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f5b580b85d0 con 0x7f5b6c1011e0 2024-05-21T15:16:10.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:10 smithi078 ceph-mon[31997]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:10.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:10 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2566135600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:10.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:10 smithi078 ceph-mon[31997]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.939+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b6c061f90 con 0x7f5b5c0a2a10 2024-05-21T15:16:10.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.946+0000 7f5b69ffb640 1 -- 172.21.15.78:0/4116730560 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257782 (secure 0 0 0) 0x7f5b6c061f90 con 0x7f5b5c0a2a10 2024-05-21T15:16:10.952 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:10.953 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-05-21T15:16:10.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b5c0a2a10 msgr2=0x7f5b5c0a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b5c0a2a10 0x7f5b5c0a4f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b600095f0 tx=0x7f5b60005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1011e0 msgr2=0x7f5b6c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1011e0 0x7f5b6c199810 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5b58031440 tx=0x7f5b58032ce0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 shutdown_connections 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b5c0a2a10 0x7f5b5c0a4f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b6c104030 0x7f5b6c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b6c1011e0 0x7f5b6c199810 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 --2- 172.21.15.78:0/4116730560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b6c1006e0 0x7f5b6c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 >> 172.21.15.78:0/4116730560 conn(0x7f5b6c0fc2e0 msgr2=0x7f5b6c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 shutdown_connections 2024-05-21T15:16:10.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:10.950+0000 7f5b72a73640 1 -- 172.21.15.78:0/4116730560 wait complete. 2024-05-21T15:16:10.966 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-05-21T15:16:10.719294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190702,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1392,"num_read_kb":855,"num_write":559,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181408,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359672,"kb_used_data":14592,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582152,"statfs":{"total":1151940427776,"available":1151572123648,"internally_reserved":0,"allocated":14942208,"data_stored":8595277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.988697"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705906+0000","last_change":"2024-05-21T15:14:12.254239+0000","last_active":"2024-05-21T15:16:08.705906+0000","last_peered":"2024-05-21T15:16:08.705906+0000","last_clean":"2024-05-21T15:16:08.705906+0000","last_became_active":"2024-05-21T15:14:12.253885+0000","last_became_peered":"2024-05-21T15:14:12.253885+0000","last_unstale":"2024-05-21T15:16:08.705906+0000","last_undegraded":"2024-05-21T15:16:08.705906+0000","last_fullsized":"2024-05-21T15:16:08.705906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:35.126836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983550+0000","last_change":"2024-05-21T15:14:12.262984+0000","last_active":"2024-05-21T15:16:08.983550+0000","last_peered":"2024-05-21T15:16:08.983550+0000","last_clean":"2024-05-21T15:16:08.983550+0000","last_became_active":"2024-05-21T15:14:12.262667+0000","last_became_peered":"2024-05-21T15:14:12.262667+0000","last_unstale":"2024-05-21T15:16:08.983550+0000","last_undegraded":"2024-05-21T15:16:08.983550+0000","last_fullsized":"2024-05-21T15:16:08.983550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:47:47.434314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983456+0000","last_change":"2024-05-21T15:14:12.260573+0000","last_active":"2024-05-21T15:16:08.983456+0000","last_peered":"2024-05-21T15:16:08.983456+0000","last_clean":"2024-05-21T15:16:08.983456+0000","last_became_active":"2024-05-21T15:14:12.259073+0000","last_became_peered":"2024-05-21T15:14:12.259073+0000","last_unstale":"2024-05-21T15:16:08.983456+0000","last_undegraded":"2024-05-21T15:16:08.983456+0000","last_fullsized":"2024-05-21T15:16:08.983456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:15:01.919464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972349+0000","last_change":"2024-05-21T15:14:12.256992+0000","last_active":"2024-05-21T15:16:08.972349+0000","last_peered":"2024-05-21T15:16:08.972349+0000","last_clean":"2024-05-21T15:16:08.972349+0000","last_became_active":"2024-05-21T15:14:12.256448+0000","last_became_peered":"2024-05-21T15:14:12.256448+0000","last_unstale":"2024-05-21T15:16:08.972349+0000","last_undegraded":"2024-05-21T15:16:08.972349+0000","last_fullsized":"2024-05-21T15:16:08.972349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:26:40.569263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905144+0000","last_change":"2024-05-21T15:14:12.251533+0000","last_active":"2024-05-21T15:16:08.905144+0000","last_peered":"2024-05-21T15:16:08.905144+0000","last_clean":"2024-05-21T15:16:08.905144+0000","last_became_active":"2024-05-21T15:14:12.251210+0000","last_became_peered":"2024-05-21T15:14:12.251210+0000","last_unstale":"2024-05-21T15:16:08.905144+0000","last_undegraded":"2024-05-21T15:16:08.905144+0000","last_fullsized":"2024-05-21T15:16:08.905144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:43:34.109179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705582+0000","last_change":"2024-05-21T15:14:12.251767+0000","last_active":"2024-05-21T15:16:08.705582+0000","last_peered":"2024-05-21T15:16:08.705582+0000","last_clean":"2024-05-21T15:16:08.705582+0000","last_became_active":"2024-05-21T15:14:12.251427+0000","last_became_peered":"2024-05-21T15:14:12.251427+0000","last_unstale":"2024-05-21T15:16:08.705582+0000","last_undegraded":"2024-05-21T15:16:08.705582+0000","last_fullsized":"2024-05-21T15:16:08.705582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:48:41.804074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975895+0000","last_change":"2024-05-21T15:14:12.253472+0000","last_active":"2024-05-21T15:16:08.975895+0000","last_peered":"2024-05-21T15:16:08.975895+0000","last_clean":"2024-05-21T15:16:08.975895+0000","last_became_active":"2024-05-21T15:14:12.252762+0000","last_became_peered":"2024-05-21T15:14:12.252762+0000","last_unstale":"2024-05-21T15:16:08.975895+0000","last_undegraded":"2024-05-21T15:16:08.975895+0000","last_fullsized":"2024-05-21T15:16:08.975895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:11:44.497404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807382+0000","last_change":"2024-05-21T15:14:12.258627+0000","last_active":"2024-05-21T15:16:08.807382+0000","last_peered":"2024-05-21T15:16:08.807382+0000","last_clean":"2024-05-21T15:16:08.807382+0000","last_became_active":"2024-05-21T15:14:12.258118+0000","last_became_peered":"2024-05-21T15:14:12.258118+0000","last_unstale":"2024-05-21T15:16:08.807382+0000","last_undegraded":"2024-05-21T15:16:08.807382+0000","last_fullsized":"2024-05-21T15:16:08.807382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:04:56.849419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979322+0000","last_change":"2024-05-21T15:14:12.255998+0000","last_active":"2024-05-21T15:16:08.979322+0000","last_peered":"2024-05-21T15:16:08.979322+0000","last_clean":"2024-05-21T15:16:08.979322+0000","last_became_active":"2024-05-21T15:14:12.255658+0000","last_became_peered":"2024-05-21T15:14:12.255658+0000","last_unstale":"2024-05-21T15:16:08.979322+0000","last_undegraded":"2024-05-21T15:16:08.979322+0000","last_fullsized":"2024-05-21T15:16:08.979322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:44:42.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.706089+0000","last_change":"2024-05-21T15:14:12.257617+0000","last_active":"2024-05-21T15:16:08.706089+0000","last_peered":"2024-05-21T15:16:08.706089+0000","last_clean":"2024-05-21T15:16:08.706089+0000","last_became_active":"2024-05-21T15:14:12.256947+0000","last_became_peered":"2024-05-21T15:14:12.256947+0000","last_unstale":"2024-05-21T15:16:08.706089+0000","last_undegraded":"2024-05-21T15:16:08.706089+0000","last_fullsized":"2024-05-21T15:16:08.706089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:45:58.174954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976285+0000","last_change":"2024-05-21T15:14:12.260454+0000","last_active":"2024-05-21T15:16:08.976285+0000","last_peered":"2024-05-21T15:16:08.976285+0000","last_clean":"2024-05-21T15:16:08.976285+0000","last_became_active":"2024-05-21T15:14:12.260126+0000","last_became_peered":"2024-05-21T15:14:12.260126+0000","last_unstale":"2024-05-21T15:16:08.976285+0000","last_undegraded":"2024-05-21T15:16:08.976285+0000","last_fullsized":"2024-05-21T15:16:08.976285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:49:45.212050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972342+0000","last_change":"2024-05-21T15:14:12.252071+0000","last_active":"2024-05-21T15:16:08.972342+0000","last_peered":"2024-05-21T15:16:08.972342+0000","last_clean":"2024-05-21T15:16:08.972342+0000","last_became_active":"2024-05-21T15:14:12.251706+0000","last_became_peered":"2024-05-21T15:14:12.251706+0000","last_unstale":"2024-05-21T15:16:08.972342+0000","last_undegraded":"2024-05-21T15:16:08.972342+0000","last_fullsized":"2024-05-21T15:16:08.972342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:02:44.349368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979900+0000","last_change":"2024-05-21T15:14:12.239877+0000","last_active":"2024-05-21T15:16:08.979900+0000","last_peered":"2024-05-21T15:16:08.979900+0000","last_clean":"2024-05-21T15:16:08.979900+0000","last_became_active":"2024-05-21T15:14:12.239615+0000","last_became_peered":"2024-05-21T15:14:12.239615+0000","last_unstale":"2024-05-21T15:16:08.979900+0000","last_undegraded":"2024-05-21T15:16:08.979900+0000","last_fullsized":"2024-05-21T15:16:08.979900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:23.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983191+0000","last_change":"2024-05-21T15:14:12.260349+0000","last_active":"2024-05-21T15:16:08.983191+0000","last_peered":"2024-05-21T15:16:08.983191+0000","last_clean":"2024-05-21T15:16:08.983191+0000","last_became_active":"2024-05-21T15:14:12.259658+0000","last_became_peered":"2024-05-21T15:14:12.259658+0000","last_unstale":"2024-05-21T15:16:08.983191+0000","last_undegraded":"2024-05-21T15:16:08.983191+0000","last_fullsized":"2024-05-21T15:16:08.983191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:59:32.326571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983078+0000","last_change":"2024-05-21T15:15:17.015535+0000","last_active":"2024-05-21T15:16:08.983078+0000","last_peered":"2024-05-21T15:16:08.983078+0000","last_clean":"2024-05-21T15:16:08.983078+0000","last_became_active":"2024-05-21T15:15:17.015057+0000","last_became_peered":"2024-05-21T15:15:17.015057+0000","last_unstale":"2024-05-21T15:16:08.983078+0000","last_undegraded":"2024-05-21T15:16:08.983078+0000","last_fullsized":"2024-05-21T15:16:08.983078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:48:14.652870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.978891+0000","last_change":"2024-05-21T15:14:12.264667+0000","last_active":"2024-05-21T15:16:08.978891+0000","last_peered":"2024-05-21T15:16:08.978891+0000","last_clean":"2024-05-21T15:16:08.978891+0000","last_became_active":"2024-05-21T15:14:12.264284+0000","last_became_peered":"2024-05-21T15:14:12.264284+0000","last_unstale":"2024-05-21T15:16:08.978891+0000","last_undegraded":"2024-05-21T15:16:08.978891+0000","last_fullsized":"2024-05-21T15:16:08.978891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:38:24.861321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705966+0000","last_change":"2024-05-21T15:14:12.259912+0000","last_active":"2024-05-21T15:16:08.705966+0000","last_peered":"2024-05-21T15:16:08.705966+0000","last_clean":"2024-05-21T15:16:08.705966+0000","last_became_active":"2024-05-21T15:14:12.259578+0000","last_became_peered":"2024-05-21T15:14:12.259578+0000","last_unstale":"2024-05-21T15:16:08.705966+0000","last_undegraded":"2024-05-21T15:16:08.705966+0000","last_fullsized":"2024-05-21T15:16:08.705966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:05:33.372481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704201+0000","last_change":"2024-05-21T15:14:12.258355+0000","last_active":"2024-05-21T15:16:08.704201+0000","last_peered":"2024-05-21T15:16:08.704201+0000","last_clean":"2024-05-21T15:16:08.704201+0000","last_became_active":"2024-05-21T15:14:12.257991+0000","last_became_peered":"2024-05-21T15:14:12.257991+0000","last_unstale":"2024-05-21T15:16:08.704201+0000","last_undegraded":"2024-05-21T15:16:08.704201+0000","last_fullsized":"2024-05-21T15:16:08.704201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:51:54.014553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976238+0000","last_change":"2024-05-21T15:14:12.258873+0000","last_active":"2024-05-21T15:16:08.976238+0000","last_peered":"2024-05-21T15:16:08.976238+0000","last_clean":"2024-05-21T15:16:08.976238+0000","last_became_active":"2024-05-21T15:14:12.258548+0000","last_became_peered":"2024-05-21T15:14:12.258548+0000","last_unstale":"2024-05-21T15:16:08.976238+0000","last_undegraded":"2024-05-21T15:16:08.976238+0000","last_fullsized":"2024-05-21T15:16:08.976238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:08:00.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":167,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704409+0000","last_change":"2024-05-21T15:15:16.626507+0000","last_active":"2024-05-21T15:16:08.704409+0000","last_peered":"2024-05-21T15:16:08.704409+0000","last_clean":"2024-05-21T15:16:08.704409+0000","last_became_active":"2024-05-21T15:15:16.615945+0000","last_became_peered":"2024-05-21T15:15:16.615945+0000","last_unstale":"2024-05-21T15:16:08.704409+0000","last_undegraded":"2024-05-21T15:16:08.704409+0000","last_fullsized":"2024-05-21T15:16:08.704409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:16:01.229449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808883+0000","last_change":"2024-05-21T15:14:12.261632+0000","last_active":"2024-05-21T15:16:08.808883+0000","last_peered":"2024-05-21T15:16:08.808883+0000","last_clean":"2024-05-21T15:16:08.808883+0000","last_became_active":"2024-05-21T15:14:12.261149+0000","last_became_peered":"2024-05-21T15:14:12.261149+0000","last_unstale":"2024-05-21T15:16:08.808883+0000","last_undegraded":"2024-05-21T15:16:08.808883+0000","last_fullsized":"2024-05-21T15:16:08.808883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:16:11.349395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809030+0000","last_change":"2024-05-21T15:14:11.567859+0000","last_active":"2024-05-21T15:16:08.809030+0000","last_peered":"2024-05-21T15:16:08.809030+0000","last_clean":"2024-05-21T15:16:08.809030+0000","last_became_active":"2024-05-21T15:14:11.566855+0000","last_became_peered":"2024-05-21T15:14:11.566855+0000","last_unstale":"2024-05-21T15:16:08.809030+0000","last_undegraded":"2024-05-21T15:16:08.809030+0000","last_fullsized":"2024-05-21T15:16:08.809030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:14:21.568436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983569+0000","last_change":"2024-05-21T15:14:12.262153+0000","last_active":"2024-05-21T15:16:08.983569+0000","last_peered":"2024-05-21T15:16:08.983569+0000","last_clean":"2024-05-21T15:16:08.983569+0000","last_became_active":"2024-05-21T15:14:12.261780+0000","last_became_peered":"2024-05-21T15:14:12.261780+0000","last_unstale":"2024-05-21T15:16:08.983569+0000","last_undegraded":"2024-05-21T15:16:08.983569+0000","last_fullsized":"2024-05-21T15:16:08.983569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:21:25.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'14","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905483+0000","last_change":"2024-05-21T15:14:11.570316+0000","last_active":"2024-05-21T15:16:08.905483+0000","last_peered":"2024-05-21T15:16:08.905483+0000","last_clean":"2024-05-21T15:16:08.905483+0000","last_became_active":"2024-05-21T15:14:11.569967+0000","last_became_peered":"2024-05-21T15:14:11.569967+0000","last_unstale":"2024-05-21T15:16:08.905483+0000","last_undegraded":"2024-05-21T15:16:08.905483+0000","last_fullsized":"2024-05-21T15:16:08.905483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:30:27.364742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":20,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979138+0000","last_change":"2024-05-21T15:14:12.261716+0000","last_active":"2024-05-21T15:16:08.979138+0000","last_peered":"2024-05-21T15:16:08.979138+0000","last_clean":"2024-05-21T15:16:08.979138+0000","last_became_active":"2024-05-21T15:14:12.261341+0000","last_became_peered":"2024-05-21T15:14:12.261341+0000","last_unstale":"2024-05-21T15:16:08.979138+0000","last_undegraded":"2024-05-21T15:16:08.979138+0000","last_fullsized":"2024-05-21T15:16:08.979138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:40:10.464974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905502+0000","last_change":"2024-05-21T15:14:11.573305+0000","last_active":"2024-05-21T15:16:08.905502+0000","last_peered":"2024-05-21T15:16:08.905502+0000","last_clean":"2024-05-21T15:16:08.905502+0000","last_became_active":"2024-05-21T15:14:11.572655+0000","last_became_peered":"2024-05-21T15:14:11.572655+0000","last_unstale":"2024-05-21T15:16:08.905502+0000","last_undegraded":"2024-05-21T15:16:08.905502+0000","last_fullsized":"2024-05-21T15:16:08.905502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:37.268440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815691+0000","last_change":"2024-05-21T15:14:12.253788+0000","last_active":"2024-05-21T15:16:08.815691+0000","last_peered":"2024-05-21T15:16:08.815691+0000","last_clean":"2024-05-21T15:16:08.815691+0000","last_became_active":"2024-05-21T15:14:12.253450+0000","last_became_peered":"2024-05-21T15:14:12.253450+0000","last_unstale":"2024-05-21T15:16:08.815691+0000","last_undegraded":"2024-05-21T15:16:08.815691+0000","last_fullsized":"2024-05-21T15:16:08.815691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:57.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":149,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430893+0000","last_change":"2024-05-21T15:14:11.567400+0000","last_active":"2024-05-21T15:16:09.430893+0000","last_peered":"2024-05-21T15:16:09.430893+0000","last_clean":"2024-05-21T15:16:09.430893+0000","last_became_active":"2024-05-21T15:14:11.565995+0000","last_became_peered":"2024-05-21T15:14:11.565995+0000","last_unstale":"2024-05-21T15:16:09.430893+0000","last_undegraded":"2024-05-21T15:16:09.430893+0000","last_fullsized":"2024-05-21T15:16:09.430893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:50.463339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905055+0000","last_change":"2024-05-21T15:14:12.251565+0000","last_active":"2024-05-21T15:16:08.905055+0000","last_peered":"2024-05-21T15:16:08.905055+0000","last_clean":"2024-05-21T15:16:08.905055+0000","last_became_active":"2024-05-21T15:14:12.251289+0000","last_became_peered":"2024-05-21T15:14:12.251289+0000","last_unstale":"2024-05-21T15:16:08.905055+0000","last_undegraded":"2024-05-21T15:16:08.905055+0000","last_fullsized":"2024-05-21T15:16:08.905055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:44.619800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705549+0000","last_change":"2024-05-21T15:14:11.575193+0000","last_active":"2024-05-21T15:16:08.705549+0000","last_peered":"2024-05-21T15:16:08.705549+0000","last_clean":"2024-05-21T15:16:08.705549+0000","last_became_active":"2024-05-21T15:14:11.574861+0000","last_became_peered":"2024-05-21T15:14:11.574861+0000","last_unstale":"2024-05-21T15:16:08.705549+0000","last_undegraded":"2024-05-21T15:16:08.705549+0000","last_fullsized":"2024-05-21T15:16:08.705549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:43:54.548408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815610+0000","last_change":"2024-05-21T15:15:17.015620+0000","last_active":"2024-05-21T15:16:08.815610+0000","last_peered":"2024-05-21T15:16:08.815610+0000","last_clean":"2024-05-21T15:16:08.815610+0000","last_became_active":"2024-05-21T15:15:17.015086+0000","last_became_peered":"2024-05-21T15:15:17.015086+0000","last_unstale":"2024-05-21T15:16:08.815610+0000","last_undegraded":"2024-05-21T15:16:08.815610+0000","last_fullsized":"2024-05-21T15:16:08.815610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:57:10.856246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979503+0000","last_change":"2024-05-21T15:14:11.572087+0000","last_active":"2024-05-21T15:16:08.979503+0000","last_peered":"2024-05-21T15:16:08.979503+0000","last_clean":"2024-05-21T15:16:08.979503+0000","last_became_active":"2024-05-21T15:14:11.571716+0000","last_became_peered":"2024-05-21T15:14:11.571716+0000","last_unstale":"2024-05-21T15:16:08.979503+0000","last_undegraded":"2024-05-21T15:16:08.979503+0000","last_fullsized":"2024-05-21T15:16:08.979503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:48:07.082766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807382+0000","last_change":"2024-05-21T15:14:12.252233+0000","last_active":"2024-05-21T15:16:08.807382+0000","last_peered":"2024-05-21T15:16:08.807382+0000","last_clean":"2024-05-21T15:16:08.807382+0000","last_became_active":"2024-05-21T15:14:12.251901+0000","last_became_peered":"2024-05-21T15:14:12.251901+0000","last_unstale":"2024-05-21T15:16:08.807382+0000","last_undegraded":"2024-05-21T15:16:08.807382+0000","last_fullsized":"2024-05-21T15:16:08.807382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:05.107436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815255+0000","last_change":"2024-05-21T15:14:11.565946+0000","last_active":"2024-05-21T15:16:08.815255+0000","last_peered":"2024-05-21T15:16:08.815255+0000","last_clean":"2024-05-21T15:16:08.815255+0000","last_became_active":"2024-05-21T15:14:11.565601+0000","last_became_peered":"2024-05-21T15:14:11.565601+0000","last_unstale":"2024-05-21T15:16:08.815255+0000","last_undegraded":"2024-05-21T15:16:08.815255+0000","last_fullsized":"2024-05-21T15:16:08.815255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:41:55.243473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704086+0000","last_change":"2024-05-21T15:14:12.255188+0000","last_active":"2024-05-21T15:16:08.704086+0000","last_peered":"2024-05-21T15:16:08.704086+0000","last_clean":"2024-05-21T15:16:08.704086+0000","last_became_active":"2024-05-21T15:14:12.254503+0000","last_became_peered":"2024-05-21T15:14:12.254503+0000","last_unstale":"2024-05-21T15:16:08.704086+0000","last_undegraded":"2024-05-21T15:16:08.704086+0000","last_fullsized":"2024-05-21T15:16:08.704086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:22:16.336740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":254,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430238+0000","last_change":"2024-05-21T15:14:11.569882+0000","last_active":"2024-05-21T15:16:09.430238+0000","last_peered":"2024-05-21T15:16:09.430238+0000","last_clean":"2024-05-21T15:16:09.430238+0000","last_became_active":"2024-05-21T15:14:11.569532+0000","last_became_peered":"2024-05-21T15:14:11.569532+0000","last_unstale":"2024-05-21T15:16:09.430238+0000","last_undegraded":"2024-05-21T15:16:09.430238+0000","last_fullsized":"2024-05-21T15:16:09.430238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:58:50.081709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979327+0000","last_change":"2024-05-21T15:14:12.262691+0000","last_active":"2024-05-21T15:16:08.979327+0000","last_peered":"2024-05-21T15:16:08.979327+0000","last_clean":"2024-05-21T15:16:08.979327+0000","last_became_active":"2024-05-21T15:14:12.261492+0000","last_became_peered":"2024-05-21T15:14:12.261492+0000","last_unstale":"2024-05-21T15:16:08.979327+0000","last_undegraded":"2024-05-21T15:16:08.979327+0000","last_fullsized":"2024-05-21T15:16:08.979327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:35:00.121256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976229+0000","last_change":"2024-05-21T15:14:11.569231+0000","last_active":"2024-05-21T15:16:08.976229+0000","last_peered":"2024-05-21T15:16:08.976229+0000","last_clean":"2024-05-21T15:16:08.976229+0000","last_became_active":"2024-05-21T15:14:11.568916+0000","last_became_peered":"2024-05-21T15:14:11.568916+0000","last_unstale":"2024-05-21T15:16:08.976229+0000","last_undegraded":"2024-05-21T15:16:08.976229+0000","last_fullsized":"2024-05-21T15:16:08.976229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:26:13.411854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983497+0000","last_change":"2024-05-21T15:14:12.256307+0000","last_active":"2024-05-21T15:16:08.983497+0000","last_peered":"2024-05-21T15:16:08.983497+0000","last_clean":"2024-05-21T15:16:08.983497+0000","last_became_active":"2024-05-21T15:14:12.255828+0000","last_became_peered":"2024-05-21T15:14:12.255828+0000","last_unstale":"2024-05-21T15:16:08.983497+0000","last_undegraded":"2024-05-21T15:16:08.983497+0000","last_fullsized":"2024-05-21T15:16:08.983497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:11:40.068779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815696+0000","last_change":"2024-05-21T15:14:11.568239+0000","last_active":"2024-05-21T15:16:08.815696+0000","last_peered":"2024-05-21T15:16:08.815696+0000","last_clean":"2024-05-21T15:16:08.815696+0000","last_became_active":"2024-05-21T15:14:11.567836+0000","last_became_peered":"2024-05-21T15:14:11.567836+0000","last_unstale":"2024-05-21T15:16:08.815696+0000","last_undegraded":"2024-05-21T15:16:08.815696+0000","last_fullsized":"2024-05-21T15:16:08.815696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:51:10.244172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979346+0000","last_change":"2024-05-21T15:14:12.257762+0000","last_active":"2024-05-21T15:16:08.979346+0000","last_peered":"2024-05-21T15:16:08.979346+0000","last_clean":"2024-05-21T15:16:08.979346+0000","last_became_active":"2024-05-21T15:14:12.256335+0000","last_became_peered":"2024-05-21T15:14:12.256335+0000","last_unstale":"2024-05-21T15:16:08.979346+0000","last_undegraded":"2024-05-21T15:16:08.979346+0000","last_fullsized":"2024-05-21T15:16:08.979346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:35.219673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":300,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.460777+0000","last_change":"2024-05-21T15:14:11.564407+0000","last_active":"2024-05-21T15:16:09.460777+0000","last_peered":"2024-05-21T15:16:09.460777+0000","last_clean":"2024-05-21T15:16:09.460777+0000","last_became_active":"2024-05-21T15:14:11.564012+0000","last_became_peered":"2024-05-21T15:14:11.564012+0000","last_unstale":"2024-05-21T15:16:09.460777+0000","last_undegraded":"2024-05-21T15:16:09.460777+0000","last_fullsized":"2024-05-21T15:16:09.460777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:34:54.177844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65676,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":184,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905245+0000","last_change":"2024-05-21T15:14:12.253820+0000","last_active":"2024-05-21T15:16:08.905245+0000","last_peered":"2024-05-21T15:16:08.905245+0000","last_clean":"2024-05-21T15:16:08.905245+0000","last_became_active":"2024-05-21T15:14:12.253360+0000","last_became_peered":"2024-05-21T15:14:12.253360+0000","last_unstale":"2024-05-21T15:16:08.905245+0000","last_undegraded":"2024-05-21T15:16:08.905245+0000","last_fullsized":"2024-05-21T15:16:08.905245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:38:19.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976498+0000","last_change":"2024-05-21T15:14:11.571161+0000","last_active":"2024-05-21T15:16:08.976498+0000","last_peered":"2024-05-21T15:16:08.976498+0000","last_clean":"2024-05-21T15:16:08.976498+0000","last_became_active":"2024-05-21T15:14:11.570837+0000","last_became_peered":"2024-05-21T15:14:11.570837+0000","last_unstale":"2024-05-21T15:16:08.976498+0000","last_undegraded":"2024-05-21T15:16:08.976498+0000","last_fullsized":"2024-05-21T15:16:08.976498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:24:55.096588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979900+0000","last_change":"2024-05-21T15:14:12.255713+0000","last_active":"2024-05-21T15:16:08.979900+0000","last_peered":"2024-05-21T15:16:08.979900+0000","last_clean":"2024-05-21T15:16:08.979900+0000","last_became_active":"2024-05-21T15:14:12.254458+0000","last_became_peered":"2024-05-21T15:14:12.254458+0000","last_unstale":"2024-05-21T15:16:08.979900+0000","last_undegraded":"2024-05-21T15:16:08.979900+0000","last_fullsized":"2024-05-21T15:16:08.979900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:55:24.692752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815443+0000","last_change":"2024-05-21T15:14:11.565916+0000","last_active":"2024-05-21T15:16:08.815443+0000","last_peered":"2024-05-21T15:16:08.815443+0000","last_clean":"2024-05-21T15:16:08.815443+0000","last_became_active":"2024-05-21T15:14:11.565460+0000","last_became_peered":"2024-05-21T15:14:11.565460+0000","last_unstale":"2024-05-21T15:16:08.815443+0000","last_undegraded":"2024-05-21T15:16:08.815443+0000","last_fullsized":"2024-05-21T15:16:08.815443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:09.982208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976147+0000","last_change":"2024-05-21T15:14:12.251856+0000","last_active":"2024-05-21T15:16:08.976147+0000","last_peered":"2024-05-21T15:16:08.976147+0000","last_clean":"2024-05-21T15:16:08.976147+0000","last_became_active":"2024-05-21T15:14:12.251496+0000","last_became_peered":"2024-05-21T15:14:12.251496+0000","last_unstale":"2024-05-21T15:16:08.976147+0000","last_undegraded":"2024-05-21T15:16:08.976147+0000","last_fullsized":"2024-05-21T15:16:08.976147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:07:17.566276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.430144+0000","last_change":"2024-05-21T15:14:11.565485+0000","last_active":"2024-05-21T15:16:09.430144+0000","last_peered":"2024-05-21T15:16:09.430144+0000","last_clean":"2024-05-21T15:16:09.430144+0000","last_became_active":"2024-05-21T15:14:11.565092+0000","last_became_peered":"2024-05-21T15:14:11.565092+0000","last_unstale":"2024-05-21T15:16:09.430144+0000","last_undegraded":"2024-05-21T15:16:09.430144+0000","last_fullsized":"2024-05-21T15:16:09.430144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:01:16.101793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975843+0000","last_change":"2024-05-21T15:14:12.262283+0000","last_active":"2024-05-21T15:16:08.975843+0000","last_peered":"2024-05-21T15:16:08.975843+0000","last_clean":"2024-05-21T15:16:08.975843+0000","last_became_active":"2024-05-21T15:14:12.261610+0000","last_became_peered":"2024-05-21T15:14:12.261610+0000","last_unstale":"2024-05-21T15:16:08.975843+0000","last_undegraded":"2024-05-21T15:16:08.975843+0000","last_fullsized":"2024-05-21T15:16:08.975843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:44:33.087115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.904945+0000","last_change":"2024-05-21T15:14:11.569905+0000","last_active":"2024-05-21T15:16:08.904945+0000","last_peered":"2024-05-21T15:16:08.904945+0000","last_clean":"2024-05-21T15:16:08.904945+0000","last_became_active":"2024-05-21T15:14:11.569221+0000","last_became_peered":"2024-05-21T15:14:11.569221+0000","last_unstale":"2024-05-21T15:16:08.904945+0000","last_undegraded":"2024-05-21T15:16:08.904945+0000","last_fullsized":"2024-05-21T15:16:08.904945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:11:04.875890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.815306+0000","last_change":"2024-05-21T15:15:16.386384+0000","last_active":"2024-05-21T15:16:08.815306+0000","last_peered":"2024-05-21T15:16:08.815306+0000","last_clean":"2024-05-21T15:16:08.815306+0000","last_became_active":"2024-05-21T15:15:16.385810+0000","last_became_peered":"2024-05-21T15:15:16.385810+0000","last_unstale":"2024-05-21T15:16:08.815306+0000","last_undegraded":"2024-05-21T15:16:08.815306+0000","last_fullsized":"2024-05-21T15:16:08.815306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:49:45.031393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905410+0000","last_change":"2024-05-21T15:14:11.569670+0000","last_active":"2024-05-21T15:16:08.905410+0000","last_peered":"2024-05-21T15:16:08.905410+0000","last_clean":"2024-05-21T15:16:08.905410+0000","last_became_active":"2024-05-21T15:14:11.569257+0000","last_became_peered":"2024-05-21T15:14:11.569257+0000","last_unstale":"2024-05-21T15:16:08.905410+0000","last_undegraded":"2024-05-21T15:16:08.905410+0000","last_fullsized":"2024-05-21T15:16:08.905410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:18.584920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705771+0000","last_change":"2024-05-21T15:15:16.374845+0000","last_active":"2024-05-21T15:16:08.705771+0000","last_peered":"2024-05-21T15:16:08.705771+0000","last_clean":"2024-05-21T15:16:08.705771+0000","last_became_active":"2024-05-21T15:15:16.374393+0000","last_became_peered":"2024-05-21T15:15:16.374393+0000","last_unstale":"2024-05-21T15:16:08.705771+0000","last_undegraded":"2024-05-21T15:16:08.705771+0000","last_fullsized":"2024-05-21T15:16:08.705771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:38:21.472453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.704254+0000","last_change":"2024-05-21T15:14:11.567849+0000","last_active":"2024-05-21T15:16:08.704254+0000","last_peered":"2024-05-21T15:16:08.704254+0000","last_clean":"2024-05-21T15:16:08.704254+0000","last_became_active":"2024-05-21T15:14:11.567476+0000","last_became_peered":"2024-05-21T15:14:11.567476+0000","last_unstale":"2024-05-21T15:16:08.704254+0000","last_undegraded":"2024-05-21T15:16:08.704254+0000","last_fullsized":"2024-05-21T15:16:08.704254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:13.516765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705800+0000","last_change":"2024-05-21T15:14:12.255165+0000","last_active":"2024-05-21T15:16:08.705800+0000","last_peered":"2024-05-21T15:16:08.705800+0000","last_clean":"2024-05-21T15:16:08.705800+0000","last_became_active":"2024-05-21T15:14:12.254743+0000","last_became_peered":"2024-05-21T15:14:12.254743+0000","last_unstale":"2024-05-21T15:16:08.705800+0000","last_undegraded":"2024-05-21T15:16:08.705800+0000","last_fullsized":"2024-05-21T15:16:08.705800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:53:11.059597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'8","reported_seq":143,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.690138+0000","last_change":"2024-05-21T15:14:11.569745+0000","last_active":"2024-05-21T15:16:09.690138+0000","last_peered":"2024-05-21T15:16:09.690138+0000","last_clean":"2024-05-21T15:16:09.690138+0000","last_became_active":"2024-05-21T15:14:11.569409+0000","last_became_peered":"2024-05-21T15:14:11.569409+0000","last_unstale":"2024-05-21T15:16:09.690138+0000","last_undegraded":"2024-05-21T15:16:09.690138+0000","last_fullsized":"2024-05-21T15:16:09.690138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:00:37.963797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1678,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705802+0000","last_change":"2024-05-21T15:14:12.254631+0000","last_active":"2024-05-21T15:16:08.705802+0000","last_peered":"2024-05-21T15:16:08.705802+0000","last_clean":"2024-05-21T15:16:08.705802+0000","last_became_active":"2024-05-21T15:14:12.254239+0000","last_became_peered":"2024-05-21T15:14:12.254239+0000","last_unstale":"2024-05-21T15:16:08.705802+0000","last_undegraded":"2024-05-21T15:16:08.705802+0000","last_fullsized":"2024-05-21T15:16:08.705802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:55:17.135550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.703987+0000","last_change":"2024-05-21T15:15:16.615801+0000","last_active":"2024-05-21T15:16:08.703987+0000","last_peered":"2024-05-21T15:16:08.703987+0000","last_clean":"2024-05-21T15:16:08.703987+0000","last_became_active":"2024-05-21T15:15:16.615323+0000","last_became_peered":"2024-05-21T15:15:16.615323+0000","last_unstale":"2024-05-21T15:16:08.703987+0000","last_undegraded":"2024-05-21T15:16:08.703987+0000","last_fullsized":"2024-05-21T15:16:08.703987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:40:15.233980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.982372+0000","last_change":"2024-05-21T15:14:11.574352+0000","last_active":"2024-05-21T15:16:08.982372+0000","last_peered":"2024-05-21T15:16:08.982372+0000","last_clean":"2024-05-21T15:16:08.982372+0000","last_became_active":"2024-05-21T15:14:11.574041+0000","last_became_peered":"2024-05-21T15:14:11.574041+0000","last_unstale":"2024-05-21T15:16:08.982372+0000","last_undegraded":"2024-05-21T15:16:08.982372+0000","last_fullsized":"2024-05-21T15:16:08.982372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:32.563218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808128+0000","last_change":"2024-05-21T15:14:12.257886+0000","last_active":"2024-05-21T15:16:08.808128+0000","last_peered":"2024-05-21T15:16:08.808128+0000","last_clean":"2024-05-21T15:16:08.808128+0000","last_became_active":"2024-05-21T15:14:12.257546+0000","last_became_peered":"2024-05-21T15:14:12.257546+0000","last_unstale":"2024-05-21T15:16:08.808128+0000","last_undegraded":"2024-05-21T15:16:08.808128+0000","last_fullsized":"2024-05-21T15:16:08.808128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:11:20.953421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972256+0000","last_change":"2024-05-21T15:14:11.568524+0000","last_active":"2024-05-21T15:16:08.972256+0000","last_peered":"2024-05-21T15:16:08.972256+0000","last_clean":"2024-05-21T15:16:08.972256+0000","last_became_active":"2024-05-21T15:14:11.567981+0000","last_became_peered":"2024-05-21T15:14:11.567981+0000","last_unstale":"2024-05-21T15:16:08.972256+0000","last_undegraded":"2024-05-21T15:16:08.972256+0000","last_fullsized":"2024-05-21T15:16:08.972256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.521409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.975897+0000","last_change":"2024-05-21T15:13:02.249260+0000","last_active":"2024-05-21T15:16:08.975897+0000","last_peered":"2024-05-21T15:16:08.975897+0000","last_clean":"2024-05-21T15:16:08.975897+0000","last_became_active":"2024-05-21T15:13:02.248637+0000","last_became_peered":"2024-05-21T15:13:02.248637+0000","last_unstale":"2024-05-21T15:16:08.975897+0000","last_undegraded":"2024-05-21T15:16:08.975897+0000","last_fullsized":"2024-05-21T15:16:08.975897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:09:41.506463+0000","last_clean_scrub_stamp":"2024-05-21T15:09:41.506463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:49:26.250910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979324+0000","last_change":"2024-05-21T15:14:12.259861+0000","last_active":"2024-05-21T15:16:08.979324+0000","last_peered":"2024-05-21T15:16:08.979324+0000","last_clean":"2024-05-21T15:16:08.979324+0000","last_became_active":"2024-05-21T15:14:12.259510+0000","last_became_peered":"2024-05-21T15:14:12.259510+0000","last_unstale":"2024-05-21T15:16:08.979324+0000","last_undegraded":"2024-05-21T15:16:08.979324+0000","last_fullsized":"2024-05-21T15:16:08.979324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:36:49.548366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976144+0000","last_change":"2024-05-21T15:14:11.570705+0000","last_active":"2024-05-21T15:16:08.976144+0000","last_peered":"2024-05-21T15:16:08.976144+0000","last_clean":"2024-05-21T15:16:08.976144+0000","last_became_active":"2024-05-21T15:14:11.570403+0000","last_became_peered":"2024-05-21T15:14:11.570403+0000","last_unstale":"2024-05-21T15:16:08.976144+0000","last_undegraded":"2024-05-21T15:16:08.976144+0000","last_fullsized":"2024-05-21T15:16:08.976144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:02:34.156258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905269+0000","last_change":"2024-05-21T15:14:11.572311+0000","last_active":"2024-05-21T15:16:08.905269+0000","last_peered":"2024-05-21T15:16:08.905269+0000","last_clean":"2024-05-21T15:16:08.905269+0000","last_became_active":"2024-05-21T15:14:11.571949+0000","last_became_peered":"2024-05-21T15:14:11.571949+0000","last_unstale":"2024-05-21T15:16:08.905269+0000","last_undegraded":"2024-05-21T15:16:08.905269+0000","last_fullsized":"2024-05-21T15:16:08.905269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:57:59.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979695+0000","last_change":"2024-05-21T15:14:12.260717+0000","last_active":"2024-05-21T15:16:08.979695+0000","last_peered":"2024-05-21T15:16:08.979695+0000","last_clean":"2024-05-21T15:16:08.979695+0000","last_became_active":"2024-05-21T15:14:12.259911+0000","last_became_peered":"2024-05-21T15:14:12.259911+0000","last_unstale":"2024-05-21T15:16:08.979695+0000","last_undegraded":"2024-05-21T15:16:08.979695+0000","last_fullsized":"2024-05-21T15:16:08.979695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:02.609299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.933483+0000","last_change":"2024-05-21T15:15:16.381952+0000","last_active":"2024-05-21T15:16:08.933483+0000","last_peered":"2024-05-21T15:16:08.933483+0000","last_clean":"2024-05-21T15:16:08.933483+0000","last_became_active":"2024-05-21T15:15:16.373966+0000","last_became_peered":"2024-05-21T15:15:16.373966+0000","last_unstale":"2024-05-21T15:16:08.933483+0000","last_undegraded":"2024-05-21T15:16:08.933483+0000","last_fullsized":"2024-05-21T15:16:08.933483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:49:05.368066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809256+0000","last_change":"2024-05-21T15:14:12.253634+0000","last_active":"2024-05-21T15:16:08.809256+0000","last_peered":"2024-05-21T15:16:08.809256+0000","last_clean":"2024-05-21T15:16:08.809256+0000","last_became_active":"2024-05-21T15:14:12.253284+0000","last_became_peered":"2024-05-21T15:14:12.253284+0000","last_unstale":"2024-05-21T15:16:08.809256+0000","last_undegraded":"2024-05-21T15:16:08.809256+0000","last_fullsized":"2024-05-21T15:16:08.809256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:28:07.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.972367+0000","last_change":"2024-05-21T15:14:11.574147+0000","last_active":"2024-05-21T15:16:08.972367+0000","last_peered":"2024-05-21T15:16:08.972367+0000","last_clean":"2024-05-21T15:16:08.972367+0000","last_became_active":"2024-05-21T15:14:11.573721+0000","last_became_peered":"2024-05-21T15:14:11.573721+0000","last_unstale":"2024-05-21T15:16:08.972367+0000","last_undegraded":"2024-05-21T15:16:08.972367+0000","last_fullsized":"2024-05-21T15:16:08.972367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:10.374364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983413+0000","last_change":"2024-05-21T15:14:12.258632+0000","last_active":"2024-05-21T15:16:08.983413+0000","last_peered":"2024-05-21T15:16:08.983413+0000","last_clean":"2024-05-21T15:16:08.983413+0000","last_became_active":"2024-05-21T15:14:12.257908+0000","last_became_peered":"2024-05-21T15:14:12.257908+0000","last_unstale":"2024-05-21T15:16:08.983413+0000","last_undegraded":"2024-05-21T15:16:08.983413+0000","last_fullsized":"2024-05-21T15:16:08.983413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:17:12.062387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.978955+0000","last_change":"2024-05-21T15:14:11.562592+0000","last_active":"2024-05-21T15:16:08.978955+0000","last_peered":"2024-05-21T15:16:08.978955+0000","last_clean":"2024-05-21T15:16:08.978955+0000","last_became_active":"2024-05-21T15:14:11.562265+0000","last_became_peered":"2024-05-21T15:14:11.562265+0000","last_unstale":"2024-05-21T15:16:08.978955+0000","last_undegraded":"2024-05-21T15:16:08.978955+0000","last_fullsized":"2024-05-21T15:16:08.978955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:24.662576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905370+0000","last_change":"2024-05-21T15:14:12.249207+0000","last_active":"2024-05-21T15:16:08.905370+0000","last_peered":"2024-05-21T15:16:08.905370+0000","last_clean":"2024-05-21T15:16:08.905370+0000","last_became_active":"2024-05-21T15:14:12.248819+0000","last_became_peered":"2024-05-21T15:14:12.248819+0000","last_unstale":"2024-05-21T15:16:08.905370+0000","last_undegraded":"2024-05-21T15:16:08.905370+0000","last_fullsized":"2024-05-21T15:16:08.905370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:38.208636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807711+0000","last_change":"2024-05-21T15:14:11.572044+0000","last_active":"2024-05-21T15:16:08.807711+0000","last_peered":"2024-05-21T15:16:08.807711+0000","last_clean":"2024-05-21T15:16:08.807711+0000","last_became_active":"2024-05-21T15:14:11.570721+0000","last_became_peered":"2024-05-21T15:14:11.570721+0000","last_unstale":"2024-05-21T15:16:08.807711+0000","last_undegraded":"2024-05-21T15:16:08.807711+0000","last_fullsized":"2024-05-21T15:16:08.807711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:34:08.413781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705475+0000","last_change":"2024-05-21T15:14:12.256327+0000","last_active":"2024-05-21T15:16:08.705475+0000","last_peered":"2024-05-21T15:16:08.705475+0000","last_clean":"2024-05-21T15:16:08.705475+0000","last_became_active":"2024-05-21T15:14:12.255962+0000","last_became_peered":"2024-05-21T15:14:12.255962+0000","last_unstale":"2024-05-21T15:16:08.705475+0000","last_undegraded":"2024-05-21T15:16:08.705475+0000","last_fullsized":"2024-05-21T15:16:08.705475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:05.076323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705571+0000","last_change":"2024-05-21T15:15:16.385658+0000","last_active":"2024-05-21T15:16:08.705571+0000","last_peered":"2024-05-21T15:16:08.705571+0000","last_clean":"2024-05-21T15:16:08.705571+0000","last_became_active":"2024-05-21T15:15:16.375122+0000","last_became_peered":"2024-05-21T15:15:16.375122+0000","last_unstale":"2024-05-21T15:16:08.705571+0000","last_undegraded":"2024-05-21T15:16:08.705571+0000","last_fullsized":"2024-05-21T15:16:08.705571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:57:59.599504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.808366+0000","last_change":"2024-05-21T15:14:12.256993+0000","last_active":"2024-05-21T15:16:08.808366+0000","last_peered":"2024-05-21T15:16:08.808366+0000","last_clean":"2024-05-21T15:16:08.808366+0000","last_became_active":"2024-05-21T15:14:12.256179+0000","last_became_peered":"2024-05-21T15:14:12.256179+0000","last_unstale":"2024-05-21T15:16:08.808366+0000","last_undegraded":"2024-05-21T15:16:08.808366+0000","last_fullsized":"2024-05-21T15:16:08.808366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:01:09.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.905244+0000","last_change":"2024-05-21T15:15:16.391272+0000","last_active":"2024-05-21T15:16:08.905244+0000","last_peered":"2024-05-21T15:16:08.905244+0000","last_clean":"2024-05-21T15:16:08.905244+0000","last_became_active":"2024-05-21T15:15:16.386921+0000","last_became_peered":"2024-05-21T15:15:16.386921+0000","last_unstale":"2024-05-21T15:16:08.905244+0000","last_undegraded":"2024-05-21T15:16:08.905244+0000","last_fullsized":"2024-05-21T15:16:08.905244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:01:55.466989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976147+0000","last_change":"2024-05-21T15:14:12.260506+0000","last_active":"2024-05-21T15:16:08.976147+0000","last_peered":"2024-05-21T15:16:08.976147+0000","last_clean":"2024-05-21T15:16:08.976147+0000","last_became_active":"2024-05-21T15:14:12.260168+0000","last_became_peered":"2024-05-21T15:14:12.260168+0000","last_unstale":"2024-05-21T15:16:08.976147+0000","last_undegraded":"2024-05-21T15:16:08.976147+0000","last_fullsized":"2024-05-21T15:16:08.976147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:55:11.321622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:09.479294+0000","last_change":"2024-05-21T15:14:11.568630+0000","last_active":"2024-05-21T15:16:09.479294+0000","last_peered":"2024-05-21T15:16:09.479294+0000","last_clean":"2024-05-21T15:16:09.479294+0000","last_became_active":"2024-05-21T15:14:11.568229+0000","last_became_peered":"2024-05-21T15:14:11.568229+0000","last_unstale":"2024-05-21T15:16:09.479294+0000","last_undegraded":"2024-05-21T15:16:09.479294+0000","last_fullsized":"2024-05-21T15:16:09.479294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:30:21.372395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976294+0000","last_change":"2024-05-21T15:14:12.259408+0000","last_active":"2024-05-21T15:16:08.976294+0000","last_peered":"2024-05-21T15:16:08.976294+0000","last_clean":"2024-05-21T15:16:08.976294+0000","last_became_active":"2024-05-21T15:14:12.259082+0000","last_became_peered":"2024-05-21T15:14:12.259082+0000","last_unstale":"2024-05-21T15:16:08.976294+0000","last_undegraded":"2024-05-21T15:16:08.976294+0000","last_fullsized":"2024-05-21T15:16:08.976294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:32:09.059307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.930782+0000","last_change":"2024-05-21T15:14:11.571077+0000","last_active":"2024-05-21T15:16:08.930782+0000","last_peered":"2024-05-21T15:16:08.930782+0000","last_clean":"2024-05-21T15:16:08.930782+0000","last_became_active":"2024-05-21T15:14:11.570664+0000","last_became_peered":"2024-05-21T15:14:11.570664+0000","last_unstale":"2024-05-21T15:16:08.930782+0000","last_undegraded":"2024-05-21T15:16:08.930782+0000","last_fullsized":"2024-05-21T15:16:08.930782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:37:16.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976028+0000","last_change":"2024-05-21T15:14:12.252972+0000","last_active":"2024-05-21T15:16:08.976028+0000","last_peered":"2024-05-21T15:16:08.976028+0000","last_clean":"2024-05-21T15:16:08.976028+0000","last_became_active":"2024-05-21T15:14:12.252591+0000","last_became_peered":"2024-05-21T15:14:12.252591+0000","last_unstale":"2024-05-21T15:16:08.976028+0000","last_undegraded":"2024-05-21T15:16:08.976028+0000","last_fullsized":"2024-05-21T15:16:08.976028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:14:16.430371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979634+0000","last_change":"2024-05-21T15:14:11.577138+0000","last_active":"2024-05-21T15:16:08.979634+0000","last_peered":"2024-05-21T15:16:08.979634+0000","last_clean":"2024-05-21T15:16:08.979634+0000","last_became_active":"2024-05-21T15:14:11.576629+0000","last_became_peered":"2024-05-21T15:14:11.576629+0000","last_unstale":"2024-05-21T15:16:08.979634+0000","last_undegraded":"2024-05-21T15:16:08.979634+0000","last_fullsized":"2024-05-21T15:16:08.979634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:10.545606+0000","last_clean_scrub_stamp":"2024-05-21T15:14:10.545606+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:20:20.134764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976368+0000","last_change":"2024-05-21T15:14:12.256174+0000","last_active":"2024-05-21T15:16:08.976368+0000","last_peered":"2024-05-21T15:16:08.976368+0000","last_clean":"2024-05-21T15:16:08.976368+0000","last_became_active":"2024-05-21T15:14:12.255839+0000","last_became_peered":"2024-05-21T15:14:12.255839+0000","last_unstale":"2024-05-21T15:16:08.976368+0000","last_undegraded":"2024-05-21T15:16:08.976368+0000","last_fullsized":"2024-05-21T15:16:08.976368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:12:40.743118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705604+0000","last_change":"2024-05-21T15:14:12.257028+0000","last_active":"2024-05-21T15:16:08.705604+0000","last_peered":"2024-05-21T15:16:08.705604+0000","last_clean":"2024-05-21T15:16:08.705604+0000","last_became_active":"2024-05-21T15:14:12.256611+0000","last_became_peered":"2024-05-21T15:14:12.256611+0000","last_unstale":"2024-05-21T15:16:08.705604+0000","last_undegraded":"2024-05-21T15:16:08.705604+0000","last_fullsized":"2024-05-21T15:16:08.705604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:08:43.881073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976078+0000","last_change":"2024-05-21T15:14:12.255724+0000","last_active":"2024-05-21T15:16:08.976078+0000","last_peered":"2024-05-21T15:16:08.976078+0000","last_clean":"2024-05-21T15:16:08.976078+0000","last_became_active":"2024-05-21T15:14:12.255272+0000","last_became_peered":"2024-05-21T15:14:12.255272+0000","last_unstale":"2024-05-21T15:16:08.976078+0000","last_undegraded":"2024-05-21T15:16:08.976078+0000","last_fullsized":"2024-05-21T15:16:08.976078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:07:07.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976048+0000","last_change":"2024-05-21T15:14:12.262508+0000","last_active":"2024-05-21T15:16:08.976048+0000","last_peered":"2024-05-21T15:16:08.976048+0000","last_clean":"2024-05-21T15:16:08.976048+0000","last_became_active":"2024-05-21T15:14:12.262155+0000","last_became_peered":"2024-05-21T15:14:12.262155+0000","last_unstale":"2024-05-21T15:16:08.976048+0000","last_undegraded":"2024-05-21T15:16:08.976048+0000","last_fullsized":"2024-05-21T15:16:08.976048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:54:19.982044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.983316+0000","last_change":"2024-05-21T15:14:12.251715+0000","last_active":"2024-05-21T15:16:08.983316+0000","last_peered":"2024-05-21T15:16:08.983316+0000","last_clean":"2024-05-21T15:16:08.983316+0000","last_became_active":"2024-05-21T15:14:12.251337+0000","last_became_peered":"2024-05-21T15:14:12.251337+0000","last_unstale":"2024-05-21T15:16:08.983316+0000","last_undegraded":"2024-05-21T15:16:08.983316+0000","last_fullsized":"2024-05-21T15:16:08.983316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:03.562179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979661+0000","last_change":"2024-05-21T15:14:12.253166+0000","last_active":"2024-05-21T15:16:08.979661+0000","last_peered":"2024-05-21T15:16:08.979661+0000","last_clean":"2024-05-21T15:16:08.979661+0000","last_became_active":"2024-05-21T15:14:12.251830+0000","last_became_peered":"2024-05-21T15:14:12.251830+0000","last_unstale":"2024-05-21T15:16:08.979661+0000","last_undegraded":"2024-05-21T15:16:08.979661+0000","last_fullsized":"2024-05-21T15:16:08.979661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:11:58.679568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705713+0000","last_change":"2024-05-21T15:14:12.256131+0000","last_active":"2024-05-21T15:16:08.705713+0000","last_peered":"2024-05-21T15:16:08.705713+0000","last_clean":"2024-05-21T15:16:08.705713+0000","last_became_active":"2024-05-21T15:14:12.255772+0000","last_became_peered":"2024-05-21T15:14:12.255772+0000","last_unstale":"2024-05-21T15:16:08.705713+0000","last_undegraded":"2024-05-21T15:16:08.705713+0000","last_fullsized":"2024-05-21T15:16:08.705713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:26:48.047278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.976512+0000","last_change":"2024-05-21T15:14:12.260829+0000","last_active":"2024-05-21T15:16:08.976512+0000","last_peered":"2024-05-21T15:16:08.976512+0000","last_clean":"2024-05-21T15:16:08.976512+0000","last_became_active":"2024-05-21T15:14:12.259686+0000","last_became_peered":"2024-05-21T15:14:12.259686+0000","last_unstale":"2024-05-21T15:16:08.976512+0000","last_undegraded":"2024-05-21T15:16:08.976512+0000","last_fullsized":"2024-05-21T15:16:08.976512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:48.480814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.809168+0000","last_change":"2024-05-21T15:14:12.262892+0000","last_active":"2024-05-21T15:16:08.809168+0000","last_peered":"2024-05-21T15:16:08.809168+0000","last_clean":"2024-05-21T15:16:08.809168+0000","last_became_active":"2024-05-21T15:14:12.262532+0000","last_became_peered":"2024-05-21T15:14:12.262532+0000","last_unstale":"2024-05-21T15:16:08.809168+0000","last_undegraded":"2024-05-21T15:16:08.809168+0000","last_fullsized":"2024-05-21T15:16:08.809168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T17:40:22.424783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.904921+0000","last_change":"2024-05-21T15:15:16.375027+0000","last_active":"2024-05-21T15:16:08.904921+0000","last_peered":"2024-05-21T15:16:08.904921+0000","last_clean":"2024-05-21T15:16:08.904921+0000","last_became_active":"2024-05-21T15:15:16.374542+0000","last_became_peered":"2024-05-21T15:15:16.374542+0000","last_unstale":"2024-05-21T15:16:08.904921+0000","last_undegraded":"2024-05-21T15:16:08.904921+0000","last_fullsized":"2024-05-21T15:16:08.904921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:55:14.399237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.807833+0000","last_change":"2024-05-21T15:14:12.259845+0000","last_active":"2024-05-21T15:16:08.807833+0000","last_peered":"2024-05-21T15:16:08.807833+0000","last_clean":"2024-05-21T15:16:08.807833+0000","last_became_active":"2024-05-21T15:14:12.259455+0000","last_became_peered":"2024-05-21T15:14:12.259455+0000","last_unstale":"2024-05-21T15:16:08.807833+0000","last_undegraded":"2024-05-21T15:16:08.807833+0000","last_fullsized":"2024-05-21T15:16:08.807833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:24:59.655178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.979108+0000","last_change":"2024-05-21T15:14:12.259713+0000","last_active":"2024-05-21T15:16:08.979108+0000","last_peered":"2024-05-21T15:16:08.979108+0000","last_clean":"2024-05-21T15:16:08.979108+0000","last_became_active":"2024-05-21T15:14:12.259057+0000","last_became_peered":"2024-05-21T15:14:12.259057+0000","last_unstale":"2024-05-21T15:16:08.979108+0000","last_undegraded":"2024-05-21T15:16:08.979108+0000","last_fullsized":"2024-05-21T15:16:08.979108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T15:46:22.377652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.980055+0000","last_change":"2024-05-21T15:14:12.254389+0000","last_active":"2024-05-21T15:16:08.980055+0000","last_peered":"2024-05-21T15:16:08.980055+0000","last_clean":"2024-05-21T15:16:08.980055+0000","last_became_active":"2024-05-21T15:14:12.253846+0000","last_became_peered":"2024-05-21T15:14:12.253846+0000","last_unstale":"2024-05-21T15:16:08.980055+0000","last_undegraded":"2024-05-21T15:16:08.980055+0000","last_fullsized":"2024-05-21T15:16:08.980055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T16:53:32.932316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-21T15:16:08.705619+0000","last_change":"2024-05-21T15:14:12.252292+0000","last_active":"2024-05-21T15:16:08.705619+0000","last_peered":"2024-05-21T15:16:08.705619+0000","last_clean":"2024-05-21T15:16:08.705619+0000","last_became_active":"2024-05-21T15:14:12.251816+0000","last_became_peered":"2024-05-21T15:14:12.251816+0000","last_unstale":"2024-05-21T15:16:08.705619+0000","last_undegraded":"2024-05-21T15:16:08.705619+0000","last_fullsized":"2024-05-21T15:16:08.705619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T15:14:11.547368+0000","last_clean_scrub_stamp":"2024-05-21T15:14:11.547368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:59:14.332010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600100,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1256,"num_read_kb":736,"num_write":376,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":672,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1800300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383716,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28356,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716796,"statfs":{"total":95995035648,"available":95965999104,"internally_reserved":0,"allocated":1368064,"data_stored":859573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28788,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716364,"statfs":{"total":95995035648,"available":95965556736,"internally_reserved":0,"allocated":1814528,"data_stored":1254662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1392640,"data_stored":859112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":465602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1576960,"data_stored":1050256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":466703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":533292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":457967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":534127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":456453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":466537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32936,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712216,"statfs":{"total":95995035648,"available":95961309184,"internally_reserved":0,"allocated":1740800,"data_stored":1190993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":199403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":394799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T15:16:10.971 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:16:11.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 -- 172.21.15.78:0/904568715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be0101f00 msgr2=0x7f6be0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 --2- 172.21.15.78:0/904568715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be0101f00 0x7f6be0102310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009900 tx=0x7f6bd002a410 comp rx=0 tx=0).stop 2024-05-21T15:16:11.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 -- 172.21.15.78:0/904568715 shutdown_connections 2024-05-21T15:16:11.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 --2- 172.21.15.78:0/904568715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6be0103400 0x7f6be0109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 --2- 172.21.15.78:0/904568715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be01028e0 0x7f6be0102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 --2- 172.21.15.78:0/904568715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be0101f00 0x7f6be0102310 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 -- 172.21.15.78:0/904568715 >> 172.21.15.78:0/904568715 conn(0x7f6be00fc2e0 msgr2=0x7f6be00fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 -- 172.21.15.78:0/904568715 shutdown_connections 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.124+0000 7f6be5644640 1 -- 172.21.15.78:0/904568715 wait complete. 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6be5644640 1 Processor -- start 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6be5644640 1 -- start start 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6be5644640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be0101f00 0x7f6be01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6be5644640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be01028e0 0x7f6be01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6be5644640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6be0103400 0x7f6be019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6be5644640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010d5f0 con 0x7f6be01028e0 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6be5644640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010d470 con 0x7f6be0103400 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6be5644640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6be010d770 con 0x7f6be0101f00 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bd7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be01028e0 0x7f6be01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bd7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be01028e0 0x7f6be01986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46934/0 (socket says 172.21.15.78:46934) 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bd7fff640 1 -- 172.21.15.78:0/1265977830 learned_addr learned my addr 172.21.15.78:0/1265977830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bdeffd640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be0101f00 0x7f6be01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bdf7fe640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6be0103400 0x7f6be019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bd7fff640 1 -- 172.21.15.78:0/1265977830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be0101f00 msgr2=0x7f6be01981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bd7fff640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be0101f00 0x7f6be01981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bd7fff640 1 -- 172.21.15.78:0/1265977830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6be0103400 msgr2=0x7f6be019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bd7fff640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6be0103400 0x7f6be019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.125+0000 7f6bd7fff640 1 -- 172.21.15.78:0/1265977830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be019fce0 con 0x7f6be01028e0 2024-05-21T15:16:11.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.126+0000 7f6bd7fff640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be01028e0 0x7f6be01986e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800e9f0 tx=0x7f6bc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:11.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.126+0000 7f6bdcff9640 1 -- 172.21.15.78:0/1265977830 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800cd80 con 0x7f6be01028e0 2024-05-21T15:16:11.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.126+0000 7f6bdcff9640 1 -- 172.21.15.78:0/1265977830 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc8005500 con 0x7f6be01028e0 2024-05-21T15:16:11.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.126+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be019ffd0 con 0x7f6be01028e0 2024-05-21T15:16:11.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.126+0000 7f6bdcff9640 1 -- 172.21.15.78:0/1265977830 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8010690 con 0x7f6be01028e0 2024-05-21T15:16:11.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.126+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be01a0430 con 0x7f6be01028e0 2024-05-21T15:16:11.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.127+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be0106310 con 0x7f6be01028e0 2024-05-21T15:16:11.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.128+0000 7f6bdcff9640 1 -- 172.21.15.78:0/1265977830 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f6bc80046c0 con 0x7f6be01028e0 2024-05-21T15:16:11.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.129+0000 7f6bdcff9640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bcc0a2d00 0x7f6bcc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.129+0000 7f6bdeffd640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bcc0a2d00 0x7f6bcc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.129+0000 7f6bdcff9640 1 -- 172.21.15.78:0/1265977830 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6bc80cfa60 con 0x7f6be01028e0 2024-05-21T15:16:11.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.130+0000 7f6bdeffd640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bcc0a2d00 0x7f6bcc0a5210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009230 tx=0x7f6bd0031400 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:11.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.133+0000 7f6bdcff9640 1 -- 172.21.15.78:0/1265977830 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bc8053850 con 0x7f6be01028e0 2024-05-21T15:16:11.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:10 smithi084 ceph-mon[36334]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:11.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:10 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2566135600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:11.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:10 smithi084 ceph-mon[36334]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:11.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:10 smithi172 ceph-mon[36402]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:11.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:10 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2566135600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:11.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:10 smithi172 ceph-mon[36402]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:11.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.319+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6be0100f20 con 0x7f6be01028e0 2024-05-21T15:16:11.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.320+0000 7f6bdcff9640 1 -- 172.21.15.78:0/1265977830 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22994 (secure 0 0 0) 0x7f6bc809c680 con 0x7f6be01028e0 2024-05-21T15:16:11.323 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:11.323 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:16:08.699024+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6801/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2012910515":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2194549734":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/3526992601":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2604499013":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:16:11.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bcc0a2d00 msgr2=0x7f6bcc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bcc0a2d00 0x7f6bcc0a5210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009230 tx=0x7f6bd0031400 comp rx=0 tx=0).stop 2024-05-21T15:16:11.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be01028e0 msgr2=0x7f6be01986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be01028e0 0x7f6be01986e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800e9f0 tx=0x7f6bc800edf0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 shutdown_connections 2024-05-21T15:16:11.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6be0103400 0x7f6be019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bcc0a2d00 0x7f6bcc0a5210 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6be01028e0 0x7f6be01986e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 --2- 172.21.15.78:0/1265977830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be0101f00 0x7f6be01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 >> 172.21.15.78:0/1265977830 conn(0x7f6be00fc2e0 msgr2=0x7f6be0103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 shutdown_connections 2024-05-21T15:16:11.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.322+0000 7f6be5644640 1 -- 172.21.15.78:0/1265977830 wait complete. 2024-05-21T15:16:11.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:16:11.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.457+0000 7f4711509640 1 -- 172.21.15.78:0/4125233823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 msgr2=0x7f470c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.457+0000 7f4711509640 1 --2- 172.21.15.78:0/4125233823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 0x7f470c102340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009900 tx=0x7f46fc02a410 comp rx=0 tx=0).stop 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.457+0000 7f4711509640 1 -- 172.21.15.78:0/4125233823 shutdown_connections 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.457+0000 7f4711509640 1 --2- 172.21.15.78:0/4125233823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f470c102910 0x7f470c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.457+0000 7f4711509640 1 --2- 172.21.15.78:0/4125233823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c103410 0x7f470c109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.457+0000 7f4711509640 1 --2- 172.21.15.78:0/4125233823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 0x7f470c102340 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.457+0000 7f4711509640 1 -- 172.21.15.78:0/4125233823 >> 172.21.15.78:0/4125233823 conn(0x7f470c0fc2f0 msgr2=0x7f470c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.457+0000 7f4711509640 1 -- 172.21.15.78:0/4125233823 shutdown_connections 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.458+0000 7f4711509640 1 -- 172.21.15.78:0/4125233823 wait complete. 2024-05-21T15:16:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.458+0000 7f4711509640 1 Processor -- start 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.458+0000 7f4711509640 1 -- start start 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f4711509640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 0x7f470c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f4711509640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f470c102910 0x7f470c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f4711509640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c103410 0x7f470c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f4711509640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f470c10d5e0 con 0x7f470c101f30 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f4711509640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f470c10d460 con 0x7f470c102910 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f4711509640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f470c10d760 con 0x7f470c103410 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 0x7f470c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 0x7f470c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46946/0 (socket says 172.21.15.78:46946) 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470affd640 1 -- 172.21.15.78:0/2784885919 learned_addr learned my addr 172.21.15.78:0/2784885919 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:11.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470a7fc640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f470c102910 0x7f470c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470b7fe640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c103410 0x7f470c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470a7fc640 1 -- 172.21.15.78:0/2784885919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c103410 msgr2=0x7f470c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470a7fc640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c103410 0x7f470c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470a7fc640 1 -- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 msgr2=0x7f470c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470a7fc640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 0x7f470c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470a7fc640 1 -- 172.21.15.78:0/2784885919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c19fce0 con 0x7f470c102910 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470b7fe640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c103410 0x7f470c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.459+0000 7f470a7fc640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f470c102910 0x7f470c1986e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4700005fd0 tx=0x7f470000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.460+0000 7f46f3fff640 1 -- 172.21.15.78:0/2784885919 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700009c30 con 0x7f470c102910 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.460+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c19ffd0 con 0x7f470c102910 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.460+0000 7f46f3fff640 1 -- 172.21.15.78:0/2784885919 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4700009dd0 con 0x7f470c102910 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.460+0000 7f46f3fff640 1 -- 172.21.15.78:0/2784885919 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700010460 con 0x7f470c102910 2024-05-21T15:16:11.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.460+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c1a0440 con 0x7f470c102910 2024-05-21T15:16:11.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.461+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c1068d0 con 0x7f470c102910 2024-05-21T15:16:11.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.462+0000 7f46f3fff640 1 -- 172.21.15.78:0/2784885919 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f4700004030 con 0x7f470c102910 2024-05-21T15:16:11.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.463+0000 7f46f3fff640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46f40a2d00 0x7f46f40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.463+0000 7f470affd640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46f40a2d00 0x7f46f40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.463+0000 7f46f3fff640 1 -- 172.21.15.78:0/2784885919 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f47000cf030 con 0x7f470c102910 2024-05-21T15:16:11.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.463+0000 7f470affd640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46f40a2d00 0x7f46f40a5210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f46fc031610 tx=0x7f46fc005d50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:11.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.468+0000 7f46f3fff640 1 -- 172.21.15.78:0/2784885919 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4700017020 con 0x7f470c102910 2024-05-21T15:16:11.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.655+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f470c02d4c0 con 0x7f470c102910 2024-05-21T15:16:11.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.656+0000 7f46f3fff640 1 -- 172.21.15.78:0/2784885919 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f470009b600 con 0x7f470c102910 2024-05-21T15:16:11.659 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:16:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46f40a2d00 msgr2=0x7f46f40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46f40a2d00 0x7f46f40a5210 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f46fc031610 tx=0x7f46fc005d50 comp rx=0 tx=0).stop 2024-05-21T15:16:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f470c102910 msgr2=0x7f470c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f470c102910 0x7f470c1986e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4700005fd0 tx=0x7f470000dde0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 shutdown_connections 2024-05-21T15:16:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f470c102910 0x7f470c1986e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46f40a2d00 0x7f46f40a5210 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c103410 0x7f470c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 --2- 172.21.15.78:0/2784885919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f470c101f30 0x7f470c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 >> 172.21.15.78:0/2784885919 conn(0x7f470c0fc2f0 msgr2=0x7f470c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 shutdown_connections 2024-05-21T15:16:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.658+0000 7f4711509640 1 -- 172.21.15.78:0/2784885919 wait complete. 2024-05-21T15:16:11.673 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:16:11.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 -- 172.21.15.78:0/825791592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 msgr2=0x7f738c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 --2- 172.21.15.78:0/825791592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 0x7f738c109cf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7384009ca0 tx=0x7f738402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 -- 172.21.15.78:0/825791592 shutdown_connections 2024-05-21T15:16:11.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 --2- 172.21.15.78:0/825791592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 0x7f738c109cf0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 --2- 172.21.15.78:0/825791592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c102bd0 0x7f738c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 --2- 172.21.15.78:0/825791592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f738c101990 0x7f738c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 -- 172.21.15.78:0/825791592 >> 172.21.15.78:0/825791592 conn(0x7f738c0fcb80 msgr2=0x7f738c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:11.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 -- 172.21.15.78:0/825791592 shutdown_connections 2024-05-21T15:16:11.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.789+0000 7f7391afe640 1 -- 172.21.15.78:0/825791592 wait complete. 2024-05-21T15:16:11.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f7391afe640 1 Processor -- start 2024-05-21T15:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f7391afe640 1 -- start start 2024-05-21T15:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f7391afe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c101990 0x7f738c0698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f7391afe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f738c102bd0 0x7f738c19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f7391afe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 0x7f738c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f7391afe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f738c10d610 con 0x7f738c1078d0 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f7391afe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f738c10d490 con 0x7f738c102bd0 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f7391afe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f738c10d790 con 0x7f738c101990 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f738bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 0x7f738c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f738bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 0x7f738c1a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46950/0 (socket says 172.21.15.78:46950) 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.790+0000 7f738bfff640 1 -- 172.21.15.78:0/2541555705 learned_addr learned my addr 172.21.15.78:0/2541555705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f738b7fe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c101990 0x7f738c0698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f738affd640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f738c102bd0 0x7f738c19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f738bfff640 1 -- 172.21.15.78:0/2541555705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c101990 msgr2=0x7f738c0698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f738bfff640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c101990 0x7f738c0698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f738bfff640 1 -- 172.21.15.78:0/2541555705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f738c102bd0 msgr2=0x7f738c19e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f738bfff640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f738c102bd0 0x7f738c19e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f738bfff640 1 -- 172.21.15.78:0/2541555705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c1a1050 con 0x7f738c1078d0 2024-05-21T15:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f738bfff640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 0x7f738c1a0830 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7384005e30 tx=0x7f7384031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f7388ff9640 1 -- 172.21.15.78:0/2541555705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384003b70 con 0x7f738c1078d0 2024-05-21T15:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c1a12e0 con 0x7f738c1078d0 2024-05-21T15:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c1a1880 con 0x7f738c1078d0 2024-05-21T15:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f7388ff9640 1 -- 172.21.15.78:0/2541555705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7384003d10 con 0x7f738c1078d0 2024-05-21T15:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.791+0000 7f7388ff9640 1 -- 172.21.15.78:0/2541555705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738402fb70 con 0x7f738c1078d0 2024-05-21T15:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.792+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73780052c0 con 0x7f738c1078d0 2024-05-21T15:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.794+0000 7f7388ff9640 1 -- 172.21.15.78:0/2541555705 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f738402fd10 con 0x7f738c1078d0 2024-05-21T15:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.794+0000 7f7388ff9640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f73740a2d00 0x7f73740a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.795+0000 7f738b7fe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f73740a2d00 0x7f73740a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.795+0000 7f7388ff9640 1 -- 172.21.15.78:0/2541555705 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f73840f04f0 con 0x7f738c1078d0 2024-05-21T15:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.795+0000 7f738b7fe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f73740a2d00 0x7f73740a5210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f738c102590 tx=0x7f737c005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:11.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.803+0000 7f7388ff9640 1 -- 172.21.15.78:0/2541555705 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73840bd2b0 con 0x7f738c1078d0 2024-05-21T15:16:11.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.989+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7378005880 con 0x7f738c1078d0 2024-05-21T15:16:11.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.990+0000 7f7388ff9640 1 -- 172.21.15.78:0/2541555705 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f73840bcbb0 con 0x7f738c1078d0 2024-05-21T15:16:11.993 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:16:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f73740a2d00 msgr2=0x7f73740a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f73740a2d00 0x7f73740a5210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f738c102590 tx=0x7f737c005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 msgr2=0x7f738c1a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 0x7f738c1a0830 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7384005e30 tx=0x7f7384031ee0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 shutdown_connections 2024-05-21T15:16:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f738c102bd0 0x7f738c19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f73740a2d00 0x7f73740a5210 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f738c1078d0 0x7f738c1a0830 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 --2- 172.21.15.78:0/2541555705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c101990 0x7f738c0698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 >> 172.21.15.78:0/2541555705 conn(0x7f738c0fcb80 msgr2=0x7f738c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 shutdown_connections 2024-05-21T15:16:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:11.992+0000 7f7391afe640 1 -- 172.21.15.78:0/2541555705 wait complete. 2024-05-21T15:16:12.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:12.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:12.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-21T15:16:12.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:12.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1265977830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:12.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:11 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2784885919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:12.008 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:16:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 -- 172.21.15.78:0/4257803353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781011f0 msgr2=0x7fd27810ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 --2- 172.21.15.78:0/4257803353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781011f0 0x7fd27810ac50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd270009ca0 tx=0x7fd27002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 -- 172.21.15.78:0/4257803353 shutdown_connections 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 --2- 172.21.15.78:0/4257803353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781011f0 0x7fd27810ac50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 --2- 172.21.15.78:0/4257803353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2781006f0 0x7fd278100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 --2- 172.21.15.78:0/4257803353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd278104040 0x7fd278104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 -- 172.21.15.78:0/4257803353 >> 172.21.15.78:0/4257803353 conn(0x7fd2780fc2f0 msgr2=0x7fd2780fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 -- 172.21.15.78:0/4257803353 shutdown_connections 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.121+0000 7fd27eb96640 1 -- 172.21.15.78:0/4257803353 wait complete. 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27eb96640 1 Processor -- start 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27eb96640 1 -- start start 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27eb96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781006f0 0x7fd278199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27eb96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2781011f0 0x7fd2781997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27eb96640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd278104040 0x7fd2781a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27eb96640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810e700 con 0x7fd2781006f0 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27eb96640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810e580 con 0x7fd278104040 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27eb96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810e880 con 0x7fd2781011f0 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27c932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781006f0 0x7fd278199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27c932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781006f0 0x7fd278199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46966/0 (socket says 172.21.15.78:46966) 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd27c932640 1 -- 172.21.15.78:0/2356692673 learned_addr learned my addr 172.21.15.78:0/2356692673 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.122+0000 7fd277fff640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2781011f0 0x7fd2781997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27d133640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd278104040 0x7fd2781a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27c932640 1 -- 172.21.15.78:0/2356692673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2781011f0 msgr2=0x7fd2781997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27c932640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2781011f0 0x7fd2781997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27c932640 1 -- 172.21.15.78:0/2356692673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd278104040 msgr2=0x7fd2781a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27c932640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd278104040 0x7fd2781a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27c932640 1 -- 172.21.15.78:0/2356692673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2781a0db0 con 0x7fd2781006f0 2024-05-21T15:16:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27d133640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd278104040 0x7fd2781a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27c932640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781006f0 0x7fd278199270 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00e980 tx=0x7fd26c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd275ffb640 1 -- 172.21.15.78:0/2356692673 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00cd80 con 0x7fd2781006f0 2024-05-21T15:16:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2781a10a0 con 0x7fd2781006f0 2024-05-21T15:16:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd275ffb640 1 -- 172.21.15.78:0/2356692673 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26c010910 con 0x7fd2781006f0 2024-05-21T15:16:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd275ffb640 1 -- 172.21.15.78:0/2356692673 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0189c0 con 0x7fd2781006f0 2024-05-21T15:16:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.123+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2781a15f0 con 0x7fd2781006f0 2024-05-21T15:16:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.124+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2781099e0 con 0x7fd2781006f0 2024-05-21T15:16:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.126+0000 7fd275ffb640 1 -- 172.21.15.78:0/2356692673 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7fd26c01f020 con 0x7fd2781006f0 2024-05-21T15:16:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.126+0000 7fd275ffb640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd2680a2d00 0x7fd2680a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.126+0000 7fd277fff640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd2680a2d00 0x7fd2680a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.126+0000 7fd275ffb640 1 -- 172.21.15.78:0/2356692673 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd26c0cfa90 con 0x7fd2781006f0 2024-05-21T15:16:12.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.127+0000 7fd277fff640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd2680a2d00 0x7fd2680a5210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd27819a310 tx=0x7fd264005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:12.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.134+0000 7fd275ffb640 1 -- 172.21.15.78:0/2356692673 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd26c09c6b0 con 0x7fd2781006f0 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1265977830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:12.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:11 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2784885919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:12.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:12.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:12.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1265977830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:12.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:11 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2784885919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:16:12.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.374+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd278062780 con 0x7fd2781006f0 2024-05-21T15:16:12.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.374+0000 7fd275ffb640 1 -- 172.21.15.78:0/2356692673 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fd26c09bfb0 con 0x7fd2781006f0 2024-05-21T15:16:12.377 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:16:12.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd2680a2d00 msgr2=0x7fd2680a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd2680a2d00 0x7fd2680a5210 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd27819a310 tx=0x7fd264005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:12.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781006f0 msgr2=0x7fd278199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781006f0 0x7fd278199270 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00e980 tx=0x7fd26c00ee40 comp rx=0 tx=0).stop 2024-05-21T15:16:12.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 shutdown_connections 2024-05-21T15:16:12.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd278104040 0x7fd2781a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd2680a2d00 0x7fd2680a5210 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2781011f0 0x7fd2781997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 --2- 172.21.15.78:0/2356692673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2781006f0 0x7fd278199270 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 >> 172.21.15.78:0/2356692673 conn(0x7fd2780fc2f0 msgr2=0x7fd278106bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:12.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 shutdown_connections 2024-05-21T15:16:12.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.378+0000 7fd27eb96640 1 -- 172.21.15.78:0/2356692673 wait complete. 2024-05-21T15:16:12.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:16:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.522+0000 7f3c5fff5640 1 -- 172.21.15.78:0/2394459071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101970 msgr2=0x7f3c58102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.522+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/2394459071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101970 0x7f3c58102670 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009900 tx=0x7f3c5402a410 comp rx=0 tx=0).stop 2024-05-21T15:16:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 -- 172.21.15.78:0/2394459071 shutdown_connections 2024-05-21T15:16:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/2394459071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c581078b0 0x7f3c58109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/2394459071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c58102bb0 0x7f3c58102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/2394459071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58101970 0x7f3c58102670 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 -- 172.21.15.78:0/2394459071 >> 172.21.15.78:0/2394459071 conn(0x7f3c580fcb80 msgr2=0x7f3c580ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 -- 172.21.15.78:0/2394459071 shutdown_connections 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 -- 172.21.15.78:0/2394459071 wait complete. 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 Processor -- start 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.523+0000 7f3c5fff5640 1 -- start start 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5fff5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c58101970 0x7f3c580fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5fff5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58102bb0 0x7f3c580fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5fff5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c581078b0 0x7f3c580fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5fff5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5810d5a0 con 0x7f3c58102bb0 2024-05-21T15:16:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5fff5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5810d420 con 0x7f3c58101970 2024-05-21T15:16:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5fff5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5810d720 con 0x7f3c581078b0 2024-05-21T15:16:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58102bb0 0x7f3c580fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58102bb0 0x7f3c580fd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46986/0 (socket says 172.21.15.78:46986) 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 -- 172.21.15.78:0/627372343 learned_addr learned my addr 172.21.15.78:0/627372343 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5e592640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c581078b0 0x7f3c580fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5dd91640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c58101970 0x7f3c580fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 -- 172.21.15.78:0/627372343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c581078b0 msgr2=0x7f3c580fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c581078b0 0x7f3c580fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 -- 172.21.15.78:0/627372343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c58101970 msgr2=0x7f3c580fe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c58101970 0x7f3c580fe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 -- 172.21.15.78:0/627372343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c580fdcd0 con 0x7f3c58102bb0 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5dd91640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c58101970 0x7f3c580fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5d590640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58102bb0 0x7f3c580fd0b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800e9f0 tx=0x7f3c4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c4effd640 1 -- 172.21.15.78:0/627372343 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800cd80 con 0x7f3c58102bb0 2024-05-21T15:16:12.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c4effd640 1 -- 172.21.15.78:0/627372343 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c48010910 con 0x7f3c58102bb0 2024-05-21T15:16:12.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c581a33b0 con 0x7f3c58102bb0 2024-05-21T15:16:12.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.524+0000 7f3c4effd640 1 -- 172.21.15.78:0/627372343 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48018990 con 0x7f3c58102bb0 2024-05-21T15:16:12.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.525+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c581a3810 con 0x7f3c58102bb0 2024-05-21T15:16:12.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.526+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c58069f50 con 0x7f3c58102bb0 2024-05-21T15:16:12.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.527+0000 7f3c4effd640 1 -- 172.21.15.78:0/627372343 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f3c48018b30 con 0x7f3c58102bb0 2024-05-21T15:16:12.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.528+0000 7f3c4effd640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c440a2d00 0x7f3c440a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.528+0000 7f3c5dd91640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c440a2d00 0x7f3c440a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.528+0000 7f3c4effd640 1 -- 172.21.15.78:0/627372343 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3c480cf830 con 0x7f3c58102bb0 2024-05-21T15:16:12.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.528+0000 7f3c5dd91640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c440a2d00 0x7f3c440a5210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009230 tx=0x7f3c54000bc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:12.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.532+0000 7f3c4effd640 1 -- 172.21.15.78:0/627372343 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c4809c450 con 0x7f3c58102bb0 2024-05-21T15:16:12.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.853+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c581a3a70 con 0x7f3c58102bb0 2024-05-21T15:16:12.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.853+0000 7f3c4effd640 1 -- 172.21.15.78:0/627372343 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11925 (secure 0 0 0) 0x7f3c4809bd50 con 0x7f3c58102bb0 2024-05-21T15:16:12.856 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:12.856 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:16:06.684612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24419,24448,24502,24526,24527]},"id":1}]} 2024-05-21T15:16:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-05-21T15:16:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c440a2d00 msgr2=0x7f3c440a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c440a2d00 0x7f3c440a5210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009230 tx=0x7f3c54000bc0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58102bb0 msgr2=0x7f3c580fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58102bb0 0x7f3c580fd0b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800e9f0 tx=0x7f3c4800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 shutdown_connections 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c440a2d00 0x7f3c440a5210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c581078b0 0x7f3c580fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c58102bb0 0x7f3c580fd0b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 --2- 172.21.15.78:0/627372343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c58101970 0x7f3c580fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.855+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 >> 172.21.15.78:0/627372343 conn(0x7f3c580fcb80 msgr2=0x7f3c58108da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.856+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 shutdown_connections 2024-05-21T15:16:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.856+0000 7f3c5fff5640 1 -- 172.21.15.78:0/627372343 wait complete. 2024-05-21T15:16:12.870 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:16:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 -- 172.21.15.78:0/4249661113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4102c00 msgr2=0x7f16c4103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/4249661113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4102c00 0x7f16c4103010 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009900 tx=0x7f16b802a410 comp rx=0 tx=0).stop 2024-05-21T15:16:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 -- 172.21.15.78:0/4249661113 shutdown_connections 2024-05-21T15:16:12.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/4249661113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16c4107900 0x7f16c4109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/4249661113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4102c00 0x7f16c4103010 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/4249661113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c41019c0 0x7f16c41026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 -- 172.21.15.78:0/4249661113 >> 172.21.15.78:0/4249661113 conn(0x7f16c40fcbb0 msgr2=0x7f16c40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 -- 172.21.15.78:0/4249661113 shutdown_connections 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.982+0000 7f16c9a3d640 1 -- 172.21.15.78:0/4249661113 wait complete. 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c9a3d640 1 Processor -- start 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c9a3d640 1 -- start start 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c9a3d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c41019c0 0x7f16c41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c9a3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16c4102c00 0x7f16c4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c9a3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4107900 0x7f16c419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c9a3d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410d610 con 0x7f16c4107900 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c9a3d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410d490 con 0x7f16c41019c0 2024-05-21T15:16:12.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c9a3d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16c410d790 con 0x7f16c4102c00 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4107900 0x7f16c419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c41019c0 0x7f16c41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4107900 0x7f16c419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47004/0 (socket says 172.21.15.78:47004) 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c37fe640 1 -- 172.21.15.78:0/852640596 learned_addr learned my addr 172.21.15.78:0/852640596 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c41019c0 0x7f16c41981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:41242/0 (socket says 172.21.15.78:41242) 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.983+0000 7f16c27fc640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16c4102c00 0x7f16c4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c37fe640 1 -- 172.21.15.78:0/852640596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16c4102c00 msgr2=0x7f16c4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c37fe640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16c4102c00 0x7f16c4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c37fe640 1 -- 172.21.15.78:0/852640596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c41019c0 msgr2=0x7f16c41981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c37fe640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c41019c0 0x7f16c41981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c37fe640 1 -- 172.21.15.78:0/852640596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c419fd20 con 0x7f16c4107900 2024-05-21T15:16:12.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c2ffd640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c41019c0 0x7f16c41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:12.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c37fe640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4107900 0x7f16c419f6b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00ebb0 tx=0x7f16bc007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:12.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c8a3b640 1 -- 172.21.15.78:0/852640596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc007ca0 con 0x7f16c4107900 2024-05-21T15:16:12.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c406a220 con 0x7f16c4107900 2024-05-21T15:16:12.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c8a3b640 1 -- 172.21.15.78:0/852640596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc007e40 con 0x7f16c4107900 2024-05-21T15:16:12.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c8a3b640 1 -- 172.21.15.78:0/852640596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0058a0 con 0x7f16c4107900 2024-05-21T15:16:12.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.984+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c406a720 con 0x7f16c4107900 2024-05-21T15:16:12.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.985+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c40682c0 con 0x7f16c4107900 2024-05-21T15:16:12.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.987+0000 7f16c8a3b640 1 -- 172.21.15.78:0/852640596 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f16bc004a40 con 0x7f16c4107900 2024-05-21T15:16:12.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.988+0000 7f16c8a3b640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16ac0a2d00 0x7f16ac0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:12.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.988+0000 7f16c2ffd640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16ac0a2d00 0x7f16ac0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:12.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.988+0000 7f16c8a3b640 1 -- 172.21.15.78:0/852640596 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f16bc0d0940 con 0x7f16c4107900 2024-05-21T15:16:12.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.988+0000 7f16c2ffd640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16ac0a2d00 0x7f16ac0a5210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f16c41025c0 tx=0x7f16b4005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:12.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:12.993+0000 7f16c8a3b640 1 -- 172.21.15.78:0/852640596 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16bc09cf10 con 0x7f16c4107900 2024-05-21T15:16:13.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Unable to set osd_memory_target on smithi084 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2541555705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi078 to 176.1M 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Unable to set osd_memory_target on smithi078 to 184744448: error parsing value: Value '184744448' is below minimum 939524096 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Updating smithi078:/etc/ceph/ceph.conf 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2356692673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:13.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:12 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/627372343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:13.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Unable to set osd_memory_target on smithi084 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2541555705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi078 to 176.1M 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Unable to set osd_memory_target on smithi078 to 184744448: error parsing value: Value '184744448' is below minimum 939524096 2024-05-21T15:16:13.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:13.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:13.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Updating smithi078:/etc/ceph/ceph.conf 2024-05-21T15:16:13.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:16:13.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:16:13.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2356692673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:13.186 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:12 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/627372343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.231+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16c41063d0 con 0x7f16c4107900 2024-05-21T15:16:13.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.235+0000 7f16c8a3b640 1 -- 172.21.15.78:0/852640596 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22994 (secure 0 0 0) 0x7f16bc0a1210 con 0x7f16c4107900 2024-05-21T15:16:13.239 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:13.239 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:16:08.699024+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6801/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2012910515":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2194549734":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/3526992601":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2604499013":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:16:13.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16ac0a2d00 msgr2=0x7f16ac0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16ac0a2d00 0x7f16ac0a5210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f16c41025c0 tx=0x7f16b4005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:13.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4107900 msgr2=0x7f16c419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4107900 0x7f16c419f6b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00ebb0 tx=0x7f16bc007590 comp rx=0 tx=0).stop 2024-05-21T15:16:13.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 shutdown_connections 2024-05-21T15:16:13.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16ac0a2d00 0x7f16ac0a5210 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c4107900 0x7f16c419f6b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16c4102c00 0x7f16c4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 --2- 172.21.15.78:0/852640596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16c41019c0 0x7f16c41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 >> 172.21.15.78:0/852640596 conn(0x7f16c40fcbb0 msgr2=0x7f16c41038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 shutdown_connections 2024-05-21T15:16:13.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.238+0000 7f16c9a3d640 1 -- 172.21.15.78:0/852640596 wait complete. 2024-05-21T15:16:13.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Unable to set osd_memory_target on smithi084 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2541555705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi078 to 176.1M 2024-05-21T15:16:13.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Unable to set osd_memory_target on smithi078 to 184744448: error parsing value: Value '184744448' is below minimum 939524096 2024-05-21T15:16:13.262 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:13.262 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:13.262 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Updating smithi078:/etc/ceph/ceph.conf 2024-05-21T15:16:13.262 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Updating smithi084:/etc/ceph/ceph.conf 2024-05-21T15:16:13.262 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: Updating smithi172:/etc/ceph/ceph.conf 2024-05-21T15:16:13.262 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2356692673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:16:13.262 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:12 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/627372343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:13.264 DEBUG:tasks.fwd_scrub:Yielding 2024-05-21T15:16:13.264 INFO:teuthology.run_tasks:Running task exec... 2024-05-21T15:16:13.273 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T15:16:13.274 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:16:13.274 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-21T15:16:13.274 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-21T15:16:13.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:16:13.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:13.275 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:16:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 -- 172.21.15.78:0/3218955443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c107e70 msgr2=0x7fa82c102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 --2- 172.21.15.78:0/3218955443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c107e70 0x7fa82c102110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa820009860 tx=0x7fa82002a340 comp rx=0 tx=0).stop 2024-05-21T15:16:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 -- 172.21.15.78:0/3218955443 shutdown_connections 2024-05-21T15:16:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 --2- 172.21.15.78:0/3218955443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82c102780 0x7fa82c10e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 --2- 172.21.15.78:0/3218955443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c107e70 0x7fa82c102110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 --2- 172.21.15.78:0/3218955443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa82c1079d0 0x7fa82c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 -- 172.21.15.78:0/3218955443 >> 172.21.15.78:0/3218955443 conn(0x7fa82c076830 msgr2=0x7fa82c076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 -- 172.21.15.78:0/3218955443 shutdown_connections 2024-05-21T15:16:13.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.393+0000 7fa830b72640 1 -- 172.21.15.78:0/3218955443 wait complete. 2024-05-21T15:16:13.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa830b72640 1 Processor -- start 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa830b72640 1 -- start start 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa830b72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c102780 0x7fa82c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa830b72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa82c1079d0 0x7fa82c19fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa830b72640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82c107e70 0x7fa82c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa830b72640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c111d10 con 0x7fa82c1079d0 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa830b72640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c111b90 con 0x7fa82c107e70 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa830b72640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c111e90 con 0x7fa82c102780 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa829d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa82c1079d0 0x7fa82c19fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa829d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa82c1079d0 0x7fa82c19fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47008/0 (socket says 172.21.15.78:47008) 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa829d9b640 1 -- 172.21.15.78:0/275436842 learned_addr learned my addr 172.21.15.78:0/275436842 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.394+0000 7fa82a59c640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c102780 0x7fa82c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa82ad9d640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82c107e70 0x7fa82c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa829d9b640 1 -- 172.21.15.78:0/275436842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c102780 msgr2=0x7fa82c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa829d9b640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c102780 0x7fa82c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa829d9b640 1 -- 172.21.15.78:0/275436842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82c107e70 msgr2=0x7fa82c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa829d9b640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82c107e70 0x7fa82c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa829d9b640 1 -- 172.21.15.78:0/275436842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c199d90 con 0x7fa82c1079d0 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa829d9b640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa82c1079d0 0x7fa82c19fa20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa82002a860 tx=0x7fa820033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa81b7fe640 1 -- 172.21.15.78:0/275436842 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820037040 con 0x7fa82c1079d0 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa830b72640 1 -- 172.21.15.78:0/275436842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c199fc0 con 0x7fa82c1079d0 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa81b7fe640 1 -- 172.21.15.78:0/275436842 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa820040070 con 0x7fa82c1079d0 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa81b7fe640 1 -- 172.21.15.78:0/275436842 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820003970 con 0x7fa82c1079d0 2024-05-21T15:16:13.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.395+0000 7fa830b72640 1 -- 172.21.15.78:0/275436842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c1a6280 con 0x7fa82c1079d0 2024-05-21T15:16:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.398+0000 7fa837a17740 1 Processor -- start 2024-05-21T15:16:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa837a17740 1 -- start start 2024-05-21T15:16:13.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa81b7fe640 1 -- 172.21.15.78:0/275436842 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fa820004050 con 0x7fa82c1079d0 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa837a17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b09620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa837a17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b09b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa837a17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b0a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa837a17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560474a6e990 con 0x56047476a410 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa837a17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560474a165b0 con 0x560474741550 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa837a17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560474a6a950 con 0x560474749730 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa82ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b09620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa82a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b09b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa82ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b09620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47022/0 (socket says 172.21.15.78:47022) 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa82ad9d640 1 -- 172.21.15.78:0/1731684066 learned_addr learned my addr 172.21.15.78:0/1731684066 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.399+0000 7fa829d9b640 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b0a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa82ad9d640 1 -- 172.21.15.78:0/1731684066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 msgr2=0x560474b0a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa82ad9d640 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b0a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa82ad9d640 1 -- 172.21.15.78:0/1731684066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 msgr2=0x560474b09b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa82ad9d640 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b09b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa82ad9d640 1 -- 172.21.15.78:0/1731684066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56047484a7e0 con 0x56047476a410 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa82a59c640 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b09b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa82ad9d640 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b09620 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa824011440 tx=0x7fa824011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d3e5cf31dcb84c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa81b7fe640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa8140a2d90 0x7fa8140a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa82a59c640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa8140a2d90 0x7fa8140a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.400+0000 7fa81b7fe640 1 -- 172.21.15.78:0/275436842 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa8200ef140 con 0x7fa82c1079d0 2024-05-21T15:16:13.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.401+0000 7fa82a59c640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa8140a2d90 0x7fa8140a52a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0096c0 tx=0x7fa81c009260 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.403+0000 7fa818ff9640 1 -- 172.21.15.78:0/1731684066 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824003c30 con 0x56047476a410 2024-05-21T15:16:13.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.403+0000 7fa818ff9640 1 -- 172.21.15.78:0/1731684066 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa824003dd0 con 0x56047476a410 2024-05-21T15:16:13.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.403+0000 7fa818ff9640 1 -- 172.21.15.78:0/1731684066 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82400fac0 con 0x56047476a410 2024-05-21T15:16:13.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 -- 172.21.15.78:0/1731684066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 msgr2=0x560474b09620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b09620 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa824011440 tx=0x7fa824011900 comp rx=0 tx=0).stop 2024-05-21T15:16:13.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 -- 172.21.15.78:0/1731684066 shutdown_connections 2024-05-21T15:16:13.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b09b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b0a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 --2- 172.21.15.78:0/1731684066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b09620 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 -- 172.21.15.78:0/1731684066 >> 172.21.15.78:0/1731684066 conn(0x56047475e560 msgr2=0x560474b05dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 -- 172.21.15.78:0/1731684066 shutdown_connections 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 -- 172.21.15.78:0/1731684066 wait complete. 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.404+0000 7fa837a17740 1 -- start start 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa837a17740 1 Processor -- start 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa837a17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474741550 0x560474b15670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa837a17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560474749730 0x560474b15bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa837a17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56047476a410 0x560474b13b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa837a17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560474b16970 con 0x560474749730 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa837a17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560474a29b40 con 0x56047476a410 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa837a17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560474b16af0 con 0x560474741550 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa82a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474741550 0x560474b15670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa829d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560474749730 0x560474b15bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa82a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474741550 0x560474b15670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:59228/0 (socket says 172.21.15.78:59228) 2024-05-21T15:16:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa82a59c640 1 -- 172.21.15.78:0/110510960 learned_addr learned my addr 172.21.15.78:0/110510960 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa837a17740 10 client.0 _ll_register_callbacks cb 0x560474a1e0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.405+0000 7fa82ad9d640 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56047476a410 0x560474b13b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.406+0000 7fa82a59c640 1 -- 172.21.15.78:0/110510960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56047476a410 msgr2=0x560474b13b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.406+0000 7fa82a59c640 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56047476a410 0x560474b13b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.406+0000 7fa82a59c640 1 -- 172.21.15.78:0/110510960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560474749730 msgr2=0x560474b15bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.406+0000 7fa82a59c640 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560474749730 0x560474b15bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.406+0000 7fa82a59c640 1 -- 172.21.15.78:0/110510960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604748aaf70 con 0x560474741550 2024-05-21T15:16:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.406+0000 7fa829d9b640 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560474749730 0x560474b15bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.407+0000 7fa82a59c640 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474741550 0x560474b15670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005fd0 tx=0x7fa81c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604db8171baca6ad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.407+0000 7fa811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:16:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.408+0000 7fa811ffb640 1 -- 172.21.15.78:0/110510960 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c009e20 con 0x560474741550 2024-05-21T15:16:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.408+0000 7fa837a17740 1 -- 172.21.15.78:0/110510960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa824003dd0 con 0x560474741550 2024-05-21T15:16:13.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.408+0000 7fa811ffb640 1 -- 172.21.15.78:0/110510960 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa81c017030 con 0x560474741550 2024-05-21T15:16:13.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.409+0000 7fa811ffb640 1 -- 172.21.15.78:0/110510960 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa81c004020 con 0x560474741550 2024-05-21T15:16:13.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.409+0000 7fa837a17740 10 client.25150 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:13.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.409+0000 7fa837a17740 1 -- 172.21.15.78:0/110510960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560474742170 con 0x560474741550 2024-05-21T15:16:13.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.409+0000 7fa811ffb640 1 -- 172.21.15.78:0/110510960 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa824003dd0 con 0x560474741550 2024-05-21T15:16:13.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.409+0000 7fa811ffb640 1 -- 172.21.15.78:0/110510960 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa81c031460 con 0x560474741550 2024-05-21T15:16:13.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.409+0000 7fa837a17740 10 client.25150 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:13.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.409+0000 7fa837a17740 10 client.25150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:13.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.410+0000 7fa837a17740 20 client.25150 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:13.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.410+0000 7fa837a17740 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5604747291f0 0x560474b26680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.410+0000 7fa837a17740 4 client.25150 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:13.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.410+0000 7fa837a17740 1 -- 172.21.15.78:0/110510960 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560474a7c0c0 con 0x5604747291f0 2024-05-21T15:16:13.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.410+0000 7fa829d9b640 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5604747291f0 0x560474b26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa811ffb640 10 client.25150 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:13.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa829d9b640 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5604747291f0 0x560474b26680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=127e956f9f0f955a server_cookie=38f536f7720347c4 in_seq=0 out_seq=0 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa811ffb640 1 -- 172.21.15.78:0/110510960 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560474a7c0c0 con 0x5604747291f0 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa811ffb640 10 client.25150 handle_command_reply: tid=0 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa837a17740 1 client.25150 shutdown 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa837a17740 1 -- 172.21.15.78:0/110510960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474741550 msgr2=0x560474b15670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa837a17740 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474741550 0x560474b15670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005fd0 tx=0x7fa81c004730 comp rx=0 tx=0).stop 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa837a17740 1 -- 172.21.15.78:0/110510960 shutdown_connections 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.411+0000 7fa837a17740 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5604747291f0 0x560474b26680 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.412+0000 7fa837a17740 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56047476a410 0x560474b13b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.412+0000 7fa811ffb640 0 client.25150 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.412+0000 7fa837a17740 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560474749730 0x560474b15bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.412+0000 7fa837a17740 1 --2- 172.21.15.78:0/110510960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474741550 0x560474b15670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.412+0000 7fa837a17740 1 -- 172.21.15.78:0/110510960 >> 172.21.15.78:0/110510960 conn(0x56047475e560 msgr2=0x560474b093f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.412+0000 7fa837a17740 1 -- 172.21.15.78:0/110510960 shutdown_connections 2024-05-21T15:16:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.412+0000 7fa837a17740 1 -- 172.21.15.78:0/110510960 wait complete. 2024-05-21T15:16:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.412+0000 7fa837a17740 20 client.25150 trim_cache size 0 max 16384 2024-05-21T15:16:13.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.421+0000 7fb297338640 1 -- 172.21.15.78:0/482383762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb290107a10 msgr2=0x7fb2901015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.421+0000 7fb297338640 1 --2- 172.21.15.78:0/482383762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb290107a10 0x7fb2901015c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009860 tx=0x7fb28c02a340 comp rx=0 tx=0).stop 2024-05-21T15:16:13.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.421+0000 7fb297338640 1 -- 172.21.15.78:0/482383762 shutdown_connections 2024-05-21T15:16:13.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.421+0000 7fb297338640 1 --2- 172.21.15.78:0/482383762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb290102170 0x7fb29010e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.421+0000 7fb297338640 1 --2- 172.21.15.78:0/482383762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb290107eb0 0x7fb290101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.421+0000 7fb297338640 1 --2- 172.21.15.78:0/482383762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb290107a10 0x7fb2901015c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.421+0000 7fb297338640 1 -- 172.21.15.78:0/482383762 >> 172.21.15.78:0/482383762 conn(0x7fb290077160 msgr2=0x7fb2900775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.422+0000 7fb297338640 1 -- 172.21.15.78:0/482383762 shutdown_connections 2024-05-21T15:16:13.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.422+0000 7fb297338640 1 -- 172.21.15.78:0/482383762 wait complete. 2024-05-21T15:16:13.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.422+0000 7fb297338640 1 Processor -- start 2024-05-21T15:16:13.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.422+0000 7fb297338640 1 -- start start 2024-05-21T15:16:13.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb297338640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb290102170 0x7fb2901a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb297338640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb290107a10 0x7fb2901a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb297338640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb290107eb0 0x7fb29019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb297338640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb290111d60 con 0x7fb290102170 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb297338640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb290111be0 con 0x7fb290107eb0 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb297338640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb290111ee0 con 0x7fb290107a10 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2950d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb290102170 0x7fb2901a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2950d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb290102170 0x7fb2901a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47056/0 (socket says 172.21.15.78:47056) 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2958d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb290107eb0 0x7fb29019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2950d4640 1 -- 172.21.15.78:0/3131727379 learned_addr learned my addr 172.21.15.78:0/3131727379 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2948d3640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb290107a10 0x7fb2901a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2958d5640 1 -- 172.21.15.78:0/3131727379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb290107a10 msgr2=0x7fb2901a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2958d5640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb290107a10 0x7fb2901a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2958d5640 1 -- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb290102170 msgr2=0x7fb2901a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2958d5640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb290102170 0x7fb2901a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2958d5640 1 -- 172.21.15.78:0/3131727379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29019bea0 con 0x7fb290107eb0 2024-05-21T15:16:13.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2950d4640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb290102170 0x7fb2901a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:13.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2958d5640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb290107eb0 0x7fb29019b8b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb28800cab0 tx=0x7fb28800cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2867fc640 1 -- 172.21.15.78:0/3131727379 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28800ae20 con 0x7fb290107eb0 2024-05-21T15:16:13.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.423+0000 7fb2867fc640 1 -- 172.21.15.78:0/3131727379 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb28800f930 con 0x7fb290107eb0 2024-05-21T15:16:13.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.424+0000 7fb2867fc640 1 -- 172.21.15.78:0/3131727379 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288018b30 con 0x7fb290107eb0 2024-05-21T15:16:13.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.424+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29019c190 con 0x7fb290107eb0 2024-05-21T15:16:13.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.424+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29019c690 con 0x7fb290107eb0 2024-05-21T15:16:13.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.425+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2900690c0 con 0x7fb290107eb0 2024-05-21T15:16:13.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.426+0000 7fb2867fc640 1 -- 172.21.15.78:0/3131727379 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb28800fae0 con 0x7fb290107eb0 2024-05-21T15:16:13.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.426+0000 7fb2867fc640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb27c0a2d80 0x7fb27c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.426+0000 7fb2950d4640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb27c0a2d80 0x7fb27c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.426+0000 7fb2867fc640 1 -- 172.21.15.78:0/3131727379 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb2880cf970 con 0x7fb290107eb0 2024-05-21T15:16:13.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.427+0000 7fb2950d4640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb27c0a2d80 0x7fb27c0a5290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb28c000c00 tx=0x7fb28c0056d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.431+0000 7fb2867fc640 1 -- 172.21.15.78:0/3131727379 <== mon.2 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb288014030 con 0x7fb290107eb0 2024-05-21T15:16:13.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa837a17740 1 Processor -- start 2024-05-21T15:16:13.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa837a17740 1 -- start start 2024-05-21T15:16:13.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa837a17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa837a17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b1bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa837a17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b1c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa837a17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1ac9a0 con 0x56047476a410 2024-05-21T15:16:13.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa837a17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1ae4e0 con 0x560474741550 2024-05-21T15:16:13.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa837a17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1ad070 con 0x560474749730 2024-05-21T15:16:13.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa82ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b1c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa82ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b1c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47070/0 (socket says 172.21.15.78:47070) 2024-05-21T15:16:13.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa82ad9d640 1 -- 172.21.15.78:0/2154464049 learned_addr learned my addr 172.21.15.78:0/2154464049 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa829d9b640 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b1bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa82a59c640 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa82ad9d640 1 -- 172.21.15.78:0/2154464049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 msgr2=0x560474b1bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.433+0000 7fa82ad9d640 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b1bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.434+0000 7fa82ad9d640 1 -- 172.21.15.78:0/2154464049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 msgr2=0x560474b159b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.434+0000 7fa82ad9d640 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b159b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.434+0000 7fa82ad9d640 1 -- 172.21.15.78:0/2154464049 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560474b1c800 con 0x56047476a410 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.434+0000 7fa82ad9d640 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b1c2c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa824009f80 tx=0x7fa82400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba116f36b50a179 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.434+0000 7fa813fff640 1 -- 172.21.15.78:0/2154464049 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824003c30 con 0x56047476a410 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.434+0000 7fa813fff640 1 -- 172.21.15.78:0/2154464049 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa824003dd0 con 0x56047476a410 2024-05-21T15:16:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.434+0000 7fa813fff640 1 -- 172.21.15.78:0/2154464049 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824004260 con 0x56047476a410 2024-05-21T15:16:13.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 -- 172.21.15.78:0/2154464049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 msgr2=0x560474b1c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b1c2c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa824009f80 tx=0x7fa82400f720 comp rx=0 tx=0).stop 2024-05-21T15:16:13.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 -- 172.21.15.78:0/2154464049 shutdown_connections 2024-05-21T15:16:13.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b1c2c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b1bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 --2- 172.21.15.78:0/2154464049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 -- 172.21.15.78:0/2154464049 >> 172.21.15.78:0/2154464049 conn(0x56047475e560 msgr2=0x560474b152a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 -- 172.21.15.78:0/2154464049 shutdown_connections 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 -- 172.21.15.78:0/2154464049 wait complete. 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.435+0000 7fa837a17740 1 -- start start 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa837a17740 1 Processor -- start 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa837a17740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b1aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa837a17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b1b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa837a17740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b8ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa837a17740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1ade10 con 0x56047476a410 2024-05-21T15:16:13.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa837a17740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1ad740 con 0x560474741550 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa837a17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1aa9a0 con 0x560474749730 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa837a17740 10 client.0 _ll_register_callbacks cb 0x7fa81c01d890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b1aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b1aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:41312/0 (socket says 172.21.15.78:41312) 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82a59c640 1 -- 172.21.15.78:0/3217054676 learned_addr learned my addr 172.21.15.78:0/3217054676 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b8ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa829d9b640 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b1b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82a59c640 1 -- 172.21.15.78:0/3217054676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 msgr2=0x560474b1b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82a59c640 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b1b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82a59c640 1 -- 172.21.15.78:0/3217054676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 msgr2=0x560474b8ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82a59c640 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b8ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.436+0000 7fa82a59c640 1 -- 172.21.15.78:0/3217054676 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560474b1b870 con 0x560474741550 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa82a59c640 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b1aad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa81c018040 tx=0x7fa81c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc38bf7f726ab68 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa811ffb640 1 -- 172.21.15.78:0/3217054676 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c0041c0 con 0x560474741550 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa811ffb640 1 -- 172.21.15.78:0/3217054676 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa81c004360 con 0x560474741550 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa837a17740 1 -- 172.21.15.78:0/3217054676 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa824003dd0 con 0x560474741550 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa811ffb640 1 -- 172.21.15.78:0/3217054676 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa81c0035f0 con 0x560474741550 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa837a17740 10 client.25175 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa837a17740 1 -- 172.21.15.78:0/3217054676 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560474b8df70 con 0x560474741550 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa811ffb640 1 -- 172.21.15.78:0/3217054676 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa824003dd0 con 0x560474741550 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa811ffb640 1 -- 172.21.15.78:0/3217054676 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa81c03a070 con 0x560474741550 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa837a17740 10 client.25175 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.437+0000 7fa837a17740 10 client.25175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.438+0000 7fa837a17740 20 client.25175 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.438+0000 7fa837a17740 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5604746fe630 0x560474b906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.438+0000 7fa837a17740 4 client.25175 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.438+0000 7fa837a17740 1 -- 172.21.15.78:0/3217054676 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560474a76410 con 0x5604746fe630 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.438+0000 7fa829d9b640 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5604746fe630 0x560474b906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.438+0000 7fa811ffb640 10 client.25175 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.438+0000 7fa829d9b640 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5604746fe630 0x560474b906d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da70dbd46520b97b server_cookie=ef77b3567962e246 in_seq=0 out_seq=0 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa811ffb640 1 -- 172.21.15.78:0/3217054676 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560474a76410 con 0x5604746fe630 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa811ffb640 10 client.25175 handle_command_reply: tid=0 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 client.25175 shutdown 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 -- 172.21.15.78:0/3217054676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 msgr2=0x560474b1aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b1aad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa81c018040 tx=0x7fa81c009b80 comp rx=0 tx=0).stop 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 -- 172.21.15.78:0/3217054676 shutdown_connections 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5604746fe630 0x560474b906d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56047476a410 0x560474b8ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa811ffb640 0 client.25175 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560474749730 0x560474b1b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 --2- 172.21.15.78:0/3217054676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560474741550 0x560474b1aad0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.439+0000 7fa837a17740 1 -- 172.21.15.78:0/3217054676 >> 172.21.15.78:0/3217054676 conn(0x56047475e560 msgr2=0x560474b0d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa837a17740 1 -- 172.21.15.78:0/3217054676 shutdown_connections 2024-05-21T15:16:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa837a17740 1 -- 172.21.15.78:0/3217054676 wait complete. 2024-05-21T15:16:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa837a17740 20 client.25175 trim_cache size 0 max 16384 2024-05-21T15:16:13.450 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:16:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "2275f818-63ef-43d3-b1db-4a135e95df40", 2024-05-21T15:16:13.451 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:16:13.451 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:13.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 -- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa8140a2d90 msgr2=0x7fa8140a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa8140a2d90 0x7fa8140a52a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0096c0 tx=0x7fa81c009260 comp rx=0 tx=0).stop 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 -- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa82c1079d0 msgr2=0x7fa82c19fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa82c1079d0 0x7fa82c19fa20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa82002a860 tx=0x7fa820033b60 comp rx=0 tx=0).stop 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 -- 172.21.15.78:0/275436842 shutdown_connections 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa82c107e70 0x7fa82c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa8140a2d90 0x7fa8140a52a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa82c1079d0 0x7fa82c19fa20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 --2- 172.21.15.78:0/275436842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c102780 0x7fa82c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 -- 172.21.15.78:0/275436842 >> 172.21.15.78:0/275436842 conn(0x7fa82c076830 msgr2=0x7fa82c105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 -- 172.21.15.78:0/275436842 shutdown_connections 2024-05-21T15:16:13.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.440+0000 7fa830b72640 1 -- 172.21.15.78:0/275436842 wait complete. 2024-05-21T15:16:13.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:16:13.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.570+0000 7f0e4fade640 1 -- 172.21.15.78:0/3622430662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e481079d0 msgr2=0x7f0e48101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.570+0000 7f0e4fade640 1 --2- 172.21.15.78:0/3622430662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e481079d0 0x7f0e48101560 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009900 tx=0x7f0e4402a410 comp rx=0 tx=0).stop 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.570+0000 7f0e4fade640 1 -- 172.21.15.78:0/3622430662 shutdown_connections 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.570+0000 7f0e4fade640 1 --2- 172.21.15.78:0/3622430662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e48101aa0 0x7f0e48101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.570+0000 7f0e4fade640 1 --2- 172.21.15.78:0/3622430662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4810c330 0x7f0e4810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.570+0000 7f0e4fade640 1 --2- 172.21.15.78:0/3622430662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e481079d0 0x7f0e48101560 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.570+0000 7f0e4fade640 1 -- 172.21.15.78:0/3622430662 >> 172.21.15.78:0/3622430662 conn(0x7f0e480770b0 msgr2=0x7f0e480774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.571+0000 7f0e4fade640 1 -- 172.21.15.78:0/3622430662 shutdown_connections 2024-05-21T15:16:13.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.571+0000 7f0e4fade640 1 -- 172.21.15.78:0/3622430662 wait complete. 2024-05-21T15:16:13.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.571+0000 7f0e4fade640 1 Processor -- start 2024-05-21T15:16:13.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.571+0000 7f0e4fade640 1 -- start start 2024-05-21T15:16:13.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.571+0000 7f0e4fade640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e48101aa0 0x7f0e48198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4fade640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e481079d0 0x7f0e48198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4fade640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4810c330 0x7f0e4819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4fade640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e48111d30 con 0x7f0e48101aa0 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4fade640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e48111bb0 con 0x7f0e481079d0 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4fade640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e48111eb0 con 0x7f0e4810c330 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e48101aa0 0x7f0e48198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e48101aa0 0x7f0e48198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47112/0 (socket says 172.21.15.78:47112) 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 -- 172.21.15.78:0/2577925819 learned_addr learned my addr 172.21.15.78:0/2577925819 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4810c330 0x7f0e4819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d079640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e481079d0 0x7f0e48198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 -- 172.21.15.78:0/2577925819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4810c330 msgr2=0x7f0e4819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4810c330 0x7f0e4819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 -- 172.21.15.78:0/2577925819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e481079d0 msgr2=0x7f0e48198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e481079d0 0x7f0e48198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 -- 172.21.15.78:0/2577925819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4819d4d0 con 0x7f0e48101aa0 2024-05-21T15:16:13.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e48101aa0 0x7f0e48198670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0e4402a930 tx=0x7f0e44003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e3effd640 1 -- 172.21.15.78:0/2577925819 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44035070 con 0x7f0e48101aa0 2024-05-21T15:16:13.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4fade640 1 -- 172.21.15.78:0/2577925819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4819d760 con 0x7f0e48101aa0 2024-05-21T15:16:13.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e3effd640 1 -- 172.21.15.78:0/2577925819 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e44031a90 con 0x7f0e48101aa0 2024-05-21T15:16:13.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e3effd640 1 -- 172.21.15.78:0/2577925819 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44038a30 con 0x7f0e48101aa0 2024-05-21T15:16:13.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.572+0000 7f0e4fade640 1 -- 172.21.15.78:0/2577925819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4819dc50 con 0x7f0e48101aa0 2024-05-21T15:16:13.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.575+0000 7f0e3effd640 1 -- 172.21.15.78:0/2577925819 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f0e44031c40 con 0x7f0e48101aa0 2024-05-21T15:16:13.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.575+0000 7f0e5697f740 1 Processor -- start 2024-05-21T15:16:13.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e5697f740 1 -- start start 2024-05-21T15:16:13.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c044390 0x559b5c3e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c3e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e5697f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b5c348d00 con 0x559b5c01b4d0 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e5697f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559b5c2f04c0 con 0x559b5c0236b0 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e5697f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559b5c344cc0 con 0x559b5c044390 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4e07b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c044390 0x559b5c3e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e3effd640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e340a2d90 0x7f0e340a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4d87a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4d079640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c3e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4e07b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c044390 0x559b5c3e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:59290/0 (socket says 172.21.15.78:59290) 2024-05-21T15:16:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4e07b640 1 -- 172.21.15.78:0/2754447620 learned_addr learned my addr 172.21.15.78:0/2754447620 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e340a2d90 0x7f0e340a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4e07b640 1 -- 172.21.15.78:0/2754447620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 msgr2=0x559b5c3e3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c3e3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e4e07b640 1 -- 172.21.15.78:0/2754447620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 msgr2=0x559b5c3e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.576+0000 7f0e3effd640 1 -- 172.21.15.78:0/2577925819 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0e440f1610 con 0x7f0e48101aa0 2024-05-21T15:16:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e4e07b640 1 -- 172.21.15.78:0/2754447620 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5c1267d0 con 0x559b5c044390 2024-05-21T15:16:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e4d079640 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c3e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c044390 0x559b5c3e2bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005c40 tx=0x7f0e4000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5039cb86c6443b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e33fff640 1 -- 172.21.15.78:0/2754447620 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40022020 con 0x559b5c044390 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e340a2d90 0x7f0e340a52a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e40023000 tx=0x7f0e40021000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e33fff640 1 -- 172.21.15.78:0/2754447620 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e40013170 con 0x559b5c044390 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.577+0000 7f0e33fff640 1 -- 172.21.15.78:0/2754447620 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e400107b0 con 0x559b5c044390 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 -- 172.21.15.78:0/2754447620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c044390 msgr2=0x559b5c3e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c044390 0x559b5c3e2bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005c40 tx=0x7f0e4000f3c0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 -- 172.21.15.78:0/2754447620 shutdown_connections 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c3e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2754447620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c044390 0x559b5c3e2bc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 -- 172.21.15.78:0/2754447620 >> 172.21.15.78:0/2754447620 conn(0x559b5c0384e0 msgr2=0x559b5c3df4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 -- 172.21.15.78:0/2754447620 shutdown_connections 2024-05-21T15:16:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.578+0000 7f0e5697f740 1 -- 172.21.15.78:0/2754447620 wait complete. 2024-05-21T15:16:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 1 -- start start 2024-05-21T15:16:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 1 Processor -- start 2024-05-21T15:16:13.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c01b4d0 0x559b5c3f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c044390 0x559b5c3f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b5c3f4490 con 0x559b5c044390 2024-05-21T15:16:13.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559b5c303b90 con 0x559b5c01b4d0 2024-05-21T15:16:13.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559b5c3f4610 con 0x559b5c0236b0 2024-05-21T15:16:13.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e5697f740 10 client.0 _ll_register_callbacks cb 0x559b5c33fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:13.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e4d079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e4e07b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c044390 0x559b5c3f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e4d079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:59296/0 (socket says 172.21.15.78:59296) 2024-05-21T15:16:13.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e4d87a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c01b4d0 0x559b5c3f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.579+0000 7f0e4d079640 1 -- 172.21.15.78:0/4011997251 learned_addr learned my addr 172.21.15.78:0/4011997251 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e4d079640 1 -- 172.21.15.78:0/4011997251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c01b4d0 msgr2=0x559b5c3f32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e4d079640 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c01b4d0 0x559b5c3f32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e4d079640 1 -- 172.21.15.78:0/4011997251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c044390 msgr2=0x559b5c3f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e4d079640 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c044390 0x559b5c3f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e4d079640 1 -- 172.21.15.78:0/4011997251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5c05c3f0 con 0x559b5c0236b0 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c01b4d0 0x559b5c3f32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c044390 0x559b5c3f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e4d079640 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f3820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e38012700 tx=0x7f0e38012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b6ded8fce9201 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:16:13.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e317fa640 1 -- 172.21.15.78:0/4011997251 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800f920 con 0x559b5c0236b0 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e5697f740 1 -- 172.21.15.78:0/4011997251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e40013170 con 0x559b5c0236b0 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e317fa640 1 -- 172.21.15.78:0/4011997251 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0e3800fac0 con 0x559b5c0236b0 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e317fa640 1 -- 172.21.15.78:0/4011997251 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e3800fcc0 con 0x559b5c0236b0 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e5697f740 10 client.25174 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.580+0000 7f0e5697f740 1 -- 172.21.15.78:0/4011997251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b5c01c0f0 con 0x559b5c0236b0 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e317fa640 1 -- 172.21.15.78:0/4011997251 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e40013170 con 0x559b5c0236b0 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e317fa640 1 -- 172.21.15.78:0/4011997251 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0e3801c420 con 0x559b5c0236b0 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e5697f740 10 client.25174 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e5697f740 10 client.25174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e5697f740 20 client.25174 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e5697f740 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559b5c002f60 0x559b5c404130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e5697f740 4 client.25174 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:13.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e5697f740 1 -- 172.21.15.78:0/4011997251 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b5c356430 con 0x559b5c002f60 2024-05-21T15:16:13.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559b5c002f60 0x559b5c404130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559b5c002f60 0x559b5c404130 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad3563b0637cd48 server_cookie=94ae3fff252060c3 in_seq=0 out_seq=0 2024-05-21T15:16:13.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.581+0000 7f0e317fa640 10 client.25174 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:13.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e317fa640 1 -- 172.21.15.78:0/4011997251 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b5c356430 con 0x559b5c002f60 2024-05-21T15:16:13.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e317fa640 10 client.25174 handle_command_reply: tid=0 2024-05-21T15:16:13.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 client.25174 shutdown 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 -- 172.21.15.78:0/4011997251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 msgr2=0x559b5c3f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f3820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e38012700 tx=0x7f0e38012bc0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 -- 172.21.15.78:0/4011997251 shutdown_connections 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559b5c002f60 0x559b5c404130 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c044390 0x559b5c3f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f3820 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e317fa640 0 client.25174 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 --2- 172.21.15.78:0/4011997251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c01b4d0 0x559b5c3f32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 -- 172.21.15.78:0/4011997251 >> 172.21.15.78:0/4011997251 conn(0x559b5c0384e0 msgr2=0x559b5c3e49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 -- 172.21.15.78:0/4011997251 shutdown_connections 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 1 -- 172.21.15.78:0/4011997251 wait complete. 2024-05-21T15:16:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.582+0000 7f0e5697f740 20 client.25174 trim_cache size 0 max 16384 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e5697f740 1 Processor -- start 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e5697f740 1 -- start start 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0384e0 0x559b5c3e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c4098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c409de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e5697f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481ac8e0 con 0x559b5c01b4d0 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e5697f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481ae420 con 0x559b5c0236b0 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e5697f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481acfb0 con 0x559b5c0384e0 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d079640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c4098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d079640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c4098a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47126/0 (socket says 172.21.15.78:47126) 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d079640 1 -- 172.21.15.78:0/1687352029 learned_addr learned my addr 172.21.15.78:0/1687352029 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c409de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0384e0 0x559b5c3e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d079640 1 -- 172.21.15.78:0/1687352029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0384e0 msgr2=0x559b5c3e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d079640 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0384e0 0x559b5c3e4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d079640 1 -- 172.21.15.78:0/1687352029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 msgr2=0x559b5c409de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d079640 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c409de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4d079640 1 -- 172.21.15.78:0/1687352029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5c3e42d0 con 0x559b5c01b4d0 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.606+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c409de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:13.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.607+0000 7f0e4d079640 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c4098a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0e38005bc0 tx=0x7f0e380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa5b5c63fece4ee server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.609+0000 7f0e337fe640 1 -- 172.21.15.78:0/1687352029 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800f920 con 0x559b5c01b4d0 2024-05-21T15:16:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.609+0000 7f0e337fe640 1 -- 172.21.15.78:0/1687352029 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3800fac0 con 0x559b5c01b4d0 2024-05-21T15:16:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.609+0000 7f0e337fe640 1 -- 172.21.15.78:0/1687352029 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e380055a0 con 0x559b5c01b4d0 2024-05-21T15:16:13.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 -- 172.21.15.78:0/1687352029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 msgr2=0x559b5c4098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c4098a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0e38005bc0 tx=0x7f0e380057f0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 -- 172.21.15.78:0/1687352029 shutdown_connections 2024-05-21T15:16:13.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0236b0 0x559b5c409de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c4098a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 --2- 172.21.15.78:0/1687352029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0384e0 0x559b5c3e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 -- 172.21.15.78:0/1687352029 >> 172.21.15.78:0/1687352029 conn(0x559b5c3ed070 msgr2=0x559b5c3e4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 -- 172.21.15.78:0/1687352029 shutdown_connections 2024-05-21T15:16:13.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.610+0000 7f0e5697f740 1 -- 172.21.15.78:0/1687352029 wait complete. 2024-05-21T15:16:13.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 1 -- start start 2024-05-21T15:16:13.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 1 Processor -- start 2024-05-21T15:16:13.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0384e0 0x559b5c3f9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481add50 con 0x559b5c01b4d0 2024-05-21T15:16:13.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481ad680 con 0x559b5c0384e0 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481aa8c0 con 0x559b5c0236b0 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e5697f740 10 client.0 _ll_register_callbacks cb 0x7f0e38009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e4d87a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e4d87a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47132/0 (socket says 172.21.15.78:47132) 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e4d87a640 1 -- 172.21.15.78:0/2662344561 learned_addr learned my addr 172.21.15.78:0/2662344561 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e4d079640 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.611+0000 7f0e4e07b640 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0384e0 0x559b5c3f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e4d87a640 1 -- 172.21.15.78:0/2662344561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 msgr2=0x559b5c3f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e4d87a640 1 -- 172.21.15.78:0/2662344561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0384e0 msgr2=0x559b5c3f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0384e0 0x559b5c3f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e4d87a640 1 -- 172.21.15.78:0/2662344561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5c3f99d0 con 0x559b5c01b4d0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e4d87a640 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3f6f10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0e440b9e20 tx=0x7f0e44031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b52fcd3bd5faa17 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e317fa640 1 -- 172.21.15.78:0/2662344561 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e440bd790 con 0x559b5c01b4d0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e317fa640 1 -- 172.21.15.78:0/2662344561 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0e440eef50 con 0x559b5c01b4d0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e5697f740 1 -- 172.21.15.78:0/2662344561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e3800fac0 con 0x559b5c01b4d0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e317fa640 1 -- 172.21.15.78:0/2662344561 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e440ef150 con 0x559b5c01b4d0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e5697f740 10 client.15606 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e5697f740 1 -- 172.21.15.78:0/2662344561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b5c46bed0 con 0x559b5c01b4d0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.612+0000 7f0e317fa640 1 -- 172.21.15.78:0/2662344561 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3800fac0 con 0x559b5c01b4d0 2024-05-21T15:16:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e317fa640 1 -- 172.21.15.78:0/2662344561 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0e440bdc20 con 0x559b5c01b4d0 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e5697f740 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e5697f740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e5697f740 20 client.15606 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559b5c044390 0x559b5c46e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e5697f740 4 client.15606 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e5697f740 1 -- 172.21.15.78:0/2662344561 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b5c350780 con 0x559b5c044390 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e4d079640 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559b5c044390 0x559b5c46e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e317fa640 10 client.15606 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.613+0000 7f0e4d079640 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559b5c044390 0x559b5c46e630 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d4002c3f61ee2e server_cookie=fcd779ff59bad7d8 in_seq=0 out_seq=0 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e317fa640 1 -- 172.21.15.78:0/2662344561 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559b5c350780 con 0x559b5c044390 2024-05-21T15:16:13.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e317fa640 10 client.15606 handle_command_reply: tid=0 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 client.15606 shutdown 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 -- 172.21.15.78:0/2662344561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 msgr2=0x559b5c3f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3f6f10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0e440b9e20 tx=0x7f0e44031490 comp rx=0 tx=0).stop 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 -- 172.21.15.78:0/2662344561 shutdown_connections 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559b5c044390 0x559b5c46e630 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b5c0384e0 0x559b5c3f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559b5c0236b0 0x559b5c3f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 --2- 172.21.15.78:0/2662344561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b5c01b4d0 0x559b5c3f6f10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e317fa640 0 client.15606 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 -- 172.21.15.78:0/2662344561 >> 172.21.15.78:0/2662344561 conn(0x559b5c3ed070 msgr2=0x559b5c3e4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 -- 172.21.15.78:0/2662344561 shutdown_connections 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 1 -- 172.21.15.78:0/2662344561 wait complete. 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.614+0000 7f0e5697f740 20 client.15606 trim_cache size 0 max 16384 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "2275f818-63ef-43d3-b1db-4a135e95df40": { 2024-05-21T15:16:13.623 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:16:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "2275f818-63ef-43d3-b1db-4a135e95df40", 2024-05-21T15:16:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:16:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:16:13.624 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:16:13.624 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 -- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e340a2d90 msgr2=0x7f0e340a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e340a2d90 0x7f0e340a52a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e40023000 tx=0x7f0e40021000 comp rx=0 tx=0).stop 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 -- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e48101aa0 msgr2=0x7f0e48198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e48101aa0 0x7f0e48198670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0e4402a930 tx=0x7f0e44003b20 comp rx=0 tx=0).stop 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 -- 172.21.15.78:0/2577925819 shutdown_connections 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e481079d0 0x7f0e48198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e340a2d90 0x7f0e340a52a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e4810c330 0x7f0e4819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 --2- 172.21.15.78:0/2577925819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e48101aa0 0x7f0e48198670 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 -- 172.21.15.78:0/2577925819 >> 172.21.15.78:0/2577925819 conn(0x7f0e480770b0 msgr2=0x7f0e48103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 -- 172.21.15.78:0/2577925819 shutdown_connections 2024-05-21T15:16:13.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.615+0000 7f0e4fade640 1 -- 172.21.15.78:0/2577925819 wait complete. 2024-05-21T15:16:13.633 INFO:tasks.cephfs.filesystem:scrub status for tag:2275f818-63ef-43d3-b1db-4a135e95df40 - {'path': '/', 'tag': '2275f818-63ef-43d3-b1db-4a135e95df40', 'options': 'recursive,force'} 2024-05-21T15:16:13.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.634+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fb2901a85e0 con 0x7fb290107eb0 2024-05-21T15:16:13.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.639+0000 7fb2867fc640 1 -- 172.21.15.78:0/3131727379 <== mon.2 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7fb28809c420 con 0x7fb290107eb0 2024-05-21T15:16:13.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb27c0a2d80 msgr2=0x7fb27c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb27c0a2d80 0x7fb27c0a5290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb28c000c00 tx=0x7fb28c0056d0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb290107eb0 msgr2=0x7fb29019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb290107eb0 0x7fb29019b8b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb28800cab0 tx=0x7fb28800cf70 comp rx=0 tx=0).stop 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 shutdown_connections 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb290107eb0 0x7fb29019b8b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb27c0a2d80 0x7fb27c0a5290 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb290107a10 0x7fb2901a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 --2- 172.21.15.78:0/3131727379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb290102170 0x7fb2901a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.642+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 >> 172.21.15.78:0/3131727379 conn(0x7fb290077160 msgr2=0x7fb290100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.643+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 shutdown_connections 2024-05-21T15:16:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.643+0000 7fb297338640 1 -- 172.21.15.78:0/3131727379 wait complete. 2024-05-21T15:16:13.658 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-21T15:16:13.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.808+0000 7fd8dc813640 1 -- 172.21.15.78:0/4112812145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8102170 msgr2=0x7fd8d810e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.808+0000 7fd8dc813640 1 --2- 172.21.15.78:0/4112812145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8102170 0x7fd8d810e4a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009ca0 tx=0x7fd8d002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.808+0000 7fd8dc813640 1 -- 172.21.15.78:0/4112812145 shutdown_connections 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.808+0000 7fd8dc813640 1 --2- 172.21.15.78:0/4112812145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8102170 0x7fd8d810e4a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.808+0000 7fd8dc813640 1 --2- 172.21.15.78:0/4112812145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8d8107eb0 0x7fd8d8101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.808+0000 7fd8dc813640 1 --2- 172.21.15.78:0/4112812145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8107a10 0x7fd8d81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.808+0000 7fd8dc813640 1 -- 172.21.15.78:0/4112812145 >> 172.21.15.78:0/4112812145 conn(0x7fd8d8077160 msgr2=0x7fd8d80775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.808+0000 7fd8dc813640 1 -- 172.21.15.78:0/4112812145 shutdown_connections 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.809+0000 7fd8dc813640 1 -- 172.21.15.78:0/4112812145 wait complete. 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.809+0000 7fd8dc813640 1 Processor -- start 2024-05-21T15:16:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.809+0000 7fd8dc813640 1 -- start start 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8dc813640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8102170 0x7fd8d8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8dc813640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8107a10 0x7fd8d8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8dc813640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8d8107eb0 0x7fd8d819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8dc813640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d8111d80 con 0x7fd8d8107a10 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8dc813640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d8111c00 con 0x7fd8d8102170 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8dc813640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d8111f00 con 0x7fd8d8107eb0 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8107a10 0x7fd8d8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8107a10 0x7fd8d8198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47148/0 (socket says 172.21.15.78:47148) 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 -- 172.21.15.78:0/1419773300 learned_addr learned my addr 172.21.15.78:0/1419773300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d659c640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8102170 0x7fd8d8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d6d9d640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8d8107eb0 0x7fd8d819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 -- 172.21.15.78:0/1419773300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8d8107eb0 msgr2=0x7fd8d819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8d8107eb0 0x7fd8d819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 -- 172.21.15.78:0/1419773300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8102170 msgr2=0x7fd8d8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8102170 0x7fd8d8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 -- 172.21.15.78:0/1419773300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d819d560 con 0x7fd8d8107a10 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d6d9d640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8d8107eb0 0x7fd8d819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8d5d9b640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8107a10 0x7fd8d8198c60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00d740 tx=0x7fd8cc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8c77fe640 1 -- 172.21.15.78:0/1419773300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc017070 con 0x7fd8d8107a10 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8c77fe640 1 -- 172.21.15.78:0/1419773300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8cc00de50 con 0x7fd8d8107a10 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d819d850 con 0x7fd8d8107a10 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8c77fe640 1 -- 172.21.15.78:0/1419773300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc003e00 con 0x7fd8d8107a10 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.810+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d81a51d0 con 0x7fd8d8107a10 2024-05-21T15:16:13.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.812+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d80690c0 con 0x7fd8d8107a10 2024-05-21T15:16:13.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.814+0000 7fd8c77fe640 1 -- 172.21.15.78:0/1419773300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fd8cc003fa0 con 0x7fd8d8107a10 2024-05-21T15:16:13.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.814+0000 7fd8c77fe640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd8c00a2d80 0x7fd8c00a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:13.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.814+0000 7fd8d659c640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd8c00a2d80 0x7fd8c00a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:13.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.814+0000 7fd8c77fe640 1 -- 172.21.15.78:0/1419773300 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd8cc0cf410 con 0x7fd8d8107a10 2024-05-21T15:16:13.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.815+0000 7fd8d659c640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd8c00a2d80 0x7fd8c00a5290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8005fd0 tx=0x7fd8c800b590 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:13.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.818+0000 7fd8c77fe640 1 -- 172.21.15.78:0/1419773300 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd8cc09bec0 con 0x7fd8d8107a10 2024-05-21T15:16:13.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:13.996+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fd8d8109ab0 con 0x7fd8d8107a10 2024-05-21T15:16:14.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.000+0000 7fd8c77fe640 1 -- 172.21.15.78:0/1419773300 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fd8cc09b7c0 con 0x7fd8d8107a10 2024-05-21T15:16:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd8c00a2d80 msgr2=0x7fd8c00a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd8c00a2d80 0x7fd8c00a5290 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8005fd0 tx=0x7fd8c800b590 comp rx=0 tx=0).stop 2024-05-21T15:16:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8107a10 msgr2=0x7fd8d8198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8107a10 0x7fd8d8198c60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00d740 tx=0x7fd8cc00dc00 comp rx=0 tx=0).stop 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 shutdown_connections 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd8c00a2d80 0x7fd8c00a5290 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8d8107eb0 0x7fd8d819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd8d8107a10 0x7fd8d8198c60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 --2- 172.21.15.78:0/1419773300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8102170 0x7fd8d8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 >> 172.21.15.78:0/1419773300 conn(0x7fd8d8077160 msgr2=0x7fd8d80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 shutdown_connections 2024-05-21T15:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.002+0000 7fd8dc813640 1 -- 172.21.15.78:0/1419773300 wait complete. 2024-05-21T15:16:14.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 -- 172.21.15.78:0/2025059787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108102190 msgr2=0x7f010810e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 --2- 172.21.15.78:0/2025059787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108102190 0x7f010810e4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0100009900 tx=0x7f010002a410 comp rx=0 tx=0).stop 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 -- 172.21.15.78:0/2025059787 shutdown_connections 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 --2- 172.21.15.78:0/2025059787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0108107ed0 0x7f0108101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 --2- 172.21.15.78:0/2025059787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108102190 0x7f010810e4c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 --2- 172.21.15.78:0/2025059787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0108107a30 0x7f01081015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 -- 172.21.15.78:0/2025059787 >> 172.21.15.78:0/2025059787 conn(0x7f01080fd210 msgr2=0x7f01080ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 -- 172.21.15.78:0/2025059787 shutdown_connections 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 -- 172.21.15.78:0/2025059787 wait complete. 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 Processor -- start 2024-05-21T15:16:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.131+0000 7f010f06c640 1 -- start start 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010f06c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0108102190 0x7f01081997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010f06c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108107a30 0x7f0108199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010f06c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0108107ed0 0x7f010819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010f06c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0108111e10 con 0x7f0108107a30 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010f06c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0108111c90 con 0x7f0108102190 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010f06c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0108111f90 con 0x7f0108107ed0 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108107a30 0x7f0108199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108107a30 0x7f0108199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47158/0 (socket says 172.21.15.78:47158) 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 -- 172.21.15.78:0/519402957 learned_addr learned my addr 172.21.15.78:0/519402957 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010ce08640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0108102190 0x7f01081997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010d609640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0108107ed0 0x7f010819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 -- 172.21.15.78:0/519402957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0108107ed0 msgr2=0x7f010819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0108107ed0 0x7f010819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 -- 172.21.15.78:0/519402957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0108102190 msgr2=0x7f01081997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0108102190 0x7f01081997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 -- 172.21.15.78:0/519402957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010819e620 con 0x7f0108107a30 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0107fff640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108107a30 0x7f0108199d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00eb90 tx=0x7f00fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0105ffb640 1 -- 172.21.15.78:0/519402957 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc003b70 con 0x7f0108107a30 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0105ffb640 1 -- 172.21.15.78:0/519402957 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00fc00ee90 con 0x7f0108107a30 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010819e910 con 0x7f0108107a30 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.132+0000 7f0105ffb640 1 -- 172.21.15.78:0/519402957 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc019820 con 0x7f0108107a30 2024-05-21T15:16:14.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.133+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081a6290 con 0x7f0108107a30 2024-05-21T15:16:14.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.134+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010804f720 con 0x7f0108107a30 2024-05-21T15:16:14.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.135+0000 7f0105ffb640 1 -- 172.21.15.78:0/519402957 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f00fc0199c0 con 0x7f0108107a30 2024-05-21T15:16:14.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.135+0000 7f0105ffb640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00f00a2d80 0x7f00f00a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.135+0000 7f010ce08640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00f00a2d80 0x7f00f00a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.135+0000 7f0105ffb640 1 -- 172.21.15.78:0/519402957 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f00fc0cfee0 con 0x7f0108107a30 2024-05-21T15:16:14.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.136+0000 7f010ce08640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00f00a2d80 0x7f00f00a5290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f010806b4c0 tx=0x7f00f8005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:14.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.140+0000 7f0105ffb640 1 -- 172.21.15.78:0/519402957 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f00fc09c930 con 0x7f0108107a30 2024-05-21T15:16:14.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.322+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f01081a64f0 con 0x7f0108107a30 2024-05-21T15:16:14.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.326+0000 7f0105ffb640 1 -- 172.21.15.78:0/519402957 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f00fc003d10 con 0x7f0108107a30 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00f00a2d80 msgr2=0x7f00f00a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00f00a2d80 0x7f00f00a5290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f010806b4c0 tx=0x7f00f8005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108107a30 msgr2=0x7f0108199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108107a30 0x7f0108199d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00eb90 tx=0x7f00fc00c6a0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 shutdown_connections 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00f00a2d80 0x7f00f00a5290 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0108107ed0 0x7f010819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0108107a30 0x7f0108199d20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 --2- 172.21.15.78:0/519402957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0108102190 0x7f01081997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.328+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 >> 172.21.15.78:0/519402957 conn(0x7f01080fd210 msgr2=0x7f0108105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.329+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 shutdown_connections 2024-05-21T15:16:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.329+0000 7f010f06c640 1 -- 172.21.15.78:0/519402957 wait complete. 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.343 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/852640596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='client.? ' entity='client.admin' 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1419773300' entity='client.admin' 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.346 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.458+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3957555063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 msgr2=0x7f6bc4101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.458+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3957555063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 0x7f6bc4101b20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009900 tx=0x7f6bb802a410 comp rx=0 tx=0).stop 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3957555063 shutdown_connections 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3957555063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bc4102190 0x7f6bc410e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3957555063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 0x7f6bc4101b20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3957555063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc4107a30 0x7f6bc41015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3957555063 >> 172.21.15.78:0/3957555063 conn(0x7f6bc40fd210 msgr2=0x7f6bc40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3957555063 shutdown_connections 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3957555063 wait complete. 2024-05-21T15:16:14.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 Processor -- start 2024-05-21T15:16:14.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.459+0000 7f6bc9b3e640 1 -- start start 2024-05-21T15:16:14.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc9b3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bc4102190 0x7f6bc4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc9b3e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc4107a30 0x7f6bc4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc9b3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 0x7f6bc419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc9b3e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4111d80 con 0x7f6bc4107ed0 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc9b3e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4111c00 con 0x7f6bc4107a30 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc9b3e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4111f00 con 0x7f6bc4102190 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 0x7f6bc419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bc4102190 0x7f6bc4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 0x7f6bc419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47176/0 (socket says 172.21.15.78:47176) 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 -- 172.21.15.78:0/3039626011 learned_addr learned my addr 172.21.15.78:0/3039626011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc2ffd640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc4107a30 0x7f6bc4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 -- 172.21.15.78:0/3039626011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bc4102190 msgr2=0x7f6bc4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bc4102190 0x7f6bc4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 -- 172.21.15.78:0/3039626011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc4107a30 msgr2=0x7f6bc4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc4107a30 0x7f6bc4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 -- 172.21.15.78:0/3039626011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc419d580 con 0x7f6bc4107ed0 2024-05-21T15:16:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc3fff640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 0x7f6bc419cf40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00eaa0 tx=0x7f6bbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.460+0000 7f6bc0ff9640 1 -- 172.21.15.78:0/3039626011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00cdb0 con 0x7f6bc4107ed0 2024-05-21T15:16:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.461+0000 7f6bc0ff9640 1 -- 172.21.15.78:0/3039626011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bbc005020 con 0x7f6bc4107ed0 2024-05-21T15:16:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.461+0000 7f6bc0ff9640 1 -- 172.21.15.78:0/3039626011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0116b0 con 0x7f6bc4107ed0 2024-05-21T15:16:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.461+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc419d870 con 0x7f6bc4107ed0 2024-05-21T15:16:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.461+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a51f0 con 0x7f6bc4107ed0 2024-05-21T15:16:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.462+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc404f720 con 0x7f6bc4107ed0 2024-05-21T15:16:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.463+0000 7f6bc0ff9640 1 -- 172.21.15.78:0/3039626011 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6bbc005560 con 0x7f6bc4107ed0 2024-05-21T15:16:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.464+0000 7f6bc0ff9640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bac0a2e00 0x7f6bac0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.464+0000 7f6bc37fe640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bac0a2e00 0x7f6bac0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.464+0000 7f6bc0ff9640 1 -- 172.21.15.78:0/3039626011 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6bbc0cfb20 con 0x7f6bc4107ed0 2024-05-21T15:16:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.464+0000 7f6bc37fe640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bac0a2e00 0x7f6bac0a5310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6bc406b4c0 tx=0x7f6bb4005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:14.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.467+0000 7f6bc0ff9640 1 -- 172.21.15.78:0/3039626011 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bbc0cfe20 con 0x7f6bc4107ed0 2024-05-21T15:16:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/852640596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:16:14.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='client.? ' entity='client.admin' 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.648 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1419773300' entity='client.admin' 2024-05-21T15:16:14.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.649 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.666+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f6bc40ff490 con 0x7f6bac0a2e00 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: Updating smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/852640596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-21T15:16:14.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='client.? ' entity='client.admin' 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1419773300' entity='client.admin' 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:14.686 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:14.687 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:14.687 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:14.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.733+0000 7f6bc0ff9640 1 -- 172.21.15.78:0/3039626011 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6bc40ff490 con 0x7f6bac0a2e00 2024-05-21T15:16:14.736 INFO:teuthology.orchestra.run.smithi078.stdout:Schedule set for path / 2024-05-21T15:16:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bac0a2e00 msgr2=0x7f6bac0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bac0a2e00 0x7f6bac0a5310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6bc406b4c0 tx=0x7f6bb4005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 msgr2=0x7f6bc419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 0x7f6bc419cf40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00eaa0 tx=0x7f6bbc00ef60 comp rx=0 tx=0).stop 2024-05-21T15:16:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 shutdown_connections 2024-05-21T15:16:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bc4107a30 0x7f6bc4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6bac0a2e00 0x7f6bac0a5310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bc4107ed0 0x7f6bc419cf40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 --2- 172.21.15.78:0/3039626011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bc4102190 0x7f6bc4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 >> 172.21.15.78:0/3039626011 conn(0x7f6bc40fd210 msgr2=0x7f6bc40fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:14.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 shutdown_connections 2024-05-21T15:16:14.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.735+0000 7f6bc9b3e640 1 -- 172.21.15.78:0/3039626011 wait complete. 2024-05-21T15:16:14.749 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-21T15:16:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 -- 172.21.15.78:0/2897631144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa088103c10 msgr2=0x7fa088104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/2897631144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa088103c10 0x7fa088104020 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa078009ca0 tx=0x7fa07802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 -- 172.21.15.78:0/2897631144 shutdown_connections 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/2897631144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa08810d060 0x7fa08810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/2897631144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08810a670 0x7fa08810ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/2897631144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa088103c10 0x7fa088104020 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 -- 172.21.15.78:0/2897631144 >> 172.21.15.78:0/2897631144 conn(0x7fa0880fd210 msgr2=0x7fa0880ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 -- 172.21.15.78:0/2897631144 shutdown_connections 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.862+0000 7fa08ca5a640 1 -- 172.21.15.78:0/2897631144 wait complete. 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08ca5a640 1 Processor -- start 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08ca5a640 1 -- start start 2024-05-21T15:16:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08ca5a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa088103c10 0x7fa088199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08ca5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08810a670 0x7fa088199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08ca5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa08810d060 0x7fa08819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08ca5a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa088112e10 con 0x7fa08810a670 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08ca5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa088112c90 con 0x7fa08810d060 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08ca5a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa088112f90 con 0x7fa088103c10 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa085d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08810a670 0x7fa088199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa085d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08810a670 0x7fa088199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47188/0 (socket says 172.21.15.78:47188) 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa085d9b640 1 -- 172.21.15.78:0/1831425985 learned_addr learned my addr 172.21.15.78:0/1831425985 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa08659c640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa088103c10 0x7fa088199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa086d9d640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa08810d060 0x7fa08819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa085d9b640 1 -- 172.21.15.78:0/1831425985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa088103c10 msgr2=0x7fa088199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa085d9b640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa088103c10 0x7fa088199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa085d9b640 1 -- 172.21.15.78:0/1831425985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa08810d060 msgr2=0x7fa08819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa085d9b640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa08810d060 0x7fa08819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.863+0000 7fa085d9b640 1 -- 172.21.15.78:0/1831425985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08819e620 con 0x7fa08810a670 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.864+0000 7fa085d9b640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08810a670 0x7fa088199d70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00d950 tx=0x7fa07c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.864+0000 7fa0777fe640 1 -- 172.21.15.78:0/1831425985 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c017030 con 0x7fa08810a670 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.864+0000 7fa0777fe640 1 -- 172.21.15.78:0/1831425985 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa07c004020 con 0x7fa08810a670 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.864+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08819e910 con 0x7fa08810a670 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.864+0000 7fa0777fe640 1 -- 172.21.15.78:0/1831425985 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0047a0 con 0x7fa08810a670 2024-05-21T15:16:14.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.864+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0881a6290 con 0x7fa08810a670 2024-05-21T15:16:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.865+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08804f720 con 0x7fa08810a670 2024-05-21T15:16:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.866+0000 7fa0777fe640 1 -- 172.21.15.78:0/1831425985 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fa07c0041d0 con 0x7fa08810a670 2024-05-21T15:16:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.867+0000 7fa0777fe640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0700a2d80 0x7fa0700a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:14.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.867+0000 7fa0777fe640 1 -- 172.21.15.78:0/1831425985 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa07c0cf090 con 0x7fa08810a670 2024-05-21T15:16:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.867+0000 7fa08659c640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0700a2d80 0x7fa0700a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.867+0000 7fa08659c640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0700a2d80 0x7fa0700a5290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa078004af0 tx=0x7fa078000f40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:14.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:14.871+0000 7fa0777fe640 1 -- 172.21.15.78:0/1831425985 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa07c09bb40 con 0x7fa08810a670 2024-05-21T15:16:15.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.069+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fa08810dc50 con 0x7fa0700a2d80 2024-05-21T15:16:15.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.130+0000 7fa0777fe640 1 -- 172.21.15.78:0/1831425985 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fa08810dc50 con 0x7fa0700a2d80 2024-05-21T15:16:15.133 INFO:teuthology.orchestra.run.smithi078.stdout:Retention added to path / 2024-05-21T15:16:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0700a2d80 msgr2=0x7fa0700a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0700a2d80 0x7fa0700a5290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa078004af0 tx=0x7fa078000f40 comp rx=0 tx=0).stop 2024-05-21T15:16:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08810a670 msgr2=0x7fa088199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08810a670 0x7fa088199d70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00d950 tx=0x7fa07c00de10 comp rx=0 tx=0).stop 2024-05-21T15:16:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 shutdown_connections 2024-05-21T15:16:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa08810d060 0x7fa08819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0700a2d80 0x7fa0700a5290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08810a670 0x7fa088199d70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 --2- 172.21.15.78:0/1831425985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa088103c10 0x7fa088199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 >> 172.21.15.78:0/1831425985 conn(0x7fa0880fd210 msgr2=0x7fa08810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 shutdown_connections 2024-05-21T15:16:15.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.132+0000 7fa08ca5a640 1 -- 172.21.15.78:0/1831425985 wait complete. 2024-05-21T15:16:15.146 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-21T15:16:15.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3005696624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e410f3a0 msgr2=0x7f96e41117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3005696624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e410f3a0 0x7f96e41117c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009ca0 tx=0x7f96dc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3005696624 shutdown_connections 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3005696624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e410ca40 0x7f96e410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3005696624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e410f3a0 0x7f96e41117c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3005696624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e4074290 0x7f96e40746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3005696624 >> 172.21.15.78:0/3005696624 conn(0x7f96e4078220 msgr2=0x7f96e4078660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3005696624 shutdown_connections 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.255+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3005696624 wait complete. 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96eb6e9640 1 Processor -- start 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96eb6e9640 1 -- start start 2024-05-21T15:16:15.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96eb6e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e4074290 0x7f96e41a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96eb6e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e410ca40 0x7f96e41a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96eb6e9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e410f3a0 0x7f96e419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96eb6e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4115060 con 0x7f96e4074290 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96eb6e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4114ee0 con 0x7f96e410ca40 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96eb6e9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41151e0 con 0x7f96e410f3a0 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96e9485640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e4074290 0x7f96e41a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96e9485640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e4074290 0x7f96e41a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47204/0 (socket says 172.21.15.78:47204) 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96e9485640 1 -- 172.21.15.78:0/3717783126 learned_addr learned my addr 172.21.15.78:0/3717783126 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96e9c86640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e410f3a0 0x7f96e419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96e9485640 1 -- 172.21.15.78:0/3717783126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e410f3a0 msgr2=0x7f96e419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96e9485640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e410f3a0 0x7f96e419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96e9485640 1 -- 172.21.15.78:0/3717783126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e410ca40 msgr2=0x7f96e41a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.256+0000 7f96e9485640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e410ca40 0x7f96e41a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.257+0000 7f96e9485640 1 -- 172.21.15.78:0/3717783126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e419bec0 con 0x7f96e4074290 2024-05-21T15:16:15.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.257+0000 7f96e9485640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e4074290 0x7f96e41a15b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e9f0 tx=0x7f96e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:15.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.257+0000 7f96d67fc640 1 -- 172.21.15.78:0/3717783126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e000cd80 con 0x7f96e4074290 2024-05-21T15:16:15.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.257+0000 7f96d67fc640 1 -- 172.21.15.78:0/3717783126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e0005500 con 0x7f96e4074290 2024-05-21T15:16:15.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.257+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e419c1b0 con 0x7f96e4074290 2024-05-21T15:16:15.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.257+0000 7f96d67fc640 1 -- 172.21.15.78:0/3717783126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0010690 con 0x7f96e4074290 2024-05-21T15:16:15.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.257+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41a8400 con 0x7f96e4074290 2024-05-21T15:16:15.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.258+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e40755b0 con 0x7f96e4074290 2024-05-21T15:16:15.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.259+0000 7f96d67fc640 1 -- 172.21.15.78:0/3717783126 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f96e00046c0 con 0x7f96e4074290 2024-05-21T15:16:15.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.260+0000 7f96d67fc640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f96d00a2e00 0x7f96d00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:15.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.260+0000 7f96e8c84640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f96d00a2e00 0x7f96d00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:15.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.260+0000 7f96d67fc640 1 -- 172.21.15.78:0/3717783126 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f96e00cfa20 con 0x7f96e4074290 2024-05-21T15:16:15.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.260+0000 7f96e8c84640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f96d00a2e00 0x7f96d00a5310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f96d8005d80 tx=0x7f96d8005c90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:15.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.265+0000 7f96d67fc640 1 -- 172.21.15.78:0/3717783126 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f96e009c4d0 con 0x7f96e4074290 2024-05-21T15:16:15.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.466+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f96e4101930 con 0x7f96d00a2e00 2024-05-21T15:16:15.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.474+0000 7f96d67fc640 1 -- 172.21.15.78:0/3717783126 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f96e4101930 con 0x7f96d00a2e00 2024-05-21T15:16:15.477 INFO:teuthology.orchestra.run.smithi078.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-21T00:00:00", "created": "2024-05-21T15:16:14", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-21T15:16:15.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f96d00a2e00 msgr2=0x7f96d00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f96d00a2e00 0x7f96d00a5310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f96d8005d80 tx=0x7f96d8005c90 comp rx=0 tx=0).stop 2024-05-21T15:16:15.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e4074290 msgr2=0x7f96e41a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e4074290 0x7f96e41a15b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e9f0 tx=0x7f96e000edf0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 shutdown_connections 2024-05-21T15:16:15.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f96e410ca40 0x7f96e41a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f96d00a2e00 0x7f96d00a5310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e410f3a0 0x7f96e419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 --2- 172.21.15.78:0/3717783126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96e4074290 0x7f96e41a15b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 >> 172.21.15.78:0/3717783126 conn(0x7f96e4078220 msgr2=0x7f96e4101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:15.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 shutdown_connections 2024-05-21T15:16:15.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.476+0000 7f96eb6e9640 1 -- 172.21.15.78:0/3717783126 wait complete. 2024-05-21T15:16:15.488 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:15 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/519402957' entity='client.admin' 2024-05-21T15:16:15.488 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:15 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:15.489 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:15 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:15.489 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:15 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:15.489 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:15 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:15.490 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.599+0000 7f32b41b5640 1 -- 172.21.15.78:0/3295751620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac1017b0 msgr2=0x7f32ac10f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.599+0000 7f32b41b5640 1 --2- 172.21.15.78:0/3295751620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac1017b0 0x7f32ac10f650 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009ca0 tx=0x7f32a402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.599+0000 7f32b41b5640 1 -- 172.21.15.78:0/3295751620 shutdown_connections 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.599+0000 7f32b41b5640 1 --2- 172.21.15.78:0/3295751620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ac108fd0 0x7f32ac101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.599+0000 7f32b41b5640 1 --2- 172.21.15.78:0/3295751620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac1017b0 0x7f32ac10f650 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.599+0000 7f32b41b5640 1 --2- 172.21.15.78:0/3295751620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac108b30 0x7f32ac100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.599+0000 7f32b41b5640 1 -- 172.21.15.78:0/3295751620 >> 172.21.15.78:0/3295751620 conn(0x7f32ac0fc850 msgr2=0x7f32ac0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.600+0000 7f32b41b5640 1 -- 172.21.15.78:0/3295751620 shutdown_connections 2024-05-21T15:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.600+0000 7f32b41b5640 1 -- 172.21.15.78:0/3295751620 wait complete. 2024-05-21T15:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.600+0000 7f32b41b5640 1 Processor -- start 2024-05-21T15:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.600+0000 7f32b41b5640 1 -- start start 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.600+0000 7f32b41b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac1017b0 0x7f32ac19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b41b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac108b30 0x7f32ac19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b41b5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ac108fd0 0x7f32ac199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b41b5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac112e90 con 0x7f32ac108b30 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b41b5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac112d10 con 0x7f32ac108fd0 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b41b5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac113010 con 0x7f32ac1017b0 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac108b30 0x7f32ac19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac108b30 0x7f32ac19fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47222/0 (socket says 172.21.15.78:47222) 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 -- 172.21.15.78:0/796336084 learned_addr learned my addr 172.21.15.78:0/796336084 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b2752640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ac108fd0 0x7f32ac199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1f51640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac1017b0 0x7f32ac19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 -- 172.21.15.78:0/796336084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac1017b0 msgr2=0x7f32ac19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac1017b0 0x7f32ac19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 -- 172.21.15.78:0/796336084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ac108fd0 msgr2=0x7f32ac199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ac108fd0 0x7f32ac199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 -- 172.21.15.78:0/796336084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac199d80 con 0x7f32ac108b30 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1f51640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac1017b0 0x7f32ac19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b1750640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac108b30 0x7f32ac19fa10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f329c005fd0 tx=0x7f329c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32a2ffd640 1 -- 172.21.15.78:0/796336084 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c017070 con 0x7f32ac108b30 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32a2ffd640 1 -- 172.21.15.78:0/796336084 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c009c30 con 0x7f32ac108b30 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac19a010 con 0x7f32ac108b30 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32a2ffd640 1 -- 172.21.15.78:0/796336084 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c003e40 con 0x7f32ac108b30 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.601+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac1a6270 con 0x7f32ac108b30 2024-05-21T15:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.603+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac103f20 con 0x7f32ac108b30 2024-05-21T15:16:15.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.603+0000 7f32a2ffd640 1 -- 172.21.15.78:0/796336084 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f329c004020 con 0x7f32ac108b30 2024-05-21T15:16:15.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.604+0000 7f32a2ffd640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f32980a2d80 0x7f32980a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:15.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.604+0000 7f32b1f51640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f32980a2d80 0x7f32980a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:15.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.604+0000 7f32a2ffd640 1 -- 172.21.15.78:0/796336084 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f329c0cf1e0 con 0x7f32ac108b30 2024-05-21T15:16:15.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.605+0000 7f32b1f51640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f32980a2d80 0x7f32980a5290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32ac06a040 tx=0x7f32a8005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:15.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.609+0000 7f32a2ffd640 1 -- 172.21.15.78:0/796336084 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f329c020030 con 0x7f32ac108b30 2024-05-21T15:16:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:15 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/519402957' entity='client.admin' 2024-05-21T15:16:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:15 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:15 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:15 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:15.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:15 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:15.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:15 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/519402957' entity='client.admin' 2024-05-21T15:16:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:15 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:16:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:15 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:16:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:15 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:16:15.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:15 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:16:15.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.806+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f32ac061f90 con 0x7f32980a2d80 2024-05-21T15:16:15.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.813+0000 7f32a2ffd640 1 -- 172.21.15.78:0/796336084 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f32ac061f90 con 0x7f32980a2d80 2024-05-21T15:16:15.816 INFO:teuthology.orchestra.run.smithi078.stdout:/ 1m 6m3h 2024-05-21T15:16:15.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.815+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f32980a2d80 msgr2=0x7f32980a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.815+0000 7f32b41b5640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f32980a2d80 0x7f32980a5290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32ac06a040 tx=0x7f32a8005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.815+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac108b30 msgr2=0x7f32ac19fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.815+0000 7f32b41b5640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac108b30 0x7f32ac19fa10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f329c005fd0 tx=0x7f329c00dde0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.815+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 shutdown_connections 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.815+0000 7f32b41b5640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32ac108fd0 0x7f32ac199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.816+0000 7f32b41b5640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f32980a2d80 0x7f32980a5290 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.816+0000 7f32b41b5640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32ac108b30 0x7f32ac19fa10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.816+0000 7f32b41b5640 1 --2- 172.21.15.78:0/796336084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac1017b0 0x7f32ac19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.816+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 >> 172.21.15.78:0/796336084 conn(0x7f32ac0fc850 msgr2=0x7f32ac0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.816+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 shutdown_connections 2024-05-21T15:16:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:15.816+0000 7f32b41b5640 1 -- 172.21.15.78:0/796336084 wait complete. 2024-05-21T15:16:15.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-21T15:16:15.856 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-21T15:16:15.867 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-21T15:16:15.879 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:16:15.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-21T15:16:15.899 INFO:teuthology.run_tasks:Running task workunit... 2024-05-21T15:16:15.913 INFO:tasks.workunit:Pulling workunits from ref d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:16:15.913 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-21T15:16:15.913 INFO:tasks.workunit:timeout=3h 2024-05-21T15:16:15.913 INFO:tasks.workunit:cleanup=True 2024-05-21T15:16:15.913 DEBUG:teuthology.orchestra.run.smithi078:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-21T15:16:15.929 INFO:teuthology.orchestra.run.smithi078.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-21T15:16:15.929 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-21T15:16:15.929 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-21T15:16:15.929 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T15:16:15.929 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-21T15:16:15.930 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-05-21 15:14:17.707019884 +0000 2024-05-21T15:16:15.930 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-05-21 15:14:17.707019884 +0000 2024-05-21T15:16:15.930 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-05-21 15:16:05.922154865 +0000 2024-05-21T15:16:15.930 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-05-21 15:14:17.707019884 +0000 2024-05-21T15:16:15.930 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-21T15:16:15.930 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-21T15:16:16.007 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-21T15:16:16.064 INFO:tasks.workunit.client.0.smithi078.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-21T15:16:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:16 smithi084 ceph-mon[36334]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:16 smithi084 ceph-mon[36334]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:16 smithi084 ceph-mon[36334]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:16 smithi172 ceph-mon[36402]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:16 smithi172 ceph-mon[36402]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:16 smithi172 ceph-mon[36402]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:16.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:16 smithi078 ceph-mon[31997]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:16.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:16 smithi078 ceph-mon[31997]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:16:16.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:16 smithi078 ceph-mon[31997]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:17 smithi084 ceph-mon[36334]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:17 smithi084 ceph-mon[36334]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:17 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:16:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:17 smithi172 ceph-mon[36402]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:17 smithi172 ceph-mon[36402]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:17 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:16:17.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:17 smithi078 ceph-mon[31997]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:17.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:17 smithi078 ceph-mon[31997]: from='client.15642 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:16:17.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:17 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:16:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:18 smithi084 ceph-mon[36334]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 93 KiB/s wr, 44 op/s 2024-05-21T15:16:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:18 smithi172 ceph-mon[36402]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 93 KiB/s wr, 44 op/s 2024-05-21T15:16:18.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:18 smithi078 ceph-mon[31997]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 93 KiB/s wr, 44 op/s 2024-05-21T15:16:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:20 smithi084 ceph-mon[36334]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 34 op/s 2024-05-21T15:16:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:20 smithi172 ceph-mon[36402]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 34 op/s 2024-05-21T15:16:20.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:20 smithi078 ceph-mon[31997]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 34 op/s 2024-05-21T15:16:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:22 smithi084 ceph-mon[36334]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-05-21T15:16:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:22 smithi172 ceph-mon[36402]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-05-21T15:16:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:22 smithi078 ceph-mon[31997]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-05-21T15:16:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:24 smithi084 ceph-mon[36334]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-05-21T15:16:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:24 smithi172 ceph-mon[36402]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-05-21T15:16:24.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:24 smithi078 ceph-mon[31997]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-05-21T15:16:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:26 smithi084 ceph-mon[36334]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-05-21T15:16:26.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:26 smithi172 ceph-mon[36402]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-05-21T15:16:26.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:26 smithi078 ceph-mon[31997]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-05-21T15:16:28.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:28 smithi084 ceph-mon[36334]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-05-21T15:16:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:28 smithi172 ceph-mon[36402]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-05-21T15:16:28.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:28 smithi078 ceph-mon[31997]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 -- 172.21.15.78:0/1294518792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 msgr2=0x7f1128100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 --2- 172.21.15.78:0/1294518792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 0x7f1128100b20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1118009ca0 tx=0x7f111802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 -- 172.21.15.78:0/1294518792 shutdown_connections 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 --2- 172.21.15.78:0/1294518792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1128101060 0x7f1128101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 --2- 172.21.15.78:0/1294518792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f112810d560 0x7f112810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 --2- 172.21.15.78:0/1294518792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 0x7f1128100b20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 -- 172.21.15.78:0/1294518792 >> 172.21.15.78:0/1294518792 conn(0x7f11280fc850 msgr2=0x7f11280feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 -- 172.21.15.78:0/1294518792 shutdown_connections 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.042+0000 7f112dd79640 1 -- 172.21.15.78:0/1294518792 wait complete. 2024-05-21T15:16:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.043+0000 7f112dd79640 1 Processor -- start 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.043+0000 7f112dd79640 1 -- start start 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f112dd79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128101060 0x7f112819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f112dd79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 0x7f112819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f112dd79640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f112810d560 0x7f1128199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f112dd79640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1128112e70 con 0x7f1128108bb0 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f112dd79640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1128112cf0 con 0x7f112810d560 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f112dd79640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1128112ff0 con 0x7f1128101060 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 0x7f112819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 0x7f112819fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51864/0 (socket says 172.21.15.78:51864) 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 -- 172.21.15.78:0/774451289 learned_addr learned my addr 172.21.15.78:0/774451289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f11277fe640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128101060 0x7f112819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1127fff640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f112810d560 0x7f1128199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 -- 172.21.15.78:0/774451289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128101060 msgr2=0x7f112819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128101060 0x7f112819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 -- 172.21.15.78:0/774451289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f112810d560 msgr2=0x7f1128199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f112810d560 0x7f1128199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 -- 172.21.15.78:0/774451289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1128199d80 con 0x7f1128108bb0 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1127fff640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f112810d560 0x7f1128199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1126ffd640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 0x7f112819fa10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f111c00d770 tx=0x7f111c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1124ff9640 1 -- 172.21.15.78:0/774451289 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c017070 con 0x7f1128108bb0 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1124ff9640 1 -- 172.21.15.78:0/774451289 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111c009c30 con 0x7f1128108bb0 2024-05-21T15:16:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f112dd79640 1 -- 172.21.15.78:0/774451289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112819a010 con 0x7f1128108bb0 2024-05-21T15:16:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f1124ff9640 1 -- 172.21.15.78:0/774451289 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c00f620 con 0x7f1128108bb0 2024-05-21T15:16:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.044+0000 7f112dd79640 1 -- 172.21.15.78:0/774451289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11281a6270 con 0x7f1128108bb0 2024-05-21T15:16:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.047+0000 7f1124ff9640 1 -- 172.21.15.78:0/774451289 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f111c01a070 con 0x7f1128108bb0 2024-05-21T15:16:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.047+0000 7f1134c19740 1 Processor -- start 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.047+0000 7f1134c19740 1 -- start start 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1134c19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8e3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1134c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd51b2f0 0x559abd8e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1134c19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd5234d0 0x559abd8e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1134c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559abd848340 con 0x559abd51b2f0 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1134c19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559abd7f0290 con 0x559abd5234d0 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1134c19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559abd844300 con 0x559abd5441b0 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd51b2f0 0x559abd8e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd51b2f0 0x559abd8e35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51868/0 (socket says 172.21.15.78:51868) 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 -- 172.21.15.78:0/2691706325 learned_addr learned my addr 172.21.15.78:0/2691706325 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1127fff640 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1126ffd640 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd5234d0 0x559abd8e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1124ff9640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11100a2d80 0x7f11100a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 -- 172.21.15.78:0/2691706325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 msgr2=0x559abd8e3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8e3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 -- 172.21.15.78:0/2691706325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd5234d0 msgr2=0x559abd8e3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd5234d0 0x559abd8e3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1127fff640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11100a2d80 0x7f11100a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 -- 172.21.15.78:0/2691706325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559abd8148e0 con 0x559abd51b2f0 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f1124ff9640 1 -- 172.21.15.78:0/774451289 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f111c0cf110 con 0x7f1128108bb0 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.048+0000 7f11277fe640 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd51b2f0 0x559abd8e35c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1118009ca0 tx=0x7f111802f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5182af9fefa6fe75 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f110dffb640 1 -- 172.21.15.78:0/2691706325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118031db0 con 0x559abd51b2f0 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f1127fff640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11100a2d80 0x7f11100a5290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x559abd8e01f0 tx=0x7f1120004730 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f110dffb640 1 -- 172.21.15.78:0/2691706325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1118036050 con 0x559abd51b2f0 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f110dffb640 1 -- 172.21.15.78:0/2691706325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180042a0 con 0x559abd51b2f0 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f1134c19740 1 -- 172.21.15.78:0/2691706325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd51b2f0 msgr2=0x559abd8e35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f1134c19740 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd51b2f0 0x559abd8e35c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1118009ca0 tx=0x7f111802f4c0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f1134c19740 1 -- 172.21.15.78:0/2691706325 shutdown_connections 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f1134c19740 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd5234d0 0x559abd8e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f1134c19740 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd51b2f0 0x559abd8e35c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.049+0000 7f1134c19740 1 --2- 172.21.15.78:0/2691706325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8e3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.050+0000 7f1134c19740 1 -- 172.21.15.78:0/2691706325 >> 172.21.15.78:0/2691706325 conn(0x559abd538300 msgr2=0x559abd8df830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.050+0000 7f1134c19740 1 -- 172.21.15.78:0/2691706325 shutdown_connections 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.050+0000 7f1134c19740 1 -- 172.21.15.78:0/2691706325 wait complete. 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.050+0000 7f1134c19740 1 -- start start 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.050+0000 7f1134c19740 1 Processor -- start 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1134c19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1134c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1134c19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1134c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559abd8effd0 con 0x559abd5234d0 2024-05-21T15:16:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1134c19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559abd803790 con 0x559abd51b2f0 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1134c19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559abd8f0150 con 0x559abd5441b0 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1134c19740 10 client.0 _ll_register_callbacks cb 0x559abd83ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8edce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51870/0 (socket says 172.21.15.78:51870) 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 -- 172.21.15.78:0/1908618957 learned_addr learned my addr 172.21.15.78:0/1908618957 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f11277fe640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1127fff640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 -- 172.21.15.78:0/1908618957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 msgr2=0x559abd8f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 -- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 msgr2=0x559abd8ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8ed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 -- 172.21.15.78:0/1908618957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559abd55c210 con 0x559abd5234d0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1127fff640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f11277fe640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1126ffd640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8edce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f111c009610 tx=0x7f111c0971d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4684f3aceb2145f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1106ffd640 1 -- 172.21.15.78:0/1908618957 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c021070 con 0x559abd5234d0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1106ffd640 1 -- 172.21.15.78:0/1908618957 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f111c09a9b0 con 0x559abd5234d0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1134c19740 1 -- 172.21.15.78:0/1908618957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1118036050 con 0x559abd5234d0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.051+0000 7f1106ffd640 1 -- 172.21.15.78:0/1908618957 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f111c09aec0 con 0x559abd5234d0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1134c19740 10 client.15654 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1134c19740 1 -- 172.21.15.78:0/1908618957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559abd628620 con 0x559abd5234d0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1106ffd640 1 -- 172.21.15.78:0/1908618957 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1118036050 con 0x559abd5234d0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1106ffd640 1 -- 172.21.15.78:0/1908618957 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f111c0972b0 con 0x559abd5234d0 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1134c19740 10 client.15654 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1134c19740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1134c19740 20 client.15654 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1134c19740 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559abd502d80 0x559abd8ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1134c19740 4 client.15654 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f1134c19740 1 -- 172.21.15.78:0/1908618957 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559abd855a70 con 0x559abd502d80 2024-05-21T15:16:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.052+0000 7f11277fe640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559abd502d80 0x559abd8ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.053+0000 7f1106ffd640 10 client.15654 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.053+0000 7f11277fe640 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559abd502d80 0x559abd8ff7e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702cb2f14c7f099e server_cookie=3141373f9c202cad in_seq=0 out_seq=0 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.053+0000 7f1106ffd640 1 -- 172.21.15.78:0/1908618957 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559abd855a70 con 0x559abd502d80 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.053+0000 7f1106ffd640 10 client.15654 handle_command_reply: tid=0 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.053+0000 7f1134c19740 1 client.15654 shutdown 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 -- 172.21.15.78:0/1908618957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 msgr2=0x559abd8edce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8edce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f111c009610 tx=0x7f111c0971d0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 -- 172.21.15.78:0/1908618957 shutdown_connections 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559abd502d80 0x559abd8ff7e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8edce0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1106ffd640 0 client.15654 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 --2- 172.21.15.78:0/1908618957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 -- 172.21.15.78:0/1908618957 >> 172.21.15.78:0/1908618957 conn(0x559abd538300 msgr2=0x559abd8e4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 -- 172.21.15.78:0/1908618957 shutdown_connections 2024-05-21T15:16:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 1 -- 172.21.15.78:0/1908618957 wait complete. 2024-05-21T15:16:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.054+0000 7f1134c19740 20 client.15654 trim_cache size 0 max 16384 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1134c19740 1 Processor -- start 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1134c19740 1 -- start start 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1134c19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1134c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1134c19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1134c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11281aeb30 con 0x559abd5234d0 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1134c19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b0670 con 0x559abd51b2f0 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1134c19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11281af200 con 0x559abd5441b0 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1126ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1126ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8e6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51882/0 (socket says 172.21.15.78:51882) 2024-05-21T15:16:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1126ffd640 1 -- 172.21.15.78:0/276304017 learned_addr learned my addr 172.21.15.78:0/276304017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1126ffd640 1 -- 172.21.15.78:0/276304017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 msgr2=0x559abd8e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1126ffd640 1 --2- 172.21.15.78:0/276304017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1126ffd640 1 -- 172.21.15.78:0/276304017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 msgr2=0x559abd8f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f11277fe640 1 --2- 172.21.15.78:0/276304017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1126ffd640 1 --2- 172.21.15.78:0/276304017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.069+0000 7f1126ffd640 1 -- 172.21.15.78:0/276304017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559abd8e4460 con 0x559abd5234d0 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.070+0000 7f1126ffd640 1 --2- 172.21.15.78:0/276304017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8e6e00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f111c010fd0 tx=0x7f111c09b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ec42c4da78e6d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.070+0000 7f110d7fa640 1 -- 172.21.15.78:0/276304017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c021050 con 0x559abd5234d0 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.070+0000 7f110d7fa640 1 -- 172.21.15.78:0/276304017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111c09f910 con 0x559abd5234d0 2024-05-21T15:16:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.070+0000 7f110d7fa640 1 -- 172.21.15.78:0/276304017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c09ac30 con 0x559abd5234d0 2024-05-21T15:16:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.070+0000 7f1134c19740 1 -- 172.21.15.78:0/276304017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 msgr2=0x559abd8e6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.070+0000 7f1134c19740 1 --2- 172.21.15.78:0/276304017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8e6e00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f111c010fd0 tx=0x7f111c09b160 comp rx=0 tx=0).stop 2024-05-21T15:16:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 -- 172.21.15.78:0/276304017 shutdown_connections 2024-05-21T15:16:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 --2- 172.21.15.78:0/276304017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5441b0 0x559abd8e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 --2- 172.21.15.78:0/276304017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5234d0 0x559abd8e6e00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 --2- 172.21.15.78:0/276304017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 -- 172.21.15.78:0/276304017 >> 172.21.15.78:0/276304017 conn(0x559abd538300 msgr2=0x559abd8ebe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 -- 172.21.15.78:0/276304017 shutdown_connections 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 -- 172.21.15.78:0/276304017 wait complete. 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 -- start start 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 Processor -- start 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8f9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5234d0 0x559abd8f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5441b0 0x559abd8f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11281affa0 con 0x559abd5441b0 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f11281af8d0 con 0x559abd51b2f0 2024-05-21T15:16:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11281acb10 con 0x559abd5234d0 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.071+0000 7f1134c19740 10 client.0 _ll_register_callbacks cb 0x7f111c010bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5441b0 0x559abd8f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5441b0 0x559abd8f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51898/0 (socket says 172.21.15.78:51898) 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 -- 172.21.15.78:0/4270526876 learned_addr learned my addr 172.21.15.78:0/4270526876 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f11277fe640 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1126ffd640 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5234d0 0x559abd8f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 -- 172.21.15.78:0/4270526876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5234d0 msgr2=0x559abd8f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5234d0 0x559abd8f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 -- 172.21.15.78:0/4270526876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 msgr2=0x559abd8f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 -- 172.21.15.78:0/4270526876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559abd8f5880 con 0x559abd5441b0 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1127fff640 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5441b0 0x559abd8f50f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1120011830 tx=0x7f11200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193fced5f349a769 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1106ffd640 1 -- 172.21.15.78:0/4270526876 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120010a40 con 0x559abd5441b0 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1106ffd640 1 -- 172.21.15.78:0/4270526876 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f1120010be0 con 0x559abd5441b0 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1134c19740 1 -- 172.21.15.78:0/4270526876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111c09f910 con 0x559abd5441b0 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1106ffd640 1 -- 172.21.15.78:0/4270526876 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1120010de0 con 0x559abd5441b0 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1134c19740 10 client.15660 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.072+0000 7f1134c19740 1 -- 172.21.15.78:0/4270526876 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559abd967aa0 con 0x559abd5441b0 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1106ffd640 1 -- 172.21.15.78:0/4270526876 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111c09f910 con 0x559abd5441b0 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1106ffd640 1 -- 172.21.15.78:0/4270526876 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f112001b570 con 0x559abd5441b0 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1134c19740 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1134c19740 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1134c19740 20 client.15660 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1134c19740 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559abd4d81c0 0x559abd96a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1134c19740 4 client.15660 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:16:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1134c19740 1 -- 172.21.15.78:0/4270526876 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559abd84fdc0 con 0x559abd4d81c0 2024-05-21T15:16:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f11277fe640 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559abd4d81c0 0x559abd96a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f1106ffd640 10 client.15660 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.073+0000 7f11277fe640 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559abd4d81c0 0x559abd96a290 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4e1e688626f44e server_cookie=a5ba42d8380e919f in_seq=0 out_seq=0 2024-05-21T15:16:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.087+0000 7f1106ffd640 1 -- 172.21.15.78:0/4270526876 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559abd84fdc0 con 0x559abd4d81c0 2024-05-21T15:16:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.087+0000 7f1106ffd640 10 client.15660 handle_command_reply: tid=0 2024-05-21T15:16:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.087+0000 7f1134c19740 1 client.15660 shutdown 2024-05-21T15:16:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 -- 172.21.15.78:0/4270526876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5441b0 msgr2=0x559abd8f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5441b0 0x559abd8f50f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1120011830 tx=0x7f11200046c0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 -- 172.21.15.78:0/4270526876 shutdown_connections 2024-05-21T15:16:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559abd4d81c0 0x559abd96a290 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559abd5441b0 0x559abd8f50f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1106ffd640 0 client.15660 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559abd5234d0 0x559abd8f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 --2- 172.21.15.78:0/4270526876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559abd51b2f0 0x559abd8f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 -- 172.21.15.78:0/4270526876 >> 172.21.15.78:0/4270526876 conn(0x559abd538300 msgr2=0x559abd8e4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 -- 172.21.15.78:0/4270526876 shutdown_connections 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 1 -- 172.21.15.78:0/4270526876 wait complete. 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f1134c19740 20 client.15660 trim_cache size 0 max 16384 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:16:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:16:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:16:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:16:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f112dd79640 1 -- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11100a2d80 msgr2=0x7f11100a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f112dd79640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11100a2d80 0x7f11100a5290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x559abd8e01f0 tx=0x7f1120004730 comp rx=0 tx=0).stop 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f112dd79640 1 -- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 msgr2=0x7f112819fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.088+0000 7f112dd79640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 0x7f112819fa10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f111c00d770 tx=0x7f111c00dc30 comp rx=0 tx=0).stop 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.089+0000 7f112dd79640 1 -- 172.21.15.78:0/774451289 shutdown_connections 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.089+0000 7f112dd79640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f112810d560 0x7f1128199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.089+0000 7f112dd79640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11100a2d80 0x7f11100a5290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.089+0000 7f112dd79640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1128108bb0 0x7f112819fa10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.089+0000 7f112dd79640 1 --2- 172.21.15.78:0/774451289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1128101060 0x7f112819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.089+0000 7f112dd79640 1 -- 172.21.15.78:0/774451289 >> 172.21.15.78:0/774451289 conn(0x7f11280fc850 msgr2=0x7f11280fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.089+0000 7f112dd79640 1 -- 172.21.15.78:0/774451289 shutdown_connections 2024-05-21T15:16:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:29.089+0000 7f112dd79640 1 -- 172.21.15.78:0/774451289 wait complete. 2024-05-21T15:16:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:30 smithi172 ceph-mon[36402]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-21T15:16:30.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:30 smithi078 ceph-mon[31997]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-21T15:16:30.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:30 smithi084 ceph-mon[36334]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-21T15:16:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:32 smithi172 ceph-mon[36402]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 KiB/s wr, 37 op/s 2024-05-21T15:16:32.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:32 smithi078 ceph-mon[31997]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 KiB/s wr, 37 op/s 2024-05-21T15:16:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:32 smithi084 ceph-mon[36334]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 KiB/s wr, 37 op/s 2024-05-21T15:16:34.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:34 smithi078 ceph-mon[31997]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-21T15:16:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:34 smithi172 ceph-mon[36402]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-21T15:16:34.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:34 smithi084 ceph-mon[36334]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-21T15:16:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:36 smithi172 ceph-mon[36402]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-05-21T15:16:36.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:36 smithi078 ceph-mon[31997]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-05-21T15:16:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:36 smithi084 ceph-mon[36334]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-05-21T15:16:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:38 smithi172 ceph-mon[36402]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-05-21T15:16:38.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:38 smithi078 ceph-mon[31997]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-05-21T15:16:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:38 smithi084 ceph-mon[36334]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-05-21T15:16:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:40 smithi172 ceph-mon[36402]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-21T15:16:40.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:40 smithi078 ceph-mon[31997]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-21T15:16:40.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:40 smithi084 ceph-mon[36334]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-05-21T15:16:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 -- 172.21.15.78:0/258217533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c102190 msgr2=0x7f0d0c10e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 --2- 172.21.15.78:0/258217533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c102190 0x7f0d0c10e4c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009ca0 tx=0x7f0d0402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 -- 172.21.15.78:0/258217533 shutdown_connections 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 --2- 172.21.15.78:0/258217533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d0c107ed0 0x7f0d0c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 --2- 172.21.15.78:0/258217533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c102190 0x7f0d0c10e4c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 --2- 172.21.15.78:0/258217533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d0c107a30 0x7f0d0c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 -- 172.21.15.78:0/258217533 >> 172.21.15.78:0/258217533 conn(0x7f0d0c0fd210 msgr2=0x7f0d0c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 -- 172.21.15.78:0/258217533 shutdown_connections 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.194+0000 7f0d1202b640 1 -- 172.21.15.78:0/258217533 wait complete. 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d1202b640 1 Processor -- start 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d1202b640 1 -- start start 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d1202b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d0c102190 0x7f0d0c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d1202b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d0c107a30 0x7f0d0c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d1202b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c107ed0 0x7f0d0c19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d1202b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c111df0 con 0x7f0d0c107ed0 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d1202b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c111c70 con 0x7f0d0c107a30 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d1202b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c111f70 con 0x7f0d0c102190 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c107ed0 0x7f0d0c19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c107ed0 0x7f0d0c19aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43586/0 (socket says 172.21.15.78:43586) 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0bfff640 1 -- 172.21.15.78:0/3094191116 learned_addr learned my addr 172.21.15.78:0/3094191116 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d0c102190 0x7f0d0c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d03fff640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d0c107a30 0x7f0d0c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0bfff640 1 -- 172.21.15.78:0/3094191116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d0c102190 msgr2=0x7f0d0c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d0c102190 0x7f0d0c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0bfff640 1 -- 172.21.15.78:0/3094191116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d0c107a30 msgr2=0x7f0d0c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d0c107a30 0x7f0d0c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d0bfff640 1 -- 172.21.15.78:0/3094191116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c1a5800 con 0x7f0d0c107ed0 2024-05-21T15:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.195+0000 7f0d03fff640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d0c107a30 0x7f0d0c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.196+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c107ed0 0x7f0d0c19aff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d04003f40 tx=0x7f0d04003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.196+0000 7f0d097fa640 1 -- 172.21.15.78:0/3094191116 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d040318c0 con 0x7f0d0c107ed0 2024-05-21T15:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.196+0000 7f0d097fa640 1 -- 172.21.15.78:0/3094191116 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d04004520 con 0x7f0d0c107ed0 2024-05-21T15:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.196+0000 7f0d1202b640 1 -- 172.21.15.78:0/3094191116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c1a5a90 con 0x7f0d0c107ed0 2024-05-21T15:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.196+0000 7f0d097fa640 1 -- 172.21.15.78:0/3094191116 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0402fe10 con 0x7f0d0c107ed0 2024-05-21T15:16:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.196+0000 7f0d1202b640 1 -- 172.21.15.78:0/3094191116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1a5fe0 con 0x7f0d0c107ed0 2024-05-21T15:16:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.198+0000 7f0d097fa640 1 -- 172.21.15.78:0/3094191116 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f0d040046d0 con 0x7f0d0c107ed0 2024-05-21T15:16:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.198+0000 7f0d18fad740 1 Processor -- start 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.198+0000 7f0d18fad740 1 -- start start 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56376337a1b0 0x563763719080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x5637637195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x563763719b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d18fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56376367e340 con 0x5637633512f0 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d18fad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563763626290 con 0x5637633594d0 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d18fad740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56376367a300 con 0x56376337a1b0 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x5637637195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d097fa640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0cf80a2d80 0x7f0cf80a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x5637637195c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43590/0 (socket says 172.21.15.78:43590) 2024-05-21T15:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 -- 172.21.15.78:0/1261599995 learned_addr learned my addr 172.21.15.78:0/1261599995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d03fff640 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x563763719b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0cf80a2d80 0x7f0cf80a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d03fff640 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56376337a1b0 0x563763719080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 -- 172.21.15.78:0/1261599995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56376337a1b0 msgr2=0x563763719080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56376337a1b0 0x563763719080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 -- 172.21.15.78:0/1261599995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 msgr2=0x563763719b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x563763719b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 -- 172.21.15.78:0/1261599995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376364a8e0 con 0x5637633512f0 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d097fa640 1 -- 172.21.15.78:0/3094191116 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0d040f0c10 con 0x7f0d0c107ed0 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x5637637195c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00cc00 tx=0x7f0cfc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14973befd7637a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.199+0000 7f0d0bfff640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0cf80a2d80 0x7f0cf80a5290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0d040ee410 tx=0x7f0d040880f0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d01ffb640 1 -- 172.21.15.78:0/1261599995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc005220 con 0x5637633512f0 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d01ffb640 1 -- 172.21.15.78:0/1261599995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cfc004c80 con 0x5637633512f0 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d01ffb640 1 -- 172.21.15.78:0/1261599995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0107e0 con 0x5637633512f0 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d18fad740 1 -- 172.21.15.78:0/1261599995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 msgr2=0x5637637195c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d18fad740 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x5637637195c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00cc00 tx=0x7f0cfc00e720 comp rx=0 tx=0).stop 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d18fad740 1 -- 172.21.15.78:0/1261599995 shutdown_connections 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d18fad740 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x563763719b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d18fad740 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x5637637195c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d18fad740 1 --2- 172.21.15.78:0/1261599995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56376337a1b0 0x563763719080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.200+0000 7f0d18fad740 1 -- 172.21.15.78:0/1261599995 >> 172.21.15.78:0/1261599995 conn(0x56376336e300 msgr2=0x563763715830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 -- 172.21.15.78:0/1261599995 shutdown_connections 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 -- 172.21.15.78:0/1261599995 wait complete. 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 -- start start 2024-05-21T15:16:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 Processor -- start 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x563763723730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633594d0 0x563763723c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56376337a1b0 0x563763729700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637637261b0 con 0x5637633512f0 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563763639790 con 0x56376337a1b0 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.201+0000 7f0d18fad740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563763726330 con 0x5637633594d0 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d18fad740 10 client.0 _ll_register_callbacks cb 0x563763674a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x563763723730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x563763723730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43598/0 (socket says 172.21.15.78:43598) 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 -- 172.21.15.78:0/3710173427 learned_addr learned my addr 172.21.15.78:0/3710173427 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633594d0 0x563763723c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56376337a1b0 0x563763729700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 -- 172.21.15.78:0/3710173427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633594d0 msgr2=0x563763723c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633594d0 0x563763723c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 -- 172.21.15.78:0/3710173427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56376337a1b0 msgr2=0x563763729700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56376337a1b0 0x563763729700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 -- 172.21.15.78:0/3710173427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563763392210 con 0x5637633512f0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d03fff640 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x563763723730 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4012bb0 tx=0x7f0cf400d6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3678daca16e55790 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0ceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0ceaffd640 1 -- 172.21.15.78:0/3710173427 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4011e50 con 0x5637633512f0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0ceaffd640 1 -- 172.21.15.78:0/3710173427 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0cf400f3d0 con 0x5637633512f0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.202+0000 7f0d18fad740 1 -- 172.21.15.78:0/3710173427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cfc004c80 con 0x5637633512f0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0ceaffd640 1 -- 172.21.15.78:0/3710173427 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0cf4027050 con 0x5637633512f0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d18fad740 10 client.15672 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d18fad740 1 -- 172.21.15.78:0/3710173427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56376345e620 con 0x5637633512f0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0ceaffd640 1 -- 172.21.15.78:0/3710173427 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cfc004c80 con 0x5637633512f0 2024-05-21T15:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0ceaffd640 1 -- 172.21.15.78:0/3710173427 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0cf40106a0 con 0x5637633512f0 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d18fad740 10 client.15672 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d18fad740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d18fad740 20 client.15672 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d18fad740 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563763338d80 0x5637637359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d18fad740 4 client.15672 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d18fad740 1 -- 172.21.15.78:0/3710173427 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56376368ba70 con 0x563763338d80 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.203+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563763338d80 0x5637637359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.204+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563763338d80 0x5637637359f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7645e18df2a09e9c server_cookie=e84bfeb189503439 in_seq=0 out_seq=0 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.204+0000 7f0ceaffd640 10 client.15672 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0ceaffd640 1 -- 172.21.15.78:0/3710173427 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56376368ba70 con 0x563763338d80 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0ceaffd640 10 client.15672 handle_command_reply: tid=0 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 client.15672 shutdown 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 -- 172.21.15.78:0/3710173427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 msgr2=0x563763723730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x563763723730 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4012bb0 tx=0x7f0cf400d6f0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 -- 172.21.15.78:0/3710173427 shutdown_connections 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563763338d80 0x5637637359f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56376337a1b0 0x563763729700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633594d0 0x563763723c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0ceaffd640 0 client.15672 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 --2- 172.21.15.78:0/3710173427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637633512f0 0x563763723730 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 -- 172.21.15.78:0/3710173427 >> 172.21.15.78:0/3710173427 conn(0x56376336e300 msgr2=0x563763718e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 -- 172.21.15.78:0/3710173427 shutdown_connections 2024-05-21T15:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 1 -- 172.21.15.78:0/3710173427 wait complete. 2024-05-21T15:16:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.205+0000 7f0d18fad740 20 client.15672 trim_cache size 0 max 16384 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d18fad740 1 Processor -- start 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d18fad740 1 -- start start 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 0x563763719520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x56376373b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376373b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d18fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1aea90 con 0x56376337a1b0 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d18fad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1b05d0 con 0x5637633594d0 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d18fad740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1af160 con 0x5637633512f0 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376373b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376373b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43612/0 (socket says 172.21.15.78:43612) 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 -- 172.21.15.78:0/1327360585 learned_addr learned my addr 172.21.15.78:0/1327360585 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 0x563763719520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 -- 172.21.15.78:0/1327360585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 msgr2=0x563763719520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 0x563763719520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x56376373b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 -- 172.21.15.78:0/1327360585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 msgr2=0x56376373b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x56376373b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.220+0000 7f0d03fff640 1 -- 172.21.15.78:0/1327360585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376371a460 con 0x56376337a1b0 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d03fff640 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376373b670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40124f0 tx=0x7f0cf4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2be3e2064ce1d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d017fa640 1 -- 172.21.15.78:0/1327360585 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4011c50 con 0x56376337a1b0 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d017fa640 1 -- 172.21.15.78:0/1327360585 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf4011df0 con 0x56376337a1b0 2024-05-21T15:16:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d017fa640 1 -- 172.21.15.78:0/1327360585 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4020aa0 con 0x56376337a1b0 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d18fad740 1 -- 172.21.15.78:0/1327360585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 msgr2=0x56376373b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d18fad740 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376373b670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40124f0 tx=0x7f0cf4001e70 comp rx=0 tx=0).stop 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d18fad740 1 -- 172.21.15.78:0/1327360585 shutdown_connections 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d18fad740 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x56376373b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d18fad740 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376373b670 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.221+0000 7f0d18fad740 1 --2- 172.21.15.78:0/1327360585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 0x563763719520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 -- 172.21.15.78:0/1327360585 >> 172.21.15.78:0/1327360585 conn(0x56376336e300 msgr2=0x563763718e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 -- 172.21.15.78:0/1327360585 shutdown_connections 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 -- 172.21.15.78:0/1327360585 wait complete. 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 -- start start 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 Processor -- start 2024-05-21T15:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 0x56376372f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x56376372cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376372cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1aff00 con 0x56376337a1b0 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1af830 con 0x5637633594d0 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.222+0000 7f0d18fad740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1aca70 con 0x5637633512f0 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d18fad740 10 client.0 _ll_register_callbacks cb 0x7f0cf40127c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376372cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376372cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43614/0 (socket says 172.21.15.78:43614) 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 -- 172.21.15.78:0/448127304 learned_addr learned my addr 172.21.15.78:0/448127304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x56376372cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 -- 172.21.15.78:0/448127304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 msgr2=0x56376372f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 0x56376372f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 -- 172.21.15.78:0/448127304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 msgr2=0x56376372cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x56376372cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 -- 172.21.15.78:0/448127304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376372d530 con 0x56376337a1b0 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d03fff640 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376372cff0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400d8a0 tx=0x7f0cf400de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4efe2e1ae74542e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0ceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0ceaffd640 1 -- 172.21.15.78:0/448127304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40103d0 con 0x56376337a1b0 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0ceaffd640 1 -- 172.21.15.78:0/448127304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0cf4010570 con 0x56376337a1b0 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d18fad740 1 -- 172.21.15.78:0/448127304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf4011df0 con 0x56376337a1b0 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0ceaffd640 1 -- 172.21.15.78:0/448127304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0cf4010770 con 0x56376337a1b0 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d18fad740 10 client.15678 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.223+0000 7f0d18fad740 1 -- 172.21.15.78:0/448127304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56376379d9a0 con 0x56376337a1b0 2024-05-21T15:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0ceaffd640 1 -- 172.21.15.78:0/448127304 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf4011df0 con 0x56376337a1b0 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0ceaffd640 1 -- 172.21.15.78:0/448127304 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0cf4035800 con 0x56376337a1b0 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0d18fad740 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0d18fad740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0d18fad740 20 client.15678 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0d18fad740 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56376330e1c0 0x56376379ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0d18fad740 4 client.15678 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0d18fad740 1 -- 172.21.15.78:0/448127304 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563763685dc0 con 0x56376330e1c0 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56376330e1c0 0x56376379ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0ceaffd640 10 client.15678 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.224+0000 7f0d0b7fe640 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56376330e1c0 0x56376379ffe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c72ea44c71aced8 server_cookie=9be5327ad917f402 in_seq=0 out_seq=0 2024-05-21T15:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0ceaffd640 1 -- 172.21.15.78:0/448127304 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563763685dc0 con 0x56376330e1c0 2024-05-21T15:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0ceaffd640 10 client.15678 handle_command_reply: tid=0 2024-05-21T15:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 client.15678 shutdown 2024-05-21T15:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 -- 172.21.15.78:0/448127304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 msgr2=0x56376372cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376372cff0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400d8a0 tx=0x7f0cf400de90 comp rx=0 tx=0).stop 2024-05-21T15:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 -- 172.21.15.78:0/448127304 shutdown_connections 2024-05-21T15:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56376330e1c0 0x56376379ffe0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5637633594d0 0x56376372cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0ceaffd640 0 client.15678 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56376337a1b0 0x56376372cff0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 --2- 172.21.15.78:0/448127304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637633512f0 0x56376372f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 -- 172.21.15.78:0/448127304 >> 172.21.15.78:0/448127304 conn(0x56376336e300 msgr2=0x563763722d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 -- 172.21.15.78:0/448127304 shutdown_connections 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.231+0000 7f0d18fad740 1 -- 172.21.15.78:0/448127304 wait complete. 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d18fad740 20 client.15678 trim_cache size 0 max 16384 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:16:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:16:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:16:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 -- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0cf80a2d80 msgr2=0x7f0cf80a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0cf80a2d80 0x7f0cf80a5290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0d040ee410 tx=0x7f0d040880f0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 -- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c107ed0 msgr2=0x7f0d0c19aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c107ed0 0x7f0d0c19aff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d04003f40 tx=0x7f0d04003f70 comp rx=0 tx=0).stop 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 -- 172.21.15.78:0/3094191116 shutdown_connections 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d0c107a30 0x7f0d0c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0cf80a2d80 0x7f0cf80a5290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d0c107ed0 0x7f0d0c19aff0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 --2- 172.21.15.78:0/3094191116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d0c102190 0x7f0d0c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 -- 172.21.15.78:0/3094191116 >> 172.21.15.78:0/3094191116 conn(0x7f0d0c0fd210 msgr2=0x7f0d0c108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 -- 172.21.15.78:0/3094191116 shutdown_connections 2024-05-21T15:16:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:42.232+0000 7f0d1202b640 1 -- 172.21.15.78:0/3094191116 wait complete. 2024-05-21T15:16:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:42 smithi172 ceph-mon[36402]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-05-21T15:16:42.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:42 smithi078 ceph-mon[31997]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-05-21T15:16:42.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:42 smithi084 ceph-mon[36334]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 16 op/s 2024-05-21T15:16:43.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:16:43.634 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:16:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 -- 172.21.15.78:0/1694932451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac101ab0 msgr2=0x7f3bac101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1694932451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac101ab0 0x7f3bac101ec0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009900 tx=0x7f3ba002a410 comp rx=0 tx=0).stop 2024-05-21T15:16:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 -- 172.21.15.78:0/1694932451 shutdown_connections 2024-05-21T15:16:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1694932451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bac10c340 0x7f3bac10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1694932451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac101ab0 0x7f3bac101ec0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1694932451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bac1079e0 0x7f3bac101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 -- 172.21.15.78:0/1694932451 >> 172.21.15.78:0/1694932451 conn(0x7f3bac0fd210 msgr2=0x7f3bac0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:43.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 -- 172.21.15.78:0/1694932451 shutdown_connections 2024-05-21T15:16:43.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 -- 172.21.15.78:0/1694932451 wait complete. 2024-05-21T15:16:43.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 Processor -- start 2024-05-21T15:16:43.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.759+0000 7f3bb273e640 1 -- start start 2024-05-21T15:16:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb273e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bac101ab0 0x7f3bac199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb273e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bac1079e0 0x7f3bac199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb273e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac10c340 0x7f3bac19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb273e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac111cf0 con 0x7f3bac10c340 2024-05-21T15:16:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb273e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac111b70 con 0x7f3bac101ab0 2024-05-21T15:16:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb273e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac111e70 con 0x7f3bac1079e0 2024-05-21T15:16:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb0cdb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac10c340 0x7f3bac19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb0cdb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac10c340 0x7f3bac19e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43640/0 (socket says 172.21.15.78:43640) 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb0cdb640 1 -- 172.21.15.78:0/1229829703 learned_addr learned my addr 172.21.15.78:0/1229829703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3babfff640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bac101ab0 0x7f3bac199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bac1079e0 0x7f3bac199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb0cdb640 1 -- 172.21.15.78:0/1229829703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bac1079e0 msgr2=0x7f3bac199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb0cdb640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bac1079e0 0x7f3bac199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb0cdb640 1 -- 172.21.15.78:0/1229829703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bac101ab0 msgr2=0x7f3bac199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb0cdb640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bac101ab0 0x7f3bac199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.760+0000 7f3bb0cdb640 1 -- 172.21.15.78:0/1229829703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac19e660 con 0x7f3bac10c340 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.761+0000 7f3bb0cdb640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac10c340 0x7f3bac19e020 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400eaa0 tx=0x7f3ba400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.761+0000 7f3ba97fa640 1 -- 172.21.15.78:0/1229829703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400cdb0 con 0x7f3bac10c340 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.761+0000 7f3ba97fa640 1 -- 172.21.15.78:0/1229829703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba4005020 con 0x7f3bac10c340 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.761+0000 7f3bb273e640 1 -- 172.21.15.78:0/1229829703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac19e950 con 0x7f3bac10c340 2024-05-21T15:16:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.761+0000 7f3ba97fa640 1 -- 172.21.15.78:0/1229829703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40116b0 con 0x7f3bac10c340 2024-05-21T15:16:43.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.761+0000 7f3bb273e640 1 -- 172.21.15.78:0/1229829703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac1a62d0 con 0x7f3bac10c340 2024-05-21T15:16:43.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3ba97fa640 1 -- 172.21.15.78:0/1229829703 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f3ba4005560 con 0x7f3bac10c340 2024-05-21T15:16:43.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3bb95eb740 1 Processor -- start 2024-05-21T15:16:43.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3bb95eb740 1 -- start start 2024-05-21T15:16:43.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94b84d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947bd4d0 0x55be94b85270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be94aead00 con 0x55be947bd4d0 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55be94a924c0 con 0x55be947e6390 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.764+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55be94ae6cc0 con 0x55be947c56b0 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947bd4d0 0x55be94b85270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3ba97fa640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b940a2d90 0x7f3b940a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947bd4d0 0x55be94b85270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43652/0 (socket says 172.21.15.78:43652) 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 -- 172.21.15.78:0/3287827436 learned_addr learned my addr 172.21.15.78:0/3287827436 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94b84d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3bb0cdb640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b940a2d90 0x7f3b940a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 -- 172.21.15.78:0/3287827436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 msgr2=0x55be94b857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b857b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 -- 172.21.15.78:0/3287827436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 msgr2=0x55be94b84d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3ba97fa640 1 -- 172.21.15.78:0/1229829703 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3ba40cf720 con 0x7f3bac10c340 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94b84d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 -- 172.21.15.78:0/3287827436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be948c87d0 con 0x55be947bd4d0 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3babfff640 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947bd4d0 0x55be94b85270 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00bd10 tx=0x7f3b9c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce06b8bafeb2d86 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3bb0cdb640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b940a2d90 0x7f3b940a52a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40cce10 tx=0x7f3ba4000f40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3b927fc640 1 -- 172.21.15.78:0/3287827436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c005020 con 0x55be947bd4d0 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3b927fc640 1 -- 172.21.15.78:0/3287827436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c0047a0 con 0x55be947bd4d0 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.765+0000 7f3b927fc640 1 -- 172.21.15.78:0/3287827436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0116e0 con 0x55be947bd4d0 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3287827436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947bd4d0 msgr2=0x55be94b85270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947bd4d0 0x55be94b85270 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00bd10 tx=0x7f3b9c0107a0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3287827436 shutdown_connections 2024-05-21T15:16:43.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947bd4d0 0x55be94b85270 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3287827436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94b84d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3287827436 >> 172.21.15.78:0/3287827436 conn(0x55be947da4e0 msgr2=0x55be94b814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:43.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3287827436 shutdown_connections 2024-05-21T15:16:43.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.766+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3287827436 wait complete. 2024-05-21T15:16:43.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 1 -- start start 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 1 Processor -- start 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947bd4d0 0x55be94b90e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947e6390 0x55be94b8f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be94b91fc0 con 0x55be947e6390 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55be94aa5b90 con 0x55be947bd4d0 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55be94b92140 con 0x55be947c56b0 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bb95eb740 10 client.0 _ll_register_callbacks cb 0x55be94ae1da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947e6390 0x55be94b8f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947e6390 0x55be94b8f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43660/0 (socket says 172.21.15.78:43660) 2024-05-21T15:16:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.767+0000 7f3bab7fe640 1 -- 172.21.15.78:0/3793251036 learned_addr learned my addr 172.21.15.78:0/3793251036 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3babfff640 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bab7fe640 1 -- 172.21.15.78:0/3793251036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 msgr2=0x55be94b913d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b913d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bab7fe640 1 -- 172.21.15.78:0/3793251036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947bd4d0 msgr2=0x55be94b90e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947bd4d0 0x55be94b90e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bab7fe640 1 -- 172.21.15.78:0/3793251036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be947fe3f0 con 0x55be947e6390 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3babfff640 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b913d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947e6390 0x55be94b8f310 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009230 tx=0x7f3ba0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488a104a9e40260a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3b8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/3793251036 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0035070 con 0x55be947e6390 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3793251036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9c0047a0 con 0x55be947e6390 2024-05-21T15:16:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/3793251036 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3ba0004300 con 0x55be947e6390 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/3793251036 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ba00035f0 con 0x55be947e6390 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bb95eb740 10 client.15690 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3793251036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55be947be0f0 con 0x55be947e6390 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.768+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/3793251036 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c0047a0 con 0x55be947e6390 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/3793251036 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ba003d7c0 con 0x55be947e6390 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3bb95eb740 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3bb95eb740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3bb95eb740 20 client.15690 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be947a4f60 0x55be94ba1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3bb95eb740 4 client.15690 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3793251036 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be94af8430 con 0x55be947a4f60 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3babfff640 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be947a4f60 0x55be94ba1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3b8b7fe640 10 client.15690 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.769+0000 7f3babfff640 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be947a4f60 0x55be94ba1c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0cddaf652ba2f4 server_cookie=a16de547e61a8ff1 in_seq=0 out_seq=0 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.770+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/3793251036 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be94af8430 con 0x55be947a4f60 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.770+0000 7f3b8b7fe640 10 client.15690 handle_command_reply: tid=0 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.770+0000 7f3bb95eb740 1 client.15690 shutdown 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3793251036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947e6390 msgr2=0x55be94b8f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947e6390 0x55be94b8f310 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009230 tx=0x7f3ba0009880 comp rx=0 tx=0).stop 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3793251036 shutdown_connections 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be947a4f60 0x55be94ba1c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947e6390 0x55be94b8f310 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947c56b0 0x55be94b913d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3b8b7fe640 0 client.15690 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/3793251036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947bd4d0 0x55be94b90e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3793251036 >> 172.21.15.78:0/3793251036 conn(0x55be947da4e0 msgr2=0x55be94b86a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3793251036 shutdown_connections 2024-05-21T15:16:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 1 -- 172.21.15.78:0/3793251036 wait complete. 2024-05-21T15:16:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.771+0000 7f3bb95eb740 20 client.15690 trim_cache size 0 max 16384 2024-05-21T15:16:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.788+0000 7f3bb95eb740 1 Processor -- start 2024-05-21T15:16:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.788+0000 7f3bb95eb740 1 -- start start 2024-05-21T15:16:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94b8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94b856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94b9b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1aeb80 con 0x55be947c56b0 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1b06c0 con 0x55be947e6390 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1af250 con 0x55be947bd4d0 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94b856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94b856c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43676/0 (socket says 172.21.15.78:43676) 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 -- 172.21.15.78:0/2725286238 learned_addr learned my addr 172.21.15.78:0/2725286238 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94b9b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94b8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 -- 172.21.15.78:0/2725286238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 msgr2=0x55be94b8a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94b8a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 -- 172.21.15.78:0/2725286238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 msgr2=0x55be94b9b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94b9b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 -- 172.21.15.78:0/2725286238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be94b81e90 con 0x55be947c56b0 2024-05-21T15:16:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94b8a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3babfff640 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94b856c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c011010 tx=0x7f3b9c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692931d07a67cbe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.789+0000 7f3b91ffb640 1 -- 172.21.15.78:0/2725286238 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c02f070 con 0x55be947c56b0 2024-05-21T15:16:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3b91ffb640 1 -- 172.21.15.78:0/2725286238 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c02a650 con 0x55be947c56b0 2024-05-21T15:16:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3b91ffb640 1 -- 172.21.15.78:0/2725286238 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c011910 con 0x55be947c56b0 2024-05-21T15:16:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 -- 172.21.15.78:0/2725286238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 msgr2=0x55be94b856c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94b856c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c011010 tx=0x7f3b9c00b410 comp rx=0 tx=0).stop 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 -- 172.21.15.78:0/2725286238 shutdown_connections 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94b9b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94b856c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/2725286238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94b8a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 -- 172.21.15.78:0/2725286238 >> 172.21.15.78:0/2725286238 conn(0x55be947da4e0 msgr2=0x55be94b87050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 -- 172.21.15.78:0/2725286238 shutdown_connections 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.790+0000 7f3bb95eb740 1 -- 172.21.15.78:0/2725286238 wait complete. 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 1 -- start start 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 1 Processor -- start 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94c065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94c06b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94c009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1afff0 con 0x55be947c56b0 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1af920 con 0x55be947e6390 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1acb60 con 0x55be947bd4d0 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bb95eb740 10 client.0 _ll_register_callbacks cb 0x7f3ba0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3babfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94c06b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3babfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94c06b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43678/0 (socket says 172.21.15.78:43678) 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3babfff640 1 -- 172.21.15.78:0/819537280 learned_addr learned my addr 172.21.15.78:0/819537280 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94c065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3babfff640 1 -- 172.21.15.78:0/819537280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 msgr2=0x55be94c065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3babfff640 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94c065d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3babfff640 1 -- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 msgr2=0x55be94c009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.791+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94c009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3babfff640 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94c009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3babfff640 1 -- 172.21.15.78:0/819537280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be94c00fd0 con 0x55be947c56b0 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94c009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3babfff640 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94c06b10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00bd10 tx=0x7f3b9c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2b3bc8241b086 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3b8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/819537280 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c03a060 con 0x55be947c56b0 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/819537280 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3b9c02a800 con 0x55be947c56b0 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 1 -- 172.21.15.78:0/819537280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9c02a650 con 0x55be947c56b0 2024-05-21T15:16:43.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/819537280 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3b9c02f040 con 0x55be947c56b0 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 10 client.15696 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 1 -- 172.21.15.78:0/819537280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55be94c0dbf0 con 0x55be947c56b0 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/819537280 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c02a650 con 0x55be947c56b0 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/819537280 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3b9c011a00 con 0x55be947c56b0 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 20 client.15696 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be9477a3a0 0x55be94c10070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 4 client.15696 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.792+0000 7f3bb95eb740 1 -- 172.21.15.78:0/819537280 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be94af2780 con 0x55be9477a3a0 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be9477a3a0 0x55be94c10070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3b8b7fe640 10 client.15696 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bab7fe640 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be9477a3a0 0x55be94c10070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6af6c59d34c231 server_cookie=5f28a8114fc2f1c4 in_seq=0 out_seq=0 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3b8b7fe640 1 -- 172.21.15.78:0/819537280 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55be94af2780 con 0x55be9477a3a0 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3b8b7fe640 10 client.15696 handle_command_reply: tid=0 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 client.15696 shutdown 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 -- 172.21.15.78:0/819537280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 msgr2=0x55be94c06b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94c06b10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00bd10 tx=0x7f3b9c00b560 comp rx=0 tx=0).stop 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 -- 172.21.15.78:0/819537280 shutdown_connections 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be9477a3a0 0x55be94c10070 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be947e6390 0x55be94c009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be947c56b0 0x55be94c06b10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 --2- 172.21.15.78:0/819537280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be947bd4d0 0x55be94c065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3b8b7fe640 0 client.15696 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.793+0000 7f3bb95eb740 1 -- 172.21.15.78:0/819537280 >> 172.21.15.78:0/819537280 conn(0x55be947da4e0 msgr2=0x55be94b87050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb95eb740 1 -- 172.21.15.78:0/819537280 shutdown_connections 2024-05-21T15:16:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb95eb740 1 -- 172.21.15.78:0/819537280 wait complete. 2024-05-21T15:16:43.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb95eb740 20 client.15696 trim_cache size 0 max 16384 2024-05-21T15:16:43.808 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:43.808 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:16:43.808 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:16:43.808 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:43.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 -- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b940a2d90 msgr2=0x7f3b940a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b940a2d90 0x7f3b940a52a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40cce10 tx=0x7f3ba4000f40 comp rx=0 tx=0).stop 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 -- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac10c340 msgr2=0x7f3bac19e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac10c340 0x7f3bac19e020 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400eaa0 tx=0x7f3ba400ef60 comp rx=0 tx=0).stop 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 -- 172.21.15.78:0/1229829703 shutdown_connections 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b940a2d90 0x7f3b940a52a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3bac10c340 0x7f3bac19e020 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bac1079e0 0x7f3bac199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 --2- 172.21.15.78:0/1229829703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bac101ab0 0x7f3bac199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 -- 172.21.15.78:0/1229829703 >> 172.21.15.78:0/1229829703 conn(0x7f3bac0fd210 msgr2=0x7f3bac0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 -- 172.21.15.78:0/1229829703 shutdown_connections 2024-05-21T15:16:43.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.794+0000 7f3bb273e640 1 -- 172.21.15.78:0/1229829703 wait complete. 2024-05-21T15:16:43.811 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:16:43.811 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:16:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 -- 172.21.15.78:0/135917268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 msgr2=0x7f9154101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 --2- 172.21.15.78:0/135917268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 0x7f9154101570 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9144009900 tx=0x7f914402a410 comp rx=0 tx=0).stop 2024-05-21T15:16:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 -- 172.21.15.78:0/135917268 shutdown_connections 2024-05-21T15:16:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 --2- 172.21.15.78:0/135917268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f915410c340 0x7f915410e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 --2- 172.21.15.78:0/135917268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9154101ab0 0x7f9154101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 --2- 172.21.15.78:0/135917268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 0x7f9154101570 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 -- 172.21.15.78:0/135917268 >> 172.21.15.78:0/135917268 conn(0x7f91540fd030 msgr2=0x7f91540ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 -- 172.21.15.78:0/135917268 shutdown_connections 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.919+0000 7f91588a6640 1 -- 172.21.15.78:0/135917268 wait complete. 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.920+0000 7f91588a6640 1 Processor -- start 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.920+0000 7f91588a6640 1 -- start start 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.920+0000 7f91588a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9154101ab0 0x7f9154199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f91588a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 0x7f9154199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f91588a6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f915410c340 0x7f915419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f91588a6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9154111d40 con 0x7f91541079e0 2024-05-21T15:16:43.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f91588a6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9154111bc0 con 0x7f915410c340 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f91588a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9154111ec0 con 0x7f9154101ab0 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 0x7f9154199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 0x7f9154199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43700/0 (socket says 172.21.15.78:43700) 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 -- 172.21.15.78:0/413370596 learned_addr learned my addr 172.21.15.78:0/413370596 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9152d9d640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f915410c340 0x7f915419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f915259c640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9154101ab0 0x7f9154199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 -- 172.21.15.78:0/413370596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9154101ab0 msgr2=0x7f9154199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9154101ab0 0x7f9154199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 -- 172.21.15.78:0/413370596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f915410c340 msgr2=0x7f915419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f915410c340 0x7f915419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 -- 172.21.15.78:0/413370596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915419e5d0 con 0x7f91541079e0 2024-05-21T15:16:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f915259c640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9154101ab0 0x7f9154199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f9149d9b640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 0x7f9154199cd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f913c005fd0 tx=0x7f913c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f914bfff640 1 -- 172.21.15.78:0/413370596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c017070 con 0x7f91541079e0 2024-05-21T15:16:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915419e8c0 con 0x7f91541079e0 2024-05-21T15:16:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f914bfff640 1 -- 172.21.15.78:0/413370596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f913c009c30 con 0x7f91541079e0 2024-05-21T15:16:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f914bfff640 1 -- 172.21.15.78:0/413370596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c003e40 con 0x7f91541079e0 2024-05-21T15:16:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.921+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541a6240 con 0x7f91541079e0 2024-05-21T15:16:43.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.923+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9154069140 con 0x7f91541079e0 2024-05-21T15:16:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.924+0000 7f914bfff640 1 -- 172.21.15.78:0/413370596 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f913c004000 con 0x7f91541079e0 2024-05-21T15:16:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.924+0000 7f914bfff640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f91400a2d90 0x7f91400a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.925+0000 7f915259c640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f91400a2d90 0x7f91400a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.925+0000 7f914bfff640 1 -- 172.21.15.78:0/413370596 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f913c0cf3b0 con 0x7f91541079e0 2024-05-21T15:16:43.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.925+0000 7f915259c640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f91400a2d90 0x7f91400a52a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f91440096c0 tx=0x7f9144005fd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:43.929+0000 7f914bfff640 1 -- 172.21.15.78:0/413370596 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f913c020030 con 0x7f91541079e0 2024-05-21T15:16:44.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.141+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91541a6410 con 0x7f91541079e0 2024-05-21T15:16:44.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.142+0000 7f914bfff640 1 -- 172.21.15.78:0/413370596 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11925 (secure 0 0 0) 0x7f913c09be60 con 0x7f91541079e0 2024-05-21T15:16:44.144 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:16:44.145 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:16:06.684612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24419,24448,24502,24526,24527]},"id":1}]} 2024-05-21T15:16:44.145 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f91400a2d90 msgr2=0x7f91400a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f91400a2d90 0x7f91400a52a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f91440096c0 tx=0x7f9144005fd0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 msgr2=0x7f9154199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 0x7f9154199cd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f913c005fd0 tx=0x7f913c00dde0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 shutdown_connections 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f915410c340 0x7f915419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f91400a2d90 0x7f91400a52a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91541079e0 0x7f9154199cd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 --2- 172.21.15.78:0/413370596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9154101ab0 0x7f9154199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 >> 172.21.15.78:0/413370596 conn(0x7f91540fd030 msgr2=0x7f915410d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 shutdown_connections 2024-05-21T15:16:44.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.144+0000 7f91588a6640 1 -- 172.21.15.78:0/413370596 wait complete. 2024-05-21T15:16:44.158 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:16:44.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 -- 172.21.15.78:0/1046196440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c107a20 msgr2=0x7fc01c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 --2- 172.21.15.78:0/1046196440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c107a20 0x7fc01c1015b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009900 tx=0x7fc00c02a410 comp rx=0 tx=0).stop 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 -- 172.21.15.78:0/1046196440 shutdown_connections 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 --2- 172.21.15.78:0/1046196440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc01c101af0 0x7fc01c101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 --2- 172.21.15.78:0/1046196440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc01c10c380 0x7fc01c10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 --2- 172.21.15.78:0/1046196440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c107a20 0x7fc01c1015b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 -- 172.21.15.78:0/1046196440 >> 172.21.15.78:0/1046196440 conn(0x7fc01c0fd210 msgr2=0x7fc01c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 -- 172.21.15.78:0/1046196440 shutdown_connections 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.268+0000 7fc0247ea640 1 -- 172.21.15.78:0/1046196440 wait complete. 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc0247ea640 1 Processor -- start 2024-05-21T15:16:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc0247ea640 1 -- start start 2024-05-21T15:16:44.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc0247ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c101af0 0x7fc01c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc0247ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc01c107a20 0x7fc01c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc0247ea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc01c10c380 0x7fc01c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc0247ea640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c111e00 con 0x7fc01c101af0 2024-05-21T15:16:44.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc0247ea640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c111c80 con 0x7fc01c107a20 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc0247ea640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c111f80 con 0x7fc01c10c380 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc022586640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c101af0 0x7fc01c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc022586640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c101af0 0x7fc01c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43708/0 (socket says 172.21.15.78:43708) 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc022586640 1 -- 172.21.15.78:0/3322204968 learned_addr learned my addr 172.21.15.78:0/3322204968 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.269+0000 7fc022d87640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc01c10c380 0x7fc01c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc022586640 1 -- 172.21.15.78:0/3322204968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc01c10c380 msgr2=0x7fc01c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc022586640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc01c10c380 0x7fc01c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc022586640 1 -- 172.21.15.78:0/3322204968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc01c107a20 msgr2=0x7fc01c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc021d85640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc01c107a20 0x7fc01c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc022586640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc01c107a20 0x7fc01c198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc022586640 1 -- 172.21.15.78:0/3322204968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c19d560 con 0x7fc01c101af0 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc022586640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c101af0 0x7fc01c198700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc00c02a930 tx=0x7fc00c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc0137fe640 1 -- 172.21.15.78:0/3322204968 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c03f070 con 0x7fc01c101af0 2024-05-21T15:16:44.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc0137fe640 1 -- 172.21.15.78:0/3322204968 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00c031c90 con 0x7fc01c101af0 2024-05-21T15:16:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc0137fe640 1 -- 172.21.15.78:0/3322204968 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c035030 con 0x7fc01c101af0 2024-05-21T15:16:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc0247ea640 1 -- 172.21.15.78:0/3322204968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c19d7f0 con 0x7fc01c101af0 2024-05-21T15:16:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.270+0000 7fc0247ea640 1 -- 172.21.15.78:0/3322204968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c19dd60 con 0x7fc01c101af0 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.272+0000 7fc0137fe640 1 -- 172.21.15.78:0/3322204968 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fc00c042070 con 0x7fc01c101af0 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc02b76f740 1 Processor -- start 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc02b76f740 1 -- start start 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057420db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x56057420e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057420e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc0137fe640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc0080a2d90 0x7fc0080a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc02b76f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560574173c40 con 0x560573e4e6b0 2024-05-21T15:16:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc02b76f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56057411b580 con 0x560573e6f390 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc02b76f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56057416fc00 con 0x560573e464d0 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc022d87640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc0080a2d90 0x7fc0080a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc021d85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057420e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc021d85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057420e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43714/0 (socket says 172.21.15.78:43714) 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc021d85640 1 -- 172.21.15.78:0/2310168639 learned_addr learned my addr 172.21.15.78:0/2310168639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc0137fe640 1 -- 172.21.15.78:0/3322204968 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc00c0efe20 con 0x7fc01c101af0 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.273+0000 7fc022586640 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x56057420e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc022d87640 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057420db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc021d85640 1 -- 172.21.15.78:0/2310168639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 msgr2=0x56057420e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc021d85640 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x56057420e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc021d85640 1 -- 172.21.15.78:0/2310168639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 msgr2=0x56057420db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc021d85640 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057420db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc021d85640 1 -- 172.21.15.78:0/2310168639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560573e873f0 con 0x560573e4e6b0 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc022d87640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc0080a2d90 0x7fc0080a52a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc014009fd0 tx=0x7fc014012af0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc021d85640 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057420e610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc01800a880 tx=0x7fc01800ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4545fe225748297b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc010ff9640 1 -- 172.21.15.78:0/2310168639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018015070 con 0x560573e4e6b0 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc010ff9640 1 -- 172.21.15.78:0/2310168639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01800cb40 con 0x560573e4e6b0 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.274+0000 7fc010ff9640 1 -- 172.21.15.78:0/2310168639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180185c0 con 0x560573e4e6b0 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 -- 172.21.15.78:0/2310168639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 msgr2=0x56057420e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057420e610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc01800a880 tx=0x7fc01800ad40 comp rx=0 tx=0).stop 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 -- 172.21.15.78:0/2310168639 shutdown_connections 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057420e610 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x56057420e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 --2- 172.21.15.78:0/2310168639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057420db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 -- 172.21.15.78:0/2310168639 >> 172.21.15.78:0/2310168639 conn(0x560573e634e0 msgr2=0x56057420a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 -- 172.21.15.78:0/2310168639 shutdown_connections 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 -- 172.21.15.78:0/2310168639 wait complete. 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.275+0000 7fc02b76f740 1 -- start start 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc02b76f740 1 Processor -- start 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x56057421a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 0x56057421aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 0x560574226730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc02b76f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56057421d280 con 0x560573e464d0 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc02b76f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56057412ec40 con 0x560573e4e6b0 2024-05-21T15:16:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc02b76f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56057421d400 con 0x560573e6f390 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc02b76f740 10 client.0 _ll_register_callbacks cb 0x560573e86be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x56057421a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x56057421a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43728/0 (socket says 172.21.15.78:43728) 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 -- 172.21.15.78:0/1089244831 learned_addr learned my addr 172.21.15.78:0/1089244831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc021d85640 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 0x56057421aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022d87640 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 0x560574226730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 -- 172.21.15.78:0/1089244831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 msgr2=0x560574226730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 0x560574226730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 -- 172.21.15.78:0/1089244831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 msgr2=0x56057421aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 0x56057421aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 -- 172.21.15.78:0/1089244831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560573f517d0 con 0x560573e464d0 2024-05-21T15:16:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022d87640 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 0x560574226730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc022586640 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x56057421a4e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0ed540 tx=0x7fc00c0bbdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdef2f2b4c49b517 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc005ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.276+0000 7fc005ffb640 1 -- 172.21.15.78:0/1089244831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c049040 con 0x560573e464d0 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc005ffb640 1 -- 172.21.15.78:0/1089244831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc00c03b4f0 con 0x560573e464d0 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 1 -- 172.21.15.78:0/1089244831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01800cb40 con 0x560573e464d0 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc005ffb640 1 -- 172.21.15.78:0/1089244831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc00c0bc1d0 con 0x560573e464d0 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 10 client.15714 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 1 -- 172.21.15.78:0/1089244831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560574229900 con 0x560573e464d0 2024-05-21T15:16:44.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc005ffb640 1 -- 172.21.15.78:0/1089244831 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01800cb40 con 0x560573e464d0 2024-05-21T15:16:44.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc005ffb640 1 -- 172.21.15.78:0/1089244831 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc00c0c0170 con 0x560573e464d0 2024-05-21T15:16:44.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 10 client.15714 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 10 client.15714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:16:44.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 20 client.15714 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x560573e2df60 0x56057422ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 4 client.15714 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc02b76f740 1 -- 172.21.15.78:0/1089244831 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560574181370 con 0x560573e2df60 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.277+0000 7fc021d85640 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x560573e2df60 0x56057422ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.278+0000 7fc021d85640 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x560573e2df60 0x56057422ce60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6d1c3963fb887b12 server_cookie=ac1d83405ab4afe2 in_seq=0 out_seq=0 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.278+0000 7fc005ffb640 10 client.15714 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc005ffb640 1 -- 172.21.15.78:0/1089244831 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560574181370 con 0x560573e2df60 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc005ffb640 10 client.15714 handle_command_reply: tid=0 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 client.15714 shutdown 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 -- 172.21.15.78:0/1089244831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 msgr2=0x56057421a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x56057421a4e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0ed540 tx=0x7fc00c0bbdd0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 -- 172.21.15.78:0/1089244831 shutdown_connections 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x560573e2df60 0x56057422ce60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 0x56057421aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc005ffb640 0 client.15714 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:16:44.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 0x560574226730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 --2- 172.21.15.78:0/1089244831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x56057421a4e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 -- 172.21.15.78:0/1089244831 >> 172.21.15.78:0/1089244831 conn(0x560573e634e0 msgr2=0x56057420f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 -- 172.21.15.78:0/1089244831 shutdown_connections 2024-05-21T15:16:44.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 1 -- 172.21.15.78:0/1089244831 wait complete. 2024-05-21T15:16:44.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.279+0000 7fc02b76f740 20 client.15714 trim_cache size 0 max 16384 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.292+0000 7fc02b76f740 1 Processor -- start 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.292+0000 7fc02b76f740 1 -- start start 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x560574209570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 0x560574226310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 0x560574232620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc02b76f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ac8c0 con 0x560573e464d0 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc02b76f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ae400 con 0x560573e4e6b0 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc02b76f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1acf90 con 0x560573e6f390 2024-05-21T15:16:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x560574209570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x560574209570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43732/0 (socket says 172.21.15.78:43732) 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 -- 172.21.15.78:0/4173414545 learned_addr learned my addr 172.21.15.78:0/4173414545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 -- 172.21.15.78:0/4173414545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 msgr2=0x560574232620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 --2- 172.21.15.78:0/4173414545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 0x560574232620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 -- 172.21.15.78:0/4173414545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 msgr2=0x560574226310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 --2- 172.21.15.78:0/4173414545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 0x560574226310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 -- 172.21.15.78:0/4173414545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560574218820 con 0x560573e464d0 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc022586640 1 --2- 172.21.15.78:0/4173414545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x560574209570 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0bbed0 tx=0x7fc00c031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e23adfa7859e7aa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc007fff640 1 -- 172.21.15.78:0/4173414545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c049070 con 0x560573e464d0 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc007fff640 1 -- 172.21.15.78:0/4173414545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00c0c4e40 con 0x560573e464d0 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.293+0000 7fc007fff640 1 -- 172.21.15.78:0/4173414545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0bbfa0 con 0x560573e464d0 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 -- 172.21.15.78:0/4173414545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 msgr2=0x560574209570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 --2- 172.21.15.78:0/4173414545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x560574209570 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0bbed0 tx=0x7fc00c031490 comp rx=0 tx=0).stop 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 -- 172.21.15.78:0/4173414545 shutdown_connections 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 --2- 172.21.15.78:0/4173414545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e4e6b0 0x560574226310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 --2- 172.21.15.78:0/4173414545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e6f390 0x560574232620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 --2- 172.21.15.78:0/4173414545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e464d0 0x560574209570 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 -- 172.21.15.78:0/4173414545 >> 172.21.15.78:0/4173414545 conn(0x560573e634e0 msgr2=0x560574219210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 -- 172.21.15.78:0/4173414545 shutdown_connections 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 -- 172.21.15.78:0/4173414545 wait complete. 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 -- start start 2024-05-21T15:16:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.294+0000 7fc02b76f740 1 Processor -- start 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x560574222950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057421dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc02b76f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057421e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc02b76f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1add30 con 0x560573e4e6b0 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc02b76f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ad660 con 0x560573e6f390 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc02b76f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1aa8a0 con 0x560573e464d0 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc02b76f740 10 client.0 _ll_register_callbacks cb 0x7fc00c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057421dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057421dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43736/0 (socket says 172.21.15.78:43736) 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 -- 172.21.15.78:0/3423267535 learned_addr learned my addr 172.21.15.78:0/3423267535 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc022586640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x560574222950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc022d87640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057421e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 -- 172.21.15.78:0/3423267535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 msgr2=0x560574222950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x560574222950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 -- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 msgr2=0x56057421e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057421e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 -- 172.21.15.78:0/3423267535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56057421e750 con 0x560573e4e6b0 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc022d87640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057421e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc022586640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x560574222950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc021d85640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057421dcd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc018011c90 tx=0x7fc018006f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d39257250af1c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc005ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc005ffb640 1 -- 172.21.15.78:0/3423267535 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01802d070 con 0x560573e4e6b0 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.295+0000 7fc005ffb640 1 -- 172.21.15.78:0/3423267535 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc01800cce0 con 0x560573e4e6b0 2024-05-21T15:16:44.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 1 -- 172.21.15.78:0/3423267535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc00c0c4e40 con 0x560573e4e6b0 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc005ffb640 1 -- 172.21.15.78:0/3423267535 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc018039050 con 0x560573e4e6b0 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 10 client.15720 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 1 -- 172.21.15.78:0/3423267535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560574294df0 con 0x560573e4e6b0 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc005ffb640 1 -- 172.21.15.78:0/3423267535 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00c0c4e40 con 0x560573e4e6b0 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc005ffb640 1 -- 172.21.15.78:0/3423267535 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc018024040 con 0x560573e4e6b0 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 10 client.15720 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 20 client.15720 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x560573e033a0 0x560574297240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 4 client.15720 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc02b76f740 1 -- 172.21.15.78:0/3423267535 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56057417b6c0 con 0x560573e033a0 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.296+0000 7fc022586640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x560573e033a0 0x560574297240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc022586640 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x560573e033a0 0x560574297240 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d34d357796d4a97 server_cookie=413ae81f7fd0811d in_seq=0 out_seq=0 2024-05-21T15:16:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc005ffb640 10 client.15720 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc005ffb640 1 -- 172.21.15.78:0/3423267535 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56057417b6c0 con 0x560573e033a0 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc005ffb640 10 client.15720 handle_command_reply: tid=0 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 client.15720 shutdown 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 -- 172.21.15.78:0/3423267535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 msgr2=0x56057421dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057421dcd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc018011c90 tx=0x7fc018006f40 comp rx=0 tx=0).stop 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 -- 172.21.15.78:0/3423267535 shutdown_connections 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x560573e033a0 0x560574297240 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560573e6f390 0x56057421e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560573e4e6b0 0x56057421dcd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 --2- 172.21.15.78:0/3423267535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560573e464d0 0x560574222950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc005ffb640 0 client.15720 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 -- 172.21.15.78:0/3423267535 >> 172.21.15.78:0/3423267535 conn(0x560573e634e0 msgr2=0x5605742092d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 -- 172.21.15.78:0/3423267535 shutdown_connections 2024-05-21T15:16:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 1 -- 172.21.15.78:0/3423267535 wait complete. 2024-05-21T15:16:44.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.297+0000 7fc02b76f740 20 client.15720 trim_cache size 0 max 16384 2024-05-21T15:16:44.307 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 -- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc0080a2d90 msgr2=0x7fc0080a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc0080a2d90 0x7fc0080a52a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc014009fd0 tx=0x7fc014012af0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 -- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c101af0 msgr2=0x7fc01c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c101af0 0x7fc01c198700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc00c02a930 tx=0x7fc00c003b90 comp rx=0 tx=0).stop 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 -- 172.21.15.78:0/3322204968 shutdown_connections 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc01c107a20 0x7fc01c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc0080a2d90 0x7fc0080a52a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc01c10c380 0x7fc01c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 --2- 172.21.15.78:0/3322204968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c101af0 0x7fc01c198700 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 -- 172.21.15.78:0/3322204968 >> 172.21.15.78:0/3322204968 conn(0x7fc01c0fd210 msgr2=0x7fc01c1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 -- 172.21.15.78:0/3322204968 shutdown_connections 2024-05-21T15:16:44.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.298+0000 7fc0247ea640 1 -- 172.21.15.78:0/3322204968 wait complete. 2024-05-21T15:16:44.311 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:16:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 -- 172.21.15.78:0/3837748241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 msgr2=0x7fc5c410e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/3837748241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 0x7fc5c410e810 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009ca0 tx=0x7fc5bc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 -- 172.21.15.78:0/3837748241 shutdown_connections 2024-05-21T15:16:44.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/3837748241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c4102270 0x7fc5c4102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/3837748241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 0x7fc5c410e810 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/3837748241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c4107a00 0x7fc5c4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 -- 172.21.15.78:0/3837748241 >> 172.21.15.78:0/3837748241 conn(0x7fc5c4076830 msgr2=0x7fc5c4076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 -- 172.21.15.78:0/3837748241 shutdown_connections 2024-05-21T15:16:44.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.422+0000 7fc5c9db5640 1 -- 172.21.15.78:0/3837748241 wait complete. 2024-05-21T15:16:44.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c9db5640 1 Processor -- start 2024-05-21T15:16:44.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c9db5640 1 -- start start 2024-05-21T15:16:44.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c9db5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c4102270 0x7fc5c41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c9db5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c4107a00 0x7fc5c41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c9db5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 0x7fc5c4198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c9db5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c4111d20 con 0x7fc5c410c3f0 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c9db5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c4111ba0 con 0x7fc5c4102270 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c9db5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c4111ea0 con 0x7fc5c4107a00 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 0x7fc5c4198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 0x7fc5c4198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43746/0 (socket says 172.21.15.78:43746) 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.423+0000 7fc5c3fff640 1 -- 172.21.15.78:0/1787927207 learned_addr learned my addr 172.21.15.78:0/1787927207 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c4102270 0x7fc5c41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c4107a00 0x7fc5c41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c3fff640 1 -- 172.21.15.78:0/1787927207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c4107a00 msgr2=0x7fc5c41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c3fff640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c4107a00 0x7fc5c41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c3fff640 1 -- 172.21.15.78:0/1787927207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c4102270 msgr2=0x7fc5c41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c3fff640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c4102270 0x7fc5c41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c3fff640 1 -- 172.21.15.78:0/1787927207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c41a7a60 con 0x7fc5c410c3f0 2024-05-21T15:16:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c4102270 0x7fc5c41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c3fff640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 0x7fc5c4198e30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc031a20 tx=0x7fc5bc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c0ff9640 1 -- 172.21.15.78:0/1787927207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc003b70 con 0x7fc5c410c3f0 2024-05-21T15:16:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c0ff9640 1 -- 172.21.15.78:0/1787927207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5bc003d10 con 0x7fc5c410c3f0 2024-05-21T15:16:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c9db5640 1 -- 172.21.15.78:0/1787927207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c41a7cf0 con 0x7fc5c410c3f0 2024-05-21T15:16:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c0ff9640 1 -- 172.21.15.78:0/1787927207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc02fc40 con 0x7fc5c410c3f0 2024-05-21T15:16:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.424+0000 7fc5c9db5640 1 -- 172.21.15.78:0/1787927207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c41a81e0 con 0x7fc5c410c3f0 2024-05-21T15:16:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c0ff9640 1 -- 172.21.15.78:0/1787927207 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fc5bc004050 con 0x7fc5c410c3f0 2024-05-21T15:16:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5d0c67740 1 Processor -- start 2024-05-21T15:16:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5d0c67740 1 -- start start 2024-05-21T15:16:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a85418390 0x562a857b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853ef4d0 0x562a857b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562a8571cc40 con 0x562a853ef4d0 2024-05-21T15:16:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562a856c4580 con 0x562a85418390 2024-05-21T15:16:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a85718c00 con 0x562a853f76b0 2024-05-21T15:16:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853ef4d0 0x562a857b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853ef4d0 0x562a857b7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43752/0 (socket says 172.21.15.78:43752) 2024-05-21T15:16:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c37fe640 1 -- 172.21.15.78:0/3686853295 learned_addr learned my addr 172.21.15.78:0/3686853295 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a85418390 0x562a857b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c0ff9640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc5ac0a2d90 0x7fc5ac0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c37fe640 1 -- 172.21.15.78:0/3686853295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 msgr2=0x562a857b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c37fe640 1 -- 172.21.15.78:0/3686853295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a85418390 msgr2=0x562a857b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a85418390 0x562a857b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c37fe640 1 -- 172.21.15.78:0/3686853295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a854303f0 con 0x562a853ef4d0 2024-05-21T15:16:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.427+0000 7fc5c3fff640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc5ac0a2d90 0x7fc5ac0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.428+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853ef4d0 0x562a857b7260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009ce0 tx=0x7fc5b40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff4356c93f05611 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.428+0000 7fc5c0ff9640 1 -- 172.21.15.78:0/1787927207 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc5bc0f0710 con 0x7fc5c410c3f0 2024-05-21T15:16:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.428+0000 7fc5a9ffb640 1 -- 172.21.15.78:0/3686853295 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4012450 con 0x562a853ef4d0 2024-05-21T15:16:44.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.428+0000 7fc5a9ffb640 1 -- 172.21.15.78:0/3686853295 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b4012ad0 con 0x562a853ef4d0 2024-05-21T15:16:44.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.428+0000 7fc5a9ffb640 1 -- 172.21.15.78:0/3686853295 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400fc60 con 0x562a853ef4d0 2024-05-21T15:16:44.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3686853295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853ef4d0 msgr2=0x562a857b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5c3fff640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc5ac0a2d90 0x7fc5ac0a52a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0edf10 tx=0x7fc5bc08fdf0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853ef4d0 0x562a857b7260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009ce0 tx=0x7fc5b40117b0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3686853295 shutdown_connections 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853ef4d0 0x562a857b7260 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3686853295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a85418390 0x562a857b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3686853295 >> 172.21.15.78:0/3686853295 conn(0x562a8540c4e0 msgr2=0x562a857b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3686853295 shutdown_connections 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3686853295 wait complete. 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 -- start start 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.429+0000 7fc5d0c67740 1 Processor -- start 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 0x562a857c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853f76b0 0x562a857c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a85418390 0x562a857c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562a857c3fb0 con 0x562a853f76b0 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562a856d7c40 con 0x562a853ef4d0 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a857c4130 con 0x562a85418390 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5d0c67740 10 client.0 _ll_register_callbacks cb 0x562a8542fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853f76b0 0x562a857c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853f76b0 0x562a857c1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43766/0 (socket says 172.21.15.78:43766) 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 -- 172.21.15.78:0/1738285609 learned_addr learned my addr 172.21.15.78:0/1738285609 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 0x562a857c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a85418390 0x562a857c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 -- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a85418390 msgr2=0x562a857c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a85418390 0x562a857c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 -- 172.21.15.78:0/1738285609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 msgr2=0x562a857c13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 0x562a857c13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 -- 172.21.15.78:0/1738285609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a854fa7d0 con 0x562a853f76b0 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a85418390 0x562a857c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853f76b0 0x562a857c1910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400f880 tx=0x7fc5b4012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9179f271747c152 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/1738285609 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4029020 con 0x562a853f76b0 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/1738285609 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc5b4012660 con 0x562a853f76b0 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.430+0000 7fc5d0c67740 1 -- 172.21.15.78:0/1738285609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b4012ad0 con 0x562a853f76b0 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/1738285609 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5b4034050 con 0x562a853f76b0 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5d0c67740 10 client.15732 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5d0c67740 1 -- 172.21.15.78:0/1738285609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a857d07d0 con 0x562a853f76b0 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/1738285609 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b4012ad0 con 0x562a853f76b0 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/1738285609 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc5b40054b0 con 0x562a853f76b0 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5d0c67740 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5d0c67740 10 client.15732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5d0c67740 20 client.15732 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x562a853d6f60 0x562a857d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5d0c67740 4 client.15732 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.431+0000 7fc5d0c67740 1 -- 172.21.15.78:0/1738285609 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a8572a370 con 0x562a853d6f60 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.432+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x562a853d6f60 0x562a857d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.432+0000 7fc5a2ffd640 10 client.15732 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.432+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x562a853d6f60 0x562a857d3c40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0948fc1b084e876 server_cookie=ba5bacd447f37f46 in_seq=0 out_seq=0 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/1738285609 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a8572a370 con 0x562a853d6f60 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5a2ffd640 10 client.15732 handle_command_reply: tid=0 2024-05-21T15:16:44.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 client.15732 shutdown 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 -- 172.21.15.78:0/1738285609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853f76b0 msgr2=0x562a857c1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853f76b0 0x562a857c1910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400f880 tx=0x7fc5b4012d00 comp rx=0 tx=0).stop 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 -- 172.21.15.78:0/1738285609 shutdown_connections 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x562a853d6f60 0x562a857d3c40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a85418390 0x562a857c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5a2ffd640 0 client.15732 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a853f76b0 0x562a857c1910 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/1738285609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 0x562a857c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 -- 172.21.15.78:0/1738285609 >> 172.21.15.78:0/1738285609 conn(0x562a8540c4e0 msgr2=0x562a857b8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 -- 172.21.15.78:0/1738285609 shutdown_connections 2024-05-21T15:16:44.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 1 -- 172.21.15.78:0/1738285609 wait complete. 2024-05-21T15:16:44.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.433+0000 7fc5d0c67740 20 client.15732 trim_cache size 0 max 16384 2024-05-21T15:16:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5d0c67740 1 Processor -- start 2024-05-21T15:16:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5d0c67740 1 -- start start 2024-05-21T15:16:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853ef4d0 0x562a857c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853f76b0 0x562a857c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41b0ca0 con 0x562a85418390 2024-05-21T15:16:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41b27e0 con 0x562a853f76b0 2024-05-21T15:16:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41b1370 con 0x562a853ef4d0 2024-05-21T15:16:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857ba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43768/0 (socket says 172.21.15.78:43768) 2024-05-21T15:16:44.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 -- 172.21.15.78:0/3935404150 learned_addr learned my addr 172.21.15.78:0/3935404150 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853ef4d0 0x562a857c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853f76b0 0x562a857c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 -- 172.21.15.78:0/3935404150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853ef4d0 msgr2=0x562a857c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853ef4d0 0x562a857c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 -- 172.21.15.78:0/3935404150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853f76b0 msgr2=0x562a857c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853f76b0 0x562a857c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 -- 172.21.15.78:0/3935404150 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a857ae260 con 0x562a85418390 2024-05-21T15:16:44.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.447+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857ba9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009f50 tx=0x7fc5b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5512ba2f2cdcd4b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5a97fa640 1 -- 172.21.15.78:0/3935404150 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8011bd0 con 0x562a85418390 2024-05-21T15:16:44.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5a97fa640 1 -- 172.21.15.78:0/3935404150 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b8011d70 con 0x562a85418390 2024-05-21T15:16:44.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5a97fa640 1 -- 172.21.15.78:0/3935404150 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8004d60 con 0x562a85418390 2024-05-21T15:16:44.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3935404150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 msgr2=0x562a857ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857ba9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009f50 tx=0x7fc5b80046c0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3935404150 shutdown_connections 2024-05-21T15:16:44.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853f76b0 0x562a857c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857ba9d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3935404150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853ef4d0 0x562a857c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3935404150 >> 172.21.15.78:0/3935404150 conn(0x562a8540c4e0 msgr2=0x562a857b7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3935404150 shutdown_connections 2024-05-21T15:16:44.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.448+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3935404150 wait complete. 2024-05-21T15:16:44.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 1 -- start start 2024-05-21T15:16:44.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 1 Processor -- start 2024-05-21T15:16:44.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 0x562a857cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41b2110 con 0x562a85418390 2024-05-21T15:16:44.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41b1a40 con 0x562a853ef4d0 2024-05-21T15:16:44.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c41aec80 con 0x562a853f76b0 2024-05-21T15:16:44.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5d0c67740 10 client.0 _ll_register_callbacks cb 0x7fc5b4009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43776/0 (socket says 172.21.15.78:43776) 2024-05-21T15:16:44.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.449+0000 7fc5c2ffd640 1 -- 172.21.15.78:0/3663606213 learned_addr learned my addr 172.21.15.78:0/3663606213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 0x562a857cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5c2ffd640 1 -- 172.21.15.78:0/3663606213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 msgr2=0x562a857cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5c2ffd640 1 -- 172.21.15.78:0/3663606213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 msgr2=0x562a857cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 0x562a857cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5c2ffd640 1 -- 172.21.15.78:0/3663606213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a857cbf30 con 0x562a85418390 2024-05-21T15:16:44.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5c2ffd640 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857cb7f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800a010 tx=0x7fc5b8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f833c7b4be2d2a0d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/3663606213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8032030 con 0x562a85418390 2024-05-21T15:16:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/3663606213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc5b8027040 con 0x562a85418390 2024-05-21T15:16:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3663606213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b8011d70 con 0x562a85418390 2024-05-21T15:16:44.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/3663606213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5b80047a0 con 0x562a85418390 2024-05-21T15:16:44.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5d0c67740 10 client.15738 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3663606213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a8583bc60 con 0x562a85418390 2024-05-21T15:16:44.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.450+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/3663606213 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b8011d70 con 0x562a85418390 2024-05-21T15:16:44.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/3663606213 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc5b8003ca0 con 0x562a85418390 2024-05-21T15:16:44.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5d0c67740 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5d0c67740 10 client.15738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:16:44.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5d0c67740 20 client.15738 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x562a853ac3a0 0x562a8583e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5d0c67740 4 client.15738 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:16:44.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3663606213 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a857246c0 con 0x562a853ac3a0 2024-05-21T15:16:44.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x562a853ac3a0 0x562a8583e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5c37fe640 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x562a853ac3a0 0x562a8583e2f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf880bdc1b6866f5 server_cookie=5b6aa5dd7fb5aee5 in_seq=0 out_seq=0 2024-05-21T15:16:44.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.451+0000 7fc5a2ffd640 10 client.15738 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:16:44.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5a2ffd640 1 -- 172.21.15.78:0/3663606213 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a857246c0 con 0x562a853ac3a0 2024-05-21T15:16:44.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5a2ffd640 10 client.15738 handle_command_reply: tid=0 2024-05-21T15:16:44.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 client.15738 shutdown 2024-05-21T15:16:44.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3663606213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 msgr2=0x562a857cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857cb7f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800a010 tx=0x7fc5b8005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:44.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3663606213 shutdown_connections 2024-05-21T15:16:44.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x562a853ac3a0 0x562a8583e2f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a85418390 0x562a857cb7f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a853f76b0 0x562a857cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5a2ffd640 0 client.15738 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:16:44.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 --2- 172.21.15.78:0/3663606213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a853ef4d0 0x562a857cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3663606213 >> 172.21.15.78:0/3663606213 conn(0x562a8540c4e0 msgr2=0x562a857b7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3663606213 shutdown_connections 2024-05-21T15:16:44.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 1 -- 172.21.15.78:0/3663606213 wait complete. 2024-05-21T15:16:44.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5d0c67740 20 client.15738 trim_cache size 0 max 16384 2024-05-21T15:16:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:16:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5c9db5640 1 -- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc5ac0a2d90 msgr2=0x7fc5ac0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc5ac0a2d90 0x7fc5ac0a52a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0edf10 tx=0x7fc5bc08fdf0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5c9db5640 1 -- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 msgr2=0x7fc5c4198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 0x7fc5c4198e30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc031a20 tx=0x7fc5bc031a50 comp rx=0 tx=0).stop 2024-05-21T15:16:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5c9db5640 1 -- 172.21.15.78:0/1787927207 shutdown_connections 2024-05-21T15:16:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc5ac0a2d90 0x7fc5ac0a52a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5c410c3f0 0x7fc5c4198e30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.452+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5c4107a00 0x7fc5c41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.453+0000 7fc5c9db5640 1 --2- 172.21.15.78:0/1787927207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5c4102270 0x7fc5c41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.453+0000 7fc5c9db5640 1 -- 172.21.15.78:0/1787927207 >> 172.21.15.78:0/1787927207 conn(0x7fc5c4076830 msgr2=0x7fc5c4103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.453+0000 7fc5c9db5640 1 -- 172.21.15.78:0/1787927207 shutdown_connections 2024-05-21T15:16:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.453+0000 7fc5c9db5640 1 -- 172.21.15.78:0/1787927207 wait complete. 2024-05-21T15:16:44.478 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:16:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/2702003342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e641013f0 msgr2=0x7f5e64101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/2702003342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e641013f0 0x7f5e64101800 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009900 tx=0x7f5e5402a410 comp rx=0 tx=0).stop 2024-05-21T15:16:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/2702003342 shutdown_connections 2024-05-21T15:16:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/2702003342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e641063f0 0x7f5e6410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/2702003342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e64105930 0x7f5e64105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/2702003342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e641013f0 0x7f5e64101800 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/2702003342 >> 172.21.15.78:0/2702003342 conn(0x7f5e64076830 msgr2=0x7f5e64076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/2702003342 shutdown_connections 2024-05-21T15:16:44.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/2702003342 wait complete. 2024-05-21T15:16:44.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 Processor -- start 2024-05-21T15:16:44.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.588+0000 7f5e6a3a8640 1 -- start start 2024-05-21T15:16:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e6a3a8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e641013f0 0x7f5e64198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e6a3a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e64105930 0x7f5e64198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e6a3a8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e641063f0 0x7f5e6419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e6a3a8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64111ce0 con 0x7f5e64105930 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e6a3a8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64111b60 con 0x7f5e641013f0 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e6a3a8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64111e60 con 0x7f5e641063f0 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e64105930 0x7f5e64198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e64105930 0x7f5e64198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43794/0 (socket says 172.21.15.78:43794) 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 -- 172.21.15.78:0/580539183 learned_addr learned my addr 172.21.15.78:0/580539183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e68945640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e641063f0 0x7f5e6419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e63fff640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e641013f0 0x7f5e64198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 -- 172.21.15.78:0/580539183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e641063f0 msgr2=0x7f5e6419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e641063f0 0x7f5e6419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 -- 172.21.15.78:0/580539183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e641013f0 msgr2=0x7f5e64198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e641013f0 0x7f5e64198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 -- 172.21.15.78:0/580539183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6419d580 con 0x7f5e64105930 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e68945640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e641063f0 0x7f5e6419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e63fff640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e641013f0 0x7f5e64198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.589+0000 7f5e637fe640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e64105930 0x7f5e64198c80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800d740 tx=0x7f5e5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.590+0000 7f5e617fa640 1 -- 172.21.15.78:0/580539183 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58017070 con 0x7f5e64105930 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.590+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/580539183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6419d870 con 0x7f5e64105930 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.590+0000 7f5e617fa640 1 -- 172.21.15.78:0/580539183 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5800de50 con 0x7f5e64105930 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.590+0000 7f5e617fa640 1 -- 172.21.15.78:0/580539183 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58004e00 con 0x7f5e64105930 2024-05-21T15:16:44.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.590+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/580539183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e641a51f0 con 0x7f5e64105930 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e617fa640 1 -- 172.21.15.78:0/580539183 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f5e58010460 con 0x7f5e64105930 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e7124d740 1 Processor -- start 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e7124d740 1 -- start start 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86b9390 0x5595f8a59f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a5a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e7124d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595f89bdc40 con 0x5595f86986b0 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e7124d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5595f8965580 con 0x5595f86b9390 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.592+0000 7f5e7124d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5595f89b9c00 con 0x5595f86904d0 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a5a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e617fa640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5e4c0a2d90 0x7f5e4c0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a5a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43810/0 (socket says 172.21.15.78:43810) 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 -- 172.21.15.78:0/3973423465 learned_addr learned my addr 172.21.15.78:0/3973423465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e68945640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5e4c0a2d90 0x7f5e4c0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e68945640 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86b9390 0x5595f8a59f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e63fff640 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e617fa640 1 -- 172.21.15.78:0/580539183 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5e580cf070 con 0x7f5e64105930 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 -- 172.21.15.78:0/3973423465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 msgr2=0x5595f8a5a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a5a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 -- 172.21.15.78:0/3973423465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86b9390 msgr2=0x5595f8a59f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86b9390 0x5595f8a59f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 -- 172.21.15.78:0/3973423465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f86d13f0 con 0x5595f86986b0 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e68945640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5e4c0a2d90 0x7f5e4c0a52a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009fd0 tx=0x7f5e5c009d80 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e637fe640 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a5a9b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5e580cc670 tx=0x7f5e580048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e5358c2aaad3c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.593+0000 7f5e4a7fc640 1 -- 172.21.15.78:0/3973423465 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58020070 con 0x5595f86986b0 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e4a7fc640 1 -- 172.21.15.78:0/3973423465 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e580a4090 con 0x5595f86986b0 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e4a7fc640 1 -- 172.21.15.78:0/3973423465 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580d10d0 con 0x5595f86986b0 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 -- 172.21.15.78:0/3973423465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 msgr2=0x5595f8a5a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a5a9b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5e580cc670 tx=0x7f5e580048a0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 -- 172.21.15.78:0/3973423465 shutdown_connections 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a5a9b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a5a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 --2- 172.21.15.78:0/3973423465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86b9390 0x5595f8a59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 -- 172.21.15.78:0/3973423465 >> 172.21.15.78:0/3973423465 conn(0x5595f86ad4e0 msgr2=0x5595f8a566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 -- 172.21.15.78:0/3973423465 shutdown_connections 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.594+0000 7f5e7124d740 1 -- 172.21.15.78:0/3973423465 wait complete. 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 1 -- start start 2024-05-21T15:16:44.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 1 Processor -- start 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86904d0 0x5595f8a64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86986b0 0x5595f8a64ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86b9390 0x5595f8a6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595f8a67220 con 0x5595f86904d0 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5595f8978c40 con 0x5595f86986b0 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5595f8a673a0 con 0x5595f86b9390 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e7124d740 10 client.0 _ll_register_callbacks cb 0x5595f86d0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e63fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86904d0 0x5595f8a64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e63fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86904d0 0x5595f8a64590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43814/0 (socket says 172.21.15.78:43814) 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.595+0000 7f5e63fff640 1 -- 172.21.15.78:0/2131298630 learned_addr learned my addr 172.21.15.78:0/2131298630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e68945640 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86b9390 0x5595f8a6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e63fff640 1 -- 172.21.15.78:0/2131298630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86b9390 msgr2=0x5595f8a6a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e63fff640 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86b9390 0x5595f8a6a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e637fe640 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86986b0 0x5595f8a64ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e63fff640 1 -- 172.21.15.78:0/2131298630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86986b0 msgr2=0x5595f8a64ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e63fff640 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86986b0 0x5595f8a64ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e63fff640 1 -- 172.21.15.78:0/2131298630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f879b7d0 con 0x5595f86904d0 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e63fff640 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86904d0 0x5595f8a64590 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009900 tx=0x7f5e54031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db6f8773809592d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e437fe640 1 -- 172.21.15.78:0/2131298630 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54035040 con 0x5595f86904d0 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e437fe640 1 -- 172.21.15.78:0/2131298630 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5e54004240 con 0x5595f86904d0 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e7124d740 1 -- 172.21.15.78:0/2131298630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e580a4090 con 0x5595f86904d0 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e437fe640 1 -- 172.21.15.78:0/2131298630 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e54004440 con 0x5595f86904d0 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e7124d740 10 client.15750 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e7124d740 1 -- 172.21.15.78:0/2131298630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595f8a739c0 con 0x5595f86904d0 2024-05-21T15:16:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.596+0000 7f5e437fe640 1 -- 172.21.15.78:0/2131298630 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e580a4090 con 0x5595f86904d0 2024-05-21T15:16:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e437fe640 1 -- 172.21.15.78:0/2131298630 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5e54004bf0 con 0x5595f86904d0 2024-05-21T15:16:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e7124d740 10 client.15750 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e7124d740 10 client.15750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:16:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e7124d740 20 client.15750 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e7124d740 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5595f8677f60 0x5595f8a76eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e7124d740 4 client.15750 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e7124d740 1 -- 172.21.15.78:0/2131298630 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595f89cb370 con 0x5595f8677f60 2024-05-21T15:16:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e637fe640 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5595f8677f60 0x5595f8a76eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e437fe640 10 client.15750 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:16:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.597+0000 7f5e637fe640 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5595f8677f60 0x5595f8a76eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=730e3e3e0bf9762f server_cookie=fc1dc03219f73ad8 in_seq=0 out_seq=0 2024-05-21T15:16:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.598+0000 7f5e437fe640 1 -- 172.21.15.78:0/2131298630 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5595f89cb370 con 0x5595f8677f60 2024-05-21T15:16:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.598+0000 7f5e437fe640 10 client.15750 handle_command_reply: tid=0 2024-05-21T15:16:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.598+0000 7f5e7124d740 1 client.15750 shutdown 2024-05-21T15:16:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.598+0000 7f5e7124d740 1 -- 172.21.15.78:0/2131298630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86904d0 msgr2=0x5595f8a64590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.598+0000 7f5e7124d740 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86904d0 0x5595f8a64590 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009900 tx=0x7f5e54031920 comp rx=0 tx=0).stop 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.598+0000 7f5e7124d740 1 -- 172.21.15.78:0/2131298630 shutdown_connections 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e7124d740 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5595f8677f60 0x5595f8a76eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e7124d740 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f86986b0 0x5595f8a64ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e437fe640 0 client.15750 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e7124d740 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86b9390 0x5595f8a6a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e7124d740 1 --2- 172.21.15.78:0/2131298630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86904d0 0x5595f8a64590 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e7124d740 1 -- 172.21.15.78:0/2131298630 >> 172.21.15.78:0/2131298630 conn(0x5595f86ad4e0 msgr2=0x5595f8a5bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e7124d740 1 -- 172.21.15.78:0/2131298630 shutdown_connections 2024-05-21T15:16:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e7124d740 1 -- 172.21.15.78:0/2131298630 wait complete. 2024-05-21T15:16:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.599+0000 7f5e7124d740 20 client.15750 trim_cache size 0 max 16384 2024-05-21T15:16:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e7124d740 1 Processor -- start 2024-05-21T15:16:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e7124d740 1 -- start start 2024-05-21T15:16:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a5ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a53570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e7124d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641ac8a0 con 0x5595f86986b0 2024-05-21T15:16:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e7124d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641ae3e0 con 0x5595f8677f60 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e7124d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641acf70 con 0x5595f86904d0 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e68945640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e68945640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43830/0 (socket says 172.21.15.78:43830) 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.612+0000 7f5e68945640 1 -- 172.21.15.78:0/3657861228 learned_addr learned my addr 172.21.15.78:0/3657861228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e637fe640 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a53570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e63fff640 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a5ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e68945640 1 -- 172.21.15.78:0/3657861228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 msgr2=0x5595f8a53570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e68945640 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a53570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e68945640 1 -- 172.21.15.78:0/3657861228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 msgr2=0x5595f8a5ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e68945640 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a5ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e68945640 1 -- 172.21.15.78:0/3657861228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f8a65240 con 0x5595f86986b0 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e63fff640 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a5ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:44.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e68945640 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6a610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005fd0 tx=0x7f5e5c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272027d0c3abdf33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e49ffb640 1 -- 172.21.15.78:0/3657861228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c004b20 con 0x5595f86986b0 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e49ffb640 1 -- 172.21.15.78:0/3657861228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5c01ede0 con 0x5595f86986b0 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.613+0000 7f5e49ffb640 1 -- 172.21.15.78:0/3657861228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c005560 con 0x5595f86986b0 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 -- 172.21.15.78:0/3657861228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 msgr2=0x5595f8a6a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6a610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005fd0 tx=0x7f5e5c01e000 comp rx=0 tx=0).stop 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 -- 172.21.15.78:0/3657861228 shutdown_connections 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6a610 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a53570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 --2- 172.21.15.78:0/3657861228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a5ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 -- 172.21.15.78:0/3657861228 >> 172.21.15.78:0/3657861228 conn(0x5595f86ad4e0 msgr2=0x5595f8a632c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 -- 172.21.15.78:0/3657861228 shutdown_connections 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 -- 172.21.15.78:0/3657861228 wait complete. 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 -- start start 2024-05-21T15:16:44.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.614+0000 7f5e7124d740 1 Processor -- start 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a70840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a70da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e7124d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e7124d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641add10 con 0x5595f86986b0 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e7124d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641ad640 con 0x5595f8677f60 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e7124d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641aa880 con 0x5595f86904d0 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e7124d740 10 client.0 _ll_register_callbacks cb 0x7f5e540315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43836/0 (socket says 172.21.15.78:43836) 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 -- 172.21.15.78:0/941505465 learned_addr learned my addr 172.21.15.78:0/941505465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e63fff640 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a70840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e637fe640 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a70da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 -- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 msgr2=0x5595f8a70da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a70da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 -- 172.21.15.78:0/941505465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 msgr2=0x5595f8a70840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a70840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 -- 172.21.15.78:0/941505465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f8a6c3c0 con 0x5595f86986b0 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e637fe640 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e68945640 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6bd00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005380 tx=0x7f5e5c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44fcb2a340b25c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e437fe640 1 -- 172.21.15.78:0/941505465 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c004cc0 con 0x5595f86986b0 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e437fe640 1 -- 172.21.15.78:0/941505465 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5e5c0054e0 con 0x5595f86986b0 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.615+0000 7f5e7124d740 1 -- 172.21.15.78:0/941505465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e5c01ede0 con 0x5595f86986b0 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e437fe640 1 -- 172.21.15.78:0/941505465 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e5c0056e0 con 0x5595f86986b0 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e7124d740 10 client.15756 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e7124d740 1 -- 172.21.15.78:0/941505465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595f8adead0 con 0x5595f86986b0 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e437fe640 1 -- 172.21.15.78:0/941505465 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5c01ede0 con 0x5595f86986b0 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e437fe640 1 -- 172.21.15.78:0/941505465 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5e5c00c040 con 0x5595f86986b0 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e7124d740 10 client.15756 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e7124d740 10 client.15756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e7124d740 20 client.15756 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e7124d740 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5595f86b9390 0x5595f8ae11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e7124d740 4 client.15756 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e7124d740 1 -- 172.21.15.78:0/941505465 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595f89c56c0 con 0x5595f86b9390 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.616+0000 7f5e63fff640 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5595f86b9390 0x5595f8ae11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e437fe640 10 client.15756 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e63fff640 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5595f86b9390 0x5595f8ae11e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfd9eb72787c7894 server_cookie=98e5a69ab6a5e22f in_seq=0 out_seq=0 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e437fe640 1 -- 172.21.15.78:0/941505465 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595f89c56c0 con 0x5595f86b9390 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e437fe640 10 client.15756 handle_command_reply: tid=0 2024-05-21T15:16:44.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 client.15756 shutdown 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 -- 172.21.15.78:0/941505465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 msgr2=0x5595f8a6bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6bd00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005380 tx=0x7f5e5c01eba0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 -- 172.21.15.78:0/941505465 shutdown_connections 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5595f86b9390 0x5595f8ae11e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595f86986b0 0x5595f8a6bd00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595f86904d0 0x5595f8a70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e437fe640 0 client.15756 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 --2- 172.21.15.78:0/941505465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5595f8677f60 0x5595f8a70840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 -- 172.21.15.78:0/941505465 >> 172.21.15.78:0/941505465 conn(0x5595f86ad4e0 msgr2=0x5595f8a59d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 -- 172.21.15.78:0/941505465 shutdown_connections 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 1 -- 172.21.15.78:0/941505465 wait complete. 2024-05-21T15:16:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.617+0000 7f5e7124d740 20 client.15756 trim_cache size 0 max 16384 2024-05-21T15:16:44.629 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:16:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5e4c0a2d90 msgr2=0x7f5e4c0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5e4c0a2d90 0x7f5e4c0a52a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009fd0 tx=0x7f5e5c009d80 comp rx=0 tx=0).stop 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e64105930 msgr2=0x7f5e64198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e64105930 0x7f5e64198c80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800d740 tx=0x7f5e5800dc00 comp rx=0 tx=0).stop 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/580539183 shutdown_connections 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5e4c0a2d90 0x7f5e4c0a52a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e641063f0 0x7f5e6419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e64105930 0x7f5e64198c80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 --2- 172.21.15.78:0/580539183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e641013f0 0x7f5e64198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/580539183 >> 172.21.15.78:0/580539183 conn(0x7f5e64076830 msgr2=0x7f5e640ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/580539183 shutdown_connections 2024-05-21T15:16:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.618+0000 7f5e6a3a8640 1 -- 172.21.15.78:0/580539183 wait complete. 2024-05-21T15:16:44.632 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:16:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:44 smithi172 ceph-mon[36402]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:16:44.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:44 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/413370596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:44.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 -- 172.21.15.78:0/3530909324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc101aa0 msgr2=0x7f6ebc101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/3530909324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc101aa0 0x7f6ebc101eb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009900 tx=0x7f6eac02a410 comp rx=0 tx=0).stop 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 -- 172.21.15.78:0/3530909324 shutdown_connections 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/3530909324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ebc10c330 0x7f6ebc10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/3530909324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc101aa0 0x7f6ebc101eb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/3530909324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ebc1079d0 0x7f6ebc101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 -- 172.21.15.78:0/3530909324 >> 172.21.15.78:0/3530909324 conn(0x7f6ebc0770b0 msgr2=0x7f6ebc0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 -- 172.21.15.78:0/3530909324 shutdown_connections 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.743+0000 7f6ec37b1640 1 -- 172.21.15.78:0/3530909324 wait complete. 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec37b1640 1 Processor -- start 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec37b1640 1 -- start start 2024-05-21T15:16:44.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec37b1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ebc101aa0 0x7f6ebc1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec37b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ebc1079d0 0x7f6ebc199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec37b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc10c330 0x7f6ebc19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec37b1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc112e00 con 0x7f6ebc10c330 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec37b1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc112c80 con 0x7f6ebc1079d0 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec37b1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc112f80 con 0x7f6ebc101aa0 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec1d4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc10c330 0x7f6ebc19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec1d4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc10c330 0x7f6ebc19dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43844/0 (socket says 172.21.15.78:43844) 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.744+0000 7f6ec1d4e640 1 -- 172.21.15.78:0/2294343905 learned_addr learned my addr 172.21.15.78:0/2294343905 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec154d640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ebc101aa0 0x7f6ebc1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ebc1079d0 0x7f6ebc199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec1d4e640 1 -- 172.21.15.78:0/2294343905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ebc101aa0 msgr2=0x7f6ebc1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec1d4e640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ebc101aa0 0x7f6ebc1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec1d4e640 1 -- 172.21.15.78:0/2294343905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ebc1079d0 msgr2=0x7f6ebc199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec1d4e640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ebc1079d0 0x7f6ebc199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec1d4e640 1 -- 172.21.15.78:0/2294343905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ebc19e620 con 0x7f6ebc10c330 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ebc1079d0 0x7f6ebc199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec1d4e640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc10c330 0x7f6ebc19dfe0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400eb10 tx=0x7f6eb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6eb27fc640 1 -- 172.21.15.78:0/2294343905 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4004b20 con 0x7f6ebc10c330 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec37b1640 1 -- 172.21.15.78:0/2294343905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ebc19e910 con 0x7f6ebc10c330 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6eb27fc640 1 -- 172.21.15.78:0/2294343905 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb4004cc0 con 0x7f6ebc10c330 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6eb27fc640 1 -- 172.21.15.78:0/2294343905 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4010760 con 0x7f6ebc10c330 2024-05-21T15:16:44.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.745+0000 7f6ec37b1640 1 -- 172.21.15.78:0/2294343905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ebc1a6290 con 0x7f6ebc10c330 2024-05-21T15:16:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.748+0000 7f6eca658740 1 Processor -- start 2024-05-21T15:16:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eb27fc640 1 -- 172.21.15.78:0/2294343905 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6eb4010900 con 0x7f6ebc10c330 2024-05-21T15:16:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eca658740 1 -- start start 2024-05-21T15:16:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eca658740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a67037390 0x558a673d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eca658740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 0x558a673d6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eca658740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a670166b0 0x558a673d67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eca658740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558a6733bc40 con 0x558a67037390 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eca658740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558a672e3580 con 0x558a670166b0 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eca658740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558a67337c00 con 0x558a6700e4d0 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6ec0d4c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a67037390 0x558a673d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6ec0d4c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a67037390 0x558a673d5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43850/0 (socket says 172.21.15.78:43850) 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6ec0d4c640 1 -- 172.21.15.78:0/3632659094 learned_addr learned my addr 172.21.15.78:0/3632659094 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6ec154d640 1 --2- 172.21.15.78:0/3632659094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 0x558a673d6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.749+0000 7f6eb27fc640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6ea80a2d90 0x7f6ea80a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ec1d4e640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6ea80a2d90 0x7f6ea80a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ec0d4c640 1 -- 172.21.15.78:0/3632659094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 msgr2=0x558a673d6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/3632659094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 0x558a673d6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ec0d4c640 1 -- 172.21.15.78:0/3632659094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a670166b0 msgr2=0x558a673d67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/3632659094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a670166b0 0x558a673d67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ec0d4c640 1 -- 172.21.15.78:0/3632659094 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6704f3f0 con 0x558a67037390 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6eb27fc640 1 -- 172.21.15.78:0/2294343905 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6eb40cfd80 con 0x7f6ebc10c330 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/3632659094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a67037390 0x558a673d5d20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a8e9252024a9ce server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ec1d4e640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6ea80a2d90 0x7f6ea80a52a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40cd410 tx=0x7f6eb406e090 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ea77fe640 1 -- 172.21.15.78:0/3632659094 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac035050 con 0x558a67037390 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ea77fe640 1 -- 172.21.15.78:0/3632659094 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eac004030 con 0x558a67037390 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.750+0000 7f6ea77fe640 1 -- 172.21.15.78:0/3632659094 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0054c0 con 0x558a67037390 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 -- 172.21.15.78:0/3632659094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a67037390 msgr2=0x558a673d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 --2- 172.21.15.78:0/3632659094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a67037390 0x558a673d5d20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac005e60 comp rx=0 tx=0).stop 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 -- 172.21.15.78:0/3632659094 shutdown_connections 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 --2- 172.21.15.78:0/3632659094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a670166b0 0x558a673d67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 --2- 172.21.15.78:0/3632659094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 0x558a673d6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 --2- 172.21.15.78:0/3632659094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a67037390 0x558a673d5d20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 -- 172.21.15.78:0/3632659094 >> 172.21.15.78:0/3632659094 conn(0x558a6702b4e0 msgr2=0x558a673d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 -- 172.21.15.78:0/3632659094 shutdown_connections 2024-05-21T15:16:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 -- 172.21.15.78:0/3632659094 wait complete. 2024-05-21T15:16:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.751+0000 7f6eca658740 1 -- start start 2024-05-21T15:16:44.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6eca658740 1 Processor -- start 2024-05-21T15:16:44.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6eca658740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a6700e4d0 0x558a673e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6eca658740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6eca658740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a67037390 0x558a673e0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6eca658740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558a673e2fb0 con 0x558a670166b0 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6eca658740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558a672f6c40 con 0x558a6700e4d0 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6eca658740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558a673e3130 con 0x558a67037390 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6eca658740 10 client.0 _ll_register_callbacks cb 0x558a6704ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43864/0 (socket says 172.21.15.78:43864) 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec154d640 1 -- 172.21.15.78:0/236510832 learned_addr learned my addr 172.21.15.78:0/236510832 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a6700e4d0 0x558a673e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a67037390 0x558a673e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec154d640 1 -- 172.21.15.78:0/236510832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a67037390 msgr2=0x558a673e0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec154d640 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a67037390 0x558a673e0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec154d640 1 -- 172.21.15.78:0/236510832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a6700e4d0 msgr2=0x558a673e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec154d640 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a6700e4d0 0x558a673e1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec154d640 1 -- 172.21.15.78:0/236510832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a671197d0 con 0x558a670166b0 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.752+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a6700e4d0 0x558a673e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6ec154d640 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e22b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8012700 tx=0x7f6eb8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23677e1c319bf43 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6ea4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/236510832 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8004320 con 0x558a670166b0 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/236510832 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6eb80119a0 con 0x558a670166b0 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 1 -- 172.21.15.78:0/236510832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eac004030 con 0x558a670166b0 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/236510832 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6eb80044c0 con 0x558a670166b0 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 10 client.15768 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 1 -- 172.21.15.78:0/236510832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a673ef7a0 con 0x558a670166b0 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/236510832 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eac004030 con 0x558a670166b0 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/236510832 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6eb8025480 con 0x558a670166b0 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 10 client.15768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 20 client.15768 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x558a66ff5f60 0x558a673f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 4 client.15768 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.753+0000 7f6eca658740 1 -- 172.21.15.78:0/236510832 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a67349370 con 0x558a66ff5f60 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.754+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x558a66ff5f60 0x558a673f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.754+0000 7f6ea4ff9640 10 client.15768 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.754+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x558a66ff5f60 0x558a673f2c00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6f012151aa82e4 server_cookie=1670285e61145f7c in_seq=0 out_seq=0 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/236510832 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a67349370 con 0x558a66ff5f60 2024-05-21T15:16:44.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6ea4ff9640 10 client.15768 handle_command_reply: tid=0 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 client.15768 shutdown 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 -- 172.21.15.78:0/236510832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 msgr2=0x558a673e22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e22b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8012700 tx=0x7f6eb8012bc0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 -- 172.21.15.78:0/236510832 shutdown_connections 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x558a66ff5f60 0x558a673f2c00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a67037390 0x558a673e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e22b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 --2- 172.21.15.78:0/236510832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a6700e4d0 0x558a673e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6ea4ff9640 0 client.15768 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 -- 172.21.15.78:0/236510832 >> 172.21.15.78:0/236510832 conn(0x558a6702b4e0 msgr2=0x558a673d7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 -- 172.21.15.78:0/236510832 shutdown_connections 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 1 -- 172.21.15.78:0/236510832 wait complete. 2024-05-21T15:16:44.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.755+0000 7f6eca658740 20 client.15768 trim_cache size 0 max 16384 2024-05-21T15:16:44.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6eca658740 1 Processor -- start 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6eca658740 1 -- start start 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6eca658740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a6700e4d0 0x558a673d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6eca658740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a670166b0 0x558a673dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6eca658740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 0x558a673d99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6eca658740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1ae8a0 con 0x558a6700e4d0 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6eca658740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1b03e0 con 0x558a67037390 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6eca658740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1aef70 con 0x558a670166b0 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a6700e4d0 0x558a673d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a6700e4d0 0x558a673d81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43866/0 (socket says 172.21.15.78:43866) 2024-05-21T15:16:44.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 -- 172.21.15.78:0/3994152412 learned_addr learned my addr 172.21.15.78:0/3994152412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec154d640 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a670166b0 0x558a673dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 -- 172.21.15.78:0/3994152412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a670166b0 msgr2=0x558a673dfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a670166b0 0x558a673dfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 -- 172.21.15.78:0/3994152412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 msgr2=0x558a673d99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 0x558a673d99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 -- 172.21.15.78:0/3994152412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a673d56b0 con 0x558a6700e4d0 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec154d640 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a670166b0 0x558a673dfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.769+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a6700e4d0 0x558a673d81f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009900 tx=0x7f6eac031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c14073dbd860c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6ea6ffd640 1 -- 172.21.15.78:0/3994152412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac057070 con 0x558a6700e4d0 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6ea6ffd640 1 -- 172.21.15.78:0/3994152412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eac005630 con 0x558a6700e4d0 2024-05-21T15:16:44.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6ea6ffd640 1 -- 172.21.15.78:0/3994152412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac04f040 con 0x558a6700e4d0 2024-05-21T15:16:44.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 -- 172.21.15.78:0/3994152412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a6700e4d0 msgr2=0x558a673d81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a6700e4d0 0x558a673d81f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009900 tx=0x7f6eac031440 comp rx=0 tx=0).stop 2024-05-21T15:16:44.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 -- 172.21.15.78:0/3994152412 shutdown_connections 2024-05-21T15:16:44.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 0x558a673d99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a670166b0 0x558a673dfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 --2- 172.21.15.78:0/3994152412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a6700e4d0 0x558a673d81f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 -- 172.21.15.78:0/3994152412 >> 172.21.15.78:0/3994152412 conn(0x558a6702b4e0 msgr2=0x558a673df7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 -- 172.21.15.78:0/3994152412 shutdown_connections 2024-05-21T15:16:44.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.770+0000 7f6eca658740 1 -- 172.21.15.78:0/3994152412 wait complete. 2024-05-21T15:16:44.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 1 -- start start 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 1 Processor -- start 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 0x558a673e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 0x558a673e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1afd10 con 0x558a670166b0 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1af640 con 0x558a67037390 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1ac880 con 0x558a6700e4d0 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6eca658740 10 client.0 _ll_register_callbacks cb 0x7f6eb80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6ec154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6ec154d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43872/0 (socket says 172.21.15.78:43872) 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.771+0000 7f6ec154d640 1 -- 172.21.15.78:0/306892186 learned_addr learned my addr 172.21.15.78:0/306892186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 0x558a673e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ec154d640 1 -- 172.21.15.78:0/306892186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 msgr2=0x558a673e87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 0x558a673e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ec154d640 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 0x558a673e87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ec154d640 1 -- 172.21.15.78:0/306892186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 msgr2=0x558a673e60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ec154d640 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 0x558a673e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ec154d640 1 -- 172.21.15.78:0/306892186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a673e66e0 con 0x558a670166b0 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ec154d640 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e5b60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000c00 tx=0x7f6eb8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396624bb5c534ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ea4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/306892186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8034060 con 0x558a670166b0 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/306892186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6eb8025c40 con 0x558a670166b0 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6eca658740 1 -- 172.21.15.78:0/306892186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558a673436c0 con 0x558a670166b0 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/306892186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6eb802b040 con 0x558a670166b0 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6eca658740 10 client.15774 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6eca658740 1 -- 172.21.15.78:0/306892186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a6745ad50 con 0x558a670166b0 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.772+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/306892186 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558a673436c0 con 0x558a670166b0 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/306892186 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6eb8004660 con 0x558a670166b0 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6eca658740 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6eca658740 10 client.15774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6eca658740 20 client.15774 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6eca658740 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x558a66fcb3a0 0x558a6745d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6eca658740 4 client.15774 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6eca658740 1 -- 172.21.15.78:0/306892186 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a672c0820 con 0x558a66fcb3a0 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x558a66fcb3a0 0x558a6745d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6ea4ff9640 10 client.15774 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6ec0d4c640 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x558a66fcb3a0 0x558a6745d540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3725599c0da776f5 server_cookie=13f3b5ca7a798227 in_seq=0 out_seq=0 2024-05-21T15:16:44.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6ea4ff9640 1 -- 172.21.15.78:0/306892186 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a672c0820 con 0x558a66fcb3a0 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.773+0000 7f6ea4ff9640 10 client.15774 handle_command_reply: tid=0 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 client.15774 shutdown 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 -- 172.21.15.78:0/306892186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 msgr2=0x558a673e5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e5b60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000c00 tx=0x7f6eb8004410 comp rx=0 tx=0).stop 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 -- 172.21.15.78:0/306892186 shutdown_connections 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x558a66fcb3a0 0x558a6745d540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a67037390 0x558a673e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a670166b0 0x558a673e5b60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ea4ff9640 0 client.15774 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 --2- 172.21.15.78:0/306892186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a6700e4d0 0x558a673e87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 -- 172.21.15.78:0/306892186 >> 172.21.15.78:0/306892186 conn(0x558a6702b4e0 msgr2=0x558a673e01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 -- 172.21.15.78:0/306892186 shutdown_connections 2024-05-21T15:16:44.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 1 -- 172.21.15.78:0/306892186 wait complete. 2024-05-21T15:16:44.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6eca658740 20 client.15774 trim_cache size 0 max 16384 2024-05-21T15:16:44.784 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 -- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6ea80a2d90 msgr2=0x7f6ea80a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6ea80a2d90 0x7f6ea80a52a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40cd410 tx=0x7f6eb406e090 comp rx=0 tx=0).stop 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 -- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc10c330 msgr2=0x7f6ebc19dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc10c330 0x7f6ebc19dfe0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400eb10 tx=0x7f6eb400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 -- 172.21.15.78:0/2294343905 shutdown_connections 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ebc1079d0 0x7f6ebc199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6ea80a2d90 0x7f6ea80a52a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ebc10c330 0x7f6ebc19dfe0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 --2- 172.21.15.78:0/2294343905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ebc101aa0 0x7f6ebc1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.774+0000 7f6ec37b1640 1 -- 172.21.15.78:0/2294343905 >> 172.21.15.78:0/2294343905 conn(0x7f6ebc0770b0 msgr2=0x7f6ebc103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.775+0000 7f6ec37b1640 1 -- 172.21.15.78:0/2294343905 shutdown_connections 2024-05-21T15:16:44.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.775+0000 7f6ec37b1640 1 -- 172.21.15.78:0/2294343905 wait complete. 2024-05-21T15:16:44.787 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:44 smithi078 ceph-mon[31997]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:16:44.787 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:44 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/413370596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:44.788 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:16:44.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:44 smithi084 ceph-mon[36334]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:16:44.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:44 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/413370596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 -- 172.21.15.78:0/4180913413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc6010d010 msgr2=0x7fbc6010f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/4180913413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc6010d010 0x7fbc6010f500 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009ca0 tx=0x7fbc5802a9e0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 -- 172.21.15.78:0/4180913413 shutdown_connections 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/4180913413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6010a660 0x7fbc6010ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/4180913413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc6010d010 0x7fbc6010f500 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/4180913413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc60103c00 0x7fbc60104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 -- 172.21.15.78:0/4180913413 >> 172.21.15.78:0/4180913413 conn(0x7fbc600770b0 msgr2=0x7fbc600774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 -- 172.21.15.78:0/4180913413 shutdown_connections 2024-05-21T15:16:44.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 -- 172.21.15.78:0/4180913413 wait complete. 2024-05-21T15:16:44.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 Processor -- start 2024-05-21T15:16:44.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.899+0000 7fbc65eb6640 1 -- start start 2024-05-21T15:16:44.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc65eb6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc60103c00 0x7fbc60199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc65eb6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc6010a660 0x7fbc60199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc65eb6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6010d010 0x7fbc6019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc65eb6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbc60112e10 con 0x7fbc60103c00 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc65eb6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbc60112c90 con 0x7fbc6010d010 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc65eb6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc60112f90 con 0x7fbc6010a660 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc60103c00 0x7fbc60199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc60103c00 0x7fbc60199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43900/0 (socket says 172.21.15.78:43900) 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5f7fe640 1 -- 172.21.15.78:0/1024306190 learned_addr learned my addr 172.21.15.78:0/1024306190 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc6010a660 0x7fbc60199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6010d010 0x7fbc6019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5f7fe640 1 -- 172.21.15.78:0/1024306190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc6010a660 msgr2=0x7fbc60199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5f7fe640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc6010a660 0x7fbc60199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5f7fe640 1 -- 172.21.15.78:0/1024306190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6010d010 msgr2=0x7fbc6019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5f7fe640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6010d010 0x7fbc6019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.900+0000 7fbc5f7fe640 1 -- 172.21.15.78:0/1024306190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6019e590 con 0x7fbc60103c00 2024-05-21T15:16:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.901+0000 7fbc5f7fe640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc60103c00 0x7fbc60199730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000d770 tx=0x7fbc5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.901+0000 7fbc5d7fa640 1 -- 172.21.15.78:0/1024306190 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50017070 con 0x7fbc60103c00 2024-05-21T15:16:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.901+0000 7fbc5d7fa640 1 -- 172.21.15.78:0/1024306190 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc50005500 con 0x7fbc60103c00 2024-05-21T15:16:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.901+0000 7fbc65eb6640 1 -- 172.21.15.78:0/1024306190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6019e880 con 0x7fbc60103c00 2024-05-21T15:16:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.901+0000 7fbc5d7fa640 1 -- 172.21.15.78:0/1024306190 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50004e00 con 0x7fbc60103c00 2024-05-21T15:16:44.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.901+0000 7fbc65eb6640 1 -- 172.21.15.78:0/1024306190 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6019ed80 con 0x7fbc60103c00 2024-05-21T15:16:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc5d7fa640 1 -- 172.21.15.78:0/1024306190 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fbc50010460 con 0x7fbc60103c00 2024-05-21T15:16:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc6ce3a740 1 Processor -- start 2024-05-21T15:16:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc6ce3a740 1 -- start start 2024-05-21T15:16:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650854c390 0x5565088eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 0x5565088eb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650852b6b0 0x5565088eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556508850c40 con 0x55650854c390 2024-05-21T15:16:44.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5565087f8580 con 0x5565085234d0 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55650884cc00 con 0x55650852b6b0 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc5ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650854c390 0x5565088eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc5ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650854c390 0x5565088eabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43902/0 (socket says 172.21.15.78:43902) 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.904+0000 7fbc5ffff640 1 -- 172.21.15.78:0/2729735030 learned_addr learned my addr 172.21.15.78:0/2729735030 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5f7fe640 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 0x5565088eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650852b6b0 0x5565088eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5ffff640 1 -- 172.21.15.78:0/2729735030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650852b6b0 msgr2=0x5565088eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650852b6b0 0x5565088eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5ffff640 1 -- 172.21.15.78:0/2729735030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 msgr2=0x5565088eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 0x5565088eb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5ffff640 1 -- 172.21.15.78:0/2729735030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565085643f0 con 0x55650854c390 2024-05-21T15:16:44.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5d7fa640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbc4c0a2d90 0x7fbc4c0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650854c390 0x5565088eabf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009ca0 tx=0x7fbc58005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1be671aafcf1042 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5f7fe640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbc4c0a2d90 0x7fbc4c0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc55ffb640 1 -- 172.21.15.78:0/2729735030 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58033050 con 0x55650854c390 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc5d7fa640 1 -- 172.21.15.78:0/1024306190 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbc500cefe0 con 0x7fbc60103c00 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc55ffb640 1 -- 172.21.15.78:0/2729735030 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc58047070 con 0x55650854c390 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.905+0000 7fbc55ffb640 1 -- 172.21.15.78:0/2729735030 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5802f8a0 con 0x55650854c390 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2729735030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650854c390 msgr2=0x5565088eabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650854c390 0x5565088eabf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009ca0 tx=0x7fbc58005d80 comp rx=0 tx=0).stop 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc5f7fe640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbc4c0a2d90 0x7fbc4c0a52a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc500cc6d0 tx=0x7fbc500bb040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2729735030 shutdown_connections 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 0x5565088eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650852b6b0 0x5565088eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2729735030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650854c390 0x5565088eabf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2729735030 >> 172.21.15.78:0/2729735030 conn(0x5565085404e0 msgr2=0x5565088e74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2729735030 shutdown_connections 2024-05-21T15:16:44.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2729735030 wait complete. 2024-05-21T15:16:44.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.906+0000 7fbc6ce3a740 1 -- start start 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc6ce3a740 1 Processor -- start 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 0x5565088f9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650852b6b0 0x5565088f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650854c390 0x556508905950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5565088fc4a0 con 0x55650852b6b0 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55650880bc40 con 0x5565085234d0 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5565088fc620 con 0x55650854c390 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc6ce3a740 10 client.0 _ll_register_callbacks cb 0x556508563be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650852b6b0 0x5565088f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650852b6b0 0x5565088f9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43914/0 (socket says 172.21.15.78:43914) 2024-05-21T15:16:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc56ffd640 1 -- 172.21.15.78:0/3255883414 learned_addr learned my addr 172.21.15.78:0/3255883414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 0x5565088f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650854c390 0x556508905950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc56ffd640 1 -- 172.21.15.78:0/3255883414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650854c390 msgr2=0x556508905950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650854c390 0x556508905950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc56ffd640 1 -- 172.21.15.78:0/3255883414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 msgr2=0x5565088f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 0x5565088f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc56ffd640 1 -- 172.21.15.78:0/3255883414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650862e7d0 con 0x55650852b6b0 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.907+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650854c390 0x556508905950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650852b6b0 0x5565088f9c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbc480126d0 tx=0x7fbc48012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f20882e054f618 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc3effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc3effd640 1 -- 172.21.15.78:0/3255883414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800f9e0 con 0x55650852b6b0 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc3effd640 1 -- 172.21.15.78:0/3255883414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbc4800fb80 con 0x55650852b6b0 2024-05-21T15:16:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3255883414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc58047070 con 0x55650852b6b0 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc3effd640 1 -- 172.21.15.78:0/3255883414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc4800fd80 con 0x55650852b6b0 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 10 client.15786 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3255883414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556508908be0 con 0x55650852b6b0 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc3effd640 1 -- 172.21.15.78:0/3255883414 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc58047070 con 0x55650852b6b0 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc3effd640 1 -- 172.21.15.78:0/3255883414 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc4801c4b0 con 0x55650852b6b0 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 10 client.15786 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 20 client.15786 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55650850af60 0x55650890c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 4 client.15786 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.908+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3255883414 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55650885e370 con 0x55650850af60 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.909+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55650850af60 0x55650890c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.909+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55650850af60 0x55650890c140 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4b1d8bd9d555bff server_cookie=783b1d3381098ef5 in_seq=0 out_seq=0 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.909+0000 7fbc3effd640 10 client.15786 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc3effd640 1 -- 172.21.15.78:0/3255883414 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55650885e370 con 0x55650850af60 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc3effd640 10 client.15786 handle_command_reply: tid=0 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 client.15786 shutdown 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3255883414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650852b6b0 msgr2=0x5565088f9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650852b6b0 0x5565088f9c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbc480126d0 tx=0x7fbc48012b90 comp rx=0 tx=0).stop 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3255883414 shutdown_connections 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55650850af60 0x55650890c140 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650854c390 0x556508905950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55650852b6b0 0x5565088f9c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc3effd640 0 client.15786 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3255883414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5565085234d0 0x5565088f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3255883414 >> 172.21.15.78:0/3255883414 conn(0x5565085404e0 msgr2=0x5565088ed3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3255883414 shutdown_connections 2024-05-21T15:16:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3255883414 wait complete. 2024-05-21T15:16:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.910+0000 7fbc6ce3a740 20 client.15786 trim_cache size 0 max 16384 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc6ce3a740 1 Processor -- start 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc6ce3a740 1 -- start start 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x556508905530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 0x5565088f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601ac7a0 con 0x5565085234d0 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601ae2e0 con 0x55650852b6b0 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601ace70 con 0x55650850af60 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x556508905530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x556508905530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43922/0 (socket says 172.21.15.78:43922) 2024-05-21T15:16:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc56ffd640 1 -- 172.21.15.78:0/3813281355 learned_addr learned my addr 172.21.15.78:0/3813281355 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc56ffd640 1 -- 172.21.15.78:0/3813281355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 msgr2=0x5565088f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc56ffd640 1 -- 172.21.15.78:0/3813281355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 msgr2=0x5565088f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 0x5565088f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc56ffd640 1 -- 172.21.15.78:0/3813281355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565088e2970 con 0x5565085234d0 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.924+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x556508905530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbc4801cb10 tx=0x7fbc4800f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcce5f386e64e2d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc557fa640 1 -- 172.21.15.78:0/3813281355 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800fb20 con 0x5565085234d0 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc557fa640 1 -- 172.21.15.78:0/3813281355 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc480055b0 con 0x5565085234d0 2024-05-21T15:16:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc557fa640 1 -- 172.21.15.78:0/3813281355 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48003850 con 0x5565085234d0 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3813281355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 msgr2=0x556508905530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x556508905530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbc4801cb10 tx=0x7fbc4800f840 comp rx=0 tx=0).stop 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3813281355 shutdown_connections 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 0x5565088f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x556508905530 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/3813281355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3813281355 >> 172.21.15.78:0/3813281355 conn(0x5565085404e0 msgr2=0x5565088f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.925+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3813281355 shutdown_connections 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/3813281355 wait complete. 2024-05-21T15:16:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 -- start start 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 Processor -- start 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x5565088e6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 0x5565088e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601adc10 con 0x5565085234d0 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601ad540 con 0x55650852b6b0 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601aa780 con 0x55650850af60 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.926+0000 7fbc6ce3a740 10 client.0 _ll_register_callbacks cb 0x7fbc480097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x5565088e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x5565088e6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43924/0 (socket says 172.21.15.78:43924) 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 -- 172.21.15.78:0/2682878554 learned_addr learned my addr 172.21.15.78:0/2682878554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 0x5565088e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 -- 172.21.15.78:0/2682878554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 msgr2=0x5565088e5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088e5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 -- 172.21.15.78:0/2682878554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 msgr2=0x5565088e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 0x5565088e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 -- 172.21.15.78:0/2682878554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556508900a10 con 0x5565085234d0 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc56ffd640 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x5565088e6080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009380 tx=0x7fbc48009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41ebeefac9b7bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc3effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc3effd640 1 -- 172.21.15.78:0/2682878554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48032030 con 0x5565085234d0 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc3effd640 1 -- 172.21.15.78:0/2682878554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbc48018040 con 0x5565085234d0 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2682878554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc480055b0 con 0x5565085234d0 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc3effd640 1 -- 172.21.15.78:0/2682878554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc4803c050 con 0x5565085234d0 2024-05-21T15:16:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc6ce3a740 10 client.15792 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2682878554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556508901c90 con 0x5565085234d0 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.927+0000 7fbc3effd640 1 -- 172.21.15.78:0/2682878554 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc480055b0 con 0x5565085234d0 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc3effd640 1 -- 172.21.15.78:0/2682878554 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc4802ec10 con 0x5565085234d0 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc6ce3a740 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc6ce3a740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc6ce3a740 20 client.15792 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55650854c390 0x5565089764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc6ce3a740 4 client.15792 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2682878554 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565088586c0 con 0x55650854c390 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55650854c390 0x5565089764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc3effd640 10 client.15792 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.928+0000 7fbc5ffff640 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55650854c390 0x5565089764c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382c8cfc090671c3 server_cookie=de2ee7873a0cb810 in_seq=0 out_seq=0 2024-05-21T15:16:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc3effd640 1 -- 172.21.15.78:0/2682878554 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565088586c0 con 0x55650854c390 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc3effd640 10 client.15792 handle_command_reply: tid=0 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 client.15792 shutdown 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2682878554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 msgr2=0x5565088e6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x5565088e6080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009380 tx=0x7fbc48009e90 comp rx=0 tx=0).stop 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2682878554 shutdown_connections 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55650854c390 0x5565089764c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55650852b6b0 0x5565088e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc3effd640 0 client.15792 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565085234d0 0x5565088e6080 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 --2- 172.21.15.78:0/2682878554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55650850af60 0x5565088e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2682878554 >> 172.21.15.78:0/2682878554 conn(0x5565085404e0 msgr2=0x5565088eaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2682878554 shutdown_connections 2024-05-21T15:16:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 1 -- 172.21.15.78:0/2682878554 wait complete. 2024-05-21T15:16:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc6ce3a740 20 client.15792 trim_cache size 0 max 16384 2024-05-21T15:16:44.939 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc65eb6640 1 -- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbc4c0a2d90 msgr2=0x7fbc4c0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbc4c0a2d90 0x7fbc4c0a52a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc500cc6d0 tx=0x7fbc500bb040 comp rx=0 tx=0).stop 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc65eb6640 1 -- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc60103c00 msgr2=0x7fbc60199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.929+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc60103c00 0x7fbc60199730 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000d770 tx=0x7fbc5000dc30 comp rx=0 tx=0).stop 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.930+0000 7fbc65eb6640 1 -- 172.21.15.78:0/1024306190 shutdown_connections 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.930+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc6010d010 0x7fbc6019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.930+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbc4c0a2d90 0x7fbc4c0a52a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.930+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc6010a660 0x7fbc60199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.930+0000 7fbc65eb6640 1 --2- 172.21.15.78:0/1024306190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc60103c00 0x7fbc60199730 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.930+0000 7fbc65eb6640 1 -- 172.21.15.78:0/1024306190 >> 172.21.15.78:0/1024306190 conn(0x7fbc600770b0 msgr2=0x7fbc60100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.930+0000 7fbc65eb6640 1 -- 172.21.15.78:0/1024306190 shutdown_connections 2024-05-21T15:16:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:44.930+0000 7fbc65eb6640 1 -- 172.21.15.78:0/1024306190 wait complete. 2024-05-21T15:16:45.944 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:16:45.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:16:45.944 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:16:46.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 -- 172.21.15.78:0/729820807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 msgr2=0x7f8ea010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 --2- 172.21.15.78:0/729820807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 0x7f8ea010f920 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009ca0 tx=0x7f8e9802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 -- 172.21.15.78:0/729820807 shutdown_connections 2024-05-21T15:16:46.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 --2- 172.21.15.78:0/729820807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ea0101000 0x7f8ea0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 --2- 172.21.15.78:0/729820807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 0x7f8ea010f920 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 --2- 172.21.15.78:0/729820807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ea0108b50 0x7f8ea0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 -- 172.21.15.78:0/729820807 >> 172.21.15.78:0/729820807 conn(0x7f8ea0076830 msgr2=0x7f8ea0076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 -- 172.21.15.78:0/729820807 shutdown_connections 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.061+0000 7f8ea6873640 1 -- 172.21.15.78:0/729820807 wait complete. 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea6873640 1 Processor -- start 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea6873640 1 -- start start 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea6873640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ea0101000 0x7f8ea0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea6873640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ea0108b50 0x7f8ea0199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea6873640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 0x7f8ea019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea6873640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0112e10 con 0x7f8ea010d500 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea6873640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0112c90 con 0x7f8ea0101000 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea6873640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0112f90 con 0x7f8ea0108b50 2024-05-21T15:16:46.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea4e10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 0x7f8ea019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea4e10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 0x7f8ea019df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43940/0 (socket says 172.21.15.78:43940) 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea4e10640 1 -- 172.21.15.78:0/3615920789 learned_addr learned my addr 172.21.15.78:0/3615920789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ea0101000 0x7f8ea0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8e97fff640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ea0108b50 0x7f8ea0199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea4e10640 1 -- 172.21.15.78:0/3615920789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ea0108b50 msgr2=0x7f8ea0199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea4e10640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ea0108b50 0x7f8ea0199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea4e10640 1 -- 172.21.15.78:0/3615920789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ea0101000 msgr2=0x7f8ea0199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea4e10640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ea0101000 0x7f8ea0199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.062+0000 7f8ea4e10640 1 -- 172.21.15.78:0/3615920789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea019e580 con 0x7f8ea010d500 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.063+0000 7f8ea4e10640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 0x7f8ea019df40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8e980319f0 tx=0x7f8e98031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.063+0000 7f8e9dffb640 1 -- 172.21.15.78:0/3615920789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98003c30 con 0x7f8ea010d500 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.063+0000 7f8e9dffb640 1 -- 172.21.15.78:0/3615920789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e98003dd0 con 0x7f8ea010d500 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.063+0000 7f8ea6873640 1 -- 172.21.15.78:0/3615920789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea019e810 con 0x7f8ea010d500 2024-05-21T15:16:46.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.063+0000 7f8e9dffb640 1 -- 172.21.15.78:0/3615920789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9802fdd0 con 0x7f8ea010d500 2024-05-21T15:16:46.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.063+0000 7f8ea6873640 1 -- 172.21.15.78:0/3615920789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea019ed60 con 0x7f8ea010d500 2024-05-21T15:16:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.065+0000 7f8e9dffb640 1 -- 172.21.15.78:0/3615920789 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f8e98004050 con 0x7f8ea010d500 2024-05-21T15:16:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8ead7fb740 1 Processor -- start 2024-05-21T15:16:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8ead7fb740 1 -- start start 2024-05-21T15:16:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 0x55710cdb3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdb3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdb40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55710cd18990 con 0x55710c9eb550 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55710ccc05b0 con 0x55710ca14410 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55710cd14950 con 0x55710c9f3730 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdb3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdb3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43956/0 (socket says 172.21.15.78:43956) 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9ffff640 1 -- 172.21.15.78:0/623074159 learned_addr learned my addr 172.21.15.78:0/623074159 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e97fff640 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 0x55710cdb3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e97fff640 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdb40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9ffff640 1 -- 172.21.15.78:0/623074159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 msgr2=0x55710cdb40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdb40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9ffff640 1 -- 172.21.15.78:0/623074159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 msgr2=0x55710cdb3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 0x55710cdb3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9dffb640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e8c0a2d90 0x7f8e8c0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e9ffff640 1 -- 172.21.15.78:0/623074159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55710caf47e0 con 0x55710c9eb550 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.066+0000 7f8e97fff640 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdb40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.067+0000 7f8ea4e10640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e8c0a2d90 0x7f8e8c0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.067+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdb3b60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000bc70 tx=0x7f8e900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b22cd60c7297588 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.067+0000 7f8e9dffb640 1 -- 172.21.15.78:0/3615920789 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8e980f07d0 con 0x7f8ea010d500 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.067+0000 7f8e967fc640 1 -- 172.21.15.78:0/623074159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90004220 con 0x55710c9eb550 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.067+0000 7f8e967fc640 1 -- 172.21.15.78:0/623074159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e9000be20 con 0x55710c9eb550 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.067+0000 7f8e967fc640 1 -- 172.21.15.78:0/623074159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90004cb0 con 0x55710c9eb550 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 -- 172.21.15.78:0/623074159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 msgr2=0x55710cdb3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ea4e10640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e8c0a2d90 0x7f8e8c0a52a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e980edfd0 tx=0x7f8e980e7820 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdb3b60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000bc70 tx=0x7f8e900107a0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 -- 172.21.15.78:0/623074159 shutdown_connections 2024-05-21T15:16:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdb40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdb3b60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/623074159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 0x55710cdb3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 -- 172.21.15.78:0/623074159 >> 172.21.15.78:0/623074159 conn(0x55710ca08560 msgr2=0x55710cdafdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 -- 172.21.15.78:0/623074159 shutdown_connections 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 -- 172.21.15.78:0/623074159 wait complete. 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 -- start start 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.068+0000 7f8ead7fb740 1 Processor -- start 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9eb550 0x55710cdbdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9f3730 0x55710cdbe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 0x55710cdc3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55710cdc0970 con 0x55710c9f3730 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55710ccd3b40 con 0x55710c9eb550 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55710cdc0af0 con 0x55710ca14410 2024-05-21T15:16:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8ead7fb740 10 client.0 _ll_register_callbacks cb 0x55710ccc80f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9f3730 0x55710cdbe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9f3730 0x55710cdbe240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43966/0 (socket says 172.21.15.78:43966) 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 -- 172.21.15.78:0/2424428368 learned_addr learned my addr 172.21.15.78:0/2424428368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9eb550 0x55710cdbdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 0x55710cdc3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 -- 172.21.15.78:0/2424428368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 msgr2=0x55710cdc3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 0x55710cdc3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 -- 172.21.15.78:0/2424428368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9eb550 msgr2=0x55710cdbdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9eb550 0x55710cdbdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 -- 172.21.15.78:0/2424428368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55710cb54f70 con 0x55710c9f3730 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 0x55710cdc3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9f3730 0x55710cdbe240 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e90000c00 tx=0x7f8e90004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91de1d01530e881a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2424428368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9002b060 con 0x55710c9f3730 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2424428368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8e90011710 con 0x55710c9f3730 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.069+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2424428368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9000be20 con 0x55710c9f3730 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2424428368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e90011910 con 0x55710c9f3730 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8ead7fb740 10 client.15804 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2424428368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55710c9ec170 con 0x55710c9f3730 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2424428368 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e9000be20 con 0x55710c9f3730 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2424428368 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8e90004a50 con 0x55710c9f3730 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8ead7fb740 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8ead7fb740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8ead7fb740 20 client.15804 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55710c9d31f0 0x55710cdd0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8ead7fb740 4 client.15804 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2424428368 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55710cd260c0 con 0x55710c9d31f0 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.070+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55710c9d31f0 0x55710cdd0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.071+0000 7f8e7f7fe640 10 client.15804 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.071+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55710c9d31f0 0x55710cdd0050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35eee7a6d7c6417 server_cookie=759abc562d1c7562 in_seq=0 out_seq=0 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.071+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2424428368 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55710cd260c0 con 0x55710c9d31f0 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.071+0000 7f8e7f7fe640 10 client.15804 handle_command_reply: tid=0 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 client.15804 shutdown 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2424428368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9f3730 msgr2=0x55710cdbe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9f3730 0x55710cdbe240 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e90000c00 tx=0x7f8e90004960 comp rx=0 tx=0).stop 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2424428368 shutdown_connections 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55710c9d31f0 0x55710cdd0050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 0x55710cdc3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9f3730 0x55710cdbe240 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2424428368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9eb550 0x55710cdbdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8e7f7fe640 0 client.15804 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2424428368 >> 172.21.15.78:0/2424428368 conn(0x55710ca08560 msgr2=0x55710cdb5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2424428368 shutdown_connections 2024-05-21T15:16:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2424428368 wait complete. 2024-05-21T15:16:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.072+0000 7f8ead7fb740 20 client.15804 trim_cache size 0 max 16384 2024-05-21T15:16:46.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8ead7fb740 1 Processor -- start 2024-05-21T15:16:46.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8ead7fb740 1 -- start start 2024-05-21T15:16:46.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdbe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9f3730 0x55710cdc3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 0x55710cdc42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01aea20 con 0x55710c9eb550 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01b0560 con 0x55710c9f3730 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01af0f0 con 0x55710ca14410 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdbe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdbe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43972/0 (socket says 172.21.15.78:43972) 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 -- 172.21.15.78:0/3144587466 learned_addr learned my addr 172.21.15.78:0/3144587466 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/3144587466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9f3730 0x55710cdc3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 -- 172.21.15.78:0/3144587466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 msgr2=0x55710cdc42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 --2- 172.21.15.78:0/3144587466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 0x55710cdc42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 -- 172.21.15.78:0/3144587466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9f3730 msgr2=0x55710cdc3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 --2- 172.21.15.78:0/3144587466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9f3730 0x55710cdc3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 -- 172.21.15.78:0/3144587466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55710cdbcaa0 con 0x55710c9eb550 2024-05-21T15:16:46.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.090+0000 7f8e97fff640 1 --2- 172.21.15.78:0/3144587466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdbe1c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e88000c00 tx=0x7f8e88015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddbe17a85f2335f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8e95ffb640 1 -- 172.21.15.78:0/3144587466 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88020e60 con 0x55710c9eb550 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8e95ffb640 1 -- 172.21.15.78:0/3144587466 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e88011db0 con 0x55710c9eb550 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8e95ffb640 1 -- 172.21.15.78:0/3144587466 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88020a80 con 0x55710c9eb550 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 -- 172.21.15.78:0/3144587466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 msgr2=0x55710cdbe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/3144587466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdbe1c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e88000c00 tx=0x7f8e88015040 comp rx=0 tx=0).stop 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 -- 172.21.15.78:0/3144587466 shutdown_connections 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/3144587466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710c9f3730 0x55710cdc3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/3144587466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710ca14410 0x55710cdc42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/3144587466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdbe1c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 -- 172.21.15.78:0/3144587466 >> 172.21.15.78:0/3144587466 conn(0x55710ca08560 msgr2=0x55710cdbdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 -- 172.21.15.78:0/3144587466 shutdown_connections 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.091+0000 7f8ead7fb740 1 -- 172.21.15.78:0/3144587466 wait complete. 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 1 -- start start 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 1 Processor -- start 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdc7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 0x55710cdc8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01afe90 con 0x55710c9eb550 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01af7c0 con 0x55710ca14410 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01aca00 con 0x55710c9f3730 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8ead7fb740 10 client.0 _ll_register_callbacks cb 0x7f8e8800e9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43976/0 (socket says 172.21.15.78:43976) 2024-05-21T15:16:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.092+0000 7f8e97fff640 1 -- 172.21.15.78:0/2374546866 learned_addr learned my addr 172.21.15.78:0/2374546866 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 0x55710cdc8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdc7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e97fff640 1 -- 172.21.15.78:0/2374546866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 msgr2=0x55710cdc7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdc7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e97fff640 1 -- 172.21.15.78:0/2374546866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 msgr2=0x55710cdc8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 0x55710cdc8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e97fff640 1 -- 172.21.15.78:0/2374546866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55710cdc8720 con 0x55710c9eb550 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e97fff640 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdca880 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x55710cdbec40 tx=0x7f8e88011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c372f5d6553451 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2374546866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88024040 con 0x55710c9eb550 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2374546866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8e88011ad0 con 0x55710c9eb550 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2374546866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e88011db0 con 0x55710c9eb550 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2374546866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e880204e0 con 0x55710c9eb550 2024-05-21T15:16:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8ead7fb740 10 client.15810 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.093+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2374546866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55710ce37f60 con 0x55710c9eb550 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2374546866 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e88011db0 con 0x55710c9eb550 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2374546866 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8e88004cc0 con 0x55710c9eb550 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8ead7fb740 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8ead7fb740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8ead7fb740 20 client.15810 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55710c9a8630 0x55710ce3a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8ead7fb740 4 client.15810 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2374546866 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55710cd20410 con 0x55710c9a8630 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55710c9a8630 0x55710ce3a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8e9ffff640 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55710c9a8630 0x55710ce3a600 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d90db874c367db5 server_cookie=433142baa16404ee in_seq=0 out_seq=0 2024-05-21T15:16:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.094+0000 7f8e7f7fe640 10 client.15810 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.095+0000 7f8e7f7fe640 1 -- 172.21.15.78:0/2374546866 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55710cd20410 con 0x55710c9a8630 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.095+0000 7f8e7f7fe640 10 client.15810 handle_command_reply: tid=0 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.095+0000 7f8ead7fb740 1 client.15810 shutdown 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.095+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2374546866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 msgr2=0x55710cdca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.095+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdca880 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x55710cdbec40 tx=0x7f8e88011f60 comp rx=0 tx=0).stop 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2374546866 shutdown_connections 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55710c9a8630 0x55710ce3a600 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55710ca14410 0x55710cdc8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55710c9f3730 0x55710cdc7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 1 --2- 172.21.15.78:0/2374546866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55710c9eb550 0x55710cdca880 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2374546866 >> 172.21.15.78:0/2374546866 conn(0x55710ca08560 msgr2=0x55710cdae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8e7f7fe640 0 client.15810 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2374546866 shutdown_connections 2024-05-21T15:16:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 1 -- 172.21.15.78:0/2374546866 wait complete. 2024-05-21T15:16:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ead7fb740 20 client.15810 trim_cache size 0 max 16384 2024-05-21T15:16:46.105 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:46.105 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:16:46.105 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c9157f69-e23b-4ae8-9245-c3b9b1a408dc", 2024-05-21T15:16:46.105 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:16:46.105 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 -- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e8c0a2d90 msgr2=0x7f8e8c0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e8c0a2d90 0x7f8e8c0a52a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e980edfd0 tx=0x7f8e980e7820 comp rx=0 tx=0).stop 2024-05-21T15:16:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 -- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 msgr2=0x7f8ea019df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 0x7f8ea019df40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8e980319f0 tx=0x7f8e98031a20 comp rx=0 tx=0).stop 2024-05-21T15:16:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 -- 172.21.15.78:0/3615920789 shutdown_connections 2024-05-21T15:16:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e8c0a2d90 0x7f8e8c0a52a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ea010d500 0x7f8ea019df40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ea0108b50 0x7f8ea0199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 --2- 172.21.15.78:0/3615920789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ea0101000 0x7f8ea0199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 -- 172.21.15.78:0/3615920789 >> 172.21.15.78:0/3615920789 conn(0x7f8ea0076830 msgr2=0x7f8ea01094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 -- 172.21.15.78:0/3615920789 shutdown_connections 2024-05-21T15:16:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.096+0000 7f8ea6873640 1 -- 172.21.15.78:0/3615920789 wait complete. 2024-05-21T15:16:46.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:16:46.111 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:16:46.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/957704004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d810c350 msgr2=0x7fe9d810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/957704004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d810c350 0x7fe9d810e770 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009ca0 tx=0x7fe9d002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/957704004 shutdown_connections 2024-05-21T15:16:46.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/957704004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d8102210 0x7fe9d8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/957704004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d810c350 0x7fe9d810e770 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/957704004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d81079f0 0x7fe9d8101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/957704004 >> 172.21.15.78:0/957704004 conn(0x7fe9d8076830 msgr2=0x7fe9d8076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/957704004 shutdown_connections 2024-05-21T15:16:46.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.221+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/957704004 wait complete. 2024-05-21T15:16:46.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9ddc0d640 1 Processor -- start 2024-05-21T15:16:46.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9ddc0d640 1 -- start start 2024-05-21T15:16:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9ddc0d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d8102210 0x7fe9d81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9ddc0d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d81079f0 0x7fe9d8199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9ddc0d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d810c350 0x7fe9d819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9ddc0d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8111d50 con 0x7fe9d8102210 2024-05-21T15:16:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9ddc0d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8111bd0 con 0x7fe9d810c350 2024-05-21T15:16:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9ddc0d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8111ed0 con 0x7fe9d81079f0 2024-05-21T15:16:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d8102210 0x7fe9d81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d8102210 0x7fe9d81997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44000/0 (socket says 172.21.15.78:44000) 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 -- 172.21.15.78:0/2621339100 learned_addr learned my addr 172.21.15.78:0/2621339100 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d810c350 0x7fe9d819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9cffff640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d81079f0 0x7fe9d8199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 -- 172.21.15.78:0/2621339100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d81079f0 msgr2=0x7fe9d8199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d81079f0 0x7fe9d8199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 -- 172.21.15.78:0/2621339100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d810c350 msgr2=0x7fe9d819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d810c350 0x7fe9d819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 -- 172.21.15.78:0/2621339100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d819e640 con 0x7fe9d8102210 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d810c350 0x7fe9d819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.222+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d8102210 0x7fe9d81997e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d770 tx=0x7fe9c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.223+0000 7fe9d57fa640 1 -- 172.21.15.78:0/2621339100 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8017070 con 0x7fe9d8102210 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.223+0000 7fe9d57fa640 1 -- 172.21.15.78:0/2621339100 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c8009c30 con 0x7fe9d8102210 2024-05-21T15:16:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.223+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/2621339100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d819e930 con 0x7fe9d8102210 2024-05-21T15:16:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.223+0000 7fe9d57fa640 1 -- 172.21.15.78:0/2621339100 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800f620 con 0x7fe9d8102210 2024-05-21T15:16:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.223+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/2621339100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d81a62b0 con 0x7fe9d8102210 2024-05-21T15:16:46.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.225+0000 7fe9d57fa640 1 -- 172.21.15.78:0/2621339100 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fe9c801a070 con 0x7fe9d8102210 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.225+0000 7fe9e4b91740 1 Processor -- start 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.225+0000 7fe9e4b91740 1 -- start start 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781ef390 0x55c27858dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d57fa640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe9c40a2d90 0x7fe9c40a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781c64d0 0x55c27858e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ce6b0 0x55c27858e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c2784f3d00 con 0x55c2781ef390 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c27849b4c0 con 0x55c2781c64d0 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c2784efcc0 con 0x55c2781ce6b0 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781ef390 0x55c27858dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe9c40a2d90 0x7fe9c40a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781ef390 0x55c27858dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44004/0 (socket says 172.21.15.78:44004) 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 -- 172.21.15.78:0/796987873 learned_addr learned my addr 172.21.15.78:0/796987873 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9cffff640 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781c64d0 0x55c27858e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d57fa640 1 -- 172.21.15.78:0/2621339100 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe9c80cf120 con 0x7fe9d8102210 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ce6b0 0x55c27858e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 -- 172.21.15.78:0/796987873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ce6b0 msgr2=0x55c27858e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ce6b0 0x55c27858e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 -- 172.21.15.78:0/796987873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781c64d0 msgr2=0x55c27858e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781c64d0 0x55c27858e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 -- 172.21.15.78:0/796987873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2782d17d0 con 0x55c2781ef390 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9cffff640 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781c64d0 0x55c27858e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe9c40a2d90 0x7fe9c40a52a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006010 tx=0x7fe9d0031d10 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.226+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781ef390 0x55c27858dd30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe9c809b1d0 tx=0x7fe9c8010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b792cecb3044b2d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9cdffb640 1 -- 172.21.15.78:0/796987873 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8021070 con 0x55c2781ef390 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9cdffb640 1 -- 172.21.15.78:0/796987873 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c809bbd0 con 0x55c2781ef390 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9cdffb640 1 -- 172.21.15.78:0/796987873 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80a4140 con 0x55c2781ef390 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9e4b91740 1 -- 172.21.15.78:0/796987873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781ef390 msgr2=0x55c27858dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781ef390 0x55c27858dd30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe9c809b1d0 tx=0x7fe9c8010cd0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9e4b91740 1 -- 172.21.15.78:0/796987873 shutdown_connections 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781c64d0 0x55c27858e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ce6b0 0x55c27858e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/796987873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781ef390 0x55c27858dd30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.227+0000 7fe9e4b91740 1 -- 172.21.15.78:0/796987873 >> 172.21.15.78:0/796987873 conn(0x55c2781e34e0 msgr2=0x55c27858a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 -- 172.21.15.78:0/796987873 shutdown_connections 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 -- 172.21.15.78:0/796987873 wait complete. 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 -- start start 2024-05-21T15:16:46.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 Processor -- start 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c278599e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c27859a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ef390 0x55c2785983b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c27859afc0 con 0x55c2781c64d0 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c2784aeb90 con 0x55c2781ce6b0 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.228+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c27859b140 con 0x55c2781ef390 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c278599e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9e4b91740 10 client.0 _ll_register_callbacks cb 0x55c2784eada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c278599e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44018/0 (socket says 172.21.15.78:44018) 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 -- 172.21.15.78:0/1912268588 learned_addr learned my addr 172.21.15.78:0/1912268588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9cffff640 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c27859a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ef390 0x55c2785983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 -- 172.21.15.78:0/1912268588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ef390 msgr2=0x55c2785983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ef390 0x55c2785983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 -- 172.21.15.78:0/1912268588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 msgr2=0x55c27859a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c27859a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 -- 172.21.15.78:0/1912268588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2782073f0 con 0x55c2781c64d0 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ef390 0x55c2785983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c278599e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80cc890 tx=0x7fe9c8097140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2b1b4e076fd343 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/1912268588 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8021070 con 0x55c2781c64d0 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9e4b91740 1 -- 172.21.15.78:0/1912268588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c809bbd0 con 0x55c2781c64d0 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/1912268588 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe9c80cca60 con 0x55c2781c64d0 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/1912268588 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9c8066560 con 0x55c2781c64d0 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9e4b91740 10 client.15822 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:46.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9e4b91740 1 -- 172.21.15.78:0/1912268588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2781c70f0 con 0x55c2781c64d0 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.229+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/1912268588 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c809bbd0 con 0x55c2781c64d0 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/1912268588 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe9c80a3cb0 con 0x55c2781c64d0 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9e4b91740 10 client.15822 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9e4b91740 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9e4b91740 20 client.15822 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c2781adf60 0x55c2785aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9e4b91740 4 client.15822 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9e4b91740 1 -- 172.21.15.78:0/1912268588 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c278501430 con 0x55c2781adf60 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9cffff640 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c2781adf60 0x55c2785aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9b6ffd640 10 client.15822 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.230+0000 7fe9cffff640 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c2781adf60 0x55c2785aa9c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9173a179370ff8b7 server_cookie=251d14fe9c26f66f in_seq=0 out_seq=0 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.231+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/1912268588 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c278501430 con 0x55c2781adf60 2024-05-21T15:16:46.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.231+0000 7fe9b6ffd640 10 client.15822 handle_command_reply: tid=0 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.231+0000 7fe9e4b91740 1 client.15822 shutdown 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 -- 172.21.15.78:0/1912268588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 msgr2=0x55c278599e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c278599e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80cc890 tx=0x7fe9c8097140 comp rx=0 tx=0).stop 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 -- 172.21.15.78:0/1912268588 shutdown_connections 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c2781adf60 0x55c2785aa9c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c27859a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781ef390 0x55c2785983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9b6ffd640 0 client.15822 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/1912268588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c278599e50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 -- 172.21.15.78:0/1912268588 >> 172.21.15.78:0/1912268588 conn(0x55c2781e34e0 msgr2=0x55c27858fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 -- 172.21.15.78:0/1912268588 shutdown_connections 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 1 -- 172.21.15.78:0/1912268588 wait complete. 2024-05-21T15:16:46.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.232+0000 7fe9e4b91740 20 client.15822 trim_cache size 0 max 16384 2024-05-21T15:16:46.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.249+0000 7fe9e4b91740 1 Processor -- start 2024-05-21T15:16:46.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.249+0000 7fe9e4b91740 1 -- start start 2024-05-21T15:16:46.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781adf60 0x55c278597b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c27858e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c2785ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81aebf0 con 0x55c2781c64d0 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81b0730 con 0x55c2781ce6b0 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81af2c0 con 0x55c2781adf60 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c27858e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c27858e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44030/0 (socket says 172.21.15.78:44030) 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 -- 172.21.15.78:0/4205610511 learned_addr learned my addr 172.21.15.78:0/4205610511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c2785ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781adf60 0x55c278597b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 -- 172.21.15.78:0/4205610511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781adf60 msgr2=0x55c278597b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781adf60 0x55c278597b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 -- 172.21.15.78:0/4205610511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 msgr2=0x55c2785ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c2785ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 -- 172.21.15.78:0/4205610511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c278585bd0 con 0x55c2781c64d0 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cffff640 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c27858e3c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80746f0 tx=0x7fe9c0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2983659e3cfbe78d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cd7fa640 1 -- 172.21.15.78:0/4205610511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0004250 con 0x55c2781c64d0 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cd7fa640 1 -- 172.21.15.78:0/4205610511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c00043f0 con 0x55c2781c64d0 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.250+0000 7fe9cd7fa640 1 -- 172.21.15.78:0/4205610511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0003850 con 0x55c2781c64d0 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 -- 172.21.15.78:0/4205610511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 msgr2=0x55c27858e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c27858e3c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80746f0 tx=0x7fe9c0005f90 comp rx=0 tx=0).stop 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 -- 172.21.15.78:0/4205610511 shutdown_connections 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c2785ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781c64d0 0x55c27858e3c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/4205610511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781adf60 0x55c278597b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 -- 172.21.15.78:0/4205610511 >> 172.21.15.78:0/4205610511 conn(0x55c2781e34e0 msgr2=0x55c27858dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 -- 172.21.15.78:0/4205610511 shutdown_connections 2024-05-21T15:16:46.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.251+0000 7fe9e4b91740 1 -- 172.21.15.78:0/4205610511 wait complete. 2024-05-21T15:16:46.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 1 -- start start 2024-05-21T15:16:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 1 Processor -- start 2024-05-21T15:16:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781adf60 0x55c2785acd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781c64d0 0x55c2785ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c278610af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81b0060 con 0x55c2781adf60 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81af990 con 0x55c2781ce6b0 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81acbd0 con 0x55c2781c64d0 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9e4b91740 10 client.0 _ll_register_callbacks cb 0x7fe9c0009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781adf60 0x55c2785acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781adf60 0x55c2785acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44044/0 (socket says 172.21.15.78:44044) 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9d77fe640 1 -- 172.21.15.78:0/2364998361 learned_addr learned my addr 172.21.15.78:0/2364998361 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.252+0000 7fe9d7fff640 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c278610af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9cffff640 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781c64d0 0x55c2785ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9d77fe640 1 -- 172.21.15.78:0/2364998361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781c64d0 msgr2=0x55c2785ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781c64d0 0x55c2785ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9d77fe640 1 -- 172.21.15.78:0/2364998361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 msgr2=0x55c278610af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:46.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c278610af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9d77fe640 1 -- 172.21.15.78:0/2364998361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2785a53a0 con 0x55c2781adf60 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9cffff640 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781c64d0 0x55c2785ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9d77fe640 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781adf60 0x55c2785acd80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009610 tx=0x7fe9c8097d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa68cb57e25b8b6d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/2364998361 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8021070 con 0x55c2781adf60 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/2364998361 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe9c809a920 con 0x55c2781adf60 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 1 -- 172.21.15.78:0/2364998361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c00043f0 con 0x55c2781adf60 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/2364998361 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9c80ccc60 con 0x55c2781adf60 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 10 client.15828 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 1 -- 172.21.15.78:0/2364998361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c278612080 con 0x55c2781adf60 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/2364998361 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c00043f0 con 0x55c2781adf60 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/2364998361 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe9c807b020 con 0x55c2781adf60 2024-05-21T15:16:46.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 20 client.15828 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c2781ef390 0x55c2786146b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 4 client.15828 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.253+0000 7fe9e4b91740 1 -- 172.21.15.78:0/2364998361 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c278593f70 con 0x55c2781ef390 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9cffff640 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c2781ef390 0x55c2786146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9b6ffd640 10 client.15828 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9cffff640 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c2781ef390 0x55c2786146b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1caf15e7e2d26c23 server_cookie=6e720801c711101d in_seq=0 out_seq=0 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9b6ffd640 1 -- 172.21.15.78:0/2364998361 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c278593f70 con 0x55c2781ef390 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9b6ffd640 10 client.15828 handle_command_reply: tid=0 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 client.15828 shutdown 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 -- 172.21.15.78:0/2364998361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781adf60 msgr2=0x55c2785acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781adf60 0x55c2785acd80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009610 tx=0x7fe9c8097d70 comp rx=0 tx=0).stop 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 -- 172.21.15.78:0/2364998361 shutdown_connections 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c2781ef390 0x55c2786146b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c2781ce6b0 0x55c278610af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2781c64d0 0x55c2785ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 --2- 172.21.15.78:0/2364998361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c2781adf60 0x55c2785acd80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9b6ffd640 0 client.15828 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.254+0000 7fe9e4b91740 1 -- 172.21.15.78:0/2364998361 >> 172.21.15.78:0/2364998361 conn(0x55c2781e34e0 msgr2=0x55c27858e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9e4b91740 1 -- 172.21.15.78:0/2364998361 shutdown_connections 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9e4b91740 1 -- 172.21.15.78:0/2364998361 wait complete. 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9e4b91740 20 client.15828 trim_cache size 0 max 16384 2024-05-21T15:16:46.264 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout: "c9157f69-e23b-4ae8-9245-c3b9b1a408dc": { 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c9157f69-e23b-4ae8-9245-c3b9b1a408dc", 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:16:46.265 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:16:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe9c40a2d90 msgr2=0x7fe9c40a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe9c40a2d90 0x7fe9c40a52a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006010 tx=0x7fe9d0031d10 comp rx=0 tx=0).stop 2024-05-21T15:16:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d8102210 msgr2=0x7fe9d81997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d8102210 0x7fe9d81997e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d770 tx=0x7fe9c800dc30 comp rx=0 tx=0).stop 2024-05-21T15:16:46.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/2621339100 shutdown_connections 2024-05-21T15:16:46.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d810c350 0x7fe9d819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe9c40a2d90 0x7fe9c40a52a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9d81079f0 0x7fe9d8199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 --2- 172.21.15.78:0/2621339100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9d8102210 0x7fe9d81997e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:46.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/2621339100 >> 172.21.15.78:0/2621339100 conn(0x7fe9d8076830 msgr2=0x7fe9d8100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:46.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/2621339100 shutdown_connections 2024-05-21T15:16:46.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:16:46.255+0000 7fe9ddc0d640 1 -- 172.21.15.78:0/2621339100 wait complete. 2024-05-21T15:16:46.269 INFO:tasks.cephfs.filesystem:scrub status for tag:c9157f69-e23b-4ae8-9245-c3b9b1a408dc - {'path': '/', 'tag': 'c9157f69-e23b-4ae8-9245-c3b9b1a408dc', 'options': 'recursive,force'} 2024-05-21T15:16:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:46 smithi078 ceph-mon[31997]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:16:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:46 smithi172 ceph-mon[36402]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:16:46.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:46 smithi084 ceph-mon[36334]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-05-21T15:16:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:47 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:16:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:47 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:47.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:47 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:16:47.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:47 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:47.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:47 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:16:47.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:47 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:16:47.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:47 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:47.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:47 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:16:47.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:47 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:47.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:47 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:16:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:47 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:16:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:47 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:47 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:16:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:47 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:16:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:47 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:16:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:48 smithi172 ceph-mon[36402]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-21T15:16:48.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:48 smithi078 ceph-mon[31997]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-21T15:16:48.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:48 smithi084 ceph-mon[36334]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-21T15:16:50.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:50 smithi078 ceph-mon[31997]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-21T15:16:50.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:50 smithi084 ceph-mon[36334]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-21T15:16:50.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:50 smithi172 ceph-mon[36402]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-21T15:16:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:52 smithi078 ceph-mon[31997]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-21T15:16:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:52 smithi084 ceph-mon[36334]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-21T15:16:52.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:52 smithi172 ceph-mon[36402]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-21T15:16:54.735 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:54 smithi172 ceph-mon[36402]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-21T15:16:54.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:54 smithi078 ceph-mon[31997]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-21T15:16:54.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:54 smithi084 ceph-mon[36334]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-05-21T15:16:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 -- 172.21.15.78:0/2094972602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 msgr2=0x7f6c3c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 --2- 172.21.15.78:0/2094972602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 0x7f6c3c102790 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009900 tx=0x7f6c3802a410 comp rx=0 tx=0).stop 2024-05-21T15:16:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 -- 172.21.15.78:0/2094972602 shutdown_connections 2024-05-21T15:16:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 --2- 172.21.15.78:0/2094972602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3c106240 0x7f6c3c106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 --2- 172.21.15.78:0/2094972602 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c3c106cc0 0x7f6c3c10e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 --2- 172.21.15.78:0/2094972602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 0x7f6c3c102790 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 -- 172.21.15.78:0/2094972602 >> 172.21.15.78:0/2094972602 conn(0x7f6c3c0fc8a0 msgr2=0x7f6c3c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 -- 172.21.15.78:0/2094972602 shutdown_connections 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.338+0000 7f6c443c7640 1 -- 172.21.15.78:0/2094972602 wait complete. 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c443c7640 1 Processor -- start 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c443c7640 1 -- start start 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c443c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 0x7f6c3c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c443c7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c3c106240 0x7f6c3c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c443c7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3c106cc0 0x7f6c3c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c443c7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c111d10 con 0x7f6c3c102380 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c443c7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c111b90 con 0x7f6c3c106cc0 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c443c7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c111e90 con 0x7f6c3c106240 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c42163640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 0x7f6c3c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c42163640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 0x7f6c3c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38698/0 (socket says 172.21.15.78:38698) 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.339+0000 7f6c42163640 1 -- 172.21.15.78:0/3414278409 learned_addr learned my addr 172.21.15.78:0/3414278409 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c42964640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3c106cc0 0x7f6c3c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c41962640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c3c106240 0x7f6c3c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c42163640 1 -- 172.21.15.78:0/3414278409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c3c106240 msgr2=0x7f6c3c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c42163640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c3c106240 0x7f6c3c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c42163640 1 -- 172.21.15.78:0/3414278409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3c106cc0 msgr2=0x7f6c3c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c42163640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3c106cc0 0x7f6c3c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c42163640 1 -- 172.21.15.78:0/3414278409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c19d590 con 0x7f6c3c102380 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c41962640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c3c106240 0x7f6c3c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c42163640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 0x7f6c3c198730 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6c38005940 tx=0x7f6c38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c2f7fe640 1 -- 172.21.15.78:0/3414278409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3803f030 con 0x7f6c3c102380 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c2f7fe640 1 -- 172.21.15.78:0/3414278409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38003ad0 con 0x7f6c3c102380 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c443c7640 1 -- 172.21.15.78:0/3414278409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c19d820 con 0x7f6c3c102380 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c2f7fe640 1 -- 172.21.15.78:0/3414278409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38035070 con 0x7f6c3c102380 2024-05-21T15:16:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.340+0000 7f6c443c7640 1 -- 172.21.15.78:0/3414278409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c19dd10 con 0x7f6c3c102380 2024-05-21T15:16:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c2f7fe640 1 -- 172.21.15.78:0/3414278409 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6c38042030 con 0x7f6c3c102380 2024-05-21T15:16:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c4b274740 1 Processor -- start 2024-05-21T15:16:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c4b274740 1 -- start start 2024-05-21T15:16:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba1cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba1d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba1d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c4b274740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5569ab982340 con 0x5569ab67e1b0 2024-05-21T15:16:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c4b274740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5569ab92a290 con 0x5569ab65d4d0 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c4b274740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5569ab97e300 con 0x5569ab6552f0 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c42964640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba1cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c42964640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba1cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38700/0 (socket says 172.21.15.78:38700) 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c42964640 1 -- 172.21.15.78:0/3555039361 learned_addr learned my addr 172.21.15.78:0/3555039361 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.344+0000 7f6c41962640 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba1d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c42163640 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba1d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c2f7fe640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6c280a2d80 0x7f6c280a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c42964640 1 -- 172.21.15.78:0/3555039361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 msgr2=0x5569aba1d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c42964640 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba1d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c42964640 1 -- 172.21.15.78:0/3555039361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 msgr2=0x5569aba1d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c42964640 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba1d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c42964640 1 -- 172.21.15.78:0/3555039361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ab94e8e0 con 0x5569ab67e1b0 2024-05-21T15:16:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c41962640 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba1d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c2f7fe640 1 -- 172.21.15.78:0/3414278409 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6c380efe20 con 0x7f6c3c102380 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c42964640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6c280a2d80 0x7f6c280a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c42964640 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba1cf50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009f70 tx=0x7f6c34010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd6cbe9607125e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c2cbfe640 1 -- 172.21.15.78:0/3555039361 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34003d30 con 0x5569ab67e1b0 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c2cbfe640 1 -- 172.21.15.78:0/3555039361 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c34004640 con 0x5569ab67e1b0 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.345+0000 7f6c2cbfe640 1 -- 172.21.15.78:0/3555039361 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3401c5f0 con 0x5569ab67e1b0 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 -- 172.21.15.78:0/3555039361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 msgr2=0x5569aba1cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c42964640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6c280a2d80 0x7f6c280a5290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c3401c000 tx=0x7f6c3401cd60 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba1cf50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009f70 tx=0x7f6c34010860 comp rx=0 tx=0).stop 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 -- 172.21.15.78:0/3555039361 shutdown_connections 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba1d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba1d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 --2- 172.21.15.78:0/3555039361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba1cf50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 -- 172.21.15.78:0/3555039361 >> 172.21.15.78:0/3555039361 conn(0x5569ab672300 msgr2=0x5569aba19830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 -- 172.21.15.78:0/3555039361 shutdown_connections 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.346+0000 7f6c4b274740 1 -- 172.21.15.78:0/3555039361 wait complete. 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 1 -- start start 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 1 Processor -- start 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba29df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba35920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5569aba2c470 con 0x5569ab67e1b0 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5569ab93d790 con 0x5569ab65d4d0 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5569aba2c5f0 con 0x5569ab6552f0 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c4b274740 10 client.0 _ll_register_callbacks cb 0x5569ab978a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42964640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba35920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42964640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba35920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38710/0 (socket says 172.21.15.78:38710) 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42964640 1 -- 172.21.15.78:0/2589769791 learned_addr learned my addr 172.21.15.78:0/2589769791 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c41962640 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba29df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42163640 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42964640 1 -- 172.21.15.78:0/2589769791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 msgr2=0x5569aba298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42964640 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42964640 1 -- 172.21.15.78:0/2589769791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 msgr2=0x5569aba29df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42964640 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba29df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42964640 1 -- 172.21.15.78:0/2589769791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ab696210 con 0x5569ab67e1b0 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.347+0000 7f6c42163640 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c42964640 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba35920 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6c34003ee0 tx=0x7f6c34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d3191402db20c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c25ffb640 1 -- 172.21.15.78:0/2589769791 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34036060 con 0x5569ab67e1b0 2024-05-21T15:16:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c25ffb640 1 -- 172.21.15.78:0/2589769791 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6c34011500 con 0x5569ab67e1b0 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 1 -- 172.21.15.78:0/2589769791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c34004640 con 0x5569ab67e1b0 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c25ffb640 1 -- 172.21.15.78:0/2589769791 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c3402d040 con 0x5569ab67e1b0 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 10 client.15840 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 1 -- 172.21.15.78:0/2589769791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5569ab762620 con 0x5569ab67e1b0 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c25ffb640 1 -- 172.21.15.78:0/2589769791 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c34004640 con 0x5569ab67e1b0 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c25ffb640 1 -- 172.21.15.78:0/2589769791 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c34004040 con 0x5569ab67e1b0 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 10 client.15840 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 20 client.15840 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5569ab63cd80 0x5569aba3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 4 client.15840 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:16:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.348+0000 7f6c4b274740 1 -- 172.21.15.78:0/2589769791 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569ab98fa70 con 0x5569ab63cd80 2024-05-21T15:16:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.349+0000 7f6c42163640 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5569ab63cd80 0x5569aba3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.349+0000 7f6c25ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.349+0000 7f6c42163640 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5569ab63cd80 0x5569aba3bae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=804405cfe2a52b97 server_cookie=7ab753fcc8fa8441 in_seq=0 out_seq=0 2024-05-21T15:16:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c25ffb640 1 -- 172.21.15.78:0/2589769791 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569ab98fa70 con 0x5569ab63cd80 2024-05-21T15:16:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c25ffb640 10 client.15840 handle_command_reply: tid=0 2024-05-21T15:16:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 client.15840 shutdown 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 -- 172.21.15.78:0/2589769791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 msgr2=0x5569aba35920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba35920 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6c34003ee0 tx=0x7f6c34017000 comp rx=0 tx=0).stop 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 -- 172.21.15.78:0/2589769791 shutdown_connections 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5569ab63cd80 0x5569aba3bae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba29df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569aba35920 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 --2- 172.21.15.78:0/2589769791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c25ffb640 0 client.15840 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 -- 172.21.15.78:0/2589769791 >> 172.21.15.78:0/2589769791 conn(0x5569ab672300 msgr2=0x5569aba1f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 -- 172.21.15.78:0/2589769791 shutdown_connections 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 1 -- 172.21.15.78:0/2589769791 wait complete. 2024-05-21T15:16:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.350+0000 7f6c4b274740 20 client.15840 trim_cache size 0 max 16384 2024-05-21T15:16:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.365+0000 7f6c4b274740 1 Processor -- start 2024-05-21T15:16:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.365+0000 7f6c4b274740 1 -- start start 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.365+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab6552f0 0x5569aba199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab65d4d0 0x5569aba230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab67e1b0 0x5569aba23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c4b274740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1ac9a0 con 0x5569ab65d4d0 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c4b274740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1ae4e0 con 0x5569ab6552f0 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c4b274740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1ad070 con 0x5569ab67e1b0 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab65d4d0 0x5569aba230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab65d4d0 0x5569aba230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38714/0 (socket says 172.21.15.78:38714) 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 -- 172.21.15.78:0/1682479656 learned_addr learned my addr 172.21.15.78:0/1682479656 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c42163640 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab6552f0 0x5569aba199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c42964640 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab67e1b0 0x5569aba23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 -- 172.21.15.78:0/1682479656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab67e1b0 msgr2=0x5569aba23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab67e1b0 0x5569aba23600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 -- 172.21.15.78:0/1682479656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab6552f0 msgr2=0x5569aba199d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab6552f0 0x5569aba199d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 -- 172.21.15.78:0/1682479656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569aba1e440 con 0x5569ab65d4d0 2024-05-21T15:16:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c41962640 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab65d4d0 0x5569aba230c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c06ad80 tx=0x7f6c300127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51fa611cd24a24e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c27fff640 1 -- 172.21.15.78:0/1682479656 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3001f070 con 0x5569ab65d4d0 2024-05-21T15:16:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c27fff640 1 -- 172.21.15.78:0/1682479656 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3001a8d0 con 0x5569ab65d4d0 2024-05-21T15:16:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.366+0000 7f6c27fff640 1 -- 172.21.15.78:0/1682479656 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300109f0 con 0x5569ab65d4d0 2024-05-21T15:16:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 -- 172.21.15.78:0/1682479656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab65d4d0 msgr2=0x5569aba230c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab65d4d0 0x5569aba230c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c06ad80 tx=0x7f6c300127c0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 -- 172.21.15.78:0/1682479656 shutdown_connections 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab67e1b0 0x5569aba23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab65d4d0 0x5569aba230c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 --2- 172.21.15.78:0/1682479656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab6552f0 0x5569aba199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 -- 172.21.15.78:0/1682479656 >> 172.21.15.78:0/1682479656 conn(0x5569ab672300 msgr2=0x5569aba192c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 -- 172.21.15.78:0/1682479656 shutdown_connections 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.367+0000 7f6c4b274740 1 -- 172.21.15.78:0/1682479656 wait complete. 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 1 -- start start 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 1 Processor -- start 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba39d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569abaa1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1ade10 con 0x5569ab67e1b0 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1ad740 con 0x5569ab65d4d0 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1aa960 con 0x5569ab6552f0 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c4b274740 10 client.0 _ll_register_callbacks cb 0x7f6c3402a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42964640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569abaa1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42964640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569abaa1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38716/0 (socket says 172.21.15.78:38716) 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42964640 1 -- 172.21.15.78:0/53336497 learned_addr learned my addr 172.21.15.78:0/53336497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c41962640 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42163640 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba39d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42964640 1 -- 172.21.15.78:0/53336497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 msgr2=0x5569aba39d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42964640 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba39d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42964640 1 -- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 msgr2=0x5569aba3a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:16:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42964640 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba3a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c42964640 1 -- 172.21.15.78:0/53336497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569abaa2130 con 0x5569ab67e1b0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.368+0000 7f6c41962640 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c42964640 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569abaa1bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6c340108a0 tx=0x7f6c34010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f569e3c6209bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c25ffb640 1 -- 172.21.15.78:0/53336497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34036040 con 0x5569ab67e1b0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c25ffb640 1 -- 172.21.15.78:0/53336497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6c34003f30 con 0x5569ab67e1b0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 1 -- 172.21.15.78:0/53336497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3001a8d0 con 0x5569ab67e1b0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c25ffb640 1 -- 172.21.15.78:0/53336497 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c34050050 con 0x5569ab67e1b0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 10 client.15846 fetch_fsmap learned FSMap version 26 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 1 -- 172.21.15.78:0/53336497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5569abaa32f0 con 0x5569ab67e1b0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c25ffb640 1 -- 172.21.15.78:0/53336497 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3001a8d0 con 0x5569ab67e1b0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c25ffb640 1 -- 172.21.15.78:0/53336497 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c34011500 con 0x5569ab67e1b0 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 10 client.15846 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:16:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:16:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 20 client.15846 populate_metadata read hostname 'smithi078' 2024-05-21T15:16:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5569ab6121c0 0x5569abaa5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:16:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 4 client.15846 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:16:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.369+0000 7f6c4b274740 1 -- 172.21.15.78:0/53336497 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569ab989dc0 con 0x5569ab6121c0 2024-05-21T15:16:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.370+0000 7f6c42163640 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5569ab6121c0 0x5569abaa5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:16:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.370+0000 7f6c25ffb640 10 client.15846 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.370+0000 7f6c42163640 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5569ab6121c0 0x5569abaa5a00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f49a7a01c12ac2d server_cookie=6a33f4614913fc38 in_seq=0 out_seq=0 2024-05-21T15:16:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c25ffb640 1 -- 172.21.15.78:0/53336497 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569ab989dc0 con 0x5569ab6121c0 2024-05-21T15:16:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c25ffb640 10 client.15846 handle_command_reply: tid=0 2024-05-21T15:16:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 client.15846 shutdown 2024-05-21T15:16:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 -- 172.21.15.78:0/53336497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 msgr2=0x5569abaa1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569abaa1bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6c340108a0 tx=0x7f6c34010d60 comp rx=0 tx=0).stop 2024-05-21T15:16:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 -- 172.21.15.78:0/53336497 shutdown_connections 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5569ab6121c0 0x5569abaa5a00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5569ab65d4d0 0x5569aba3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5569ab67e1b0 0x5569abaa1bf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c25ffb640 0 client.15846 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 --2- 172.21.15.78:0/53336497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569ab6552f0 0x5569aba39d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 -- 172.21.15.78:0/53336497 >> 172.21.15.78:0/53336497 conn(0x5569ab672300 msgr2=0x5569aba27f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 -- 172.21.15.78:0/53336497 shutdown_connections 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 1 -- 172.21.15.78:0/53336497 wait complete. 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.378+0000 7f6c4b274740 20 client.15846 trim_cache size 0 max 16384 2024-05-21T15:16:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:16:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:16:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:16:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:16:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 -- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6c280a2d80 msgr2=0x7f6c280a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6c280a2d80 0x7f6c280a5290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c3401c000 tx=0x7f6c3401cd60 comp rx=0 tx=0).stop 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 -- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 msgr2=0x7f6c3c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 0x7f6c3c198730 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6c38005940 tx=0x7f6c38004270 comp rx=0 tx=0).stop 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 -- 172.21.15.78:0/3414278409 shutdown_connections 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c3c106cc0 0x7f6c3c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6c280a2d80 0x7f6c280a5290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c3c106240 0x7f6c3c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 --2- 172.21.15.78:0/3414278409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c3c102380 0x7f6c3c198730 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 -- 172.21.15.78:0/3414278409 >> 172.21.15.78:0/3414278409 conn(0x7f6c3c0fc8a0 msgr2=0x7f6c3c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 -- 172.21.15.78:0/3414278409 shutdown_connections 2024-05-21T15:16:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:16:55.379+0000 7f6c443c7640 1 -- 172.21.15.78:0/3414278409 wait complete. 2024-05-21T15:16:56.698 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:56 smithi084 ceph-mon[36334]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-21T15:16:56.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:56 smithi078 ceph-mon[31997]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-21T15:16:56.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:56 smithi172 ceph-mon[36402]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-21T15:16:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:16:58 smithi078 ceph-mon[31997]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-21T15:16:59.024 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:16:58 smithi172 ceph-mon[36402]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-21T15:16:59.024 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:16:58 smithi084 ceph-mon[36334]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-21T15:17:00.447 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:17:00 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:17:00.140+0000 7f7cc5697700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:17:00.447 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:17:00 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:17:00.140+0000 7f7cc5697700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:17:00.448 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:17:00 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:17:00.140+0000 7f7cc5697700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:17:00.448 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:17:00 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:17:00.140+0000 7f7cc5697700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:17:00.448 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:17:00 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:17:00.140+0000 7f7cc5697700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T15:17:00.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:00 smithi078 ceph-mon[31997]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T15:17:00.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:00 smithi084 ceph-mon[36334]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T15:17:00.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:00 smithi172 ceph-mon[36402]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-21T15:17:02.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:02 smithi078 ceph-mon[31997]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-21T15:17:02.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:02 smithi078 ceph-mon[31997]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-05-21T15:17:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:02 smithi084 ceph-mon[36334]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-21T15:17:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:02 smithi084 ceph-mon[36334]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-05-21T15:17:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:02 smithi172 ceph-mon[36402]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-05-21T15:17:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:02 smithi172 ceph-mon[36402]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-05-21T15:17:04.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:04 smithi078 ceph-mon[31997]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-21T15:17:04.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:04 smithi084 ceph-mon[36334]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-21T15:17:04.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:04 smithi172 ceph-mon[36402]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-21T15:17:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:06 smithi172 ceph-mon[36402]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 27 op/s 2024-05-21T15:17:06.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:06 smithi078 ceph-mon[31997]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 27 op/s 2024-05-21T15:17:06.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:06 smithi084 ceph-mon[36334]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 27 op/s 2024-05-21T15:17:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1814619449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 msgr2=0x7fbd88106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1814619449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 0x7fbd88106460 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009900 tx=0x7fbd7c02a410 comp rx=0 tx=0).stop 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1814619449 shutdown_connections 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1814619449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88106af0 0x7fbd8810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1814619449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 0x7fbd88106460 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1814619449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88101aa0 0x7fbd88101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1814619449 >> 172.21.15.78:0/1814619449 conn(0x7fbd880fd0a0 msgr2=0x7fbd880ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1814619449 shutdown_connections 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.479+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1814619449 wait complete. 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd8c99d640 1 Processor -- start 2024-05-21T15:17:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd8c99d640 1 -- start start 2024-05-21T15:17:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd8c99d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88101aa0 0x7fbd88103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd8c99d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 0x7fbd88103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd8c99d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88106af0 0x7fbd88103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd8c99d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88111b50 con 0x7fbd88106030 2024-05-21T15:17:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd8c99d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881119d0 con 0x7fbd88106af0 2024-05-21T15:17:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd8c99d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88111cd0 con 0x7fbd88101aa0 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 0x7fbd88103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 0x7fbd88103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60972/0 (socket says 172.21.15.78:60972) 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd86ffd640 1 -- 172.21.15.78:0/1921831776 learned_addr learned my addr 172.21.15.78:0/1921831776 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd877fe640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88101aa0 0x7fbd88103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.480+0000 7fbd87fff640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88106af0 0x7fbd88103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd86ffd640 1 -- 172.21.15.78:0/1921831776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88101aa0 msgr2=0x7fbd88103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88101aa0 0x7fbd88103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd86ffd640 1 -- 172.21.15.78:0/1921831776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88106af0 msgr2=0x7fbd88103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88106af0 0x7fbd88103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd86ffd640 1 -- 172.21.15.78:0/1921831776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd881a7960 con 0x7fbd88106030 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 0x7fbd88103580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005940 tx=0x7fbd7c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd84ff9640 1 -- 172.21.15.78:0/1921831776 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c03f030 con 0x7fbd88106030 2024-05-21T15:17:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1921831776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd881a7bf0 con 0x7fbd88106030 2024-05-21T15:17:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd84ff9640 1 -- 172.21.15.78:0/1921831776 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7c0044c0 con 0x7fbd88106030 2024-05-21T15:17:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd84ff9640 1 -- 172.21.15.78:0/1921831776 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c035070 con 0x7fbd88106030 2024-05-21T15:17:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.481+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1921831776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd881a80e0 con 0x7fbd88106030 2024-05-21T15:17:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd94b7e740 1 Processor -- start 2024-05-21T15:17:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd84ff9640 1 -- 172.21.15.78:0/1921831776 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fbd7c042030 con 0x7fbd88106030 2024-05-21T15:17:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd94b7e740 1 -- start start 2024-05-21T15:17:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d221b0 0x55ec580c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 0x55ec580c17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 0x55ec580c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ec58026340 con 0x55ec57d221b0 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ec57fce290 con 0x55ec57d014d0 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec58022300 con 0x55ec57cf92f0 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd87fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d221b0 0x55ec580c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd87fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d221b0 0x55ec580c1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60982/0 (socket says 172.21.15.78:60982) 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd87fff640 1 -- 172.21.15.78:0/2485630491 learned_addr learned my addr 172.21.15.78:0/2485630491 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd84ff9640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd700a2e10 0x7fbd700a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 0x55ec580c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.484+0000 7fbd877fe640 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 0x55ec580c17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd87fff640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd700a2e10 0x7fbd700a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd87fff640 1 -- 172.21.15.78:0/2485630491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 msgr2=0x55ec580c17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd87fff640 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 0x55ec580c17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd87fff640 1 -- 172.21.15.78:0/2485630491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 msgr2=0x55ec580c1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd87fff640 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 0x55ec580c1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd87fff640 1 -- 172.21.15.78:0/2485630491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec57ff28e0 con 0x55ec57d221b0 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd84ff9640 1 -- 172.21.15.78:0/1921831776 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd7c0efac0 con 0x7fbd88106030 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd87fff640 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d221b0 0x55ec580c1280 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbd80014a60 tx=0x7fbd80014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178656608b0c4a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd6dffb640 1 -- 172.21.15.78:0/2485630491 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80025070 con 0x55ec57d221b0 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd6dffb640 1 -- 172.21.15.78:0/2485630491 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd80020470 con 0x55ec57d221b0 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.485+0000 7fbd6dffb640 1 -- 172.21.15.78:0/2485630491 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000e600 con 0x55ec57d221b0 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2485630491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d221b0 msgr2=0x55ec580c1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd87fff640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd700a2e10 0x7fbd700a5320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbd80022000 tx=0x7fbd8001f000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d221b0 0x55ec580c1280 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbd80014a60 tx=0x7fbd80014f20 comp rx=0 tx=0).stop 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2485630491 shutdown_connections 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 0x55ec580c1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 0x55ec580c17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2485630491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d221b0 0x55ec580c1280 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2485630491 >> 172.21.15.78:0/2485630491 conn(0x55ec57d16300 msgr2=0x55ec580bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.486+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2485630491 shutdown_connections 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2485630491 wait complete. 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 -- start start 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 Processor -- start 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57cf92f0 0x55ec580cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 0x55ec580cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ec580ce3b0 con 0x55ec57cf92f0 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ec57fe1790 con 0x55ec57d014d0 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.487+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec580ce530 con 0x55ec57d221b0 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd94b7e740 10 client.0 _ll_register_callbacks cb 0x55ec5801ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57cf92f0 0x55ec580cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57cf92f0 0x55ec580cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60994/0 (socket says 172.21.15.78:60994) 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 -- 172.21.15.78:0/3122012631 learned_addr learned my addr 172.21.15.78:0/3122012631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd87fff640 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 0x55ec580cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 -- 172.21.15.78:0/3122012631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 msgr2=0x55ec580cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 -- 172.21.15.78:0/3122012631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 msgr2=0x55ec580cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 0x55ec580cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 -- 172.21.15.78:0/3122012631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec57d3a210 con 0x55ec57cf92f0 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd87fff640 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd877fe640 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57cf92f0 0x55ec580cd2e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbd780126e0 tx=0x7fbd78012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68060865280bcf24 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd66ffd640 1 -- 172.21.15.78:0/3122012631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78012190 con 0x55ec57cf92f0 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd66ffd640 1 -- 172.21.15.78:0/3122012631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd78004020 con 0x55ec57cf92f0 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3122012631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd80020470 con 0x55ec57cf92f0 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd66ffd640 1 -- 172.21.15.78:0/3122012631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd78004220 con 0x55ec57cf92f0 2024-05-21T15:17:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd94b7e740 10 client.15864 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.488+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3122012631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec57e06620 con 0x55ec57cf92f0 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd66ffd640 1 -- 172.21.15.78:0/3122012631 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd80020470 con 0x55ec57cf92f0 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd66ffd640 1 -- 172.21.15.78:0/3122012631 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbd780038f0 con 0x55ec57cf92f0 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd94b7e740 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd94b7e740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd94b7e740 20 client.15864 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ec57ce0d80 0x55ec580ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd94b7e740 4 client.15864 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3122012631 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec58033a70 con 0x55ec57ce0d80 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.489+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ec57ce0d80 0x55ec580ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.490+0000 7fbd66ffd640 10 client.15864 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.490+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ec57ce0d80 0x55ec580ddc10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4214291f6b0a9c0 server_cookie=fb7b0928a74539f in_seq=0 out_seq=0 2024-05-21T15:17:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.490+0000 7fbd66ffd640 1 -- 172.21.15.78:0/3122012631 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec58033a70 con 0x55ec57ce0d80 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.490+0000 7fbd66ffd640 10 client.15864 handle_command_reply: tid=0 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.490+0000 7fbd94b7e740 1 client.15864 shutdown 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3122012631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57cf92f0 msgr2=0x55ec580cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57cf92f0 0x55ec580cd2e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbd780126e0 tx=0x7fbd78012ba0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3122012631 shutdown_connections 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ec57ce0d80 0x55ec580ddc10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d014d0 0x55ec580cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd66ffd640 0 client.15864 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3122012631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57cf92f0 0x55ec580cd2e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3122012631 >> 172.21.15.78:0/3122012631 conn(0x55ec57d16300 msgr2=0x55ec580c1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3122012631 shutdown_connections 2024-05-21T15:17:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3122012631 wait complete. 2024-05-21T15:17:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.491+0000 7fbd94b7e740 20 client.15864 trim_cache size 0 max 16384 2024-05-21T15:17:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.508+0000 7fbd94b7e740 1 Processor -- start 2024-05-21T15:17:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.508+0000 7fbd94b7e740 1 -- start start 2024-05-21T15:17:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.508+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 0x55ec580c1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec580d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d221b0 0x55ec580d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881b0ad0 con 0x55ec57d014d0 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881b2610 con 0x55ec57d221b0 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881b11a0 con 0x55ec57cf92f0 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec580d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec580d37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32772/0 (socket says 172.21.15.78:32772) 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 -- 172.21.15.78:0/3504821821 learned_addr learned my addr 172.21.15.78:0/3504821821 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd87fff640 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d221b0 0x55ec580d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd877fe640 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 0x55ec580c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 -- 172.21.15.78:0/3504821821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 msgr2=0x55ec580c1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 0x55ec580c1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 -- 172.21.15.78:0/3504821821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d221b0 msgr2=0x55ec580d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d221b0 0x55ec580d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 -- 172.21.15.78:0/3504821821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec580d4240 con 0x55ec57d014d0 2024-05-21T15:17:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec580d37c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0bbdf0 tx=0x7fbd7c0b8230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6823d1d5eebcdced server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd6d7fa640 1 -- 172.21.15.78:0/3504821821 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c049070 con 0x55ec57d014d0 2024-05-21T15:17:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd6d7fa640 1 -- 172.21.15.78:0/3504821821 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7c0c0210 con 0x55ec57d014d0 2024-05-21T15:17:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.509+0000 7fbd6d7fa640 1 -- 172.21.15.78:0/3504821821 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0bc0c0 con 0x55ec57d014d0 2024-05-21T15:17:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3504821821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 msgr2=0x55ec580d37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec580d37c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0bbdf0 tx=0x7fbd7c0b8230 comp rx=0 tx=0).stop 2024-05-21T15:17:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3504821821 shutdown_connections 2024-05-21T15:17:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57d221b0 0x55ec580d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec580d37c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/3504821821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57cf92f0 0x55ec580c1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3504821821 >> 172.21.15.78:0/3504821821 conn(0x55ec57d16300 msgr2=0x55ec580c1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3504821821 shutdown_connections 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.510+0000 7fbd94b7e740 1 -- 172.21.15.78:0/3504821821 wait complete. 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 1 -- start start 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 1 Processor -- start 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57cf92f0 0x55ec5813e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec5813ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881b1f40 con 0x55ec57d014d0 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881b1870 con 0x55ec57cf92f0 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881aeab0 con 0x55ec57d221b0 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd94b7e740 10 client.0 _ll_register_callbacks cb 0x7fbd78009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.511+0000 7fbd86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec5813ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec5813ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32776/0 (socket says 172.21.15.78:32776) 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd86ffd640 1 -- 172.21.15.78:0/2868219199 learned_addr learned my addr 172.21.15.78:0/2868219199 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd877fe640 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57cf92f0 0x55ec5813e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd87fff640 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd86ffd640 1 -- 172.21.15.78:0/2868219199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 msgr2=0x55ec580d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580d2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd86ffd640 1 -- 172.21.15.78:0/2868219199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57cf92f0 msgr2=0x55ec5813e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57cf92f0 0x55ec5813e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd86ffd640 1 -- 172.21.15.78:0/2868219199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec580d35e0 con 0x55ec57d014d0 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd86ffd640 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec5813ee40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0b82c0 tx=0x7fbd7c0bbd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8eb7b4a944d365 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd66ffd640 1 -- 172.21.15.78:0/2868219199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0d8030 con 0x55ec57d014d0 2024-05-21T15:17:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd66ffd640 1 -- 172.21.15.78:0/2868219199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd7c0bc0c0 con 0x55ec57d014d0 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2868219199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7c0c0210 con 0x55ec57d014d0 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd66ffd640 1 -- 172.21.15.78:0/2868219199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd7c049050 con 0x55ec57d014d0 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd94b7e740 10 client.15870 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2868219199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec58145fc0 con 0x55ec57d014d0 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.512+0000 7fbd66ffd640 1 -- 172.21.15.78:0/2868219199 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7c0c0210 con 0x55ec57d014d0 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd66ffd640 1 -- 172.21.15.78:0/2868219199 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbd7c0c4640 con 0x55ec57d014d0 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd94b7e740 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd94b7e740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd94b7e740 20 client.15870 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ec57cb61c0 0x55ec58148660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd94b7e740 4 client.15870 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2868219199 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec5802ddc0 con 0x55ec57cb61c0 2024-05-21T15:17:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd877fe640 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ec57cb61c0 0x55ec58148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd66ffd640 10 client.15870 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.513+0000 7fbd877fe640 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ec57cb61c0 0x55ec58148660 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceee01fe9bd533d0 server_cookie=ab03704931768cb5 in_seq=0 out_seq=0 2024-05-21T15:17:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd66ffd640 1 -- 172.21.15.78:0/2868219199 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec5802ddc0 con 0x55ec57cb61c0 2024-05-21T15:17:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd66ffd640 10 client.15870 handle_command_reply: tid=0 2024-05-21T15:17:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 client.15870 shutdown 2024-05-21T15:17:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2868219199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 msgr2=0x55ec5813ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec5813ee40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0b82c0 tx=0x7fbd7c0bbd00 comp rx=0 tx=0).stop 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2868219199 shutdown_connections 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ec57cb61c0 0x55ec58148660 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec57d221b0 0x55ec580d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec57d014d0 0x55ec5813ee40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd66ffd640 0 client.15870 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 --2- 172.21.15.78:0/2868219199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec57cf92f0 0x55ec5813e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2868219199 >> 172.21.15.78:0/2868219199 conn(0x55ec57d16300 msgr2=0x55ec580c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2868219199 shutdown_connections 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 1 -- 172.21.15.78:0/2868219199 wait complete. 2024-05-21T15:17:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.520+0000 7fbd94b7e740 20 client.15870 trim_cache size 0 max 16384 2024-05-21T15:17:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:17:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:17:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:17:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd700a2e10 msgr2=0x7fbd700a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd700a2e10 0x7fbd700a5320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbd80022000 tx=0x7fbd8001f000 comp rx=0 tx=0).stop 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 msgr2=0x7fbd88103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 0x7fbd88103580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005940 tx=0x7fbd7c0040c0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1921831776 shutdown_connections 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88106af0 0x7fbd88103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd700a2e10 0x7fbd700a5320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88106030 0x7fbd88103580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 --2- 172.21.15.78:0/1921831776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88101aa0 0x7fbd88103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1921831776 >> 172.21.15.78:0/1921831776 conn(0x7fbd880fd0a0 msgr2=0x7fbd880ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1921831776 shutdown_connections 2024-05-21T15:17:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:08.521+0000 7fbd8c99d640 1 -- 172.21.15.78:0/1921831776 wait complete. 2024-05-21T15:17:08.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:08 smithi078 ceph-mon[31997]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-21T15:17:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:08 smithi084 ceph-mon[36334]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-21T15:17:08.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:08 smithi172 ceph-mon[36402]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-05-21T15:17:09.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:09 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]: dispatch 2024-05-21T15:17:09.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:09 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-05-21T15:17:09.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:09 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-21T15:17:09.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:09 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]: dispatch 2024-05-21T15:17:09.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:09 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-05-21T15:17:09.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:09 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-21T15:17:09.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:09 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]: dispatch 2024-05-21T15:17:09.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:09 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-05-21T15:17:09.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:09 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-05-21T15:17:10.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:10 smithi078 ceph-mon[31997]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-21T15:17:10.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:10 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]': finished 2024-05-21T15:17:10.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:10 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-05-21T15:17:10.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:10 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-21T15:17:10.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:10 smithi078 ceph-mon[31997]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T15:17:10.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:10 smithi084 ceph-mon[36334]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-21T15:17:10.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:10 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]': finished 2024-05-21T15:17:10.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:10 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-05-21T15:17:10.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:10 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-21T15:17:10.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:10 smithi084 ceph-mon[36334]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T15:17:10.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:10 smithi172 ceph-mon[36402]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-21T15:17:10.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:10 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 9]}]': finished 2024-05-21T15:17:10.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:10 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-05-21T15:17:10.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:10 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-05-21T15:17:10.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:10 smithi172 ceph-mon[36402]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T15:17:11.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:11 smithi078 ceph-mon[31997]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T15:17:11.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:11 smithi078 ceph-mon[31997]: pgmap v36: 97 pgs: 3 peering, 94 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 37 op/s 2024-05-21T15:17:11.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:11 smithi084 ceph-mon[36334]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T15:17:11.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:11 smithi084 ceph-mon[36334]: pgmap v36: 97 pgs: 3 peering, 94 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 37 op/s 2024-05-21T15:17:11.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:11 smithi172 ceph-mon[36402]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T15:17:11.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:11 smithi172 ceph-mon[36402]: pgmap v36: 97 pgs: 3 peering, 94 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 35 KiB/s wr, 37 op/s 2024-05-21T15:17:12.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:12 smithi078 ceph-mon[31997]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-21T15:17:12.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:12 smithi084 ceph-mon[36334]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-21T15:17:12.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:12 smithi172 ceph-mon[36402]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-21T15:17:13.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:13 smithi078 ceph-mon[31997]: pgmap v37: 97 pgs: 3 peering, 94 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-21T15:17:13.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:13 smithi084 ceph-mon[36334]: pgmap v37: 97 pgs: 3 peering, 94 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-21T15:17:13.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:13 smithi172 ceph-mon[36402]: pgmap v37: 97 pgs: 3 peering, 94 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-21T15:17:14.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:14 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:17:14.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:14 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:17:14.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:14 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:17:16.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:17:16.271 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:17:16.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 -- 172.21.15.78:0/703973379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 msgr2=0x7efe9010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 --2- 172.21.15.78:0/703973379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 0x7efe9010e750 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efe88009ca0 tx=0x7efe8802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 -- 172.21.15.78:0/703973379 shutdown_connections 2024-05-21T15:17:16.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 --2- 172.21.15.78:0/703973379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 0x7efe9010e750 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 --2- 172.21.15.78:0/703973379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe90101aa0 0x7efe90101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 --2- 172.21.15.78:0/703973379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe901079d0 0x7efe90101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 -- 172.21.15.78:0/703973379 >> 172.21.15.78:0/703973379 conn(0x7efe900770b0 msgr2=0x7efe900774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:16.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 -- 172.21.15.78:0/703973379 shutdown_connections 2024-05-21T15:17:16.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.405+0000 7efe94c37640 1 -- 172.21.15.78:0/703973379 wait complete. 2024-05-21T15:17:16.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.406+0000 7efe94c37640 1 Processor -- start 2024-05-21T15:17:16.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.406+0000 7efe94c37640 1 -- start start 2024-05-21T15:17:16.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe94c37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe90101aa0 0x7efe90199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe94c37640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe901079d0 0x7efe90199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe94c37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 0x7efe9019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe94c37640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe90112e00 con 0x7efe9010c330 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe94c37640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe90112c80 con 0x7efe90101aa0 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe94c37640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe90112f80 con 0x7efe901079d0 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe901079d0 0x7efe90199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 0x7efe9019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe901079d0 0x7efe90199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:52194/0 (socket says 172.21.15.78:52194) 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 0x7efe9019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46164/0 (socket says 172.21.15.78:46164) 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8dd9b640 1 -- 172.21.15.78:0/2062774286 learned_addr learned my addr 172.21.15.78:0/2062774286 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8e59c640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe90101aa0 0x7efe90199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8ed9d640 1 -- 172.21.15.78:0/2062774286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe901079d0 msgr2=0x7efe90199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8ed9d640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe901079d0 0x7efe90199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8ed9d640 1 -- 172.21.15.78:0/2062774286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe90101aa0 msgr2=0x7efe90199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8ed9d640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe90101aa0 0x7efe90199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8ed9d640 1 -- 172.21.15.78:0/2062774286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9019e5c0 con 0x7efe9010c330 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.407+0000 7efe8e59c640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe90101aa0 0x7efe90199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.408+0000 7efe8ed9d640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 0x7efe9019df80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efe8802ade0 tx=0x7efe880319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.408+0000 7efe7f7fe640 1 -- 172.21.15.78:0/2062774286 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88003b70 con 0x7efe9010c330 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.408+0000 7efe94c37640 1 -- 172.21.15.78:0/2062774286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9019e850 con 0x7efe9010c330 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.408+0000 7efe7f7fe640 1 -- 172.21.15.78:0/2062774286 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe88003d10 con 0x7efe9010c330 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.408+0000 7efe7f7fe640 1 -- 172.21.15.78:0/2062774286 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8802fcb0 con 0x7efe9010c330 2024-05-21T15:17:16.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.408+0000 7efe94c37640 1 -- 172.21.15.78:0/2062774286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe901a6230 con 0x7efe9010c330 2024-05-21T15:17:16.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.411+0000 7efe9bbb2740 1 Processor -- start 2024-05-21T15:17:16.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe7f7fe640 1 -- 172.21.15.78:0/2062774286 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7efe88004050 con 0x7efe9010c330 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe9bbb2740 1 -- start start 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef66850390 0x55ef66beed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66bef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef66b54d00 con 0x55ef66850390 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ef66afc4c0 con 0x55ef6682f6b0 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ef66b50cc0 con 0x55ef668274d0 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef66850390 0x55ef66beed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef66850390 0x55ef66beed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46180/0 (socket says 172.21.15.78:46180) 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.412+0000 7efe8dd9b640 1 -- 172.21.15.78:0/113510643 learned_addr learned my addr 172.21.15.78:0/113510643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8e59c640 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66bef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe7f7fe640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe780a2ea0 0x7efe780a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8dd9b640 1 -- 172.21.15.78:0/113510643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 msgr2=0x55ef66bef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66bef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8dd9b640 1 -- 172.21.15.78:0/113510643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 msgr2=0x55ef66bef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8ed9d640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe780a2ea0 0x7efe780a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8dd9b640 1 -- 172.21.15.78:0/113510643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef669327d0 con 0x55ef66850390 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe7f7fe640 1 -- 172.21.15.78:0/2062774286 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7efe880f0850 con 0x7efe9010c330 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef66850390 0x55ef66beed30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efe84012630 tx=0x7efe8400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed2d6d28ce2e2ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe7cff9640 1 -- 172.21.15.78:0/113510643 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400e510 con 0x55ef66850390 2024-05-21T15:17:16.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe7cff9640 1 -- 172.21.15.78:0/113510643 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe8400eb90 con 0x55ef66850390 2024-05-21T15:17:16.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.413+0000 7efe7cff9640 1 -- 172.21.15.78:0/113510643 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84010aa0 con 0x55ef66850390 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 -- 172.21.15.78:0/113510643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef66850390 msgr2=0x55ef66beed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe8ed9d640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe780a2ea0 0x7efe780a53b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe880edf30 tx=0x7efe8802a8c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef66850390 0x55ef66beed30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efe84012630 tx=0x7efe8400b8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 -- 172.21.15.78:0/113510643 shutdown_connections 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66bef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/113510643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef66850390 0x55ef66beed30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 -- 172.21.15.78:0/113510643 >> 172.21.15.78:0/113510643 conn(0x55ef668444e0 msgr2=0x55ef66beb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 -- 172.21.15.78:0/113510643 shutdown_connections 2024-05-21T15:17:16.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.414+0000 7efe9bbb2740 1 -- 172.21.15.78:0/113510643 wait complete. 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 1 -- start start 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 1 Processor -- start 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668274d0 0x55ef66bfad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bfb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef66850390 0x55ef66bf9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef66c00490 con 0x55ef668274d0 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ef66b0fb90 con 0x55ef6682f6b0 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ef66c00610 con 0x55ef66850390 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe9bbb2740 10 client.0 _ll_register_callbacks cb 0x55ef66b4bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668274d0 0x55ef66bfad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.415+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668274d0 0x55ef66bfad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46184/0 (socket says 172.21.15.78:46184) 2024-05-21T15:17:16.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8dd9b640 1 -- 172.21.15.78:0/1055242456 learned_addr learned my addr 172.21.15.78:0/1055242456 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8e59c640 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bfb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef66850390 0x55ef66bf9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8dd9b640 1 -- 172.21.15.78:0/1055242456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef66850390 msgr2=0x55ef66bf9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef66850390 0x55ef66bf9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8dd9b640 1 -- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 msgr2=0x55ef66bfb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bfb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8dd9b640 1 -- 172.21.15.78:0/1055242456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef668683f0 con 0x55ef668274d0 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8e59c640 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bfb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668274d0 0x55ef66bfad70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe84005d00 tx=0x7efe84010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75de82f16f552f75 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe75ffb640 1 -- 172.21.15.78:0/1055242456 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400e710 con 0x55ef668274d0 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe75ffb640 1 -- 172.21.15.78:0/1055242456 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7efe8400e8b0 con 0x55ef668274d0 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.416+0000 7efe9bbb2740 1 -- 172.21.15.78:0/1055242456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe8400eb90 con 0x55ef668274d0 2024-05-21T15:17:16.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe75ffb640 1 -- 172.21.15.78:0/1055242456 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe84005040 con 0x55ef668274d0 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe9bbb2740 10 client.15882 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe9bbb2740 1 -- 172.21.15.78:0/1055242456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef668280f0 con 0x55ef668274d0 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe75ffb640 1 -- 172.21.15.78:0/1055242456 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe8400eb90 con 0x55ef668274d0 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe75ffb640 1 -- 172.21.15.78:0/1055242456 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efe84004660 con 0x55ef668274d0 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe9bbb2740 10 client.15882 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe9bbb2740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe9bbb2740 20 client.15882 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ef6680ef60 0x55ef66c0ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe9bbb2740 4 client.15882 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe9bbb2740 1 -- 172.21.15.78:0/1055242456 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef66b62430 con 0x55ef6680ef60 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.417+0000 7efe8e59c640 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ef6680ef60 0x55ef66c0ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.418+0000 7efe8e59c640 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ef6680ef60 0x55ef66c0ff90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14ffd112fa19cfb server_cookie=2da0e75786e557e5 in_seq=0 out_seq=0 2024-05-21T15:17:16.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.418+0000 7efe75ffb640 10 client.15882 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe75ffb640 1 -- 172.21.15.78:0/1055242456 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef66b62430 con 0x55ef6680ef60 2024-05-21T15:17:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe75ffb640 10 client.15882 handle_command_reply: tid=0 2024-05-21T15:17:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 client.15882 shutdown 2024-05-21T15:17:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 -- 172.21.15.78:0/1055242456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668274d0 msgr2=0x55ef66bfad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668274d0 0x55ef66bfad70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7efe84005d00 tx=0x7efe84010680 comp rx=0 tx=0).stop 2024-05-21T15:17:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 -- 172.21.15.78:0/1055242456 shutdown_connections 2024-05-21T15:17:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ef6680ef60 0x55ef66c0ff90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6682f6b0 0x55ef66bfb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe75ffb640 0 client.15882 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef66850390 0x55ef66bf9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/1055242456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668274d0 0x55ef66bfad70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 -- 172.21.15.78:0/1055242456 >> 172.21.15.78:0/1055242456 conn(0x55ef668444e0 msgr2=0x55ef66beeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 -- 172.21.15.78:0/1055242456 shutdown_connections 2024-05-21T15:17:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 1 -- 172.21.15.78:0/1055242456 wait complete. 2024-05-21T15:17:16.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.419+0000 7efe9bbb2740 20 client.15882 trim_cache size 0 max 16384 2024-05-21T15:17:16.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.437+0000 7efe9bbb2740 1 Processor -- start 2024-05-21T15:17:16.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.437+0000 7efe9bbb2740 1 -- start start 2024-05-21T15:17:16.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668444e0 0x55ef66bf81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6680ef60 0x55ef66bf4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66bfb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe901aca60 con 0x55ef668444e0 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe901ae5a0 con 0x55ef6680ef60 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe901ad130 con 0x55ef668274d0 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668444e0 0x55ef66bf81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668444e0 0x55ef66bf81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46198/0 (socket says 172.21.15.78:46198) 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 -- 172.21.15.78:0/3110732142 learned_addr learned my addr 172.21.15.78:0/3110732142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8e59c640 1 --2- 172.21.15.78:0/3110732142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6680ef60 0x55ef66bf4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 -- 172.21.15.78:0/3110732142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 msgr2=0x55ef66bfb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/3110732142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66bfb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 -- 172.21.15.78:0/3110732142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6680ef60 msgr2=0x55ef66bf4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/3110732142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6680ef60 0x55ef66bf4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 -- 172.21.15.78:0/3110732142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef66c002e0 con 0x55ef668444e0 2024-05-21T15:17:16.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.438+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/3110732142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668444e0 0x55ef66bf81c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efe8400b450 tx=0x7efe84013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce11329a5de77d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:16.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe77fff640 1 -- 172.21.15.78:0/3110732142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe840122e0 con 0x55ef668444e0 2024-05-21T15:17:16.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe77fff640 1 -- 172.21.15.78:0/3110732142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe84029040 con 0x55ef668444e0 2024-05-21T15:17:16.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe77fff640 1 -- 172.21.15.78:0/3110732142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400e770 con 0x55ef668444e0 2024-05-21T15:17:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe9bbb2740 1 -- 172.21.15.78:0/3110732142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668444e0 msgr2=0x55ef66bf81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/3110732142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668444e0 0x55ef66bf81c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efe8400b450 tx=0x7efe84013f90 comp rx=0 tx=0).stop 2024-05-21T15:17:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe9bbb2740 1 -- 172.21.15.78:0/3110732142 shutdown_connections 2024-05-21T15:17:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/3110732142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef6680ef60 0x55ef66bf4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/3110732142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66bfb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/3110732142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef668444e0 0x55ef66bf81c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.439+0000 7efe9bbb2740 1 -- 172.21.15.78:0/3110732142 >> 172.21.15.78:0/3110732142 conn(0x55ef66bf4c00 msgr2=0x55ef66bfa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.440+0000 7efe9bbb2740 1 -- 172.21.15.78:0/3110732142 shutdown_connections 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.440+0000 7efe9bbb2740 1 -- 172.21.15.78:0/3110732142 wait complete. 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.440+0000 7efe9bbb2740 1 -- start start 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.440+0000 7efe9bbb2740 1 Processor -- start 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6680ef60 0x55ef66c025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66c02b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe9bbb2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef668444e0 0x55ef66c03070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe901aded0 con 0x55ef6680ef60 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe901ad800 con 0x55ef668444e0 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe9bbb2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe901aaa40 con 0x55ef668274d0 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6680ef60 0x55ef66c025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6680ef60 0x55ef66c025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46204/0 (socket says 172.21.15.78:46204) 2024-05-21T15:17:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 -- 172.21.15.78:0/2376614379 learned_addr learned my addr 172.21.15.78:0/2376614379 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe9bbb2740 10 client.0 _ll_register_callbacks cb 0x7efe800095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8e59c640 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66c02b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 -- 172.21.15.78:0/2376614379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 msgr2=0x55ef66c02b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66c02b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 -- 172.21.15.78:0/2376614379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef668444e0 msgr2=0x55ef66c03070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef668444e0 0x55ef66c03070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 -- 172.21.15.78:0/2376614379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef66c762a0 con 0x55ef6680ef60 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8e59c640 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66c02b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe8dd9b640 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6680ef60 0x55ef66c025f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efe84004c50 tx=0x7efe84012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7989b5629b75dd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe75ffb640 1 -- 172.21.15.78:0/2376614379 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe840055f0 con 0x55ef6680ef60 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe75ffb640 1 -- 172.21.15.78:0/2376614379 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7efe8400e510 con 0x55ef6680ef60 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.441+0000 7efe9bbb2740 1 -- 172.21.15.78:0/2376614379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe84029040 con 0x55ef6680ef60 2024-05-21T15:17:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe75ffb640 1 -- 172.21.15.78:0/2376614379 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe8400e710 con 0x55ef6680ef60 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe9bbb2740 10 client.15888 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe9bbb2740 1 -- 172.21.15.78:0/2376614379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef66c77510 con 0x55ef6680ef60 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe75ffb640 1 -- 172.21.15.78:0/2376614379 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe84029040 con 0x55ef6680ef60 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe75ffb640 1 -- 172.21.15.78:0/2376614379 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efe84010460 con 0x55ef6680ef60 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe9bbb2740 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe9bbb2740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe9bbb2740 20 client.15888 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ef6682f6b0 0x55ef66c79ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe9bbb2740 4 client.15888 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe9bbb2740 1 -- 172.21.15.78:0/2376614379 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef66b5c780 con 0x55ef6682f6b0 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.442+0000 7efe8e59c640 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ef6682f6b0 0x55ef66c79ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe75ffb640 10 client.15888 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe8e59c640 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ef6682f6b0 0x55ef66c79ae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ad51068c8fcd687 server_cookie=bb91372a07373dba in_seq=0 out_seq=0 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe75ffb640 1 -- 172.21.15.78:0/2376614379 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef66b5c780 con 0x55ef6682f6b0 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe75ffb640 10 client.15888 handle_command_reply: tid=0 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 client.15888 shutdown 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 -- 172.21.15.78:0/2376614379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6680ef60 msgr2=0x55ef66c025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6680ef60 0x55ef66c025f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efe84004c50 tx=0x7efe84012510 comp rx=0 tx=0).stop 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 -- 172.21.15.78:0/2376614379 shutdown_connections 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ef6682f6b0 0x55ef66c79ae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef668444e0 0x55ef66c03070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe75ffb640 0 client.15888 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef668274d0 0x55ef66c02b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 --2- 172.21.15.78:0/2376614379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef6680ef60 0x55ef66c025f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.443+0000 7efe9bbb2740 1 -- 172.21.15.78:0/2376614379 >> 172.21.15.78:0/2376614379 conn(0x55ef66bf4c00 msgr2=0x55ef66bfb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe9bbb2740 1 -- 172.21.15.78:0/2376614379 shutdown_connections 2024-05-21T15:17:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe9bbb2740 1 -- 172.21.15.78:0/2376614379 wait complete. 2024-05-21T15:17:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe9bbb2740 20 client.15888 trim_cache size 0 max 16384 2024-05-21T15:17:16.452 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:17:16.452 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:17:16.452 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:17:16.452 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:17:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 -- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe780a2ea0 msgr2=0x7efe780a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe780a2ea0 0x7efe780a53b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe880edf30 tx=0x7efe8802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 -- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 msgr2=0x7efe9019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 0x7efe9019df80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efe8802ade0 tx=0x7efe880319f0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 -- 172.21.15.78:0/2062774286 shutdown_connections 2024-05-21T15:17:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe780a2ea0 0x7efe780a53b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe9010c330 0x7efe9019df80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe901079d0 0x7efe90199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 --2- 172.21.15.78:0/2062774286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe90101aa0 0x7efe90199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.444+0000 7efe94c37640 1 -- 172.21.15.78:0/2062774286 >> 172.21.15.78:0/2062774286 conn(0x7efe900770b0 msgr2=0x7efe90103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.445+0000 7efe94c37640 1 -- 172.21.15.78:0/2062774286 shutdown_connections 2024-05-21T15:17:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.445+0000 7efe94c37640 1 -- 172.21.15.78:0/2062774286 wait complete. 2024-05-21T15:17:16.458 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:17:16.459 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:17:16.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 -- 172.21.15.78:0/150296255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 msgr2=0x7f394c101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 --2- 172.21.15.78:0/150296255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 0x7f394c101ba0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3940009900 tx=0x7f394002a410 comp rx=0 tx=0).stop 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 -- 172.21.15.78:0/150296255 shutdown_connections 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 --2- 172.21.15.78:0/150296255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f394c102790 0x7f394c10e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 --2- 172.21.15.78:0/150296255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f394c107e30 0x7f394c102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 --2- 172.21.15.78:0/150296255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 0x7f394c101ba0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 -- 172.21.15.78:0/150296255 >> 172.21.15.78:0/150296255 conn(0x7f394c0fc7c0 msgr2=0x7f394c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 -- 172.21.15.78:0/150296255 shutdown_connections 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.588+0000 7f3952b65640 1 -- 172.21.15.78:0/150296255 wait complete. 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.589+0000 7f3952b65640 1 Processor -- start 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.589+0000 7f3952b65640 1 -- start start 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.589+0000 7f3952b65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 0x7f394c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.589+0000 7f3952b65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f394c107e30 0x7f394c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.589+0000 7f3952b65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f394c19dcc0 0x7f394c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.589+0000 7f3952b65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f394c112df0 con 0x7f394c107990 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.589+0000 7f3952b65640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f394c112c70 con 0x7f394c19dcc0 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.589+0000 7f3952b65640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f394c112f70 con 0x7f394c107e30 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 0x7f394c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 0x7f394c1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46228/0 (socket says 172.21.15.78:46228) 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 -- 172.21.15.78:0/2667330496 learned_addr learned my addr 172.21.15.78:0/2667330496 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f394bfff640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f394c107e30 0x7f394c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3951102640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f394c19dcc0 0x7f394c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 -- 172.21.15.78:0/2667330496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f394c107e30 msgr2=0x7f394c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f394c107e30 0x7f394c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 -- 172.21.15.78:0/2667330496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f394c19dcc0 msgr2=0x7f394c19e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f394c19dcc0 0x7f394c19e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 -- 172.21.15.78:0/2667330496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c19e710 con 0x7f394c107990 2024-05-21T15:17:16.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3951102640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f394c19dcc0 0x7f394c19e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:16.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3950901640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 0x7f394c1994a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3940005940 tx=0x7f39400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:16.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3949ffb640 1 -- 172.21.15.78:0/2667330496 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394003f030 con 0x7f394c107990 2024-05-21T15:17:16.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c19e9a0 con 0x7f394c107990 2024-05-21T15:17:16.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3949ffb640 1 -- 172.21.15.78:0/2667330496 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3940003ce0 con 0x7f394c107990 2024-05-21T15:17:16.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.590+0000 7f3949ffb640 1 -- 172.21.15.78:0/2667330496 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940035070 con 0x7f394c107990 2024-05-21T15:17:16.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.591+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c1a61a0 con 0x7f394c107990 2024-05-21T15:17:16.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.592+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c0682c0 con 0x7f394c107990 2024-05-21T15:17:16.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.594+0000 7f3949ffb640 1 -- 172.21.15.78:0/2667330496 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f3940042030 con 0x7f394c107990 2024-05-21T15:17:16.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.594+0000 7f3949ffb640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f393c0a2ea0 0x7f393c0a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:16.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.595+0000 7f3949ffb640 1 -- 172.21.15.78:0/2667330496 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f39400efec0 con 0x7f394c107990 2024-05-21T15:17:16.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.595+0000 7f394bfff640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f393c0a2ea0 0x7f393c0a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:16 smithi078 ceph-mon[31997]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s 2024-05-21T15:17:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:16 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:16 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:16 smithi078 ceph-mon[31997]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-21T15:17:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:16 smithi078 ceph-mon[31997]: Cluster is now healthy 2024-05-21T15:17:16.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:16 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:17:16.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:16 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:17:16.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:16 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.595+0000 7f394bfff640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f393c0a2ea0 0x7f393c0a53b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f394c0fdc90 tx=0x7f393800b390 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.602+0000 7f3949ffb640 1 -- 172.21.15.78:0/2667330496 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f39400bc8a0 con 0x7f394c107990 2024-05-21T15:17:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:16 smithi084 ceph-mon[36334]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s 2024-05-21T15:17:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:16 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:16 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:16 smithi084 ceph-mon[36334]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-21T15:17:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:16 smithi084 ceph-mon[36334]: Cluster is now healthy 2024-05-21T15:17:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:16 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:17:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:16 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:17:16.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:16 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:16 smithi172 ceph-mon[36402]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s 2024-05-21T15:17:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:16 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:16 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:16 smithi172 ceph-mon[36402]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-21T15:17:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:16 smithi172 ceph-mon[36402]: Cluster is now healthy 2024-05-21T15:17:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:16 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:17:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:16 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:17:16.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:16 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:17:16.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.868+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f394c1a6430 con 0x7f394c107990 2024-05-21T15:17:16.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.869+0000 7f3949ffb640 1 -- 172.21.15.78:0/2667330496 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11925 (secure 0 0 0) 0x7f39400bc1a0 con 0x7f394c107990 2024-05-21T15:17:16.872 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:17:16.872 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:16:06.684612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24419,24448,24502,24526,24527]},"id":1}]} 2024-05-21T15:17:16.872 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f393c0a2ea0 msgr2=0x7f393c0a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f393c0a2ea0 0x7f393c0a53b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f394c0fdc90 tx=0x7f393800b390 comp rx=0 tx=0).stop 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 msgr2=0x7f394c1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 0x7f394c1994a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3940005940 tx=0x7f39400042f0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 shutdown_connections 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f394c19dcc0 0x7f394c19e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f393c0a2ea0 0x7f393c0a53b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f394c107e30 0x7f394c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 --2- 172.21.15.78:0/2667330496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f394c107990 0x7f394c1994a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:16.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 >> 172.21.15.78:0/2667330496 conn(0x7f394c0fc7c0 msgr2=0x7f394c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:16.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 shutdown_connections 2024-05-21T15:17:16.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:16.872+0000 7f3952b65640 1 -- 172.21.15.78:0/2667330496 wait complete. 2024-05-21T15:17:16.890 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 -- 172.21.15.78:0/3370045510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f410c350 msgr2=0x7fe4f410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/3370045510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f410c350 0x7fe4f410e770 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ca0 tx=0x7fe4ec02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 -- 172.21.15.78:0/3370045510 shutdown_connections 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/3370045510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f410c350 0x7fe4f410e770 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/3370045510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4f4101ac0 0x7fe4f4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/3370045510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f41079f0 0x7fe4f4101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 -- 172.21.15.78:0/3370045510 >> 172.21.15.78:0/3370045510 conn(0x7fe4f4077130 msgr2=0x7fe4f4077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 -- 172.21.15.78:0/3370045510 shutdown_connections 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 -- 172.21.15.78:0/3370045510 wait complete. 2024-05-21T15:17:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.039+0000 7fe4fc54b640 1 Processor -- start 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4fc54b640 1 -- start start 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4fc54b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4f4101ac0 0x7fe4f419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4fc54b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f41079f0 0x7fe4f419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4fc54b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f410c350 0x7fe4f419b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4fc54b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4111d20 con 0x7fe4f41079f0 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4fc54b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4111ba0 con 0x7fe4f410c350 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4fc54b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4111ea0 con 0x7fe4f4101ac0 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4f9ae6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f41079f0 0x7fe4f419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4fa2e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4f4101ac0 0x7fe4f419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4f9ae6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f41079f0 0x7fe4f419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46240/0 (socket says 172.21.15.78:46240) 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/2900461201 learned_addr learned my addr 172.21.15.78:0/2900461201 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4faae8640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f410c350 0x7fe4f419b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/2900461201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4f4101ac0 msgr2=0x7fe4f419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4f4101ac0 0x7fe4f419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/2900461201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f410c350 msgr2=0x7fe4f419b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f410c350 0x7fe4f419b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.040+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/2900461201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f41a58b0 con 0x7fe4f41079f0 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.041+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f41079f0 0x7fe4f419aae0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4005d00 tx=0x7fe4e400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.041+0000 7fe4eb7fe640 1 -- 172.21.15.78:0/2900461201 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4017070 con 0x7fe4f41079f0 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.041+0000 7fe4fc54b640 1 -- 172.21.15.78:0/2900461201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f41a5ba0 con 0x7fe4f41079f0 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.041+0000 7fe4eb7fe640 1 -- 172.21.15.78:0/2900461201 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e4009c30 con 0x7fe4f41079f0 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.041+0000 7fe4eb7fe640 1 -- 172.21.15.78:0/2900461201 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400f620 con 0x7fe4f41079f0 2024-05-21T15:17:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.041+0000 7fe4fc54b640 1 -- 172.21.15.78:0/2900461201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f41a60f0 con 0x7fe4f41079f0 2024-05-21T15:17:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.044+0000 7fe4eb7fe640 1 -- 172.21.15.78:0/2900461201 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fe4e401a070 con 0x7fe4f41079f0 2024-05-21T15:17:17.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.045+0000 7fe5033f1740 1 Processor -- start 2024-05-21T15:17:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.045+0000 7fe5033f1740 1 -- start start 2024-05-21T15:17:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.045+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a79390 0x564c13e17d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.045+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 0x564c13e18260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.045+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.045+0000 7fe5033f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564c13d7dc40 con 0x564c13a586b0 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.045+0000 7fe5033f1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564c13d25580 con 0x564c13a79390 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.045+0000 7fe5033f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564c13d79c00 con 0x564c13a504d0 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4faae8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a79390 0x564c13e17d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4faae8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a79390 0x564c13e17d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:48720/0 (socket says 172.21.15.78:48720) 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4f9ae6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4faae8640 1 -- 172.21.15.78:0/2576696075 learned_addr learned my addr 172.21.15.78:0/2576696075 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4eb7fe640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4e00a2e20 0x7fe4e00a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/2576696075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 0x564c13e18260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4faae8640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4e00a2e20 0x7fe4e00a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4eb7fe640 1 -- 172.21.15.78:0/2900461201 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4e40cf110 con 0x7fe4f41079f0 2024-05-21T15:17:17.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/2576696075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 msgr2=0x564c13e18260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/2576696075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 0x564c13e18260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/2576696075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a79390 msgr2=0x564c13e17d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/2576696075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a79390 0x564c13e17d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/2576696075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c13a913f0 con 0x564c13a586b0 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4faae8640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4e00a2e20 0x7fe4e00a5330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x564c13e14e90 tx=0x7fe4ec02a8c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.046+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/2576696075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e187a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40cc680 tx=0x7fe4e400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea8578d417a8ec1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.047+0000 7fe4e8bfe640 1 -- 172.21.15.78:0/2576696075 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4021070 con 0x564c13a586b0 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.047+0000 7fe4e8bfe640 1 -- 172.21.15.78:0/2576696075 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e4009de0 con 0x564c13a586b0 2024-05-21T15:17:17.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.047+0000 7fe4e8bfe640 1 -- 172.21.15.78:0/2576696075 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e409b8f0 con 0x564c13a586b0 2024-05-21T15:17:17.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 -- 172.21.15.78:0/2576696075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 msgr2=0x564c13e187a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 --2- 172.21.15.78:0/2576696075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e187a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40cc680 tx=0x7fe4e400bed0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 -- 172.21.15.78:0/2576696075 shutdown_connections 2024-05-21T15:17:17.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 --2- 172.21.15.78:0/2576696075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e187a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 --2- 172.21.15.78:0/2576696075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 0x564c13e18260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 --2- 172.21.15.78:0/2576696075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a79390 0x564c13e17d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 -- 172.21.15.78:0/2576696075 >> 172.21.15.78:0/2576696075 conn(0x564c13a6d4e0 msgr2=0x564c13e144d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 -- 172.21.15.78:0/2576696075 shutdown_connections 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 -- 172.21.15.78:0/2576696075 wait complete. 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 -- start start 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.048+0000 7fe5033f1740 1 Processor -- start 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a504d0 0x564c13e23d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a586b0 0x564c13e242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a79390 0x564c13e22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe5033f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564c13e24fb0 con 0x564c13a79390 2024-05-21T15:17:17.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe5033f1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564c13d38c40 con 0x564c13a504d0 2024-05-21T15:17:17.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe5033f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564c13e25130 con 0x564c13a586b0 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe5033f1740 10 client.0 _ll_register_callbacks cb 0x564c13a90be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4faae8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a79390 0x564c13e22230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4fa2e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a504d0 0x564c13e23d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4faae8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a79390 0x564c13e22230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46254/0 (socket says 172.21.15.78:46254) 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4fa2e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a504d0 0x564c13e23d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:48734/0 (socket says 172.21.15.78:48734) 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4faae8640 1 -- 172.21.15.78:0/1845477332 learned_addr learned my addr 172.21.15.78:0/1845477332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a586b0 0x564c13e242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4faae8640 1 -- 172.21.15.78:0/1845477332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a586b0 msgr2=0x564c13e242a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4faae8640 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a586b0 0x564c13e242a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4faae8640 1 -- 172.21.15.78:0/1845477332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a504d0 msgr2=0x564c13e23d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4faae8640 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a504d0 0x564c13e23d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4faae8640 1 -- 172.21.15.78:0/1845477332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c13b5b7d0 con 0x564c13a79390 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.049+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a504d0 0x564c13e23d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe4faae8640 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a79390 0x564c13e22230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ca0 tx=0x7fe4ec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfcc68c7aad1290 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe4ddffb640 1 -- 172.21.15.78:0/1845477332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec051030 con 0x564c13a79390 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe5033f1740 1 -- 172.21.15.78:0/1845477332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e4009de0 con 0x564c13a79390 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe4ddffb640 1 -- 172.21.15.78:0/1845477332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4ec047070 con 0x564c13a79390 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe4ddffb640 1 -- 172.21.15.78:0/1845477332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4ec0319a0 con 0x564c13a79390 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe5033f1740 10 client.15906 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe5033f1740 1 -- 172.21.15.78:0/1845477332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c13e31740 con 0x564c13a79390 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe4ddffb640 1 -- 172.21.15.78:0/1845477332 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e4009de0 con 0x564c13a79390 2024-05-21T15:17:17.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe4ddffb640 1 -- 172.21.15.78:0/1845477332 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe4ec04a050 con 0x564c13a79390 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.050+0000 7fe5033f1740 10 client.15906 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.051+0000 7fe5033f1740 10 client.15906 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.051+0000 7fe5033f1740 20 client.15906 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.051+0000 7fe5033f1740 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564c13a37f60 0x564c13e34c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.051+0000 7fe5033f1740 4 client.15906 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.051+0000 7fe5033f1740 1 -- 172.21.15.78:0/1845477332 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c13d8b370 con 0x564c13a37f60 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.051+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564c13a37f60 0x564c13e34c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.051+0000 7fe4ddffb640 10 client.15906 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:17:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.051+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564c13a37f60 0x564c13e34c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6382abcea1dd9bc server_cookie=d0218ec5d5c5fd37 in_seq=0 out_seq=0 2024-05-21T15:17:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe4ddffb640 1 -- 172.21.15.78:0/1845477332 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c13d8b370 con 0x564c13a37f60 2024-05-21T15:17:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe4ddffb640 10 client.15906 handle_command_reply: tid=0 2024-05-21T15:17:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 client.15906 shutdown 2024-05-21T15:17:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 -- 172.21.15.78:0/1845477332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a79390 msgr2=0x564c13e22230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a79390 0x564c13e22230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009ca0 tx=0x7fe4ec0095c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 -- 172.21.15.78:0/1845477332 shutdown_connections 2024-05-21T15:17:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564c13a37f60 0x564c13e34c10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a79390 0x564c13e22230 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe4ddffb640 0 client.15906 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:17:17.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a586b0 0x564c13e242a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 --2- 172.21.15.78:0/1845477332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a504d0 0x564c13e23d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.052+0000 7fe5033f1740 1 -- 172.21.15.78:0/1845477332 >> 172.21.15.78:0/1845477332 conn(0x564c13a6d4e0 msgr2=0x564c13e19a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.053+0000 7fe5033f1740 1 -- 172.21.15.78:0/1845477332 shutdown_connections 2024-05-21T15:17:17.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.053+0000 7fe5033f1740 1 -- 172.21.15.78:0/1845477332 wait complete. 2024-05-21T15:17:17.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.053+0000 7fe5033f1740 20 client.15906 trim_cache size 0 max 16384 2024-05-21T15:17:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.067+0000 7fe5033f1740 1 Processor -- start 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.067+0000 7fe5033f1740 1 -- start start 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.067+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a37f60 0x564c13e1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.067+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a504d0 0x564c13e21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.067+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a586b0 0x564c13e183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.067+0000 7fe5033f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41aec60 con 0x564c13a504d0 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.067+0000 7fe5033f1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41b07a0 con 0x564c13a586b0 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.067+0000 7fe5033f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41af330 con 0x564c13a37f60 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a504d0 0x564c13e21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a504d0 0x564c13e21b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46266/0 (socket says 172.21.15.78:46266) 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/3207131714 learned_addr learned my addr 172.21.15.78:0/3207131714 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a37f60 0x564c13e1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4faae8640 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a586b0 0x564c13e183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/3207131714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a37f60 msgr2=0x564c13e1da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a37f60 0x564c13e1da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/3207131714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a586b0 msgr2=0x564c13e183b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a586b0 0x564c13e183b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 -- 172.21.15.78:0/3207131714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c13e150c0 con 0x564c13a504d0 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a504d0 0x564c13e21b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40996e0 tx=0x7fe4e4097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733ead50e2d04adf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4dffff640 1 -- 172.21.15.78:0/3207131714 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4021070 con 0x564c13a504d0 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4dffff640 1 -- 172.21.15.78:0/3207131714 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e409b430 con 0x564c13a504d0 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.068+0000 7fe4dffff640 1 -- 172.21.15.78:0/3207131714 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e40ccb80 con 0x564c13a504d0 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.069+0000 7fe5033f1740 1 -- 172.21.15.78:0/3207131714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a504d0 msgr2=0x564c13e21b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.069+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a504d0 0x564c13e21b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40996e0 tx=0x7fe4e4097040 comp rx=0 tx=0).stop 2024-05-21T15:17:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.069+0000 7fe5033f1740 1 -- 172.21.15.78:0/3207131714 shutdown_connections 2024-05-21T15:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.069+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a586b0 0x564c13e183b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.069+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a504d0 0x564c13e21b10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.069+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3207131714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a37f60 0x564c13e1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 -- 172.21.15.78:0/3207131714 >> 172.21.15.78:0/3207131714 conn(0x564c13a6d4e0 msgr2=0x564c13e17ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 -- 172.21.15.78:0/3207131714 shutdown_connections 2024-05-21T15:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 -- 172.21.15.78:0/3207131714 wait complete. 2024-05-21T15:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 -- start start 2024-05-21T15:17:17.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 Processor -- start 2024-05-21T15:17:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a37f60 0x564c13e275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 0x564c13e27af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41b00d0 con 0x564c13a586b0 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41afa00 con 0x564c13a37f60 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41acc40 con 0x564c13a504d0 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe5033f1740 10 client.0 _ll_register_callbacks cb 0x7fe4ec009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe4faae8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.070+0000 7fe4faae8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e9af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46272/0 (socket says 172.21.15.78:46272) 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4faae8640 1 -- 172.21.15.78:0/3891376078 learned_addr learned my addr 172.21.15.78:0/3891376078 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4f9ae6640 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 0x564c13e27af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a37f60 0x564c13e275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4faae8640 1 -- 172.21.15.78:0/3891376078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 msgr2=0x564c13e27af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4faae8640 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 0x564c13e27af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4faae8640 1 -- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a37f60 msgr2=0x564c13e275b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4faae8640 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a37f60 0x564c13e275b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4faae8640 1 -- 172.21.15.78:0/3891376078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c13e9b4c0 con 0x564c13a586b0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a37f60 0x564c13e275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4faae8640 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e9af80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec02ade0 tx=0x7fe4ec0305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd48e29c230ab827 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4ddffb640 1 -- 172.21.15.78:0/3891376078 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec062020 con 0x564c13a586b0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe4ddffb640 1 -- 172.21.15.78:0/3891376078 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4ec047070 con 0x564c13a586b0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.071+0000 7fe5033f1740 1 -- 172.21.15.78:0/3891376078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e409b430 con 0x564c13a586b0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe4ddffb640 1 -- 172.21.15.78:0/3891376078 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4ec06a050 con 0x564c13a586b0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe5033f1740 10 client.15912 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe5033f1740 1 -- 172.21.15.78:0/3891376078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c13e9c5a0 con 0x564c13a586b0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe4ddffb640 1 -- 172.21.15.78:0/3891376078 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e409b430 con 0x564c13a586b0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe4ddffb640 1 -- 172.21.15.78:0/3891376078 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe4ec04a050 con 0x564c13a586b0 2024-05-21T15:17:17.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe5033f1740 10 client.15912 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe5033f1740 10 client.15912 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe5033f1740 20 client.15912 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564c13a79390 0x564c13e9ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe5033f1740 4 client.15912 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe5033f1740 1 -- 172.21.15.78:0/3891376078 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c13d856c0 con 0x564c13a79390 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.072+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564c13a79390 0x564c13e9ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.073+0000 7fe4fa2e7640 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564c13a79390 0x564c13e9ec10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34b5cc37440e27e9 server_cookie=c8c3422df35209b8 in_seq=0 out_seq=0 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.073+0000 7fe4ddffb640 10 client.15912 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.073+0000 7fe4ddffb640 1 -- 172.21.15.78:0/3891376078 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c13d856c0 con 0x564c13a79390 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.073+0000 7fe4ddffb640 10 client.15912 handle_command_reply: tid=0 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.073+0000 7fe5033f1740 1 client.15912 shutdown 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.073+0000 7fe5033f1740 1 -- 172.21.15.78:0/3891376078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 msgr2=0x564c13e9af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.073+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e9af80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec02ade0 tx=0x7fe4ec0305c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 1 -- 172.21.15.78:0/3891376078 shutdown_connections 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564c13a79390 0x564c13e9ec10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564c13a586b0 0x564c13e9af80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c13a504d0 0x564c13e27af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4ddffb640 0 client.15912 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 1 --2- 172.21.15.78:0/3891376078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564c13a37f60 0x564c13e275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 1 -- 172.21.15.78:0/3891376078 >> 172.21.15.78:0/3891376078 conn(0x564c13a6d4e0 msgr2=0x564c13e19fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 1 -- 172.21.15.78:0/3891376078 shutdown_connections 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 1 -- 172.21.15.78:0/3891376078 wait complete. 2024-05-21T15:17:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe5033f1740 20 client.15912 trim_cache size 0 max 16384 2024-05-21T15:17:17.083 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:17.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 -- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4e00a2e20 msgr2=0x7fe4e00a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4e00a2e20 0x7fe4e00a5330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x564c13e14e90 tx=0x7fe4ec02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 -- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f41079f0 msgr2=0x7fe4f419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f41079f0 0x7fe4f419aae0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4005d00 tx=0x7fe4e400dde0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 -- 172.21.15.78:0/2900461201 shutdown_connections 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4f410c350 0x7fe4f419b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4e00a2e20 0x7fe4e00a5330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4f41079f0 0x7fe4f419aae0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 --2- 172.21.15.78:0/2900461201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4f4101ac0 0x7fe4f419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 -- 172.21.15.78:0/2900461201 >> 172.21.15.78:0/2900461201 conn(0x7fe4f4077130 msgr2=0x7fe4f4103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 -- 172.21.15.78:0/2900461201 shutdown_connections 2024-05-21T15:17:17.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.074+0000 7fe4fc54b640 1 -- 172.21.15.78:0/2900461201 wait complete. 2024-05-21T15:17:17.089 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:17:17.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.219+0000 7f52b60c3640 1 -- 172.21.15.78:0/1475681269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 msgr2=0x7f52b010f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.219+0000 7f52b60c3640 1 --2- 172.21.15.78:0/1475681269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 0x7f52b010f4e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009ca0 tx=0x7f52a802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.219+0000 7f52b60c3640 1 -- 172.21.15.78:0/1475681269 shutdown_connections 2024-05-21T15:17:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.219+0000 7f52b60c3640 1 --2- 172.21.15.78:0/1475681269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 0x7f52b010f4e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.219+0000 7f52b60c3640 1 --2- 172.21.15.78:0/1475681269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b010a600 0x7f52b010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.219+0000 7f52b60c3640 1 --2- 172.21.15.78:0/1475681269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52b0103ba0 0x7f52b0103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.219+0000 7f52b60c3640 1 -- 172.21.15.78:0/1475681269 >> 172.21.15.78:0/1475681269 conn(0x7f52b00770b0 msgr2=0x7f52b00774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.220+0000 7f52b60c3640 1 -- 172.21.15.78:0/1475681269 shutdown_connections 2024-05-21T15:17:17.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.220+0000 7f52b60c3640 1 -- 172.21.15.78:0/1475681269 wait complete. 2024-05-21T15:17:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.220+0000 7f52b60c3640 1 Processor -- start 2024-05-21T15:17:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52b60c3640 1 -- start start 2024-05-21T15:17:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52b60c3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b0103ba0 0x7f52b0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52b60c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52b010a600 0x7f52b0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52b60c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 0x7f52b019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52b60c3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112da0 con 0x7f52b010cff0 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52b60c3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112c20 con 0x7f52b010a600 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52b60c3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112f20 con 0x7f52b0103ba0 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 0x7f52b019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52b010a600 0x7f52b0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52b010a600 0x7f52b0199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:48752/0 (socket says 172.21.15.78:48752) 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52aeffd640 1 -- 172.21.15.78:0/2547535309 learned_addr learned my addr 172.21.15.78:0/2547535309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 0x7f52b019df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46290/0 (socket says 172.21.15.78:46290) 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.221+0000 7f52af7fe640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b0103ba0 0x7f52b0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52aeffd640 1 -- 172.21.15.78:0/2547535309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b0103ba0 msgr2=0x7f52b0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52aeffd640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b0103ba0 0x7f52b0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52aeffd640 1 -- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 msgr2=0x7f52b019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52aeffd640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 0x7f52b019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52aeffd640 1 -- 172.21.15.78:0/2547535309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b019e5a0 con 0x7f52b010a600 2024-05-21T15:17:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52aeffd640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52b010a600 0x7f52b0199ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f52a400e9f0 tx=0x7f52a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52acff9640 1 -- 172.21.15.78:0/2547535309 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a400cd80 con 0x7f52b010a600 2024-05-21T15:17:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52acff9640 1 -- 172.21.15.78:0/2547535309 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a4005500 con 0x7f52b010a600 2024-05-21T15:17:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52b60c3640 1 -- 172.21.15.78:0/2547535309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b019e890 con 0x7f52b010a600 2024-05-21T15:17:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52acff9640 1 -- 172.21.15.78:0/2547535309 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4010690 con 0x7f52b010a600 2024-05-21T15:17:17.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.222+0000 7f52b60c3640 1 -- 172.21.15.78:0/2547535309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b019ed90 con 0x7f52b010a600 2024-05-21T15:17:17.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.225+0000 7f52bcf66740 1 Processor -- start 2024-05-21T15:17:17.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52bcf66740 1 -- start start 2024-05-21T15:17:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc25d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c85e4d0 0x55c97cc26260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52bcf66740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c97cb8bc40 con 0x55c97c85e4d0 2024-05-21T15:17:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52bcf66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c97cb33580 con 0x55c97c887390 2024-05-21T15:17:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52bcf66740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c97cb87c00 con 0x55c97c8666b0 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c85e4d0 0x55c97cc26260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52af7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c85e4d0 0x55c97cc26260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46298/0 (socket says 172.21.15.78:46298) 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52af7fe640 1 -- 172.21.15.78:0/2735538175 learned_addr learned my addr 172.21.15.78:0/2735538175 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52acff9640 1 -- 172.21.15.78:0/2547535309 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f52a4010830 con 0x7f52b010a600 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52affff640 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc25d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52af7fe640 1 -- 172.21.15.78:0/2735538175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 msgr2=0x55c97cc267a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52aeffd640 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52af7fe640 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc267a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52af7fe640 1 -- 172.21.15.78:0/2735538175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 msgr2=0x55c97cc25d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52af7fe640 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc25d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52af7fe640 1 -- 172.21.15.78:0/2735538175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97c89f3f0 con 0x55c97c85e4d0 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52affff640 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc25d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:17.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.226+0000 7f52aeffd640 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.227+0000 7f52af7fe640 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c85e4d0 0x55c97cc26260 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009cf0 tx=0x7f52a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4488deebd9a43f78 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.227+0000 7f5295ffb640 1 -- 172.21.15.78:0/2735538175 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0004230 con 0x55c97c85e4d0 2024-05-21T15:17:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.227+0000 7f52acff9640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52980a2ea0 0x7f52980a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.227+0000 7f5295ffb640 1 -- 172.21.15.78:0/2735538175 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a00043d0 con 0x55c97c85e4d0 2024-05-21T15:17:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.227+0000 7f5295ffb640 1 -- 172.21.15.78:0/2735538175 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a00198c0 con 0x55c97c85e4d0 2024-05-21T15:17:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.227+0000 7f52affff640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52980a2ea0 0x7f52980a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.227+0000 7f52acff9640 1 -- 172.21.15.78:0/2547535309 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f52a40cfb20 con 0x7f52b010a600 2024-05-21T15:17:17.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 -- 172.21.15.78:0/2735538175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c85e4d0 msgr2=0x55c97cc26260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52affff640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52980a2ea0 0x7f52980a53b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55c97cc22e90 tx=0x7f52a8004a80 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c85e4d0 0x55c97cc26260 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009cf0 tx=0x7f52a0011810 comp rx=0 tx=0).stop 2024-05-21T15:17:17.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 -- 172.21.15.78:0/2735538175 shutdown_connections 2024-05-21T15:17:17.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c85e4d0 0x55c97cc26260 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 --2- 172.21.15.78:0/2735538175 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc25d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 -- 172.21.15.78:0/2735538175 >> 172.21.15.78:0/2735538175 conn(0x55c97c87b4e0 msgr2=0x55c97cc224d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 -- 172.21.15.78:0/2735538175 shutdown_connections 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.228+0000 7f52bcf66740 1 -- 172.21.15.78:0/2735538175 wait complete. 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 1 -- start start 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 1 Processor -- start 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc31dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc32330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c887390 0x55c97cc30250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c97cc32fb0 con 0x55c97c887390 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c97cb46c40 con 0x55c97c85e4d0 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c97cc33130 con 0x55c97c8666b0 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52bcf66740 10 client.0 _ll_register_callbacks cb 0x55c97c89ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c887390 0x55c97cc30250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c887390 0x55c97cc30250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46312/0 (socket says 172.21.15.78:46312) 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52affff640 1 -- 172.21.15.78:0/1211301594 learned_addr learned my addr 172.21.15.78:0/1211301594 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52aeffd640 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc32330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.229+0000 7f52af7fe640 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc31dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f52affff640 1 -- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 msgr2=0x55c97cc32330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f52affff640 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc32330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f52affff640 1 -- 172.21.15.78:0/1211301594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 msgr2=0x55c97cc31dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f52affff640 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc31dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f52affff640 1 -- 172.21.15.78:0/1211301594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97c9697d0 con 0x55c97c887390 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f52aeffd640 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc32330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f52affff640 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c887390 0x55c97cc30250 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009ca0 tx=0x7f52a802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6f6242837dcedc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f528effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f528effd640 1 -- 172.21.15.78:0/1211301594 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8031450 con 0x55c97c887390 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f528effd640 1 -- 172.21.15.78:0/1211301594 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f52a80315f0 con 0x55c97c887390 2024-05-21T15:17:17.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.230+0000 7f52bcf66740 1 -- 172.21.15.78:0/1211301594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a00043d0 con 0x55c97c887390 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f528effd640 1 -- 172.21.15.78:0/1211301594 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52a80317f0 con 0x55c97c887390 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f52bcf66740 10 client.15924 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f52bcf66740 1 -- 172.21.15.78:0/1211301594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c97cc3f760 con 0x55c97c887390 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f528effd640 1 -- 172.21.15.78:0/1211301594 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a8057030 con 0x55c97c887390 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f528effd640 1 -- 172.21.15.78:0/1211301594 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f52a8049690 con 0x55c97c887390 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f52bcf66740 10 client.15924 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f52bcf66740 10 client.15924 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f52bcf66740 20 client.15924 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55c97c845f60 0x55c97cc42d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f52bcf66740 4 client.15924 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.231+0000 7f52bcf66740 1 -- 172.21.15.78:0/1211301594 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c97cb99370 con 0x55c97c845f60 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.232+0000 7f52af7fe640 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55c97c845f60 0x55c97cc42d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.232+0000 7f528effd640 10 client.15924 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:17:17.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.232+0000 7f52af7fe640 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55c97c845f60 0x55c97cc42d20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b5704ddeea55fde server_cookie=b5ffb13af39d33ee in_seq=0 out_seq=0 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f528effd640 1 -- 172.21.15.78:0/1211301594 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c97cb99370 con 0x55c97c845f60 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f528effd640 10 client.15924 handle_command_reply: tid=0 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 client.15924 shutdown 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 -- 172.21.15.78:0/1211301594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c887390 msgr2=0x55c97cc30250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c887390 0x55c97cc30250 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009ca0 tx=0x7f52a802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 -- 172.21.15.78:0/1211301594 shutdown_connections 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55c97c845f60 0x55c97cc42d20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c887390 0x55c97cc30250 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c8666b0 0x55c97cc32330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1211301594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc31dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f528effd640 0 client.15924 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:17:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 -- 172.21.15.78:0/1211301594 >> 172.21.15.78:0/1211301594 conn(0x55c97c87b4e0 msgr2=0x55c97cc27a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 -- 172.21.15.78:0/1211301594 shutdown_connections 2024-05-21T15:17:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.233+0000 7f52bcf66740 1 -- 172.21.15.78:0/1211301594 wait complete. 2024-05-21T15:17:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.234+0000 7f52bcf66740 20 client.15924 trim_cache size 0 max 16384 2024-05-21T15:17:17.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52bcf66740 1 Processor -- start 2024-05-21T15:17:17.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52bcf66740 1 -- start start 2024-05-21T15:17:17.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c85e4d0 0x55c97cc2a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c8666b0 0x55c97cc484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc48a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52bcf66740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01aebd0 con 0x55c97c8666b0 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52bcf66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01b0710 con 0x55c97c887390 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52bcf66740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01af2a0 con 0x55c97c85e4d0 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.247+0000 7f52aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c8666b0 0x55c97cc484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c8666b0 0x55c97cc484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46320/0 (socket says 172.21.15.78:46320) 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52aeffd640 1 -- 172.21.15.78:0/1403754461 learned_addr learned my addr 172.21.15.78:0/1403754461 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52af7fe640 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c85e4d0 0x55c97cc2a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52affff640 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc48a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52aeffd640 1 -- 172.21.15.78:0/1403754461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c85e4d0 msgr2=0x55c97cc2a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52aeffd640 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c85e4d0 0x55c97cc2a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52aeffd640 1 -- 172.21.15.78:0/1403754461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 msgr2=0x55c97cc48a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52aeffd640 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc48a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52aeffd640 1 -- 172.21.15.78:0/1403754461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97cc326a0 con 0x55c97c8666b0 2024-05-21T15:17:17.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52aeffd640 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c8666b0 0x55c97cc484e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c6a0 tx=0x7f52a409bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc62a0cd6cfa0a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52957fa640 1 -- 172.21.15.78:0/1403754461 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a409c430 con 0x55c97c8666b0 2024-05-21T15:17:17.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52957fa640 1 -- 172.21.15.78:0/1403754461 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a409bd30 con 0x55c97c8666b0 2024-05-21T15:17:17.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.248+0000 7f52957fa640 1 -- 172.21.15.78:0/1403754461 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40cd570 con 0x55c97c8666b0 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 -- 172.21.15.78:0/1403754461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c8666b0 msgr2=0x55c97cc484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c8666b0 0x55c97cc484e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c6a0 tx=0x7f52a409bb00 comp rx=0 tx=0).stop 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 -- 172.21.15.78:0/1403754461 shutdown_connections 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c887390 0x55c97cc48a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c8666b0 0x55c97cc484e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 --2- 172.21.15.78:0/1403754461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c85e4d0 0x55c97cc2a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 -- 172.21.15.78:0/1403754461 >> 172.21.15.78:0/1403754461 conn(0x55c97c87b4e0 msgr2=0x55c97cc299e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 -- 172.21.15.78:0/1403754461 shutdown_connections 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.249+0000 7f52bcf66740 1 -- 172.21.15.78:0/1403754461 wait complete. 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 1 -- start start 2024-05-21T15:17:17.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 1 Processor -- start 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c887390 0x55c97cc386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c81b3a0 0x55c97cc35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01b0040 con 0x55c97c81b3a0 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01af970 con 0x55c97c85e4d0 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01acbb0 con 0x55c97c887390 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.250+0000 7f52bcf66740 10 client.0 _ll_register_callbacks cb 0x7f52a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c81b3a0 0x55c97cc35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c81b3a0 0x55c97cc35660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46328/0 (socket says 172.21.15.78:46328) 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 -- 172.21.15.78:0/495138830 learned_addr learned my addr 172.21.15.78:0/495138830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52aeffd640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c887390 0x55c97cc386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52af7fe640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 -- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c887390 msgr2=0x55c97cc386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c887390 0x55c97cc386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 -- 172.21.15.78:0/495138830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 msgr2=0x55c97cc381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 -- 172.21.15.78:0/495138830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97cc35cd0 con 0x55c97c81b3a0 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52aeffd640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c887390 0x55c97cc386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52af7fe640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52affff640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c81b3a0 0x55c97cc35660 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f52a8004af0 tx=0x7f52a80307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb5df923dfcb214 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f528effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f528effd640 1 -- 172.21.15.78:0/495138830 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8049430 con 0x55c97c81b3a0 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f528effd640 1 -- 172.21.15.78:0/495138830 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f52a8042070 con 0x55c97c81b3a0 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.251+0000 7f52bcf66740 1 -- 172.21.15.78:0/495138830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a409bd30 con 0x55c97c81b3a0 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f528effd640 1 -- 172.21.15.78:0/495138830 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f52a8068050 con 0x55c97c81b3a0 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f52bcf66740 10 client.15930 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f52bcf66740 1 -- 172.21.15.78:0/495138830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c97ccaa960 con 0x55c97c81b3a0 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f528effd640 1 -- 172.21.15.78:0/495138830 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a8057040 con 0x55c97c81b3a0 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f528effd640 1 -- 172.21.15.78:0/495138830 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f52a8004660 con 0x55c97c81b3a0 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f52bcf66740 10 client.15930 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f52bcf66740 10 client.15930 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f52bcf66740 20 client.15930 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f52bcf66740 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55c97ccab040 0x55c97ccad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f52bcf66740 4 client.15930 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.252+0000 7f52bcf66740 1 -- 172.21.15.78:0/495138830 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f52a409bd30 con 0x55c97ccab040 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.253+0000 7f52af7fe640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55c97ccab040 0x55c97ccad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.253+0000 7f528effd640 10 client.15930 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.253+0000 7f52af7fe640 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55c97ccab040 0x55c97ccad460 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef54c6ac902f7afe server_cookie=3d9191f4c9c55246 in_seq=0 out_seq=0 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.253+0000 7f528effd640 1 -- 172.21.15.78:0/495138830 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f52a409bd30 con 0x55c97ccab040 2024-05-21T15:17:17.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.253+0000 7f528effd640 10 client.15930 handle_command_reply: tid=0 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.253+0000 7f52bcf66740 1 client.15930 shutdown 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.253+0000 7f52bcf66740 1 -- 172.21.15.78:0/495138830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c81b3a0 msgr2=0x55c97cc35660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c81b3a0 0x55c97cc35660 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f52a8004af0 tx=0x7f52a80307d0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 -- 172.21.15.78:0/495138830 shutdown_connections 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55c97ccab040 0x55c97ccad460 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97c81b3a0 0x55c97cc35660 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97c887390 0x55c97cc386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 --2- 172.21.15.78:0/495138830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97c85e4d0 0x55c97cc381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 -- 172.21.15.78:0/495138830 >> 172.21.15.78:0/495138830 conn(0x55c97c87b4e0 msgr2=0x55c97cc25ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f528effd640 0 client.15930 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:17:17.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 -- 172.21.15.78:0/495138830 shutdown_connections 2024-05-21T15:17:17.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 1 -- 172.21.15.78:0/495138830 wait complete. 2024-05-21T15:17:17.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52bcf66740 20 client.15930 trim_cache size 0 max 16384 2024-05-21T15:17:17.262 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:17.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52b60c3640 1 -- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52980a2ea0 msgr2=0x7f52980a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.254+0000 7f52b60c3640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52980a2ea0 0x7f52980a53b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55c97cc22e90 tx=0x7f52a8004a80 comp rx=0 tx=0).stop 2024-05-21T15:17:17.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 -- 172.21.15.78:0/2547535309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52b010a600 msgr2=0x7f52b0199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52b010a600 0x7f52b0199ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f52a400e9f0 tx=0x7f52a400edf0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 -- 172.21.15.78:0/2547535309 shutdown_connections 2024-05-21T15:17:17.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52b010a600 0x7f52b0199ca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52980a2ea0 0x7f52980a53b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52b010cff0 0x7f52b019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 --2- 172.21.15.78:0/2547535309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b0103ba0 0x7f52b0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 -- 172.21.15.78:0/2547535309 >> 172.21.15.78:0/2547535309 conn(0x7f52b00770b0 msgr2=0x7f52b0100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 -- 172.21.15.78:0/2547535309 shutdown_connections 2024-05-21T15:17:17.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.255+0000 7f52b60c3640 1 -- 172.21.15.78:0/2547535309 wait complete. 2024-05-21T15:17:17.272 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 -- 172.21.15.78:0/2201951752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c10c350 msgr2=0x7fe14c10e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 --2- 172.21.15.78:0/2201951752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c10c350 0x7fe14c10e770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe144009ca0 tx=0x7fe14402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 -- 172.21.15.78:0/2201951752 shutdown_connections 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 --2- 172.21.15.78:0/2201951752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c10c350 0x7fe14c10e770 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 --2- 172.21.15.78:0/2201951752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101b10 0x7fe14c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 --2- 172.21.15.78:0/2201951752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe14c1079f0 0x7fe14c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 -- 172.21.15.78:0/2201951752 >> 172.21.15.78:0/2201951752 conn(0x7fe14c0fd210 msgr2=0x7fe14c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 -- 172.21.15.78:0/2201951752 shutdown_connections 2024-05-21T15:17:17.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.384+0000 7fe1515b7640 1 -- 172.21.15.78:0/2201951752 wait complete. 2024-05-21T15:17:17.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.385+0000 7fe1515b7640 1 Processor -- start 2024-05-21T15:17:17.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.385+0000 7fe1515b7640 1 -- start start 2024-05-21T15:17:17.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.385+0000 7fe1515b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c101b10 0x7fe14c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.385+0000 7fe1515b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c1079f0 0x7fe14c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.385+0000 7fe1515b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe14c10c350 0x7fe14c19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.385+0000 7fe1515b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111d50 con 0x7fe14c101b10 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.385+0000 7fe1515b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111bd0 con 0x7fe14c10c350 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.385+0000 7fe1515b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111ed0 con 0x7fe14c1079f0 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe14c10c350 0x7fe14c19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe14c10c350 0x7fe14c19aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:48810/0 (socket says 172.21.15.78:48810) 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c1079f0 0x7fe14c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14b7fe640 1 -- 172.21.15.78:0/792817979 learned_addr learned my addr 172.21.15.78:0/792817979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14affd640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c101b10 0x7fe14c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14a7fc640 1 -- 172.21.15.78:0/792817979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe14c10c350 msgr2=0x7fe14c19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe14c10c350 0x7fe14c19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14a7fc640 1 -- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c101b10 msgr2=0x7fe14c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c101b10 0x7fe14c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14a7fc640 1 -- 172.21.15.78:0/792817979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c198340 con 0x7fe14c1079f0 2024-05-21T15:17:17.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe14c10c350 0x7fe14c19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14affd640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c101b10 0x7fe14c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c1079f0 0x7fe14c19aa90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe14c06b3d0 tx=0x7fe14000db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe133fff640 1 -- 172.21.15.78:0/792817979 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140017070 con 0x7fe14c1079f0 2024-05-21T15:17:17.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe1515b7640 1 -- 172.21.15.78:0/792817979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c198630 con 0x7fe14c1079f0 2024-05-21T15:17:17.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe133fff640 1 -- 172.21.15.78:0/792817979 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe140009d00 con 0x7fe14c1079f0 2024-05-21T15:17:17.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.386+0000 7fe133fff640 1 -- 172.21.15.78:0/792817979 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140003cd0 con 0x7fe14c1079f0 2024-05-21T15:17:17.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.387+0000 7fe1515b7640 1 -- 172.21.15.78:0/792817979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c198bd0 con 0x7fe14c1079f0 2024-05-21T15:17:17.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.389+0000 7fe133fff640 1 -- 172.21.15.78:0/792817979 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fe140009820 con 0x7fe14c1079f0 2024-05-21T15:17:17.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.389+0000 7fe158457740 1 Processor -- start 2024-05-21T15:17:17.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.389+0000 7fe158457740 1 -- start start 2024-05-21T15:17:17.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe133fff640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe1340a2ea0 0x7fe1340a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe158457740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x5643472f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe158457740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f324d0 0x5643472fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe158457740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643472fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe158457740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56434725fc40 con 0x564346f5b390 2024-05-21T15:17:17.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe158457740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564347207580 con 0x564346f3a6b0 2024-05-21T15:17:17.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe158457740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56434725bc00 con 0x564346f324d0 2024-05-21T15:17:17.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe133fff640 1 -- 172.21.15.78:0/792817979 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe1400cf410 con 0x7fe14c1079f0 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe1340a2ea0 0x7fe1340a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe14affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x5643472f9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe14affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x5643472f9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46374/0 (socket says 172.21.15.78:46374) 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe14affd640 1 -- 172.21.15.78:0/1288885742 learned_addr learned my addr 172.21.15.78:0/1288885742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.390+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f324d0 0x5643472fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14affd640 1 -- 172.21.15.78:0/1288885742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f324d0 msgr2=0x5643472fa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643472fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14affd640 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f324d0 0x5643472fa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14affd640 1 -- 172.21.15.78:0/1288885742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 msgr2=0x5643472fa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14affd640 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643472fa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14affd640 1 -- 172.21.15.78:0/1288885742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564346f733f0 con 0x564346f5b390 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643472fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe1340a2ea0 0x7fe1340a53b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1440094e0 tx=0x7fe1440099e0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe14affd640 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x5643472f9d20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe13c010400 tx=0x7fe13c0108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14932d13429c58e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe1317fa640 1 -- 172.21.15.78:0/1288885742 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c011760 con 0x564346f5b390 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe1317fa640 1 -- 172.21.15.78:0/1288885742 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c011de0 con 0x564346f5b390 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.391+0000 7fe1317fa640 1 -- 172.21.15.78:0/1288885742 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c011760 con 0x564346f5b390 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 -- 172.21.15.78:0/1288885742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 msgr2=0x5643472f9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x5643472f9d20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe13c010400 tx=0x7fe13c0108c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 -- 172.21.15.78:0/1288885742 shutdown_connections 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643472fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f324d0 0x5643472fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 --2- 172.21.15.78:0/1288885742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x5643472f9d20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 -- 172.21.15.78:0/1288885742 >> 172.21.15.78:0/1288885742 conn(0x564346f4f4e0 msgr2=0x5643472f64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 -- 172.21.15.78:0/1288885742 shutdown_connections 2024-05-21T15:17:17.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.392+0000 7fe158457740 1 -- 172.21.15.78:0/1288885742 wait complete. 2024-05-21T15:17:17.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 1 -- start start 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 1 Processor -- start 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347305d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643473062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 0x564347304240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564347306fb0 con 0x564346f324d0 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56434721ac40 con 0x564346f3a6b0 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564347307130 con 0x564346f5b390 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe158457740 10 client.0 _ll_register_callbacks cb 0x564346f72be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347305d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347305d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46378/0 (socket says 172.21.15.78:46378) 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 0x564347304240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14affd640 1 -- 172.21.15.78:0/2291917417 learned_addr learned my addr 172.21.15.78:0/2291917417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643473062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14affd640 1 -- 172.21.15.78:0/2291917417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 msgr2=0x564347304240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14affd640 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 0x564347304240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14affd640 1 -- 172.21.15.78:0/2291917417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 msgr2=0x5643473062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.393+0000 7fe14affd640 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643473062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe14affd640 1 -- 172.21.15.78:0/2291917417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434703d7d0 con 0x564346f324d0 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe14affd640 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347305d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe13c011fd0 tx=0x7fe13c01aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa578b8809b7a93b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe12eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe12eff5640 1 -- 172.21.15.78:0/2291917417 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c011760 con 0x564346f324d0 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe12eff5640 1 -- 172.21.15.78:0/2291917417 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe13c011560 con 0x564346f324d0 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe158457740 1 -- 172.21.15.78:0/2291917417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe13c011de0 con 0x564346f324d0 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe12eff5640 1 -- 172.21.15.78:0/2291917417 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe13c00bc30 con 0x564346f324d0 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe158457740 10 client.15942 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe158457740 1 -- 172.21.15.78:0/2291917417 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564347313720 con 0x564346f324d0 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.394+0000 7fe12eff5640 1 -- 172.21.15.78:0/2291917417 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c011de0 con 0x564346f324d0 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.395+0000 7fe12eff5640 1 -- 172.21.15.78:0/2291917417 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe13c0046c0 con 0x564346f324d0 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.395+0000 7fe158457740 10 client.15942 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.395+0000 7fe158457740 10 client.15942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.395+0000 7fe158457740 20 client.15942 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.395+0000 7fe158457740 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564346f19f60 0x564347316b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.395+0000 7fe158457740 4 client.15942 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.395+0000 7fe158457740 1 -- 172.21.15.78:0/2291917417 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56434726d370 con 0x564346f19f60 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.395+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564346f19f60 0x564347316b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.396+0000 7fe12eff5640 10 client.15942 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:17:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.396+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564346f19f60 0x564347316b90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb1d72e8a8544612 server_cookie=5e85cd9c4ae5ec4b in_seq=0 out_seq=0 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.396+0000 7fe12eff5640 1 -- 172.21.15.78:0/2291917417 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56434726d370 con 0x564346f19f60 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.396+0000 7fe12eff5640 10 client.15942 handle_command_reply: tid=0 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.396+0000 7fe158457740 1 client.15942 shutdown 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 -- 172.21.15.78:0/2291917417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 msgr2=0x564347305d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347305d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe13c011fd0 tx=0x7fe13c01aa90 comp rx=0 tx=0).stop 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 -- 172.21.15.78:0/2291917417 shutdown_connections 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564346f19f60 0x564347316b90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x5643473062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 0x564347304240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 --2- 172.21.15.78:0/2291917417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347305d80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 -- 172.21.15.78:0/2291917417 >> 172.21.15.78:0/2291917417 conn(0x564346f4f4e0 msgr2=0x5643472fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe12eff5640 0 client.15942 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:17:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 -- 172.21.15.78:0/2291917417 shutdown_connections 2024-05-21T15:17:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 1 -- 172.21.15.78:0/2291917417 wait complete. 2024-05-21T15:17:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.397+0000 7fe158457740 20 client.15942 trim_cache size 0 max 16384 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.410+0000 7fe158457740 1 Processor -- start 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe158457740 1 -- start start 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe158457740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f324d0 0x5643472fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe158457740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f3a6b0 0x56434731c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe158457740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x56434731c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe158457740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1b0e80 con 0x564346f5b390 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe158457740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1b29c0 con 0x564346f324d0 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe158457740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1b1550 con 0x564346f3a6b0 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x56434731c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x56434731c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46394/0 (socket says 172.21.15.78:46394) 2024-05-21T15:17:17.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14b7fe640 1 -- 172.21.15.78:0/1729760898 learned_addr learned my addr 172.21.15.78:0/1729760898 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f3a6b0 0x56434731c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14affd640 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f324d0 0x5643472fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14b7fe640 1 -- 172.21.15.78:0/1729760898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f3a6b0 msgr2=0x56434731c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f3a6b0 0x56434731c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14b7fe640 1 -- 172.21.15.78:0/1729760898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f324d0 msgr2=0x5643472fe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f324d0 0x5643472fe0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.411+0000 7fe14b7fe640 1 -- 172.21.15.78:0/1729760898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643472f7590 con 0x564346f5b390 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.412+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x56434731c7f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe144003be0 tx=0x7fe144005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b714f196a3d54dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.412+0000 7fe130ff9640 1 -- 172.21.15.78:0/1729760898 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144043040 con 0x564346f5b390 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.412+0000 7fe130ff9640 1 -- 172.21.15.78:0/1729760898 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe144045070 con 0x564346f5b390 2024-05-21T15:17:17.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.412+0000 7fe130ff9640 1 -- 172.21.15.78:0/1729760898 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14402fcb0 con 0x564346f5b390 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 -- 172.21.15.78:0/1729760898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 msgr2=0x56434731c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x56434731c7f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe144003be0 tx=0x7fe144005d80 comp rx=0 tx=0).stop 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 -- 172.21.15.78:0/1729760898 shutdown_connections 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f5b390 0x56434731c7f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f3a6b0 0x56434731c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 --2- 172.21.15.78:0/1729760898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f324d0 0x5643472fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 -- 172.21.15.78:0/1729760898 >> 172.21.15.78:0/1729760898 conn(0x564346f4f4e0 msgr2=0x5643472fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 -- 172.21.15.78:0/1729760898 shutdown_connections 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 -- 172.21.15.78:0/1729760898 wait complete. 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 -- start start 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 Processor -- start 2024-05-21T15:17:17.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.413+0000 7fe158457740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347309220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe158457740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x564347309760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe158457740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 0x564347309cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe158457740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1b22f0 con 0x564346f324d0 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe158457740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1b1c20 con 0x564346f3a6b0 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347309220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe158457740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1aee60 con 0x564346f5b390 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347309220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46408/0 (socket says 172.21.15.78:46408) 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 -- 172.21.15.78:0/2833776846 learned_addr learned my addr 172.21.15.78:0/2833776846 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe158457740 10 client.0 _ll_register_callbacks cb 0x7fe13c00be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x564347309760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14b7fe640 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 0x564347309cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 -- 172.21.15.78:0/2833776846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 msgr2=0x564347309cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 0x564347309cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 -- 172.21.15.78:0/2833776846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 msgr2=0x564347309760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x564347309760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 -- 172.21.15.78:0/2833776846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434737d090 con 0x564346f324d0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe14affd640 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347309220 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe13c01a9c0 tx=0x7fe13c004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bce8325b6cfbfc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe12eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe12eff5640 1 -- 172.21.15.78:0/2833776846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c005850 con 0x564346f324d0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.414+0000 7fe12eff5640 1 -- 172.21.15.78:0/2833776846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe13c011560 con 0x564346f324d0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 1 -- 172.21.15.78:0/2833776846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe144045070 con 0x564346f324d0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe12eff5640 1 -- 172.21.15.78:0/2833776846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe13c00bc10 con 0x564346f324d0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 10 client.15948 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 1 -- 172.21.15.78:0/2833776846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56434737e310 con 0x564346f324d0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe12eff5640 1 -- 172.21.15.78:0/2833776846 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c011b10 con 0x564346f324d0 2024-05-21T15:17:17.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe12eff5640 1 -- 172.21.15.78:0/2833776846 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe13c01c8b0 con 0x564346f324d0 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 10 client.15948 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 10 client.15948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 20 client.15948 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564346eef3a0 0x564347380a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 4 client.15948 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.415+0000 7fe158457740 1 -- 172.21.15.78:0/2833776846 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe144045070 con 0x564346eef3a0 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564346eef3a0 0x564347380a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe14a7fc640 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564346eef3a0 0x564347380a90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c173c70a2a54fdd0 server_cookie=b94b467c94d6e0be in_seq=0 out_seq=0 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe12eff5640 10 client.15948 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe12eff5640 1 -- 172.21.15.78:0/2833776846 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe144045070 con 0x564346eef3a0 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe12eff5640 10 client.15948 handle_command_reply: tid=0 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe158457740 1 client.15948 shutdown 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe158457740 1 -- 172.21.15.78:0/2833776846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 msgr2=0x564347309220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe158457740 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347309220 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe13c01a9c0 tx=0x7fe13c004890 comp rx=0 tx=0).stop 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe158457740 1 -- 172.21.15.78:0/2833776846 shutdown_connections 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe158457740 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564346eef3a0 0x564347380a90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe158457740 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564346f3a6b0 0x564347309760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.416+0000 7fe158457740 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564346f5b390 0x564347309cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe158457740 1 --2- 172.21.15.78:0/2833776846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564346f324d0 0x564347309220 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe158457740 1 -- 172.21.15.78:0/2833776846 >> 172.21.15.78:0/2833776846 conn(0x564346f4f4e0 msgr2=0x564347303ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe12eff5640 0 client.15948 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe158457740 1 -- 172.21.15.78:0/2833776846 shutdown_connections 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe158457740 1 -- 172.21.15.78:0/2833776846 wait complete. 2024-05-21T15:17:17.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe158457740 20 client.15948 trim_cache size 0 max 16384 2024-05-21T15:17:17.426 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe1515b7640 1 -- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe1340a2ea0 msgr2=0x7fe1340a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe1515b7640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe1340a2ea0 0x7fe1340a53b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1440094e0 tx=0x7fe1440099e0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe1515b7640 1 -- 172.21.15.78:0/792817979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c1079f0 msgr2=0x7fe14c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe1515b7640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c1079f0 0x7fe14c19aa90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe14c06b3d0 tx=0x7fe14000db30 comp rx=0 tx=0).stop 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe1515b7640 1 -- 172.21.15.78:0/792817979 shutdown_connections 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe1515b7640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe14c10c350 0x7fe14c19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe1515b7640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe1340a2ea0 0x7fe1340a53b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.417+0000 7fe1515b7640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c1079f0 0x7fe14c19aa90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.418+0000 7fe1515b7640 1 --2- 172.21.15.78:0/792817979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe14c101b10 0x7fe14c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.418+0000 7fe1515b7640 1 -- 172.21.15.78:0/792817979 >> 172.21.15.78:0/792817979 conn(0x7fe14c0fd210 msgr2=0x7fe14c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.418+0000 7fe1515b7640 1 -- 172.21.15.78:0/792817979 shutdown_connections 2024-05-21T15:17:17.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.418+0000 7fe1515b7640 1 -- 172.21.15.78:0/792817979 wait complete. 2024-05-21T15:17:17.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:17:17.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 -- 172.21.15.78:0/339985262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 msgr2=0x7ff9a810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 --2- 172.21.15.78:0/339985262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 0x7ff9a810ca20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff998009900 tx=0x7ff99802a410 comp rx=0 tx=0).stop 2024-05-21T15:17:17.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 -- 172.21.15.78:0/339985262 shutdown_connections 2024-05-21T15:17:17.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 --2- 172.21.15.78:0/339985262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a810cff0 0x7ff9a810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 --2- 172.21.15.78:0/339985262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 0x7ff9a810ca20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 --2- 172.21.15.78:0/339985262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a8103ba0 0x7ff9a8103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 -- 172.21.15.78:0/339985262 >> 172.21.15.78:0/339985262 conn(0x7ff9a80770b0 msgr2=0x7ff9a80774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 -- 172.21.15.78:0/339985262 shutdown_connections 2024-05-21T15:17:17.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.560+0000 7ff9af95e640 1 -- 172.21.15.78:0/339985262 wait complete. 2024-05-21T15:17:17.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9af95e640 1 Processor -- start 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9af95e640 1 -- start start 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9af95e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a8103ba0 0x7ff9a81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9af95e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 0x7ff9a8199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9af95e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a810cff0 0x7ff9a819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9af95e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8112da0 con 0x7ff9a810a600 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9af95e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8112c20 con 0x7ff9a810cff0 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9af95e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8112f20 con 0x7ff9a8103ba0 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9acef9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 0x7ff9a8199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9acef9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 0x7ff9a8199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46436/0 (socket says 172.21.15.78:46436) 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9acef9640 1 -- 172.21.15.78:0/1362846578 learned_addr learned my addr 172.21.15.78:0/1362846578 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a8103ba0 0x7ff9a81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.561+0000 7ff9adefb640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a810cff0 0x7ff9a819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff9acef9640 1 -- 172.21.15.78:0/1362846578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a8103ba0 msgr2=0x7ff9a81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff9acef9640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a8103ba0 0x7ff9a81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff9acef9640 1 -- 172.21.15.78:0/1362846578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a810cff0 msgr2=0x7ff9a819dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff9acef9640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a810cff0 0x7ff9a819dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff9acef9640 1 -- 172.21.15.78:0/1362846578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a819e600 con 0x7ff9a810a600 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff9acef9640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 0x7ff9a8199d00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff998006010 tx=0x7ff9980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff99e7fc640 1 -- 172.21.15.78:0/1362846578 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99803f030 con 0x7ff9a810a600 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff9af95e640 1 -- 172.21.15.78:0/1362846578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a819e890 con 0x7ff9a810a600 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff99e7fc640 1 -- 172.21.15.78:0/1362846578 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff998003ce0 con 0x7ff9a810a600 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff99e7fc640 1 -- 172.21.15.78:0/1362846578 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998035070 con 0x7ff9a810a600 2024-05-21T15:17:17.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.562+0000 7ff9af95e640 1 -- 172.21.15.78:0/1362846578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a819edd0 con 0x7ff9a810a600 2024-05-21T15:17:17.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9b6809740 1 Processor -- start 2024-05-21T15:17:17.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9b6809740 1 -- start start 2024-05-21T15:17:17.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d487390 0x55954d825d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d826260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 0x55954d8267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9b6809740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55954d78bc40 con 0x55954d45e4d0 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9b6809740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55954d733580 con 0x55954d4666b0 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9b6809740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55954d787c00 con 0x55954d487390 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9ad6fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d826260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9ad6fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d826260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46440/0 (socket says 172.21.15.78:46440) 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.566+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/1604649974 learned_addr learned my addr 172.21.15.78:0/1604649974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9adefb640 1 --2- 172.21.15.78:0/1604649974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d487390 0x55954d825d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff99e7fc640 1 -- 172.21.15.78:0/1362846578 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7ff998042030 con 0x7ff9a810a600 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/1604649974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d487390 msgr2=0x55954d825d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/1604649974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d487390 0x55954d825d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/1604649974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 msgr2=0x55954d8267a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:17:17.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/1604649974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 0x55954d8267a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/1604649974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954d49f3f0 con 0x55954d45e4d0 2024-05-21T15:17:17.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/1604649974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d826260 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400b760 tx=0x7ff9a400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca7d360a9b12fde server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9937fe640 1 -- 172.21.15.78:0/1604649974 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4003dd0 con 0x55954d45e4d0 2024-05-21T15:17:17.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9937fe640 1 -- 172.21.15.78:0/1604649974 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a4015070 con 0x55954d45e4d0 2024-05-21T15:17:17.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.567+0000 7ff9937fe640 1 -- 172.21.15.78:0/1604649974 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4011440 con 0x55954d45e4d0 2024-05-21T15:17:17.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.568+0000 7ff99e7fc640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff9940a2e20 0x7ff9940a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.568+0000 7ff99e7fc640 1 -- 172.21.15.78:0/1362846578 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7ff9980ef5b0 con 0x7ff9a810a600 2024-05-21T15:17:17.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.568+0000 7ff9b6809740 1 -- 172.21.15.78:0/1604649974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 msgr2=0x55954d826260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.568+0000 7ff9b6809740 1 --2- 172.21.15.78:0/1604649974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d826260 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400b760 tx=0x7ff9a400bc20 comp rx=0 tx=0).stop 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.568+0000 7ff9adefb640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff9940a2e20 0x7ff9940a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 -- 172.21.15.78:0/1604649974 shutdown_connections 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 --2- 172.21.15.78:0/1604649974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 0x55954d8267a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 --2- 172.21.15.78:0/1604649974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d826260 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 --2- 172.21.15.78:0/1604649974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d487390 0x55954d825d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 -- 172.21.15.78:0/1604649974 >> 172.21.15.78:0/1604649974 conn(0x55954d47b4e0 msgr2=0x55954d8224d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 -- 172.21.15.78:0/1604649974 shutdown_connections 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 -- 172.21.15.78:0/1604649974 wait complete. 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9adefb640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff9940a2e20 0x7ff9940a5330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009370 tx=0x7ff9a00092c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 -- start start 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.569+0000 7ff9b6809740 1 Processor -- start 2024-05-21T15:17:17.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d830360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d4666b0 0x55954d8308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d836500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9b6809740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55954d8330c0 con 0x55954d45e4d0 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9b6809740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55954d746c40 con 0x55954d487390 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9b6809740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55954d833240 con 0x55954d4666b0 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9b6809740 10 client.0 _ll_register_callbacks cb 0x55954d49ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9ad6fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d830360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9ad6fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d830360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46450/0 (socket says 172.21.15.78:46450) 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/77410732 learned_addr learned my addr 172.21.15.78:0/77410732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9adefb640 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d836500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9acef9640 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d4666b0 0x55954d8308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/77410732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d4666b0 msgr2=0x55954d8308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d4666b0 0x55954d8308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/77410732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 msgr2=0x55954d836500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d836500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.570+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/77410732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55954d5697d0 con 0x55954d45e4d0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.571+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d830360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4018000 tx=0x7ff9a4004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fbecb502f45cc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.571+0000 7ff990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.571+0000 7ff990ff9640 1 -- 172.21.15.78:0/77410732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4028060 con 0x55954d45e4d0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.571+0000 7ff990ff9640 1 -- 172.21.15.78:0/77410732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7ff9a40050b0 con 0x55954d45e4d0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.571+0000 7ff9b6809740 1 -- 172.21.15.78:0/77410732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a4015070 con 0x55954d45e4d0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.571+0000 7ff990ff9640 1 -- 172.21.15.78:0/77410732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9a400be70 con 0x55954d45e4d0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.571+0000 7ff9b6809740 10 client.15960 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.571+0000 7ff9b6809740 1 -- 172.21.15.78:0/77410732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55954d83f7b0 con 0x55954d45e4d0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.572+0000 7ff990ff9640 1 -- 172.21.15.78:0/77410732 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a4015070 con 0x55954d45e4d0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.572+0000 7ff990ff9640 1 -- 172.21.15.78:0/77410732 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9a4022070 con 0x55954d45e4d0 2024-05-21T15:17:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff9b6809740 10 client.15960 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff9b6809740 10 client.15960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:17:17.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff9b6809740 20 client.15960 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff9b6809740 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55954d445f60 0x55954d842f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff9b6809740 4 client.15960 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:17.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff9b6809740 1 -- 172.21.15.78:0/77410732 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55954d799370 con 0x55954d445f60 2024-05-21T15:17:17.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff9acef9640 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55954d445f60 0x55954d842f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff9acef9640 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55954d445f60 0x55954d842f60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acb478a345e90675 server_cookie=a6f18d9ddd4d4075 in_seq=0 out_seq=0 2024-05-21T15:17:17.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.573+0000 7ff990ff9640 10 client.15960 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:17:17.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.574+0000 7ff990ff9640 1 -- 172.21.15.78:0/77410732 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55954d799370 con 0x55954d445f60 2024-05-21T15:17:17.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.574+0000 7ff990ff9640 10 client.15960 handle_command_reply: tid=0 2024-05-21T15:17:17.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.574+0000 7ff9b6809740 1 client.15960 shutdown 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.574+0000 7ff9b6809740 1 -- 172.21.15.78:0/77410732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 msgr2=0x55954d830360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.574+0000 7ff9b6809740 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d830360 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4018000 tx=0x7ff9a4004ec0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 1 -- 172.21.15.78:0/77410732 shutdown_connections 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55954d445f60 0x55954d842f60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d836500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d4666b0 0x55954d8308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 1 --2- 172.21.15.78:0/77410732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d45e4d0 0x55954d830360 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 1 -- 172.21.15.78:0/77410732 >> 172.21.15.78:0/77410732 conn(0x55954d47b4e0 msgr2=0x55954d827a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff990ff9640 0 client.15960 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 1 -- 172.21.15.78:0/77410732 shutdown_connections 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 1 -- 172.21.15.78:0/77410732 wait complete. 2024-05-21T15:17:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.575+0000 7ff9b6809740 20 client.15960 trim_cache size 0 max 16384 2024-05-21T15:17:17.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.588+0000 7ff9b6809740 1 Processor -- start 2024-05-21T15:17:17.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.588+0000 7ff9b6809740 1 -- start start 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d4666b0 0x55954d836470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d82f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d8299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9b6809740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81b0e30 con 0x55954d4666b0 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9b6809740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81b2970 con 0x55954d487390 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9b6809740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81b1500 con 0x55954d41b3a0 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9ad6fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d4666b0 0x55954d836470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9acef9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d82f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9ad6fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d4666b0 0x55954d836470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46460/0 (socket says 172.21.15.78:46460) 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9acef9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d82f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:48886/0 (socket says 172.21.15.78:48886) 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/3399472475 learned_addr learned my addr 172.21.15.78:0/3399472475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9adefb640 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d8299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9acef9640 1 -- 172.21.15.78:0/3399472475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 msgr2=0x55954d8299d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9acef9640 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d8299d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9acef9640 1 -- 172.21.15.78:0/3399472475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d4666b0 msgr2=0x55954d836470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9acef9640 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d4666b0 0x55954d836470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9acef9640 1 -- 172.21.15.78:0/3399472475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55954d826ac0 con 0x55954d487390 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d4666b0 0x55954d836470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.589+0000 7ff9acef9640 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d82f130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ecac0 tx=0x7ff998003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19773282db4dfea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.590+0000 7ff992ffd640 1 -- 172.21.15.78:0/3399472475 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998049040 con 0x55954d487390 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.590+0000 7ff992ffd640 1 -- 172.21.15.78:0/3399472475 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff99803b580 con 0x55954d487390 2024-05-21T15:17:17.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.590+0000 7ff992ffd640 1 -- 172.21.15.78:0/3399472475 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998003800 con 0x55954d487390 2024-05-21T15:17:17.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.590+0000 7ff9b6809740 1 -- 172.21.15.78:0/3399472475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 msgr2=0x55954d82f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.590+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d82f130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ecac0 tx=0x7ff998003e90 comp rx=0 tx=0).stop 2024-05-21T15:17:17.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.590+0000 7ff9b6809740 1 -- 172.21.15.78:0/3399472475 shutdown_connections 2024-05-21T15:17:17.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.590+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d487390 0x55954d82f130 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d8299d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3399472475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d4666b0 0x55954d836470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 -- 172.21.15.78:0/3399472475 >> 172.21.15.78:0/3399472475 conn(0x55954d47b4e0 msgr2=0x55954d82ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 -- 172.21.15.78:0/3399472475 shutdown_connections 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 -- 172.21.15.78:0/3399472475 wait complete. 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 -- start start 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 Processor -- start 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d837970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.591+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 0x55954d837eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9b6809740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d487390 0x55954d8383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9b6809740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81b22a0 con 0x55954d487390 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9b6809740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81b1bd0 con 0x55954d4666b0 2024-05-21T15:17:17.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9b6809740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a81aee30 con 0x55954d41b3a0 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9b6809740 10 client.0 _ll_register_callbacks cb 0x7ff9a4004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9ad6fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d837970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9ad6fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d837970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:52432/0 (socket says 172.21.15.78:52432) 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9ad6fa640 1 -- 172.21.15.78:0/3122148091 learned_addr learned my addr 172.21.15.78:0/3122148091 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9acef9640 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 0x55954d837eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9adefb640 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d487390 0x55954d8383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9acef9640 1 -- 172.21.15.78:0/3122148091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 msgr2=0x55954d837970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9acef9640 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d837970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9acef9640 1 -- 172.21.15.78:0/3122148091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d487390 msgr2=0x55954d8383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9acef9640 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d487390 0x55954d8383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9acef9640 1 -- 172.21.15.78:0/3122148091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55954d8a9110 con 0x55954d4666b0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d837970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff9acef9640 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 0x55954d837eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ef520 tx=0x7ff9980c46c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ccbe1e46787fb36 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.592+0000 7ff990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff990ff9640 1 -- 172.21.15.78:0/3122148091 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998049070 con 0x55954d4666b0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff990ff9640 1 -- 172.21.15.78:0/3122148091 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7ff9980b7fc0 con 0x55954d4666b0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 1 -- 172.21.15.78:0/3122148091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55954d7936c0 con 0x55954d4666b0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff990ff9640 1 -- 172.21.15.78:0/3122148091 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9980bb720 con 0x55954d4666b0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 10 client.25478 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 1 -- 172.21.15.78:0/3122148091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55954d8aa380 con 0x55954d4666b0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff990ff9640 1 -- 172.21.15.78:0/3122148091 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9980bc050 con 0x55954d4666b0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff990ff9640 1 -- 172.21.15.78:0/3122148091 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9980c4130 con 0x55954d4666b0 2024-05-21T15:17:17.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 10 client.25478 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 10 client.25478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 20 client.25478 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55954d8aaa60 0x55954d8ace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 4 client.25478 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.593+0000 7ff9b6809740 1 -- 172.21.15.78:0/3122148091 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55954d7936c0 con 0x55954d8aaa60 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.594+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55954d8aaa60 0x55954d8ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.594+0000 7ff990ff9640 10 client.25478 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.594+0000 7ff9ad6fa640 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55954d8aaa60 0x55954d8ace80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3c5ef7660c3f369 server_cookie=2cecaf239a0a7c24 in_seq=0 out_seq=0 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.594+0000 7ff990ff9640 1 -- 172.21.15.78:0/3122148091 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55954d7936c0 con 0x55954d8aaa60 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.594+0000 7ff990ff9640 10 client.25478 handle_command_reply: tid=0 2024-05-21T15:17:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.594+0000 7ff9b6809740 1 client.25478 shutdown 2024-05-21T15:17:17.603 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:17 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2667330496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:17:17.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 -- 172.21.15.78:0/3122148091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 msgr2=0x55954d837eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 0x55954d837eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ef520 tx=0x7ff9980c46c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 -- 172.21.15.78:0/3122148091 shutdown_connections 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55954d8aaa60 0x55954d8ace80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55954d4666b0 0x55954d837eb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55954d487390 0x55954d8383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 --2- 172.21.15.78:0/3122148091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55954d41b3a0 0x55954d837970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 -- 172.21.15.78:0/3122148091 >> 172.21.15.78:0/3122148091 conn(0x55954d47b4e0 msgr2=0x55954d825ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff990ff9640 0 client.25478 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 -- 172.21.15.78:0/3122148091 shutdown_connections 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 1 -- 172.21.15.78:0/3122148091 wait complete. 2024-05-21T15:17:17.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.595+0000 7ff9b6809740 20 client.25478 trim_cache size 0 max 16384 2024-05-21T15:17:17.605 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:17.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 -- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff9940a2e20 msgr2=0x7ff9940a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff9940a2e20 0x7ff9940a5330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009370 tx=0x7ff9a00092c0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 -- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 msgr2=0x7ff9a8199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 0x7ff9a8199d00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff998006010 tx=0x7ff9980042f0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 -- 172.21.15.78:0/1362846578 shutdown_connections 2024-05-21T15:17:17.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a810cff0 0x7ff9a819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff9940a2e20 0x7ff9940a5330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a810a600 0x7ff9a8199d00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 --2- 172.21.15.78:0/1362846578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9a8103ba0 0x7ff9a81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 -- 172.21.15.78:0/1362846578 >> 172.21.15.78:0/1362846578 conn(0x7ff9a80770b0 msgr2=0x7ff9a8100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 -- 172.21.15.78:0/1362846578 shutdown_connections 2024-05-21T15:17:17.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.596+0000 7ff9af95e640 1 -- 172.21.15.78:0/1362846578 wait complete. 2024-05-21T15:17:17.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:17:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:17 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2667330496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:17:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:17 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2667330496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.729+0000 7fb3720ca640 1 -- 172.21.15.78:0/1649751981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10d010 msgr2=0x7fb36c10f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.729+0000 7fb3720ca640 1 --2- 172.21.15.78:0/1649751981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10d010 0x7fb36c10f500 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb364009ca0 tx=0x7fb36402a9e0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.730+0000 7fb3720ca640 1 -- 172.21.15.78:0/1649751981 shutdown_connections 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.730+0000 7fb3720ca640 1 --2- 172.21.15.78:0/1649751981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c10a660 0x7fb36c10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.730+0000 7fb3720ca640 1 --2- 172.21.15.78:0/1649751981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10d010 0x7fb36c10f500 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.730+0000 7fb3720ca640 1 --2- 172.21.15.78:0/1649751981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb36c103c00 0x7fb36c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.730+0000 7fb3720ca640 1 -- 172.21.15.78:0/1649751981 >> 172.21.15.78:0/1649751981 conn(0x7fb36c0770b0 msgr2=0x7fb36c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.730+0000 7fb3720ca640 1 -- 172.21.15.78:0/1649751981 shutdown_connections 2024-05-21T15:17:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.730+0000 7fb3720ca640 1 -- 172.21.15.78:0/1649751981 wait complete. 2024-05-21T15:17:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.730+0000 7fb3720ca640 1 Processor -- start 2024-05-21T15:17:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb3720ca640 1 -- start start 2024-05-21T15:17:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb3720ca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c103c00 0x7fb36c1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb3720ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10a660 0x7fb36c101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb3720ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb36c10d010 0x7fb36c101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb3720ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c112dc0 con 0x7fb36c10a660 2024-05-21T15:17:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb3720ca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c112c40 con 0x7fb36c103c00 2024-05-21T15:17:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb3720ca640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c112f40 con 0x7fb36c10d010 2024-05-21T15:17:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb36affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10a660 0x7fb36c101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb36affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10a660 0x7fb36c101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46476/0 (socket says 172.21.15.78:46476) 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb36affd640 1 -- 172.21.15.78:0/4024131314 learned_addr learned my addr 172.21.15.78:0/4024131314 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c103c00 0x7fb36c1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.731+0000 7fb36bfff640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb36c10d010 0x7fb36c101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb36affd640 1 -- 172.21.15.78:0/4024131314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb36c10d010 msgr2=0x7fb36c101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb36affd640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb36c10d010 0x7fb36c101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb36affd640 1 -- 172.21.15.78:0/4024131314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c103c00 msgr2=0x7fb36c1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb36affd640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c103c00 0x7fb36c1014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb36affd640 1 -- 172.21.15.78:0/4024131314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c1a7a80 con 0x7fb36c10a660 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb36affd640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10a660 0x7fb36c101a00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb35800d950 tx=0x7fb35800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb368ff9640 1 -- 172.21.15.78:0/4024131314 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358017070 con 0x7fb36c10a660 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb3720ca640 1 -- 172.21.15.78:0/4024131314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36c1a7d70 con 0x7fb36c10a660 2024-05-21T15:17:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb3720ca640 1 -- 172.21.15.78:0/4024131314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36c1a8280 con 0x7fb36c10a660 2024-05-21T15:17:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb368ff9640 1 -- 172.21.15.78:0/4024131314 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb358009c30 con 0x7fb36c10a660 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.732+0000 7fb368ff9640 1 -- 172.21.15.78:0/4024131314 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35800f6b0 con 0x7fb36c10a660 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb378f6c740 1 Processor -- start 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb378f6c740 1 -- start start 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb368ff9640 1 -- 172.21.15.78:0/4024131314 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fb35800f850 con 0x7fb36c10a660 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f72390 0x562a91310d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f494d0 0x562a91311260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a913117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb378f6c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562a91276c40 con 0x562a90f516b0 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb378f6c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562a9121e580 con 0x562a90f72390 2024-05-21T15:17:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.736+0000 7fb378f6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a91272c00 con 0x562a90f494d0 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f72390 0x562a91310d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a913117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f72390 0x562a91310d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:48924/0 (socket says 172.21.15.78:48924) 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36bfff640 1 -- 172.21.15.78:0/142982588 learned_addr learned my addr 172.21.15.78:0/142982588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f494d0 0x562a91311260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36bfff640 1 -- 172.21.15.78:0/142982588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f494d0 msgr2=0x562a91311260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36bfff640 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f494d0 0x562a91311260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36bfff640 1 -- 172.21.15.78:0/142982588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 msgr2=0x562a913117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36bfff640 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a913117a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36bfff640 1 -- 172.21.15.78:0/142982588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a90f8a3f0 con 0x562a90f72390 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.737+0000 7fb36affd640 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a913117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.738+0000 7fb368ff9640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2f20 0x7fb3500a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.738+0000 7fb36bfff640 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f72390 0x562a91310d20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb364035000 tx=0x7fb364005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a42ba113c8220be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.738+0000 7fb361ffb640 1 -- 172.21.15.78:0/142982588 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb364033050 con 0x562a90f72390 2024-05-21T15:17:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.738+0000 7fb368ff9640 1 -- 172.21.15.78:0/4024131314 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb3580cf5a0 con 0x7fb36c10a660 2024-05-21T15:17:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.738+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2f20 0x7fb3500a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.738+0000 7fb361ffb640 1 -- 172.21.15.78:0/142982588 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb364047070 con 0x562a90f72390 2024-05-21T15:17:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.738+0000 7fb361ffb640 1 -- 172.21.15.78:0/142982588 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36402f640 con 0x562a90f72390 2024-05-21T15:17:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 -- 172.21.15.78:0/142982588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f72390 msgr2=0x562a91310d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f72390 0x562a91310d20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb364035000 tx=0x7fb364005c50 comp rx=0 tx=0).stop 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2f20 0x7fb3500a5430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x562a913122e0 tx=0x7fb35c00b390 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 -- 172.21.15.78:0/142982588 shutdown_connections 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a913117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f494d0 0x562a91311260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 --2- 172.21.15.78:0/142982588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f72390 0x562a91310d20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 -- 172.21.15.78:0/142982588 >> 172.21.15.78:0/142982588 conn(0x562a90f664e0 msgr2=0x562a9130d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 -- 172.21.15.78:0/142982588 shutdown_connections 2024-05-21T15:17:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 -- 172.21.15.78:0/142982588 wait complete. 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.739+0000 7fb378f6c740 1 -- start start 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb378f6c740 1 Processor -- start 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 0x562a9131b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9131b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f72390 0x562a91327500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb378f6c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562a9131e000 con 0x562a90f72390 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb378f6c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562a91231c40 con 0x562a90f494d0 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb378f6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a9131e180 con 0x562a90f516b0 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb378f6c740 10 client.0 _ll_register_callbacks cb 0x562a90f89be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb36affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9131b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb36affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9131b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:52448/0 (socket says 172.21.15.78:52448) 2024-05-21T15:17:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb36affd640 1 -- 172.21.15.78:0/1923168634 learned_addr learned my addr 172.21.15.78:0/1923168634 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb36bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f72390 0x562a91327500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.740+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 0x562a9131b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb36bfff640 1 -- 172.21.15.78:0/1923168634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 msgr2=0x562a9131b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb36bfff640 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9131b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb36bfff640 1 -- 172.21.15.78:0/1923168634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 msgr2=0x562a9131b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb36bfff640 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 0x562a9131b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb36bfff640 1 -- 172.21.15.78:0/1923168634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a910547d0 con 0x562a90f72390 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb36affd640 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9131b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb36bfff640 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f72390 0x562a91327500 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb364005cc0 tx=0x7fb36402e390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ce68566b0f19a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb34effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb34effd640 1 -- 172.21.15.78:0/1923168634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb364003d80 con 0x562a90f72390 2024-05-21T15:17:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb34effd640 1 -- 172.21.15.78:0/1923168634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb36402f640 con 0x562a90f72390 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb378f6c740 1 -- 172.21.15.78:0/1923168634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb364047070 con 0x562a90f72390 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.741+0000 7fb34effd640 1 -- 172.21.15.78:0/1923168634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb364033020 con 0x562a90f72390 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb378f6c740 10 client.15978 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb378f6c740 1 -- 172.21.15.78:0/1923168634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a9132a710 con 0x562a90f72390 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb34effd640 1 -- 172.21.15.78:0/1923168634 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36402fd20 con 0x562a90f72390 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb34effd640 1 -- 172.21.15.78:0/1923168634 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3640045a0 con 0x562a90f72390 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb378f6c740 10 client.15978 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb378f6c740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb378f6c740 20 client.15978 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562a90f30f60 0x562a9132da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb378f6c740 4 client.15978 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb378f6c740 1 -- 172.21.15.78:0/1923168634 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb364047070 con 0x562a90f30f60 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.742+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562a90f30f60 0x562a9132da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.743+0000 7fb34effd640 10 client.15978 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.743+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562a90f30f60 0x562a9132da70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39439eb3a7c2ba12 server_cookie=4ec6be44f8d1a659 in_seq=0 out_seq=0 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb34effd640 1 -- 172.21.15.78:0/1923168634 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb364047070 con 0x562a90f30f60 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb34effd640 10 client.15978 handle_command_reply: tid=0 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 client.15978 shutdown 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 -- 172.21.15.78:0/1923168634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f72390 msgr2=0x562a91327500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f72390 0x562a91327500 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb364005cc0 tx=0x7fb36402e390 comp rx=0 tx=0).stop 2024-05-21T15:17:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 -- 172.21.15.78:0/1923168634 shutdown_connections 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562a90f30f60 0x562a9132da70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb34effd640 0 client.15978 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f72390 0x562a91327500 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9131b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1923168634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 0x562a9131b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 -- 172.21.15.78:0/1923168634 >> 172.21.15.78:0/1923168634 conn(0x562a90f664e0 msgr2=0x562a91312a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 -- 172.21.15.78:0/1923168634 shutdown_connections 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 1 -- 172.21.15.78:0/1923168634 wait complete. 2024-05-21T15:17:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.744+0000 7fb378f6c740 20 client.15978 trim_cache size 0 max 16384 2024-05-21T15:17:17.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.770+0000 7fb378f6c740 1 Processor -- start 2024-05-21T15:17:17.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb378f6c740 1 -- start start 2024-05-21T15:17:17.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f30f60 0x562a91327470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 0x562a9131cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a9131d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb378f6c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb35400e950 con 0x562a90f516b0 2024-05-21T15:17:17.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb378f6c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb354010490 con 0x562a90f494d0 2024-05-21T15:17:17.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb378f6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb35400f020 con 0x562a90f30f60 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb36bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a9131d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb36bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a9131d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46496/0 (socket says 172.21.15.78:46496) 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb36bfff640 1 -- 172.21.15.78:0/2280919633 learned_addr learned my addr 172.21.15.78:0/2280919633 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f30f60 0x562a91327470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.771+0000 7fb36affd640 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 0x562a9131cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb36b7fe640 1 -- 172.21.15.78:0/2280919633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 msgr2=0x562a9131cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 0x562a9131cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb36b7fe640 1 -- 172.21.15.78:0/2280919633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 msgr2=0x562a9131d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a9131d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb36b7fe640 1 -- 172.21.15.78:0/2280919633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a9131df50 con 0x562a90f30f60 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb36bfff640 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a9131d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f30f60 0x562a91327470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x562a9131e180 tx=0x7fb35c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de74baa309431270 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb3617fa640 1 -- 172.21.15.78:0/2280919633 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c02d070 con 0x562a90f30f60 2024-05-21T15:17:17.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb3617fa640 1 -- 172.21.15.78:0/2280919633 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35c028890 con 0x562a90f30f60 2024-05-21T15:17:17.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.772+0000 7fb3617fa640 1 -- 172.21.15.78:0/2280919633 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c027b00 con 0x562a90f30f60 2024-05-21T15:17:17.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.773+0000 7fb378f6c740 1 -- 172.21.15.78:0/2280919633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f30f60 msgr2=0x562a91327470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.773+0000 7fb378f6c740 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f30f60 0x562a91327470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x562a9131e180 tx=0x7fb35c015000 comp rx=0 tx=0).stop 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.773+0000 7fb378f6c740 1 -- 172.21.15.78:0/2280919633 shutdown_connections 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.774+0000 7fb378f6c740 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f494d0 0x562a9131cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.774+0000 7fb378f6c740 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f516b0 0x562a9131d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.774+0000 7fb378f6c740 1 --2- 172.21.15.78:0/2280919633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f30f60 0x562a91327470 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.774+0000 7fb378f6c740 1 -- 172.21.15.78:0/2280919633 >> 172.21.15.78:0/2280919633 conn(0x562a90f664e0 msgr2=0x562a9131c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.774+0000 7fb378f6c740 1 -- 172.21.15.78:0/2280919633 shutdown_connections 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.774+0000 7fb378f6c740 1 -- 172.21.15.78:0/2280919633 wait complete. 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.774+0000 7fb378f6c740 1 -- start start 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.775+0000 7fb378f6c740 1 Processor -- start 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.775+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f30f60 0x562a9132e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.775+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f494d0 0x562a9132ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.775+0000 7fb378f6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9132efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.775+0000 7fb378f6c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb35400fdc0 con 0x562a90f494d0 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.775+0000 7fb378f6c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb35400f6f0 con 0x562a90f30f60 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.775+0000 7fb36affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f494d0 0x562a9132ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.775+0000 7fb378f6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb35400c930 con 0x562a90f516b0 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f494d0 0x562a9132ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46498/0 (socket says 172.21.15.78:46498) 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36affd640 1 -- 172.21.15.78:0/1321706247 learned_addr learned my addr 172.21.15.78:0/1321706247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb378f6c740 10 client.0 _ll_register_callbacks cb 0x7fb3580d2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36bfff640 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9132efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36affd640 1 -- 172.21.15.78:0/1321706247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 msgr2=0x562a9132efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36affd640 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9132efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36affd640 1 -- 172.21.15.78:0/1321706247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f30f60 msgr2=0x562a9132e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36affd640 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f30f60 0x562a9132e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36affd640 1 -- 172.21.15.78:0/1321706247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a91311cb0 con 0x562a90f494d0 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb36affd640 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f494d0 0x562a9132ea80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb358000c00 tx=0x7fb35800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d26b8e1fdcdd95a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb34effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb34effd640 1 -- 172.21.15.78:0/1321706247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358020040 con 0x562a90f494d0 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb34effd640 1 -- 172.21.15.78:0/1321706247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb3580a4190 con 0x562a90f494d0 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.776+0000 7fb378f6c740 1 -- 172.21.15.78:0/1321706247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb35c028890 con 0x562a90f494d0 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb34effd640 1 -- 172.21.15.78:0/1321706247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb35809b3a0 con 0x562a90f494d0 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb378f6c740 10 client.15984 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:17.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb378f6c740 1 -- 172.21.15.78:0/1321706247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a91312310 con 0x562a90f494d0 2024-05-21T15:17:17.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb34effd640 1 -- 172.21.15.78:0/1321706247 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35c028890 con 0x562a90f494d0 2024-05-21T15:17:17.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb34effd640 1 -- 172.21.15.78:0/1321706247 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3580cce20 con 0x562a90f494d0 2024-05-21T15:17:17.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb378f6c740 10 client.15984 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:17.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb378f6c740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:17.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb378f6c740 20 client.15984 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:17.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562a90f72390 0x562a913981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:17.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb378f6c740 4 client.15984 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:17.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.777+0000 7fb378f6c740 1 -- 172.21.15.78:0/1321706247 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a91284370 con 0x562a90f72390 2024-05-21T15:17:17.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.778+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562a90f72390 0x562a913981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:17.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.778+0000 7fb36b7fe640 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562a90f72390 0x562a913981d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e5b21c748a586a server_cookie=5a53aa7d6c6ee407 in_seq=0 out_seq=0 2024-05-21T15:17:17.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.778+0000 7fb34effd640 10 client.15984 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:17.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.778+0000 7fb34effd640 1 -- 172.21.15.78:0/1321706247 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a91284370 con 0x562a90f72390 2024-05-21T15:17:17.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.778+0000 7fb34effd640 10 client.15984 handle_command_reply: tid=0 2024-05-21T15:17:17.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.778+0000 7fb378f6c740 1 client.15984 shutdown 2024-05-21T15:17:17.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 -- 172.21.15.78:0/1321706247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f494d0 msgr2=0x562a9132ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f494d0 0x562a9132ea80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb358000c00 tx=0x7fb35800bf60 comp rx=0 tx=0).stop 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 -- 172.21.15.78:0/1321706247 shutdown_connections 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562a90f72390 0x562a913981d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a90f516b0 0x562a9132efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562a90f494d0 0x562a9132ea80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 --2- 172.21.15.78:0/1321706247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562a90f30f60 0x562a9132e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 -- 172.21.15.78:0/1321706247 >> 172.21.15.78:0/1321706247 conn(0x562a90f664e0 msgr2=0x562a9131d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb34effd640 0 client.15984 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 -- 172.21.15.78:0/1321706247 shutdown_connections 2024-05-21T15:17:17.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 1 -- 172.21.15.78:0/1321706247 wait complete. 2024-05-21T15:17:17.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.779+0000 7fb378f6c740 20 client.15984 trim_cache size 0 max 16384 2024-05-21T15:17:17.789 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:17.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 -- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2f20 msgr2=0x7fb3500a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2f20 0x7fb3500a5430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x562a913122e0 tx=0x7fb35c00b390 comp rx=0 tx=0).stop 2024-05-21T15:17:17.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 -- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10a660 msgr2=0x7fb36c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:17.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10a660 0x7fb36c101a00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb35800d950 tx=0x7fb35800de10 comp rx=0 tx=0).stop 2024-05-21T15:17:17.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 -- 172.21.15.78:0/4024131314 shutdown_connections 2024-05-21T15:17:17.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2f20 0x7fb3500a5430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb36c10d010 0x7fb36c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb36c10a660 0x7fb36c101a00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 --2- 172.21.15.78:0/4024131314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb36c103c00 0x7fb36c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:17.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 -- 172.21.15.78:0/4024131314 >> 172.21.15.78:0/4024131314 conn(0x7fb36c0770b0 msgr2=0x7fb36c100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:17.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 -- 172.21.15.78:0/4024131314 shutdown_connections 2024-05-21T15:17:17.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:17.780+0000 7fb3720ca640 1 -- 172.21.15.78:0/4024131314 wait complete. 2024-05-21T15:17:18.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:18 smithi078 ceph-mon[31997]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s 2024-05-21T15:17:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:18 smithi084 ceph-mon[36334]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s 2024-05-21T15:17:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:18 smithi172 ceph-mon[36402]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s 2024-05-21T15:17:18.799 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:17:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:17:18.799 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:17:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 -- 172.21.15.78:0/3221939278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c101b10 msgr2=0x7f9a2c101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/3221939278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c101b10 0x7f9a2c101f20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009900 tx=0x7f9a1c02a410 comp rx=0 tx=0).stop 2024-05-21T15:17:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 -- 172.21.15.78:0/3221939278 shutdown_connections 2024-05-21T15:17:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/3221939278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a2c10c350 0x7f9a2c10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/3221939278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c101b10 0x7f9a2c101f20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:18.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/3221939278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c1079f0 0x7f9a2c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 -- 172.21.15.78:0/3221939278 >> 172.21.15.78:0/3221939278 conn(0x7f9a2c0fd210 msgr2=0x7f9a2c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 -- 172.21.15.78:0/3221939278 shutdown_connections 2024-05-21T15:17:18.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.976+0000 7f9a33c8d640 1 -- 172.21.15.78:0/3221939278 wait complete. 2024-05-21T15:17:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.977+0000 7f9a33c8d640 1 Processor -- start 2024-05-21T15:17:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.977+0000 7f9a33c8d640 1 -- start start 2024-05-21T15:17:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.977+0000 7f9a33c8d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a2c101b10 0x7f9a2c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.977+0000 7f9a33c8d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c1079f0 0x7f9a2c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.977+0000 7f9a33c8d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c10c350 0x7f9a2c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.977+0000 7f9a3222a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c10c350 0x7f9a2c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a3222a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c10c350 0x7f9a2c19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44626/0 (socket says 172.21.15.78:44626) 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a3222a640 1 -- 172.21.15.78:0/2579514744 learned_addr learned my addr 172.21.15.78:0/2579514744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c112e10 con 0x7f9a2c10c350 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c112c90 con 0x7f9a2c1079f0 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c112f90 con 0x7f9a2c101b10 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a31a29640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a2c101b10 0x7f9a2c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a31228640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c1079f0 0x7f9a2c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a3222a640 1 -- 172.21.15.78:0/2579514744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a2c101b10 msgr2=0x7f9a2c199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a3222a640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a2c101b10 0x7f9a2c199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a3222a640 1 -- 172.21.15.78:0/2579514744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c1079f0 msgr2=0x7f9a2c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a3222a640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c1079f0 0x7f9a2c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a3222a640 1 -- 172.21.15.78:0/2579514744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c19e5f0 con 0x7f9a2c10c350 2024-05-21T15:17:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a31228640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c1079f0 0x7f9a2c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a3222a640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c10c350 0x7f9a2c19dfb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400eb10 tx=0x7f9a2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.978+0000 7f9a22ffd640 1 -- 172.21.15.78:0/2579514744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24004b20 con 0x7f9a2c10c350 2024-05-21T15:17:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.979+0000 7f9a22ffd640 1 -- 172.21.15.78:0/2579514744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a24004cc0 con 0x7f9a2c10c350 2024-05-21T15:17:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.979+0000 7f9a22ffd640 1 -- 172.21.15.78:0/2579514744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24010760 con 0x7f9a2c10c350 2024-05-21T15:17:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.979+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c19e8e0 con 0x7f9a2c10c350 2024-05-21T15:17:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.980+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c19ed50 con 0x7f9a2c10c350 2024-05-21T15:17:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.983+0000 7f9a22ffd640 1 -- 172.21.15.78:0/2579514744 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f9a24005020 con 0x7f9a2c10c350 2024-05-21T15:17:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.984+0000 7f9a3ac0b740 1 Processor -- start 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.984+0000 7f9a3ac0b740 1 -- start start 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.984+0000 7f9a22ffd640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9a180a2e20 0x7f9a180a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.984+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee628620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a22ffd640 1 -- 172.21.15.78:0/2579514744 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9a240cfec0 con 0x7f9a2c10c350 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a31228640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9a180a2e20 0x7f9a180a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a31a29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee628620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a31a29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee628620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44640/0 (socket says 172.21.15.78:44640) 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a31a29640 1 -- 172.21.15.78:0/3243672217 learned_addr learned my addr 172.21.15.78:0/3243672217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee628b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee6290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3243672217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561aee58d990 con 0x561aee289410 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3243672217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561aee5355b0 con 0x561aee268730 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3243672217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561aee589950 con 0x561aee260550 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a3222a640 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee6290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a31228640 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee628b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.985+0000 7f9a31228640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9a180a2e20 0x7f9a180a5330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005940 tx=0x7f9a1c0058d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a31228640 1 -- 172.21.15.78:0/3243672217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 msgr2=0x561aee6290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a31228640 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee6290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a31228640 1 -- 172.21.15.78:0/3243672217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 msgr2=0x561aee628620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a31228640 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee628620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a31228640 1 -- 172.21.15.78:0/3243672217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aee3697e0 con 0x561aee260550 2024-05-21T15:17:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a3222a640 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee6290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a31a29640 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee628620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a31228640 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee628b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c02a930 tx=0x7f9a1c03d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e8d88d7a20b542 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a17fff640 1 -- 172.21.15.78:0/3243672217 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c04c070 con 0x561aee260550 2024-05-21T15:17:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a17fff640 1 -- 172.21.15.78:0/3243672217 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c005520 con 0x561aee260550 2024-05-21T15:17:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.986+0000 7f9a17fff640 1 -- 172.21.15.78:0/3243672217 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c035040 con 0x561aee260550 2024-05-21T15:17:19.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.987+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3243672217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 msgr2=0x561aee628b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.987+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee628b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c02a930 tx=0x7f9a1c03d000 comp rx=0 tx=0).stop 2024-05-21T15:17:19.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.987+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3243672217 shutdown_connections 2024-05-21T15:17:19.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.987+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee6290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.987+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee628b60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.987+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3243672217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee628620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.987+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3243672217 >> 172.21.15.78:0/3243672217 conn(0x561aee27d560 msgr2=0x561aee624dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.988+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3243672217 shutdown_connections 2024-05-21T15:17:19.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.988+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3243672217 wait complete. 2024-05-21T15:17:19.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.988+0000 7f9a3ac0b740 1 -- start start 2024-05-21T15:17:19.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.988+0000 7f9a3ac0b740 1 Processor -- start 2024-05-21T15:17:19.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee6347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee634d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee632bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a3ac0b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561aee635910 con 0x561aee289410 2024-05-21T15:17:19.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a31228640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee634d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a31a29640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee6347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a3222a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee632bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a31228640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee634d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:48812/0 (socket says 172.21.15.78:48812) 2024-05-21T15:17:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a31228640 1 -- 172.21.15.78:0/665084691 learned_addr learned my addr 172.21.15.78:0/665084691 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561aee548b40 con 0x561aee268730 2024-05-21T15:17:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.989+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561aee635a90 con 0x561aee260550 2024-05-21T15:17:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a3ac0b740 10 client.0 _ll_register_callbacks cb 0x561aee53d0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a31a29640 1 -- 172.21.15.78:0/665084691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 msgr2=0x561aee634d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a31a29640 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee634d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a31a29640 1 -- 172.21.15.78:0/665084691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 msgr2=0x561aee632bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a31a29640 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee632bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a31a29640 1 -- 172.21.15.78:0/665084691 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aee3c9f70 con 0x561aee260550 2024-05-21T15:17:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a31228640 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee634d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c005520 con 0x561aee260550 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a3222a640 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee632bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.990+0000 7f9a31a29640 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee6347c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x561aee625790 tx=0x7f9a28012aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb56954b75a115e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.991+0000 7f9a157fa640 10 client.25513 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.991+0000 7f9a157fa640 1 -- 172.21.15.78:0/665084691 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280117f0 con 0x561aee260550 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.991+0000 7f9a157fa640 1 -- 172.21.15.78:0/665084691 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9a28011990 con 0x561aee260550 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.991+0000 7f9a157fa640 1 -- 172.21.15.78:0/665084691 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9a28011c10 con 0x561aee260550 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.991+0000 7f9a3ac0b740 10 client.25513 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.991+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=26}) -- 0x561aee261170 con 0x561aee260550 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.991+0000 7f9a157fa640 1 -- 172.21.15.78:0/665084691 <== mon.1 v2:172.21.15.172:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9a28026470 con 0x561aee260550 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a3ac0b740 10 client.25513 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a3ac0b740 10 client.25513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a3ac0b740 20 client.25513 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561aee2481f0 0x561aee644f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a3ac0b740 4 client.25513 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aee59b0c0 con 0x561aee2481f0 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a31228640 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561aee2481f0 0x561aee644f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a157fa640 10 client.25513 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.992+0000 7f9a31228640 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561aee2481f0 0x561aee644f90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ac5e14f5faab47 server_cookie=3cde0d89391e07fb in_seq=0 out_seq=0 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.993+0000 7f9a157fa640 1 -- 172.21.15.78:0/665084691 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561aee59b0c0 con 0x561aee2481f0 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.993+0000 7f9a157fa640 10 client.25513 handle_command_reply: tid=0 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.993+0000 7f9a3ac0b740 1 client.25513 shutdown 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.993+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 msgr2=0x561aee6347c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.993+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee6347c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x561aee625790 tx=0x7f9a28012aa0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.993+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 shutdown_connections 2024-05-21T15:17:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.993+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561aee2481f0 0x561aee644f90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.994+0000 7f9a157fa640 0 client.25513 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.994+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee268730 0x561aee634d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.994+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee289410 0x561aee632bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.994+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/665084691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee6347c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.994+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 >> 172.21.15.78:0/665084691 conn(0x561aee27d560 msgr2=0x561aee62a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.994+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 shutdown_connections 2024-05-21T15:17:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.994+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/665084691 wait complete. 2024-05-21T15:17:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:18.994+0000 7f9a3ac0b740 20 client.25513 trim_cache size 0 max 16384 2024-05-21T15:17:19.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.029+0000 7f9a3ac0b740 1 Processor -- start 2024-05-21T15:17:19.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a3ac0b740 1 -- start start 2024-05-21T15:17:19.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee2481f0 0x561aee62a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee260550 0x561aee62c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee268730 0x561aee62c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a3ac0b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1aea90 con 0x561aee2481f0 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a3ac0b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1b05d0 con 0x561aee260550 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a3ac0b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1af160 con 0x561aee268730 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a31a29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee2481f0 0x561aee62a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a31a29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee2481f0 0x561aee62a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44662/0 (socket says 172.21.15.78:44662) 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a31a29640 1 -- 172.21.15.78:0/4167840265 learned_addr learned my addr 172.21.15.78:0/4167840265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a31228640 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee260550 0x561aee62c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a3222a640 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee268730 0x561aee62c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a31228640 1 -- 172.21.15.78:0/4167840265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee268730 msgr2=0x561aee62c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.030+0000 7f9a31228640 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee268730 0x561aee62c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.031+0000 7f9a31228640 1 -- 172.21.15.78:0/4167840265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee2481f0 msgr2=0x561aee62a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.031+0000 7f9a31228640 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee2481f0 0x561aee62a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.031+0000 7f9a31228640 1 -- 172.21.15.78:0/4167840265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aee6257c0 con 0x561aee260550 2024-05-21T15:17:19.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.031+0000 7f9a31a29640 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee2481f0 0x561aee62a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:19.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.031+0000 7f9a31228640 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee260550 0x561aee62c2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005f70 tx=0x7f9a1c004900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44f8a7a32dd2aff server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.031+0000 7f9a177fe640 1 -- 172.21.15.78:0/4167840265 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c040040 con 0x561aee260550 2024-05-21T15:17:19.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.031+0000 7f9a177fe640 1 -- 172.21.15.78:0/4167840265 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c05fe50 con 0x561aee260550 2024-05-21T15:17:19.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.031+0000 7f9a177fe640 1 -- 172.21.15.78:0/4167840265 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c04c040 con 0x561aee260550 2024-05-21T15:17:19.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.032+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/4167840265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee260550 msgr2=0x561aee62c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.032+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee260550 0x561aee62c2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005f70 tx=0x7f9a1c004900 comp rx=0 tx=0).stop 2024-05-21T15:17:19.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.032+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/4167840265 shutdown_connections 2024-05-21T15:17:19.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.032+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee260550 0x561aee62c2d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.032+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee268730 0x561aee62c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.032+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/4167840265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee2481f0 0x561aee62a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.033+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/4167840265 >> 172.21.15.78:0/4167840265 conn(0x561aee27d560 msgr2=0x561aee62a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.033+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/4167840265 shutdown_connections 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.033+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/4167840265 wait complete. 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.033+0000 7f9a3ac0b740 1 -- start start 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.033+0000 7f9a3ac0b740 1 Processor -- start 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee2481f0 0x561aee638100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee638640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a3ac0b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee268730 0x561aee6af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a31a29640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee2481f0 0x561aee638100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a3222a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee268730 0x561aee6af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a31228640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee638640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a31a29640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee2481f0 0x561aee638100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:48830/0 (socket says 172.21.15.78:48830) 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a3222a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee268730 0x561aee6af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44674/0 (socket says 172.21.15.78:44674) 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a31228640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee638640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:59510/0 (socket says 172.21.15.78:59510) 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1aff00 con 0x561aee268730 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a31a29640 1 -- 172.21.15.78:0/3610439678 learned_addr learned my addr 172.21.15.78:0/3610439678 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1af830 con 0x561aee2481f0 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.034+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1aca70 con 0x561aee260550 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a31a29640 1 -- 172.21.15.78:0/3610439678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 msgr2=0x561aee638640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a31a29640 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee638640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a31a29640 1 -- 172.21.15.78:0/3610439678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee268730 msgr2=0x561aee6af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a31a29640 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee268730 0x561aee6af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a31a29640 1 -- 172.21.15.78:0/3610439678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aee6afc50 con 0x561aee2481f0 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a3ac0b740 10 client.0 _ll_register_callbacks cb 0x7f9a28009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:19.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a31228640 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee638640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a3222a640 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee268730 0x561aee6af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c05fe50 con 0x561aee2481f0 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a31a29640 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee2481f0 0x561aee638100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a28012ae0 tx=0x7f9a280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e794197939dc07a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a157fa640 10 client.25511 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.035+0000 7f9a157fa640 1 -- 172.21.15.78:0/3610439678 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28004040 con 0x561aee2481f0 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.036+0000 7f9a157fa640 1 -- 172.21.15.78:0/3610439678 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9a280041e0 con 0x561aee2481f0 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.036+0000 7f9a157fa640 1 -- 172.21.15.78:0/3610439678 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9a28004440 con 0x561aee2481f0 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a3ac0b740 10 client.25511 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:19.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({fsmap=26}) -- 0x561aee6b0d70 con 0x561aee2481f0 2024-05-21T15:17:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a157fa640 1 -- 172.21.15.78:0/3610439678 <== mon.2 v2:172.21.15.84:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9a280035f0 con 0x561aee2481f0 2024-05-21T15:17:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a3ac0b740 10 client.25511 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a3ac0b740 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a3ac0b740 20 client.25511 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561aee289410 0x561aee6b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a3ac0b740 4 client.25511 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:17:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.037+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561aee595410 con 0x561aee289410 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.038+0000 7f9a31228640 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561aee289410 0x561aee6b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.038+0000 7f9a157fa640 10 client.25511 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.038+0000 7f9a31228640 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561aee289410 0x561aee6b3390 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8501b14a5038715d server_cookie=9551ea3314883073 in_seq=0 out_seq=0 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.039+0000 7f9a157fa640 1 -- 172.21.15.78:0/3610439678 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561aee595410 con 0x561aee289410 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.039+0000 7f9a157fa640 10 client.25511 handle_command_reply: tid=0 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.039+0000 7f9a3ac0b740 1 client.25511 shutdown 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.039+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee2481f0 msgr2=0x561aee638100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.039+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee2481f0 0x561aee638100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a28012ae0 tx=0x7f9a280046c0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 shutdown_connections 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561aee289410 0x561aee6b3390 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561aee268730 0x561aee6af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a157fa640 0 client.25511 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:19.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561aee260550 0x561aee638640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a3ac0b740 1 --2- 172.21.15.78:0/3610439678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561aee2481f0 0x561aee638100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 >> 172.21.15.78:0/3610439678 conn(0x561aee27d560 msgr2=0x561aee634210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 shutdown_connections 2024-05-21T15:17:19.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.040+0000 7f9a3ac0b740 1 -- 172.21.15.78:0/3610439678 wait complete. 2024-05-21T15:17:19.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.041+0000 7f9a3ac0b740 20 client.25511 trim_cache size 0 max 16384 2024-05-21T15:17:19.047 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:17:19.047 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:17:19.048 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "a4783c14-30a5-4f83-aea8-d0a1682d7bfe", 2024-05-21T15:17:19.048 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:17:19.048 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.041+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9a180a2e20 msgr2=0x7f9a180a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.041+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9a180a2e20 0x7f9a180a5330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c005940 tx=0x7f9a1c0058d0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.041+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c10c350 msgr2=0x7f9a2c19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.041+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c10c350 0x7f9a2c19dfb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400eb10 tx=0x7f9a2400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.042+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 shutdown_connections 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.042+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a2c1079f0 0x7f9a2c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.042+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9a180a2e20 0x7f9a180a5330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.042+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a2c10c350 0x7f9a2c19dfb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.042+0000 7f9a33c8d640 1 --2- 172.21.15.78:0/2579514744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a2c101b10 0x7f9a2c199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.042+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 >> 172.21.15.78:0/2579514744 conn(0x7f9a2c0fd210 msgr2=0x7f9a2c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.042+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 shutdown_connections 2024-05-21T15:17:19.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.042+0000 7f9a33c8d640 1 -- 172.21.15.78:0/2579514744 wait complete. 2024-05-21T15:17:19.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:17:19.061 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:17:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.183+0000 7f16d5d84640 1 -- 172.21.15.78:0/1210359591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0108ad0 msgr2=0x7f16d0100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.183+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1210359591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0108ad0 0x7f16d0100b80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c002a410 comp rx=0 tx=0).stop 2024-05-21T15:17:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.183+0000 7f16d5d84640 1 -- 172.21.15.78:0/1210359591 shutdown_connections 2024-05-21T15:17:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.183+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1210359591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d0101770 0x7f16d010f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.183+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1210359591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d0108f70 0x7f16d01010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.183+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1210359591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0108ad0 0x7f16d0100b80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.183+0000 7f16d5d84640 1 -- 172.21.15.78:0/1210359591 >> 172.21.15.78:0/1210359591 conn(0x7f16d0076830 msgr2=0x7f16d0076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.184+0000 7f16d5d84640 1 -- 172.21.15.78:0/1210359591 shutdown_connections 2024-05-21T15:17:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.184+0000 7f16d5d84640 1 -- 172.21.15.78:0/1210359591 wait complete. 2024-05-21T15:17:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.184+0000 7f16d5d84640 1 Processor -- start 2024-05-21T15:17:19.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.184+0000 7f16d5d84640 1 -- start start 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16d5d84640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d0108ad0 0x7f16d0199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16d5d84640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d0199580 0x7f16d019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16d5d84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d019dda0 0x7f16d019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16d5d84640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112e60 con 0x7f16d019dda0 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16d5d84640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112ce0 con 0x7f16d0199580 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16d5d84640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112fe0 con 0x7f16d0108ad0 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d019dda0 0x7f16d019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d019dda0 0x7f16d019e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44694/0 (socket says 172.21.15.78:44694) 2024-05-21T15:17:19.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d0199580 0x7f16d019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16cffff640 1 -- 172.21.15.78:0/1528908351 learned_addr learned my addr 172.21.15.78:0/1528908351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16cf7fe640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d0108ad0 0x7f16d0199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16ceffd640 1 -- 172.21.15.78:0/1528908351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d0108ad0 msgr2=0x7f16d0199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d0108ad0 0x7f16d0199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16ceffd640 1 -- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d019dda0 msgr2=0x7f16d019e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d019dda0 0x7f16d019e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16ceffd640 1 -- 172.21.15.78:0/1528908351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d0101c90 con 0x7f16d0199580 2024-05-21T15:17:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.185+0000 7f16cffff640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d019dda0 0x7f16d019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.186+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d0199580 0x7f16d019d860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f16c400e9c0 tx=0x7f16c400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.186+0000 7f16ccff9640 1 -- 172.21.15.78:0/1528908351 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c400ccc0 con 0x7f16d0199580 2024-05-21T15:17:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.186+0000 7f16d5d84640 1 -- 172.21.15.78:0/1528908351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d0101f80 con 0x7f16d0199580 2024-05-21T15:17:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.186+0000 7f16ccff9640 1 -- 172.21.15.78:0/1528908351 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c4005020 con 0x7f16d0199580 2024-05-21T15:17:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.187+0000 7f16ccff9640 1 -- 172.21.15.78:0/1528908351 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4004a00 con 0x7f16d0199580 2024-05-21T15:17:19.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.187+0000 7f16d5d84640 1 -- 172.21.15.78:0/1528908351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d01024d0 con 0x7f16d0199580 2024-05-21T15:17:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.189+0000 7f16ccff9640 1 -- 172.21.15.78:0/1528908351 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f16c40051d0 con 0x7f16d0199580 2024-05-21T15:17:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16dcc1f740 1 Processor -- start 2024-05-21T15:17:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16dcc1f740 1 -- start start 2024-05-21T15:17:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2bea4390 0x564b2c242d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 0x564b2c243270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be836b0 0x564b2c2437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564b2c1a8d00 con 0x564b2bea4390 2024-05-21T15:17:19.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564b2c1504c0 con 0x564b2be836b0 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564b2c1a4cc0 con 0x564b2be7b4d0 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2bea4390 0x564b2c242d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2bea4390 0x564b2c242d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44704/0 (socket says 172.21.15.78:44704) 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.190+0000 7f16cffff640 1 -- 172.21.15.78:0/238524334 learned_addr learned my addr 172.21.15.78:0/238524334 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16ccff9640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16b80a2f20 0x7f16b80a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16cf7fe640 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 0x564b2c243270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16ceffd640 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be836b0 0x564b2c2437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16ccff9640 1 -- 172.21.15.78:0/1528908351 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f16c40cf660 con 0x7f16d0199580 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16cffff640 1 -- 172.21.15.78:0/238524334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 msgr2=0x564b2c243270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16cffff640 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 0x564b2c243270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16cffff640 1 -- 172.21.15.78:0/238524334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be836b0 msgr2=0x564b2c2437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16cffff640 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be836b0 0x564b2c2437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16cffff640 1 -- 172.21.15.78:0/238524334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b2bf867d0 con 0x564b2bea4390 2024-05-21T15:17:19.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16cffff640 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2bea4390 0x564b2c242d30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f16c80118b0 tx=0x7f16c8011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549121deb64ddbf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.191+0000 7f16cffff640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16b80a2f20 0x7f16b80a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.192+0000 7f16cffff640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16b80a2f20 0x7f16b80a5430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16c8012fd0 tx=0x7f16c8016000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.192+0000 7f16b5ffb640 1 -- 172.21.15.78:0/238524334 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c801e070 con 0x564b2bea4390 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.192+0000 7f16b5ffb640 1 -- 172.21.15.78:0/238524334 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c8010430 con 0x564b2bea4390 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.192+0000 7f16b5ffb640 1 -- 172.21.15.78:0/238524334 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8024640 con 0x564b2bea4390 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 -- 172.21.15.78:0/238524334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2bea4390 msgr2=0x564b2c242d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2bea4390 0x564b2c242d30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f16c80118b0 tx=0x7f16c8011d70 comp rx=0 tx=0).stop 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 -- 172.21.15.78:0/238524334 shutdown_connections 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be836b0 0x564b2c2437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 0x564b2c243270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/238524334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2bea4390 0x564b2c242d30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 -- 172.21.15.78:0/238524334 >> 172.21.15.78:0/238524334 conn(0x564b2be984e0 msgr2=0x564b2c23f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 -- 172.21.15.78:0/238524334 shutdown_connections 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.193+0000 7f16dcc1f740 1 -- 172.21.15.78:0/238524334 wait complete. 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 1 -- start start 2024-05-21T15:17:19.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 1 Processor -- start 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 0x564b2c24ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be836b0 0x564b2c24f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c24d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564b2c24ffc0 con 0x564b2be836b0 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564b2c163b90 con 0x564b2bea4390 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564b2c250140 con 0x564b2be7b4d0 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16dcc1f740 10 client.0 _ll_register_callbacks cb 0x564b2c19fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be836b0 0x564b2c24f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be836b0 0x564b2c24f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44714/0 (socket says 172.21.15.78:44714) 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.194+0000 7f16ceffd640 1 -- 172.21.15.78:0/3623604855 learned_addr learned my addr 172.21.15.78:0/3623604855 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16cf7fe640 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 0x564b2c24ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16cffff640 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c24d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16ceffd640 1 -- 172.21.15.78:0/3623604855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 msgr2=0x564b2c24ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16ceffd640 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 0x564b2c24ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16ceffd640 1 -- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 msgr2=0x564b2c24d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16ceffd640 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c24d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16ceffd640 1 -- 172.21.15.78:0/3623604855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b2bebc3f0 con 0x564b2be836b0 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16cffff640 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c24d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16ceffd640 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be836b0 0x564b2c24f360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f16c4097cd0 tx=0x7f16c406a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681e690c288d3483 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16aeffd640 1 -- 172.21.15.78:0/3623604855 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c401c040 con 0x564b2be836b0 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16aeffd640 1 -- 172.21.15.78:0/3623604855 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f16c4004c60 con 0x564b2be836b0 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16dcc1f740 1 -- 172.21.15.78:0/3623604855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c8010430 con 0x564b2be836b0 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16aeffd640 1 -- 172.21.15.78:0/3623604855 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16c4004e60 con 0x564b2be836b0 2024-05-21T15:17:19.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16dcc1f740 10 client.16014 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.195+0000 7f16dcc1f740 1 -- 172.21.15.78:0/3623604855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b2be7c0f0 con 0x564b2be836b0 2024-05-21T15:17:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16aeffd640 1 -- 172.21.15.78:0/3623604855 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c8010430 con 0x564b2be836b0 2024-05-21T15:17:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16aeffd640 1 -- 172.21.15.78:0/3623604855 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f16c40cce50 con 0x564b2be836b0 2024-05-21T15:17:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16dcc1f740 10 client.16014 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16dcc1f740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:19.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16dcc1f740 20 client.16014 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564b2be62f60 0x564b2c25fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16dcc1f740 4 client.16014 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16dcc1f740 1 -- 172.21.15.78:0/3623604855 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b2c1b6430 con 0x564b2be62f60 2024-05-21T15:17:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.196+0000 7f16cf7fe640 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564b2be62f60 0x564b2c25fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.197+0000 7f16aeffd640 10 client.16014 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:19.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.197+0000 7f16cf7fe640 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564b2be62f60 0x564b2c25fa70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1cb6ab4fc84a4b server_cookie=a9955a96f28a2fca in_seq=0 out_seq=0 2024-05-21T15:17:19.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.197+0000 7f16aeffd640 1 -- 172.21.15.78:0/3623604855 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b2c1b6430 con 0x564b2be62f60 2024-05-21T15:17:19.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.197+0000 7f16aeffd640 10 client.16014 handle_command_reply: tid=0 2024-05-21T15:17:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.197+0000 7f16dcc1f740 1 client.16014 shutdown 2024-05-21T15:17:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 -- 172.21.15.78:0/3623604855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be836b0 msgr2=0x564b2c24f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be836b0 0x564b2c24f360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f16c4097cd0 tx=0x7f16c406a7a0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 -- 172.21.15.78:0/3623604855 shutdown_connections 2024-05-21T15:17:19.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564b2be62f60 0x564b2c25fa70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16aeffd640 0 client.16014 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c24d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be836b0 0x564b2c24f360 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/3623604855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be7b4d0 0x564b2c24ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 -- 172.21.15.78:0/3623604855 >> 172.21.15.78:0/3623604855 conn(0x564b2be984e0 msgr2=0x564b2c244a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 -- 172.21.15.78:0/3623604855 shutdown_connections 2024-05-21T15:17:19.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 1 -- 172.21.15.78:0/3623604855 wait complete. 2024-05-21T15:17:19.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.198+0000 7f16dcc1f740 20 client.16014 trim_cache size 0 max 16384 2024-05-21T15:17:19.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.216+0000 7f16dcc1f740 1 Processor -- start 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.216+0000 7f16dcc1f740 1 -- start start 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.216+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be62f60 0x564b2c24c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.216+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be7b4d0 0x564b2c24f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.216+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c246ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.216+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b0f00 con 0x564b2be7b4d0 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.216+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b2a40 con 0x564b2bea4390 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.216+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b15d0 con 0x564b2be62f60 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be7b4d0 0x564b2c24f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be7b4d0 0x564b2c24f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44724/0 (socket says 172.21.15.78:44724) 2024-05-21T15:17:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 -- 172.21.15.78:0/1938099258 learned_addr learned my addr 172.21.15.78:0/1938099258 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16cffff640 1 --2- 172.21.15.78:0/1938099258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c246ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 -- 172.21.15.78:0/1938099258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be62f60 msgr2=0x564b2c24c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1938099258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be62f60 0x564b2c24c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 -- 172.21.15.78:0/1938099258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 msgr2=0x564b2c246ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1938099258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c246ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 -- 172.21.15.78:0/1938099258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b2c2400b0 con 0x564b2be7b4d0 2024-05-21T15:17:19.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1938099258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be7b4d0 0x564b2c24f0f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f16c40981f0 tx=0x7f16c4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205ba6b1c0cba84c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16b57fa640 1 -- 172.21.15.78:0/1938099258 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c401c070 con 0x564b2be7b4d0 2024-05-21T15:17:19.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16b57fa640 1 -- 172.21.15.78:0/1938099258 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c409b870 con 0x564b2be7b4d0 2024-05-21T15:17:19.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.217+0000 7f16b57fa640 1 -- 172.21.15.78:0/1938099258 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c40860c0 con 0x564b2be7b4d0 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1938099258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be7b4d0 msgr2=0x564b2c24f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1938099258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be7b4d0 0x564b2c24f0f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f16c40981f0 tx=0x7f16c4009f60 comp rx=0 tx=0).stop 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1938099258 shutdown_connections 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1938099258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2bea4390 0x564b2c246ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1938099258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be7b4d0 0x564b2c24f0f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1938099258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2be62f60 0x564b2c24c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1938099258 >> 172.21.15.78:0/1938099258 conn(0x564b2be984e0 msgr2=0x564b2c24e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1938099258 shutdown_connections 2024-05-21T15:17:19.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.218+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1938099258 wait complete. 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.219+0000 7f16dcc1f740 1 -- start start 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.219+0000 7f16dcc1f740 1 Processor -- start 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.219+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be62f60 0x564b2c251040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be7b4d0 0x564b2c251580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16dcc1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2bea4390 0x564b2c251ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b2370 con 0x564b2be62f60 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16cf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be62f60 0x564b2c251040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01b1ca0 con 0x564b2be7b4d0 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16cf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be62f60 0x564b2c251040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44734/0 (socket says 172.21.15.78:44734) 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16cf7fe640 1 -- 172.21.15.78:0/1687630839 learned_addr learned my addr 172.21.15.78:0/1687630839 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16dcc1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01aeee0 con 0x564b2bea4390 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16dcc1f740 10 client.0 _ll_register_callbacks cb 0x7f16c40ccc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:19.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be7b4d0 0x564b2c251580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16cffff640 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2bea4390 0x564b2c251ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16ceffd640 1 -- 172.21.15.78:0/1687630839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2bea4390 msgr2=0x564b2c251ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2bea4390 0x564b2c251ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16ceffd640 1 -- 172.21.15.78:0/1687630839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be62f60 msgr2=0x564b2c251040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be62f60 0x564b2c251040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16ceffd640 1 -- 172.21.15.78:0/1687630839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b2c2c6e10 con 0x564b2be7b4d0 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16cf7fe640 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be62f60 0x564b2c251040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16ceffd640 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be7b4d0 0x564b2c251580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16c40981f0 tx=0x7f16c4097b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c44992c26371f8c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.220+0000 7f16aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16aeffd640 1 -- 172.21.15.78:0/1687630839 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c406c070 con 0x564b2be7b4d0 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16aeffd640 1 -- 172.21.15.78:0/1687630839 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f16c40ba1b0 con 0x564b2be7b4d0 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1687630839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c409b870 con 0x564b2be7b4d0 2024-05-21T15:17:19.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16aeffd640 1 -- 172.21.15.78:0/1687630839 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16c401c040 con 0x564b2be7b4d0 2024-05-21T15:17:19.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 10 client.25526 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:19.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1687630839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b2c2c8040 con 0x564b2be7b4d0 2024-05-21T15:17:19.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16aeffd640 1 -- 172.21.15.78:0/1687630839 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c40a44e0 con 0x564b2be7b4d0 2024-05-21T15:17:19.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16aeffd640 1 -- 172.21.15.78:0/1687630839 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f16c40cf2d0 con 0x564b2be7b4d0 2024-05-21T15:17:19.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 10 client.25526 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:19.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 10 client.25526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:19.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 20 client.25526 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:19.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564b2be383a0 0x564b2c2ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 4 client.25526 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:17:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.221+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1687630839 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f16c409b870 con 0x564b2be383a0 2024-05-21T15:17:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.222+0000 7f16cf7fe640 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564b2be383a0 0x564b2c2ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.222+0000 7f16aeffd640 10 client.25526 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.222+0000 7f16cf7fe640 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564b2be383a0 0x564b2c2ca4d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69fd7b69596e5df6 server_cookie=3d6a4dde2694201d in_seq=0 out_seq=0 2024-05-21T15:17:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16aeffd640 1 -- 172.21.15.78:0/1687630839 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f16c409b870 con 0x564b2be383a0 2024-05-21T15:17:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16aeffd640 10 client.25526 handle_command_reply: tid=0 2024-05-21T15:17:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 client.25526 shutdown 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be7b4d0 msgr2=0x564b2c251580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be7b4d0 0x564b2c251580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16c40981f0 tx=0x7f16c4097b10 comp rx=0 tx=0).stop 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1687630839 shutdown_connections 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564b2be383a0 0x564b2c2ca4d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564b2be7b4d0 0x564b2c251580 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16aeffd640 0 client.25526 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b2bea4390 0x564b2c251ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 --2- 172.21.15.78:0/1687630839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564b2be62f60 0x564b2c251040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1687630839 >> 172.21.15.78:0/1687630839 conn(0x564b2be984e0 msgr2=0x564b2c24d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1687630839 shutdown_connections 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 1 -- 172.21.15.78:0/1687630839 wait complete. 2024-05-21T15:17:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.223+0000 7f16dcc1f740 20 client.25526 trim_cache size 0 max 16384 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout: "a4783c14-30a5-4f83-aea8-d0a1682d7bfe": { 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "a4783c14-30a5-4f83-aea8-d0a1682d7bfe", 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:17:19.234 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 -- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16b80a2f20 msgr2=0x7f16b80a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16b80a2f20 0x7f16b80a5430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16c8012fd0 tx=0x7f16c8016000 comp rx=0 tx=0).stop 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 -- 172.21.15.78:0/1528908351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d0199580 msgr2=0x7f16d019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d0199580 0x7f16d019d860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f16c400e9c0 tx=0x7f16c400ee80 comp rx=0 tx=0).stop 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 -- 172.21.15.78:0/1528908351 shutdown_connections 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d0199580 0x7f16d019d860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f16b80a2f20 0x7f16b80a5430 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d019dda0 0x7f16d019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 --2- 172.21.15.78:0/1528908351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d0108ad0 0x7f16d0199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:19.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 -- 172.21.15.78:0/1528908351 >> 172.21.15.78:0/1528908351 conn(0x7f16d0076830 msgr2=0x7f16d00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:19.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 -- 172.21.15.78:0/1528908351 shutdown_connections 2024-05-21T15:17:19.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:19.224+0000 7f16d5d84640 1 -- 172.21.15.78:0/1528908351 wait complete. 2024-05-21T15:17:19.239 INFO:tasks.cephfs.filesystem:scrub status for tag:a4783c14-30a5-4f83-aea8-d0a1682d7bfe - {'path': '/', 'tag': 'a4783c14-30a5-4f83-aea8-d0a1682d7bfe', 'options': 'recursive,force'} 2024-05-21T15:17:20.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:20 smithi078 ceph-mon[31997]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 773 B/s wr, 11 op/s 2024-05-21T15:17:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:20 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:17:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:20 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:20 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:17:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:20 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:20 smithi084 ceph-mon[36334]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 773 B/s wr, 11 op/s 2024-05-21T15:17:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:20 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:17:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:20 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:20 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:17:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:20 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:20 smithi172 ceph-mon[36402]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 773 B/s wr, 11 op/s 2024-05-21T15:17:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:20 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:17:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:20 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:20.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:20 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:17:20.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:20 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 -- 172.21.15.78:0/2709922068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c107ec0 msgr2=0x7fda0c101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 --2- 172.21.15.78:0/2709922068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c107ec0 0x7fda0c101b10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fda00009900 tx=0x7fda0002a410 comp rx=0 tx=0).stop 2024-05-21T15:17:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 -- 172.21.15.78:0/2709922068 shutdown_connections 2024-05-21T15:17:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 --2- 172.21.15.78:0/2709922068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda0c102180 0x7fda0c10e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 --2- 172.21.15.78:0/2709922068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c107ec0 0x7fda0c101b10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 --2- 172.21.15.78:0/2709922068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0c107a20 0x7fda0c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 -- 172.21.15.78:0/2709922068 >> 172.21.15.78:0/2709922068 conn(0x7fda0c078120 msgr2=0x7fda0c078560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 -- 172.21.15.78:0/2709922068 shutdown_connections 2024-05-21T15:17:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 -- 172.21.15.78:0/2709922068 wait complete. 2024-05-21T15:17:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.630+0000 7fda1181b640 1 Processor -- start 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda1181b640 1 -- start start 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda1181b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c102180 0x7fda0c1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda1181b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda0c107a20 0x7fda0c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda1181b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0c107ec0 0x7fda0c19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda1181b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c110c90 con 0x7fda0c102180 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda1181b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c110b10 con 0x7fda0c107ec0 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda1181b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c110e10 con 0x7fda0c107a20 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c102180 0x7fda0c1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c102180 0x7fda0c1975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44750/0 (socket says 172.21.15.78:44750) 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0affd640 1 -- 172.21.15.78:0/2201886906 learned_addr learned my addr 172.21.15.78:0/2201886906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0b7fe640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0c107ec0 0x7fda0c19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda0c107a20 0x7fda0c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0affd640 1 -- 172.21.15.78:0/2201886906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda0c107a20 msgr2=0x7fda0c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0affd640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda0c107a20 0x7fda0c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0affd640 1 -- 172.21.15.78:0/2201886906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0c107ec0 msgr2=0x7fda0c19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0affd640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0c107ec0 0x7fda0c19bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.631+0000 7fda0affd640 1 -- 172.21.15.78:0/2201886906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c19c430 con 0x7fda0c102180 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.632+0000 7fda0affd640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c102180 0x7fda0c1975d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009f70 tx=0x7fd9fc00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.632+0000 7fda10819640 1 -- 172.21.15.78:0/2201886906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc024070 con 0x7fda0c102180 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.632+0000 7fda10819640 1 -- 172.21.15.78:0/2201886906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc01f440 con 0x7fda0c102180 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.632+0000 7fda1181b640 1 -- 172.21.15.78:0/2201886906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c19c720 con 0x7fda0c102180 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.632+0000 7fda10819640 1 -- 172.21.15.78:0/2201886906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc01d5a0 con 0x7fda0c102180 2024-05-21T15:17:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.632+0000 7fda1181b640 1 -- 172.21.15.78:0/2201886906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1a40a0 con 0x7fda0c102180 2024-05-21T15:17:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.634+0000 7fda10819640 1 -- 172.21.15.78:0/2201886906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fd9fc027070 con 0x7fda0c102180 2024-05-21T15:17:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.635+0000 7fda10819640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd9f40a2e10 0x7fd9f40a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.635+0000 7fda10819640 1 -- 172.21.15.78:0/2201886906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd9fc0d7b70 con 0x7fda0c102180 2024-05-21T15:17:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.635+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd9f40a2e10 0x7fd9f40a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda187a4740 1 Processor -- start 2024-05-21T15:17:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda187a4740 1 -- start start 2024-05-21T15:17:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda187a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013f61b0 0x562201795080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda187a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013cd2f0 0x5622017955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd9f40a2e10 0x7fd9f40a5320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda00005940 tx=0x7fda000058d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda187a4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x562201795b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda187a4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622016fa340 con 0x5622013d54d0 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda187a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5622016a2290 con 0x5622013cd2f0 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda187a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5622016f6300 con 0x5622013f61b0 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x562201795b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x562201795b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44758/0 (socket says 172.21.15.78:44758) 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda0b7fe640 1 -- 172.21.15.78:0/1840943771 learned_addr learned my addr 172.21.15.78:0/1840943771 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda0affd640 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013f61b0 0x562201795080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.636+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013cd2f0 0x5622017955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fda0b7fe640 1 -- 172.21.15.78:0/1840943771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013f61b0 msgr2=0x562201795080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fda0b7fe640 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013f61b0 0x562201795080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fda0b7fe640 1 -- 172.21.15.78:0/1840943771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013cd2f0 msgr2=0x5622017955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fda0b7fe640 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013cd2f0 0x5622017955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fda0b7fe640 1 -- 172.21.15.78:0/1840943771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622016c68e0 con 0x5622013d54d0 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013cd2f0 0x5622017955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fda0b7fe640 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x562201795b00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fda04010370 tx=0x7fda04010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200c2701fb79af1a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fd9f1ffb640 1 -- 172.21.15.78:0/1840943771 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04016070 con 0x5622013d54d0 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fd9f1ffb640 1 -- 172.21.15.78:0/1840943771 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda04004d20 con 0x5622013d54d0 2024-05-21T15:17:21.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.637+0000 7fd9f1ffb640 1 -- 172.21.15.78:0/1840943771 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04005480 con 0x5622013d54d0 2024-05-21T15:17:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 -- 172.21.15.78:0/1840943771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 msgr2=0x562201795b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x562201795b00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fda04010370 tx=0x7fda04010830 comp rx=0 tx=0).stop 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 -- 172.21.15.78:0/1840943771 shutdown_connections 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013cd2f0 0x5622017955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x562201795b00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 --2- 172.21.15.78:0/1840943771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013f61b0 0x562201795080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 -- 172.21.15.78:0/1840943771 >> 172.21.15.78:0/1840943771 conn(0x5622013ea300 msgr2=0x562201791830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 -- 172.21.15.78:0/1840943771 shutdown_connections 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.638+0000 7fda187a4740 1 -- 172.21.15.78:0/1840943771 wait complete. 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.639+0000 7fda187a4740 1 -- start start 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.639+0000 7fda187a4740 1 Processor -- start 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.639+0000 7fda187a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.639+0000 7fda187a4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x5622017a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.639+0000 7fda187a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x56220179f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.639+0000 7fda187a4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5622017a21b0 con 0x5622013d54d0 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.639+0000 7fda187a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5622016b5790 con 0x5622013f61b0 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x5622017a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x5622017a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44770/0 (socket says 172.21.15.78:44770) 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 -- 172.21.15.78:0/2067121594 learned_addr learned my addr 172.21.15.78:0/2067121594 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5622017a2330 con 0x5622013cd2f0 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0affd640 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0b7fe640 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x56220179f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda187a4740 10 client.0 _ll_register_callbacks cb 0x5622016f0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 -- 172.21.15.78:0/2067121594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 msgr2=0x5622017a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 -- 172.21.15.78:0/2067121594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 msgr2=0x56220179f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x56220179f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 -- 172.21.15.78:0/2067121594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56220140e210 con 0x5622013d54d0 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0affd640 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x5622017a16b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fda0c06b460 tx=0x7fda00009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1a0a5628d3ab25 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fd9eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fd9eaffd640 1 -- 172.21.15.78:0/2067121594 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0004d020 con 0x5622013d54d0 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fd9eaffd640 1 -- 172.21.15.78:0/2067121594 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fda00004240 con 0x5622013d54d0 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.640+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda04004d20 con 0x5622013d54d0 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.641+0000 7fd9eaffd640 1 -- 172.21.15.78:0/2067121594 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda00058050 con 0x5622013d54d0 2024-05-21T15:17:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.641+0000 7fda187a4740 10 client.16032 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.641+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622014da620 con 0x5622013d54d0 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.641+0000 7fd9eaffd640 1 -- 172.21.15.78:0/2067121594 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda04004d20 con 0x5622013d54d0 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fd9eaffd640 1 -- 172.21.15.78:0/2067121594 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda00004a50 con 0x5622013d54d0 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fda187a4740 10 client.16032 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fda187a4740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fda187a4740 20 client.16032 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fda187a4740 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5622013b4d80 0x5622017b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fda187a4740 4 client.16032 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562201707a70 con 0x5622013b4d80 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fda0affd640 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5622013b4d80 0x5622017b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fd9eaffd640 10 client.16032 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.642+0000 7fda0affd640 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5622013b4d80 0x5622017b19b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b415d19b3a2e7f server_cookie=942624175780d8cd in_seq=0 out_seq=0 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fd9eaffd640 1 -- 172.21.15.78:0/2067121594 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562201707a70 con 0x5622013b4d80 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fd9eaffd640 10 client.16032 handle_command_reply: tid=0 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 client.16032 shutdown 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 msgr2=0x5622017a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x5622017a16b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fda0c06b460 tx=0x7fda00009260 comp rx=0 tx=0).stop 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 shutdown_connections 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5622013b4d80 0x5622017b19b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fd9eaffd640 0 client.16032 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x56220179f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013d54d0 0x5622017a16b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 --2- 172.21.15.78:0/2067121594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.643+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 >> 172.21.15.78:0/2067121594 conn(0x5622013ea300 msgr2=0x562201796da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.644+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 shutdown_connections 2024-05-21T15:17:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.644+0000 7fda187a4740 1 -- 172.21.15.78:0/2067121594 wait complete. 2024-05-21T15:17:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.644+0000 7fda187a4740 20 client.16032 trim_cache size 0 max 16384 2024-05-21T15:17:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda187a4740 1 Processor -- start 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda187a4740 1 -- start start 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda187a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda187a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x5622017b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda187a4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56220138a1c0 0x5622017a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda187a4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1aca90 con 0x56220138a1c0 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda187a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1ae5d0 con 0x5622013f61b0 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda187a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1ad160 con 0x5622013cd2f0 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56220138a1c0 0x5622017a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56220138a1c0 0x5622017a75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44782/0 (socket says 172.21.15.78:44782) 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.673+0000 7fda0b7fe640 1 -- 172.21.15.78:0/3257837343 learned_addr learned my addr 172.21.15.78:0/3257837343 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fda0affd640 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fda0affd640 1 -- 172.21.15.78:0/3257837343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 msgr2=0x5622017b76f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fda0affd640 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x5622017b76f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fda0affd640 1 -- 172.21.15.78:0/3257837343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56220138a1c0 msgr2=0x5622017a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fda0affd640 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56220138a1c0 0x5622017a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fda0affd640 1 -- 172.21.15.78:0/3257837343 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56220179b1b0 con 0x5622013cd2f0 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fda0b7fe640 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56220138a1c0 0x5622017a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fda0affd640 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017b71b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0d5190 tx=0x7fd9fc019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6c28200dccad31 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fd9f17fa640 1 -- 172.21.15.78:0/3257837343 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0acb00 con 0x5622013cd2f0 2024-05-21T15:17:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fd9f17fa640 1 -- 172.21.15.78:0/3257837343 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc0a2240 con 0x5622013cd2f0 2024-05-21T15:17:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.674+0000 7fd9f17fa640 1 -- 172.21.15.78:0/3257837343 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0d5360 con 0x5622013cd2f0 2024-05-21T15:17:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 -- 172.21.15.78:0/3257837343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 msgr2=0x5622017b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017b71b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0d5190 tx=0x7fd9fc019040 comp rx=0 tx=0).stop 2024-05-21T15:17:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 -- 172.21.15.78:0/3257837343 shutdown_connections 2024-05-21T15:17:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x5622017b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56220138a1c0 0x5622017a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 --2- 172.21.15.78:0/3257837343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622013cd2f0 0x5622017b71b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 -- 172.21.15.78:0/3257837343 >> 172.21.15.78:0/3257837343 conn(0x5622013ea300 msgr2=0x56220179dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 -- 172.21.15.78:0/3257837343 shutdown_connections 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.675+0000 7fda187a4740 1 -- 172.21.15.78:0/3257837343 wait complete. 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 1 -- start start 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 1 Processor -- start 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56220138a1c0 0x5622017a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013cd2f0 0x5622017a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x562201817d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1adf00 con 0x5622013cd2f0 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1ad830 con 0x5622013f61b0 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1aaa70 con 0x56220138a1c0 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.676+0000 7fda187a4740 10 client.0 _ll_register_callbacks cb 0x7fda00040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013cd2f0 0x5622017a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013cd2f0 0x5622017a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44794/0 (socket says 172.21.15.78:44794) 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 -- 172.21.15.78:0/1224086857 learned_addr learned my addr 172.21.15.78:0/1224086857 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0affd640 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56220138a1c0 0x5622017a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0b7fe640 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x562201817d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 -- 172.21.15.78:0/1224086857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56220138a1c0 msgr2=0x5622017a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56220138a1c0 0x5622017a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 -- 172.21.15.78:0/1224086857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 msgr2=0x562201817d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x562201817d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 -- 172.21.15.78:0/1224086857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562201818290 con 0x5622013cd2f0 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0affd640 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56220138a1c0 0x5622017a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda0a7fc640 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013cd2f0 0x5622017a2b20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fda00009900 tx=0x7fda00003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513abae4889e4d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fd9eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fd9eaffd640 1 -- 172.21.15.78:0/1224086857 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0004d040 con 0x5622013cd2f0 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fd9eaffd640 1 -- 172.21.15.78:0/1224086857 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fda00004ad0 con 0x5622013cd2f0 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.677+0000 7fda187a4740 1 -- 172.21.15.78:0/1224086857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9fc0a2240 con 0x5622013cd2f0 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fd9eaffd640 1 -- 172.21.15.78:0/1224086857 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda0006a050 con 0x5622013cd2f0 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fda187a4740 10 client.16038 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fda187a4740 1 -- 172.21.15.78:0/1224086857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622018193f0 con 0x5622013cd2f0 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fd9eaffd640 1 -- 172.21.15.78:0/1224086857 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc0a2240 con 0x5622013cd2f0 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fd9eaffd640 1 -- 172.21.15.78:0/1224086857 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda00049480 con 0x5622013cd2f0 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fda187a4740 10 client.16038 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fda187a4740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fda187a4740 20 client.16038 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fda187a4740 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562201819b90 0x56220181bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fda187a4740 4 client.16038 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:17:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.678+0000 7fda187a4740 1 -- 172.21.15.78:0/1224086857 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562201701dc0 con 0x562201819b90 2024-05-21T15:17:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.679+0000 7fda0affd640 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562201819b90 0x56220181bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.679+0000 7fd9eaffd640 10 client.16038 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.679+0000 7fda0affd640 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562201819b90 0x56220181bfb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11105abc4d290325 server_cookie=53d4f08b32a6476a in_seq=0 out_seq=0 2024-05-21T15:17:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fd9eaffd640 1 -- 172.21.15.78:0/1224086857 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562201701dc0 con 0x562201819b90 2024-05-21T15:17:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fd9eaffd640 10 client.16038 handle_command_reply: tid=0 2024-05-21T15:17:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 client.16038 shutdown 2024-05-21T15:17:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 -- 172.21.15.78:0/1224086857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013cd2f0 msgr2=0x5622017a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013cd2f0 0x5622017a2b20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fda00009900 tx=0x7fda00003940 comp rx=0 tx=0).stop 2024-05-21T15:17:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 -- 172.21.15.78:0/1224086857 shutdown_connections 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562201819b90 0x56220181bfb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5622013f61b0 0x562201817d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5622013cd2f0 0x5622017a2b20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 --2- 172.21.15.78:0/1224086857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56220138a1c0 0x5622017a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 -- 172.21.15.78:0/1224086857 >> 172.21.15.78:0/1224086857 conn(0x5622013ea300 msgr2=0x562201796cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fd9eaffd640 0 client.16038 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 -- 172.21.15.78:0/1224086857 shutdown_connections 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.683+0000 7fda187a4740 1 -- 172.21.15.78:0/1224086857 wait complete. 2024-05-21T15:17:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda187a4740 20 client.16038 trim_cache size 0 max 16384 2024-05-21T15:17:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:17:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:17:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:17:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 -- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd9f40a2e10 msgr2=0x7fd9f40a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd9f40a2e10 0x7fd9f40a5320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda00005940 tx=0x7fda000058d0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 -- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c102180 msgr2=0x7fda0c1975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c102180 0x7fda0c1975d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009f70 tx=0x7fd9fc00edd0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 -- 172.21.15.78:0/2201886906 shutdown_connections 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0c107ec0 0x7fda0c19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd9f40a2e10 0x7fd9f40a5320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda0c107a20 0x7fda0c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 --2- 172.21.15.78:0/2201886906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda0c102180 0x7fda0c1975d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 -- 172.21.15.78:0/2201886906 >> 172.21.15.78:0/2201886906 conn(0x7fda0c078120 msgr2=0x7fda0c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 -- 172.21.15.78:0/2201886906 shutdown_connections 2024-05-21T15:17:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:21.684+0000 7fda1181b640 1 -- 172.21.15.78:0/2201886906 wait complete. 2024-05-21T15:17:22.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:22 smithi078 ceph-mon[31997]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-21T15:17:22.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:22 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:17:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:22 smithi084 ceph-mon[36334]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-21T15:17:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:22 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:17:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:22 smithi172 ceph-mon[36402]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-05-21T15:17:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:22 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:17:24.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:24 smithi078 ceph-mon[31997]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-21T15:17:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:24 smithi084 ceph-mon[36334]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-21T15:17:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:24 smithi172 ceph-mon[36402]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-21T15:17:26.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:26 smithi078 ceph-mon[31997]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:17:26.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:26 smithi084 ceph-mon[36334]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:17:26.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:26 smithi172 ceph-mon[36402]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:17:28.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:28 smithi078 ceph-mon[31997]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:17:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:28 smithi084 ceph-mon[36334]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:17:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:28 smithi172 ceph-mon[36402]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-21T15:17:30.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:30 smithi078 ceph-mon[31997]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:17:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:30 smithi084 ceph-mon[36334]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:17:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:30 smithi172 ceph-mon[36402]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T15:17:32.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:32 smithi078 ceph-mon[31997]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:17:32.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:32 smithi084 ceph-mon[36334]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:17:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:32 smithi172 ceph-mon[36402]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T15:17:34.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:34 smithi078 ceph-mon[31997]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T15:17:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:34 smithi084 ceph-mon[36334]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T15:17:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:34 smithi172 ceph-mon[36402]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T15:17:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.822+0000 7f5234479640 1 -- 172.21.15.78:0/1427379004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107a20 msgr2=0x7f522c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.822+0000 7f5234479640 1 --2- 172.21.15.78:0/1427379004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107a20 0x7f522c1015d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5228009900 tx=0x7f522802a410 comp rx=0 tx=0).stop 2024-05-21T15:17:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.822+0000 7f5234479640 1 -- 172.21.15.78:0/1427379004 shutdown_connections 2024-05-21T15:17:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.822+0000 7f5234479640 1 --2- 172.21.15.78:0/1427379004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522c102180 0x7f522c10e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.822+0000 7f5234479640 1 --2- 172.21.15.78:0/1427379004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f522c107ec0 0x7f522c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.822+0000 7f5234479640 1 --2- 172.21.15.78:0/1427379004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107a20 0x7f522c1015d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.822+0000 7f5234479640 1 -- 172.21.15.78:0/1427379004 >> 172.21.15.78:0/1427379004 conn(0x7f522c077130 msgr2=0x7f522c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.823+0000 7f5234479640 1 -- 172.21.15.78:0/1427379004 shutdown_connections 2024-05-21T15:17:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.823+0000 7f5234479640 1 -- 172.21.15.78:0/1427379004 wait complete. 2024-05-21T15:17:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.823+0000 7f5234479640 1 Processor -- start 2024-05-21T15:17:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.823+0000 7f5234479640 1 -- start start 2024-05-21T15:17:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5234479640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522c102180 0x7f522c1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5234479640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f522c107a20 0x7f522c1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5234479640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107ec0 0x7f522c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5234479640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f522c111d70 con 0x7f522c107ec0 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5234479640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f522c111bf0 con 0x7f522c102180 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5234479640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f522c111ef0 con 0x7f522c107a20 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107ec0 0x7f522c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232215640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522c102180 0x7f522c1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107ec0 0x7f522c19b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47946/0 (socket says 172.21.15.78:47946) 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 -- 172.21.15.78:0/2578209559 learned_addr learned my addr 172.21.15.78:0/2578209559 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5231a14640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f522c107a20 0x7f522c1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 -- 172.21.15.78:0/2578209559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f522c107a20 msgr2=0x7f522c1a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f522c107a20 0x7f522c1a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 -- 172.21.15.78:0/2578209559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522c102180 msgr2=0x7f522c1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522c102180 0x7f522c1a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 -- 172.21.15.78:0/2578209559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c19bf60 con 0x7f522c107ec0 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5231a14640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f522c107a20 0x7f522c1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232215640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522c102180 0x7f522c1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.824+0000 7f5232a16640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107ec0 0x7f522c19b830 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f522400eb10 tx=0x7f522400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.825+0000 7f52237fe640 1 -- 172.21.15.78:0/2578209559 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224004b20 con 0x7f522c107ec0 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.825+0000 7f5234479640 1 -- 172.21.15.78:0/2578209559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c19c250 con 0x7f522c107ec0 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.825+0000 7f52237fe640 1 -- 172.21.15.78:0/2578209559 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5224004cc0 con 0x7f522c107ec0 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.825+0000 7f52237fe640 1 -- 172.21.15.78:0/2578209559 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224010760 con 0x7f522c107ec0 2024-05-21T15:17:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.825+0000 7f5234479640 1 -- 172.21.15.78:0/2578209559 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c1a8420 con 0x7f522c107ec0 2024-05-21T15:17:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.828+0000 7f52237fe640 1 -- 172.21.15.78:0/2578209559 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f5224005020 con 0x7f522c107ec0 2024-05-21T15:17:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.828+0000 7f523b3f5740 1 Processor -- start 2024-05-21T15:17:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f523b3f5740 1 -- start start 2024-05-21T15:17:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 0x5586148e6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 0x5586148e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f523b3f5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55861484b340 con 0x5586145264d0 2024-05-21T15:17:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f523b3f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5586147f3290 con 0x5586145471b0 2024-05-21T15:17:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f523b3f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558614847300 con 0x55861451e2f0 2024-05-21T15:17:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f52237fe640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52180a2e10 0x7f52180a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f5231a14640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f5231a14640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47956/0 (socket says 172.21.15.78:47956) 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.829+0000 7f5231a14640 1 -- 172.21.15.78:0/3743298661 learned_addr learned my addr 172.21.15.78:0/3743298661 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f52237fe640 1 -- 172.21.15.78:0/2578209559 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f52240cfdd0 con 0x7f522c107ec0 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5231a14640 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 0x5586148e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5232a16640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52180a2e10 0x7f52180a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5232215640 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 0x5586148e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5231a14640 1 -- 172.21.15.78:0/3743298661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 msgr2=0x5586148e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5231a14640 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 0x5586148e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5231a14640 1 -- 172.21.15.78:0/3743298661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 msgr2=0x5586148e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5231a14640 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 0x5586148e6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5231a14640 1 -- 172.21.15.78:0/3743298661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586148178e0 con 0x5586145264d0 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5231a14640 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148e6b00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f521c012760 tx=0x7f521c009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcce0998827ba508 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5232a16640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52180a2e10 0x7f52180a5320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f52240cd3a0 tx=0x7f5224005b40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5220bfe640 1 -- 172.21.15.78:0/3743298661 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c004030 con 0x5586145264d0 2024-05-21T15:17:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5220bfe640 1 -- 172.21.15.78:0/3743298661 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521c010cb0 con 0x5586145264d0 2024-05-21T15:17:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.830+0000 7f5220bfe640 1 -- 172.21.15.78:0/3743298661 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c003970 con 0x5586145264d0 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.831+0000 7f523b3f5740 1 -- 172.21.15.78:0/3743298661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 msgr2=0x5586148e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.831+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148e6b00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f521c012760 tx=0x7f521c009730 comp rx=0 tx=0).stop 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.831+0000 7f523b3f5740 1 -- 172.21.15.78:0/3743298661 shutdown_connections 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.831+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148e6b00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.831+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 0x5586148e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.831+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3743298661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 0x5586148e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.831+0000 7f523b3f5740 1 -- 172.21.15.78:0/3743298661 >> 172.21.15.78:0/3743298661 conn(0x55861453b300 msgr2=0x5586148e2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 -- 172.21.15.78:0/3743298661 shutdown_connections 2024-05-21T15:17:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 -- 172.21.15.78:0/3743298661 wait complete. 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 -- start start 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 Processor -- start 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 0x5586148f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148f2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 0x5586148f0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586148f31b0 con 0x5586145264d0 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558614806790 con 0x5586145471b0 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586148f3330 con 0x55861451e2f0 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f523b3f5740 10 client.0 _ll_register_callbacks cb 0x558614841a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.832+0000 7f5232215640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5232215640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47970/0 (socket says 172.21.15.78:47970) 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5232215640 1 -- 172.21.15.78:0/3184850607 learned_addr learned my addr 172.21.15.78:0/3184850607 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5231a14640 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 0x5586148f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5231a14640 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 0x5586148f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5232215640 1 -- 172.21.15.78:0/3184850607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 msgr2=0x5586148f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5232215640 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 0x5586148f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5232215640 1 -- 172.21.15.78:0/3184850607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 msgr2=0x5586148f0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5232215640 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 0x5586148f0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5232215640 1 -- 172.21.15.78:0/3184850607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55861455f210 con 0x5586145264d0 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5232215640 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148f2710 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f522c102710 tx=0x7f52280317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfa36498a1e0c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5215ffb640 1 -- 172.21.15.78:0/3184850607 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228035040 con 0x5586145264d0 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f5215ffb640 1 -- 172.21.15.78:0/3184850607 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f5228004040 con 0x5586145264d0 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.833+0000 7f523b3f5740 1 -- 172.21.15.78:0/3184850607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f521c010cb0 con 0x5586145264d0 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f5215ffb640 1 -- 172.21.15.78:0/3184850607 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5228004240 con 0x5586145264d0 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f523b3f5740 10 client.16050 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f523b3f5740 1 -- 172.21.15.78:0/3184850607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55861462b620 con 0x5586145264d0 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f5215ffb640 1 -- 172.21.15.78:0/3184850607 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521c010cb0 con 0x5586145264d0 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f5215ffb640 1 -- 172.21.15.78:0/3184850607 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5228004ac0 con 0x5586145264d0 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f523b3f5740 10 client.16050 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f523b3f5740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f523b3f5740 20 client.16050 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558614505d80 0x558614902a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f523b3f5740 4 client.16050 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.834+0000 7f523b3f5740 1 -- 172.21.15.78:0/3184850607 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558614858a70 con 0x558614505d80 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.835+0000 7f5231a14640 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558614505d80 0x558614902a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.835+0000 7f5215ffb640 10 client.16050 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.835+0000 7f5231a14640 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558614505d80 0x558614902a00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cb8e39389e6f44f server_cookie=a59d79faf4f36aa0 in_seq=0 out_seq=0 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f5215ffb640 1 -- 172.21.15.78:0/3184850607 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558614858a70 con 0x558614505d80 2024-05-21T15:17:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f5215ffb640 10 client.16050 handle_command_reply: tid=0 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 client.16050 shutdown 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 -- 172.21.15.78:0/3184850607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 msgr2=0x5586148f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148f2710 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f522c102710 tx=0x7f52280317e0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 -- 172.21.15.78:0/3184850607 shutdown_connections 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558614505d80 0x558614902a00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586145471b0 0x5586148f0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145264d0 0x5586148f2710 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3184850607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55861451e2f0 0x5586148f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 -- 172.21.15.78:0/3184850607 >> 172.21.15.78:0/3184850607 conn(0x55861453b300 msgr2=0x5586148e7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f5215ffb640 0 client.16050 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 -- 172.21.15.78:0/3184850607 shutdown_connections 2024-05-21T15:17:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 1 -- 172.21.15.78:0/3184850607 wait complete. 2024-05-21T15:17:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.836+0000 7f523b3f5740 20 client.16050 trim_cache size 0 max 16384 2024-05-21T15:17:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.851+0000 7f523b3f5740 1 Processor -- start 2024-05-21T15:17:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.851+0000 7f523b3f5740 1 -- start start 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55861451e2f0 0x5586148e8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145471b0 0x5586149081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586144db1c0 0x558614908720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f523b3f5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b0e40 con 0x5586145471b0 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f523b3f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b2980 con 0x55861451e2f0 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f523b3f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b1510 con 0x5586144db1c0 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145471b0 0x5586149081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145471b0 0x5586149081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47982/0 (socket says 172.21.15.78:47982) 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 -- 172.21.15.78:0/3674778995 learned_addr learned my addr 172.21.15.78:0/3674778995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5231a14640 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55861451e2f0 0x5586148e8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5231a14640 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586144db1c0 0x558614908720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 -- 172.21.15.78:0/3674778995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586144db1c0 msgr2=0x558614908720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586144db1c0 0x558614908720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 -- 172.21.15.78:0/3674778995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55861451e2f0 msgr2=0x5586148e8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55861451e2f0 0x5586148e8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 -- 172.21.15.78:0/3674778995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586148dda60 con 0x5586145471b0 2024-05-21T15:17:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.852+0000 7f5232215640 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145471b0 0x5586149081e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5228009900 tx=0x7f522802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b696acf5e216c40 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.853+0000 7f5217fff640 1 -- 172.21.15.78:0/3674778995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228035030 con 0x5586145471b0 2024-05-21T15:17:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.853+0000 7f5217fff640 1 -- 172.21.15.78:0/3674778995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5228004c30 con 0x5586145471b0 2024-05-21T15:17:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.853+0000 7f5217fff640 1 -- 172.21.15.78:0/3674778995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228051070 con 0x5586145471b0 2024-05-21T15:17:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 -- 172.21.15.78:0/3674778995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145471b0 msgr2=0x5586149081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145471b0 0x5586149081e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5228009900 tx=0x7f522802a410 comp rx=0 tx=0).stop 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 -- 172.21.15.78:0/3674778995 shutdown_connections 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586144db1c0 0x558614908720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586145471b0 0x5586149081e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 --2- 172.21.15.78:0/3674778995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55861451e2f0 0x5586148e8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 -- 172.21.15.78:0/3674778995 >> 172.21.15.78:0/3674778995 conn(0x55861453b300 msgr2=0x5586148e7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 -- 172.21.15.78:0/3674778995 shutdown_connections 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.854+0000 7f523b3f5740 1 -- 172.21.15.78:0/3674778995 wait complete. 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 1 -- start start 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 1 Processor -- start 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586144db1c0 0x5586148f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55861451e2f0 0x5586148f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586145471b0 0x558614962d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b22b0 con 0x55861451e2f0 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1b1be0 con 0x5586144db1c0 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f522c1aee20 con 0x5586145471b0 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f523b3f5740 10 client.0 _ll_register_callbacks cb 0x7f521c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f5232215640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55861451e2f0 0x5586148f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f5232215640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55861451e2f0 0x5586148f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47986/0 (socket says 172.21.15.78:47986) 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f5232215640 1 -- 172.21.15.78:0/2578820456 learned_addr learned my addr 172.21.15.78:0/2578820456 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.855+0000 7f5231a14640 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586145471b0 0x558614962d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5232215640 1 -- 172.21.15.78:0/2578820456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586145471b0 msgr2=0x558614962d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5232215640 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586145471b0 0x558614962d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5232215640 1 -- 172.21.15.78:0/2578820456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586144db1c0 msgr2=0x5586148f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5232215640 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586144db1c0 0x5586148f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5232215640 1 -- 172.21.15.78:0/2578820456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586149632e0 con 0x55861451e2f0 2024-05-21T15:17:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5231a14640 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586145471b0 0x558614962d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5232215640 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55861451e2f0 0x5586148f40e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f52280035f0 tx=0x7f5228003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66052adda26f6d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5215ffb640 1 -- 172.21.15.78:0/2578820456 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228051030 con 0x55861451e2f0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5215ffb640 1 -- 172.21.15.78:0/2578820456 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f5228004960 con 0x55861451e2f0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f523b3f5740 1 -- 172.21.15.78:0/2578820456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5228004c30 con 0x55861451e2f0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f5215ffb640 1 -- 172.21.15.78:0/2578820456 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5228035070 con 0x55861451e2f0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f523b3f5740 10 client.16056 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.856+0000 7f523b3f5740 1 -- 172.21.15.78:0/2578820456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55861496a610 con 0x55861451e2f0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f5215ffb640 1 -- 172.21.15.78:0/2578820456 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5228004c30 con 0x55861451e2f0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f5215ffb640 1 -- 172.21.15.78:0/2578820456 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5228005370 con 0x55861451e2f0 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f523b3f5740 10 client.16056 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f523b3f5740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f523b3f5740 20 client.16056 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f523b3f5740 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55861496abc0 0x55861496cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f523b3f5740 4 client.16056 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:17:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f523b3f5740 1 -- 172.21.15.78:0/2578820456 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558614902fa0 con 0x55861496abc0 2024-05-21T15:17:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.857+0000 7f5231a14640 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55861496abc0 0x55861496cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.858+0000 7f5215ffb640 10 client.16056 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.858+0000 7f5231a14640 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55861496abc0 0x55861496cfe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439077d6751671d server_cookie=42e07f6db61b2959 in_seq=0 out_seq=0 2024-05-21T15:17:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f5215ffb640 1 -- 172.21.15.78:0/2578820456 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558614902fa0 con 0x55861496abc0 2024-05-21T15:17:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f5215ffb640 10 client.16056 handle_command_reply: tid=0 2024-05-21T15:17:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 client.16056 shutdown 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 -- 172.21.15.78:0/2578820456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55861451e2f0 msgr2=0x5586148f40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55861451e2f0 0x5586148f40e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f52280035f0 tx=0x7f5228003900 comp rx=0 tx=0).stop 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 -- 172.21.15.78:0/2578820456 shutdown_connections 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55861496abc0 0x55861496cfe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586145471b0 0x558614962d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f5215ffb640 0 client.16056 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55861451e2f0 0x5586148f40e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 --2- 172.21.15.78:0/2578820456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586144db1c0 0x5586148f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 -- 172.21.15.78:0/2578820456 >> 172.21.15.78:0/2578820456 conn(0x55861453b300 msgr2=0x5586148e5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 -- 172.21.15.78:0/2578820456 shutdown_connections 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 1 -- 172.21.15.78:0/2578820456 wait complete. 2024-05-21T15:17:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.862+0000 7f523b3f5740 20 client.16056 trim_cache size 0 max 16384 2024-05-21T15:17:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:17:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:17:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:17:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:17:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 -- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52180a2e10 msgr2=0x7f52180a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52180a2e10 0x7f52180a5320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f52240cd3a0 tx=0x7f5224005b40 comp rx=0 tx=0).stop 2024-05-21T15:17:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 -- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107ec0 msgr2=0x7f522c19b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107ec0 0x7f522c19b830 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f522400eb10 tx=0x7f522400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 -- 172.21.15.78:0/2578209559 shutdown_connections 2024-05-21T15:17:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f52180a2e10 0x7f52180a5320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522c107ec0 0x7f522c19b830 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f522c107a20 0x7f522c1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 --2- 172.21.15.78:0/2578209559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522c102180 0x7f522c1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 -- 172.21.15.78:0/2578209559 >> 172.21.15.78:0/2578209559 conn(0x7f522c077130 msgr2=0x7f522c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 -- 172.21.15.78:0/2578209559 shutdown_connections 2024-05-21T15:17:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:34.863+0000 7f5234479640 1 -- 172.21.15.78:0/2578209559 wait complete. 2024-05-21T15:17:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:36 smithi078 ceph-mon[31997]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:17:36.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:36 smithi084 ceph-mon[36334]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:17:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:36 smithi172 ceph-mon[36402]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T15:17:38.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:38 smithi078 ceph-mon[31997]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:17:38.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:38 smithi084 ceph-mon[36334]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:17:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:38 smithi172 ceph-mon[36402]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:17:40.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:40 smithi078 ceph-mon[31997]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:17:40.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:40 smithi084 ceph-mon[36334]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:17:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:40 smithi172 ceph-mon[36402]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:17:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:42 smithi078 ceph-mon[31997]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-21T15:17:42.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:42 smithi078 ceph-mon[31997]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-21T15:17:42.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:42 smithi084 ceph-mon[36334]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-21T15:17:42.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:42 smithi084 ceph-mon[36334]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-21T15:17:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:42 smithi172 ceph-mon[36402]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-05-21T15:17:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:42 smithi172 ceph-mon[36402]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-21T15:17:42.786 INFO:tasks.workunit.client.0.smithi078.stderr:Note: switching to 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'. 2024-05-21T15:17:42.786 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-05-21T15:17:42.786 INFO:tasks.workunit.client.0.smithi078.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-21T15:17:42.786 INFO:tasks.workunit.client.0.smithi078.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-21T15:17:42.786 INFO:tasks.workunit.client.0.smithi078.stderr:state without impacting any branches by switching back to a branch. 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr: git switch -c 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr:Or undo this operation with: 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr: git switch - 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-05-21T15:17:42.787 INFO:tasks.workunit.client.0.smithi078.stderr:HEAD is now at d2b507c724e Merge PR #56984 into wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-21T15:17:42.793 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-21T15:17:42.857 INFO:tasks.workunit.client.0.smithi078.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-21T15:17:42.860 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T15:17:42.860 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-21T15:17:43.497 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-21T15:17:43.551 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-21T15:17:43.590 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T15:17:43.592 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T15:17:43.592 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-21T15:17:43.631 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T15:17:43.634 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:17:43.634 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-21T15:17:43.690 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-05-21T15:17:43.692 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-05-21T15:17:43.692 DEBUG:teuthology.orchestra.run.smithi078:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2b507c724ef6c8295ee0a35d185cdab167fd61d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-21T15:17:43.752 INFO:tasks.workunit.client.0.smithi078.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-21T15:17:43.753 INFO:tasks.workunit.client.0.smithi078.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-05-21T15:17:43.753 INFO:tasks.workunit.client.0.smithi078.stderr:+ trap -- retry EXIT 2024-05-21T15:17:43.753 INFO:tasks.workunit.client.0.smithi078.stderr:+ rm -rf ffsb 2024-05-21T15:17:43.754 INFO:tasks.workunit.client.0.smithi078.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-05-21T15:17:43.757 INFO:tasks.workunit.client.0.smithi078.stderr:Cloning into 'ffsb'... 2024-05-21T15:17:44.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:44 smithi078 ceph-mon[31997]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-21T15:17:44.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:44 smithi084 ceph-mon[36334]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-21T15:17:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:44 smithi172 ceph-mon[36402]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-21T15:17:44.749 INFO:tasks.workunit.client.0.smithi078.stderr:+ trap - EXIT 2024-05-21T15:17:44.749 INFO:tasks.workunit.client.0.smithi078.stderr:+ cd ffsb 2024-05-21T15:17:44.749 INFO:tasks.workunit.client.0.smithi078.stderr:+ ./configure 2024-05-21T15:17:44.898 INFO:tasks.workunit.client.0.smithi078.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-21T15:17:45.909 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether build environment is sane... yes 2024-05-21T15:17:45.920 INFO:tasks.workunit.client.0.smithi078.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-21T15:17:45.920 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gawk... gawk 2024-05-21T15:17:45.940 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether make sets $(MAKE)... yes 2024-05-21T15:17:45.976 INFO:tasks.workunit.client.0.smithi078.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-21T15:17:45.976 INFO:tasks.workunit.client.0.smithi078.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-21T15:17:45.990 INFO:tasks.workunit.client.0.smithi078.stdout:checking for style of include used by make... GNU 2024-05-21T15:17:45.997 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc... gcc 2024-05-21T15:17:46.056 INFO:tasks.workunit.client.0.smithi078.stdout:checking for C compiler default output file name... a.out 2024-05-21T15:17:46.058 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether the C compiler works... yes 2024-05-21T15:17:46.062 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether we are cross compiling... no 2024-05-21T15:17:46.101 INFO:tasks.workunit.client.0.smithi078.stdout:checking for suffix of executables... 2024-05-21T15:17:46.136 INFO:tasks.workunit.client.0.smithi078.stdout:checking for suffix of object files... o 2024-05-21T15:17:46.168 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether we are using the GNU C compiler... yes 2024-05-21T15:17:46.200 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether gcc accepts -g... yes 2024-05-21T15:17:46.279 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-21T15:17:46.404 INFO:tasks.workunit.client.0.smithi078.stdout:checking dependency style of gcc... gcc3 2024-05-21T15:17:46.460 INFO:tasks.workunit.client.0.smithi078.stdout:checking how to run the C preprocessor... gcc -E 2024-05-21T15:17:46.507 INFO:tasks.workunit.client.0.smithi078.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-21T15:17:46.507 INFO:tasks.workunit.client.0.smithi078.stdout:checking for egrep... /usr/bin/grep -E 2024-05-21T15:17:46.516 INFO:tasks.workunit.client.0.smithi078.stdout:checking for AIX... no 2024-05-21T15:17:46.538 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc... (cached) gcc 2024-05-21T15:17:46.547 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-21T15:17:46.547 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-21T15:17:46.547 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-21T15:17:46.547 INFO:tasks.workunit.client.0.smithi078.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-21T15:17:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:46 smithi078 ceph-mon[31997]: pgmap v53: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 112 KiB/s wr, 21 op/s 2024-05-21T15:17:46.614 INFO:tasks.workunit.client.0.smithi078.stdout:checking for main in -lm... yes 2024-05-21T15:17:46.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:46 smithi084 ceph-mon[36334]: pgmap v53: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 112 KiB/s wr, 21 op/s 2024-05-21T15:17:46.666 INFO:tasks.workunit.client.0.smithi078.stdout:checking for main in -lpthread... yes 2024-05-21T15:17:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:46 smithi172 ceph-mon[36402]: pgmap v53: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 112 KiB/s wr, 21 op/s 2024-05-21T15:17:46.824 INFO:tasks.workunit.client.0.smithi078.stdout:checking for ANSI C header files... yes 2024-05-21T15:17:46.879 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-21T15:17:46.927 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/types.h... yes 2024-05-21T15:17:46.975 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/stat.h... yes 2024-05-21T15:17:47.022 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdlib.h... yes 2024-05-21T15:17:47.070 INFO:tasks.workunit.client.0.smithi078.stdout:checking for string.h... yes 2024-05-21T15:17:47.119 INFO:tasks.workunit.client.0.smithi078.stdout:checking for memory.h... yes 2024-05-21T15:17:47.167 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strings.h... yes 2024-05-21T15:17:47.216 INFO:tasks.workunit.client.0.smithi078.stdout:checking for inttypes.h... yes 2024-05-21T15:17:47.265 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdint.h... yes 2024-05-21T15:17:47.316 INFO:tasks.workunit.client.0.smithi078.stdout:checking for unistd.h... yes 2024-05-21T15:17:47.382 INFO:tasks.workunit.client.0.smithi078.stdout:checking pthread.h usability... yes 2024-05-21T15:17:47.405 INFO:tasks.workunit.client.0.smithi078.stdout:checking pthread.h presence... yes 2024-05-21T15:17:47.406 INFO:tasks.workunit.client.0.smithi078.stdout:checking for pthread.h... yes 2024-05-21T15:17:47.459 INFO:tasks.workunit.client.0.smithi078.stdout:checking fcntl.h usability... yes 2024-05-21T15:17:47.480 INFO:tasks.workunit.client.0.smithi078.stdout:checking fcntl.h presence... yes 2024-05-21T15:17:47.480 INFO:tasks.workunit.client.0.smithi078.stdout:checking for fcntl.h... yes 2024-05-21T15:17:47.535 INFO:tasks.workunit.client.0.smithi078.stdout:checking limits.h usability... yes 2024-05-21T15:17:47.555 INFO:tasks.workunit.client.0.smithi078.stdout:checking limits.h presence... yes 2024-05-21T15:17:47.555 INFO:tasks.workunit.client.0.smithi078.stdout:checking for limits.h... yes 2024-05-21T15:17:47.571 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdint.h... (cached) yes 2024-05-21T15:17:47.616 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/time.h usability... yes 2024-05-21T15:17:47.636 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/time.h presence... yes 2024-05-21T15:17:47.636 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/time.h... yes 2024-05-21T15:17:47.652 INFO:tasks.workunit.client.0.smithi078.stdout:checking for unistd.h... (cached) yes 2024-05-21T15:17:47.710 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/vfs.h usability... yes 2024-05-21T15:17:47.732 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/vfs.h presence... yes 2024-05-21T15:17:47.732 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/vfs.h... yes 2024-05-21T15:17:47.781 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/limits.h usability... no 2024-05-21T15:17:47.803 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/limits.h presence... no 2024-05-21T15:17:47.804 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/limits.h... no 2024-05-21T15:17:47.842 INFO:tasks.workunit.client.0.smithi078.stdout:checking for an ANSI C-conforming const... yes 2024-05-21T15:17:47.882 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-21T15:17:47.882 INFO:tasks.workunit.client.0.smithi078.stdout:checking for function prototypes... yes 2024-05-21T15:17:47.916 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether setvbuf arguments are reversed... no 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 -- 172.21.15.78:0/3128842406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 msgr2=0x7f772c1015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 --2- 172.21.15.78:0/3128842406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 0x7f772c1015e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f771c009900 tx=0x7f771c02a410 comp rx=0 tx=0).stop 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 -- 172.21.15.78:0/3128842406 shutdown_connections 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 --2- 172.21.15.78:0/3128842406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c107ed0 0x7f772c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 --2- 172.21.15.78:0/3128842406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f772c102190 0x7f772c10e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 --2- 172.21.15.78:0/3128842406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 0x7f772c1015e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 -- 172.21.15.78:0/3128842406 >> 172.21.15.78:0/3128842406 conn(0x7f772c0fd210 msgr2=0x7f772c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 -- 172.21.15.78:0/3128842406 shutdown_connections 2024-05-21T15:17:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 -- 172.21.15.78:0/3128842406 wait complete. 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.965+0000 7f7731614640 1 Processor -- start 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f7731614640 1 -- start start 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f7731614640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c102190 0x7f772c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f7731614640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 0x7f772c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f7731614640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f772c107ed0 0x7f772c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f7731614640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f772c111dd0 con 0x7f772c107a30 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f7731614640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f772c111c50 con 0x7f772c102190 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f7731614640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f772c111f50 con 0x7f772c107ed0 2024-05-21T15:17:47.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 0x7f772c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 0x7f772c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35878/0 (socket says 172.21.15.78:35878) 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772a7fc640 1 -- 172.21.15.78:0/463532413 learned_addr learned my addr 172.21.15.78:0/463532413 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772affd640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c102190 0x7f772c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772b7fe640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f772c107ed0 0x7f772c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772a7fc640 1 -- 172.21.15.78:0/463532413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f772c107ed0 msgr2=0x7f772c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772a7fc640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f772c107ed0 0x7f772c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772a7fc640 1 -- 172.21.15.78:0/463532413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c102190 msgr2=0x7f772c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.966+0000 7f772a7fc640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c102190 0x7f772c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.967+0000 7f772a7fc640 1 -- 172.21.15.78:0/463532413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c1a5940 con 0x7f772c107a30 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.967+0000 7f772affd640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c102190 0x7f772c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.967+0000 7f772a7fc640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 0x7f772c19ab70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f771400e9f0 tx=0x7f771400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.967+0000 7f7723fff640 1 -- 172.21.15.78:0/463532413 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771400ccc0 con 0x7f772c107a30 2024-05-21T15:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.967+0000 7f7723fff640 1 -- 172.21.15.78:0/463532413 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77140035f0 con 0x7f772c107a30 2024-05-21T15:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.967+0000 7f7731614640 1 -- 172.21.15.78:0/463532413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c1a5c30 con 0x7f772c107a30 2024-05-21T15:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.967+0000 7f7723fff640 1 -- 172.21.15.78:0/463532413 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771400ce60 con 0x7f772c107a30 2024-05-21T15:17:47.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.967+0000 7f7731614640 1 -- 172.21.15.78:0/463532413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c1a6180 con 0x7f772c107a30 2024-05-21T15:17:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.969+0000 7f7723fff640 1 -- 172.21.15.78:0/463532413 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f77140037a0 con 0x7f772c107a30 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.970+0000 7f7723fff640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f77180a2cf0 0x7f77180a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.970+0000 7f772affd640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f77180a2cf0 0x7f77180a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.970+0000 7f7723fff640 1 -- 172.21.15.78:0/463532413 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f77140cf740 con 0x7f772c107a30 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.970+0000 7f772affd640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f77180a2cf0 0x7f77180a5200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f771c02aa00 tx=0x7f771c031510 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f773859a740 1 Processor -- start 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f773859a740 1 -- start start 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f773859a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f773859a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f773859a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f773859a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dea712340 con 0x558dea40e1b0 2024-05-21T15:17:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f773859a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558dea6ba290 con 0x558dea3ed4d0 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f773859a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558dea70e300 con 0x558dea3e52f0 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f772affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f772affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35888/0 (socket says 172.21.15.78:35888) 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.971+0000 7f772affd640 1 -- 172.21.15.78:0/446158503 learned_addr learned my addr 172.21.15.78:0/446158503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772b7fe640 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772a7fc640 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772affd640 1 -- 172.21.15.78:0/446158503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 msgr2=0x558dea7ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772affd640 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772affd640 1 -- 172.21.15.78:0/446158503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 msgr2=0x558dea7adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772affd640 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772affd640 1 -- 172.21.15.78:0/446158503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dea6de8e0 con 0x558dea40e1b0 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772a7fc640 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772b7fe640 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f772affd640 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7ad080 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f771c033000 tx=0x7f771c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c04542c50547f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f77217fa640 1 -- 172.21.15.78:0/446158503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c035040 con 0x558dea40e1b0 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f77217fa640 1 -- 172.21.15.78:0/446158503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c04e070 con 0x558dea40e1b0 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.972+0000 7f77217fa640 1 -- 172.21.15.78:0/446158503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c032cf0 con 0x558dea40e1b0 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 -- 172.21.15.78:0/446158503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 msgr2=0x558dea7ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7ad080 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f771c033000 tx=0x7f771c02a410 comp rx=0 tx=0).stop 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 -- 172.21.15.78:0/446158503 shutdown_connections 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 --2- 172.21.15.78:0/446158503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7ad080 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 -- 172.21.15.78:0/446158503 >> 172.21.15.78:0/446158503 conn(0x558dea402300 msgr2=0x558dea7a9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 -- 172.21.15.78:0/446158503 shutdown_connections 2024-05-21T15:17:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.973+0000 7f773859a740 1 -- 172.21.15.78:0/446158503 wait complete. 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 1 -- start start 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 1 Processor -- start 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea3ed4d0 0x558dea7b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea40e1b0 0x558dea7b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558dea7ba150 con 0x558dea3ed4d0 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558dea6cd790 con 0x558dea40e1b0 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558dea7ba2d0 con 0x558dea3e52f0 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f773859a740 10 client.0 _ll_register_callbacks cb 0x558dea708a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea3ed4d0 0x558dea7b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea3ed4d0 0x558dea7b9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35900/0 (socket says 172.21.15.78:35900) 2024-05-21T15:17:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772a7fc640 1 -- 172.21.15.78:0/1880630394 learned_addr learned my addr 172.21.15.78:0/1880630394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772affd640 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772b7fe640 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea40e1b0 0x558dea7b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772a7fc640 1 -- 172.21.15.78:0/1880630394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 msgr2=0x558dea7b90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772a7fc640 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7b90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772a7fc640 1 -- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea40e1b0 msgr2=0x558dea7b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772a7fc640 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea40e1b0 0x558dea7b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772a7fc640 1 -- 172.21.15.78:0/1880630394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dea426210 con 0x558dea3ed4d0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.974+0000 7f772b7fe640 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea40e1b0 0x558dea7b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f772a7fc640 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea3ed4d0 0x558dea7b9610 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f771409bb90 tx=0x7f771409bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d55e3d16e9383c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f77127fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f77127fc640 1 -- 172.21.15.78:0/1880630394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714098270 con 0x558dea3ed4d0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f77127fc640 1 -- 172.21.15.78:0/1880630394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f77140a43e0 con 0x558dea3ed4d0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 1 -- 172.21.15.78:0/1880630394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771c04e070 con 0x558dea3ed4d0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f77127fc640 1 -- 172.21.15.78:0/1880630394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7714098410 con 0x558dea3ed4d0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 10 client.16068 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 1 -- 172.21.15.78:0/1880630394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558dea4f2620 con 0x558dea3ed4d0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f77127fc640 1 -- 172.21.15.78:0/1880630394 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c04e070 con 0x558dea3ed4d0 2024-05-21T15:17:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f77127fc640 1 -- 172.21.15.78:0/1880630394 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f771409b050 con 0x558dea3ed4d0 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 10 client.16068 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 20 client.16068 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558dea3ccd80 0x558dea7c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 4 client.16068 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.975+0000 7f773859a740 1 -- 172.21.15.78:0/1880630394 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dea71fa70 con 0x558dea3ccd80 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.976+0000 7f772affd640 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558dea3ccd80 0x558dea7c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.976+0000 7f77127fc640 10 client.16068 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.976+0000 7f772affd640 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558dea3ccd80 0x558dea7c9980 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caff62f95d7ba956 server_cookie=a127c657f000ce49 in_seq=0 out_seq=0 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f77127fc640 1 -- 172.21.15.78:0/1880630394 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558dea71fa70 con 0x558dea3ccd80 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f77127fc640 10 client.16068 handle_command_reply: tid=0 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 client.16068 shutdown 2024-05-21T15:17:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 -- 172.21.15.78:0/1880630394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea3ed4d0 msgr2=0x558dea7b9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea3ed4d0 0x558dea7b9610 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f771409bb90 tx=0x7f771409bc00 comp rx=0 tx=0).stop 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 -- 172.21.15.78:0/1880630394 shutdown_connections 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558dea3ccd80 0x558dea7c9980 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea40e1b0 0x558dea7b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f77127fc640 0 client.16068 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea3ed4d0 0x558dea7b9610 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 --2- 172.21.15.78:0/1880630394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7b90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 -- 172.21.15.78:0/1880630394 >> 172.21.15.78:0/1880630394 conn(0x558dea402300 msgr2=0x558dea7ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 -- 172.21.15.78:0/1880630394 shutdown_connections 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 1 -- 172.21.15.78:0/1880630394 wait complete. 2024-05-21T15:17:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.977+0000 7f773859a740 20 client.16068 trim_cache size 0 max 16384 2024-05-21T15:17:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.993+0000 7f773859a740 1 Processor -- start 2024-05-21T15:17:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.993+0000 7f773859a740 1 -- start start 2024-05-21T15:17:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f773859a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f773859a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f773859a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f773859a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1aecf0 con 0x558dea40e1b0 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f773859a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1b0830 con 0x558dea3ed4d0 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f773859a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1af3c0 con 0x558dea3e52f0 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:51950/0 (socket says 172.21.15.78:51950) 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7bfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35910/0 (socket says 172.21.15.78:35910) 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 -- 172.21.15.78:0/1598371506 learned_addr learned my addr 172.21.15.78:0/1598371506 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772affd640 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 -- 172.21.15.78:0/1598371506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 msgr2=0x558dea7b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 -- 172.21.15.78:0/1598371506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 msgr2=0x558dea7bfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7bfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 -- 172.21.15.78:0/1598371506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dea7bffe0 con 0x558dea3ed4d0 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772b7fe640 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f772a7fc640 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bf560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f771409b900 tx=0x7f7714003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=196b2f5b3237dead server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f7720ff9640 1 -- 172.21.15.78:0/1598371506 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714098270 con 0x558dea3ed4d0 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f7720ff9640 1 -- 172.21.15.78:0/1598371506 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771409adc0 con 0x558dea3ed4d0 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.994+0000 7f7720ff9640 1 -- 172.21.15.78:0/1598371506 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771409b9d0 con 0x558dea3ed4d0 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 -- 172.21.15.78:0/1598371506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 msgr2=0x558dea7bf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bf560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f771409b900 tx=0x7f7714003e30 comp rx=0 tx=0).stop 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 -- 172.21.15.78:0/1598371506 shutdown_connections 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bf560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea7bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 --2- 172.21.15.78:0/1598371506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 -- 172.21.15.78:0/1598371506 >> 172.21.15.78:0/1598371506 conn(0x558dea402300 msgr2=0x558dea7b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 -- 172.21.15.78:0/1598371506 shutdown_connections 2024-05-21T15:17:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.995+0000 7f773859a740 1 -- 172.21.15.78:0/1598371506 wait complete. 2024-05-21T15:17:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 1 -- start start 2024-05-21T15:17:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 1 Processor -- start 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea82fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1b0160 con 0x558dea40e1b0 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1afa90 con 0x558dea3ed4d0 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1accd0 con 0x558dea3e52f0 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f773859a740 10 client.0 _ll_register_callbacks cb 0x7f771400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f772b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea82fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f772a7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f772b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea82fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35918/0 (socket says 172.21.15.78:35918) 2024-05-21T15:17:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f772b7fe640 1 -- 172.21.15.78:0/1050663321 learned_addr learned my addr 172.21.15.78:0/1050663321 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.996+0000 7f772affd640 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f772b7fe640 1 -- 172.21.15.78:0/1050663321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 msgr2=0x558dea7be6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f772b7fe640 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7be6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f772b7fe640 1 -- 172.21.15.78:0/1050663321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 msgr2=0x558dea7bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f772b7fe640 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f772b7fe640 1 -- 172.21.15.78:0/1050663321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dea830260 con 0x558dea40e1b0 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f772a7fc640 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f772b7fe640 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea82fd20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7724009fd0 tx=0x7f7724012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3ef21f6a19e452 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f77127fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f77127fc640 1 -- 172.21.15.78:0/1050663321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724004d20 con 0x558dea40e1b0 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 1 -- 172.21.15.78:0/1050663321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771409adc0 con 0x558dea40e1b0 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f77127fc640 1 -- 172.21.15.78:0/1050663321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7724011ad0 con 0x558dea40e1b0 2024-05-21T15:17:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f77127fc640 1 -- 172.21.15.78:0/1050663321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7724011cd0 con 0x558dea40e1b0 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 10 client.16074 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 1 -- 172.21.15.78:0/1050663321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558dea831530 con 0x558dea40e1b0 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f77127fc640 1 -- 172.21.15.78:0/1050663321 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771409adc0 con 0x558dea40e1b0 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f77127fc640 1 -- 172.21.15.78:0/1050663321 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f772401b070 con 0x558dea40e1b0 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 10 client.16074 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 20 client.16074 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558dea3a21c0 0x558dea833d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 4 client.16074 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.997+0000 7f773859a740 1 -- 172.21.15.78:0/1050663321 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dea719dc0 con 0x558dea3a21c0 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.998+0000 7f772affd640 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558dea3a21c0 0x558dea833d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.998+0000 7f77127fc640 10 client.16074 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:47.998+0000 7f772affd640 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558dea3a21c0 0x558dea833d70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e51381f5332b85b server_cookie=692a32cdfe33cdaa in_seq=0 out_seq=0 2024-05-21T15:17:48.007 INFO:tasks.workunit.client.0.smithi078.stdout:checking for system... yes 2024-05-21T15:17:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f77127fc640 1 -- 172.21.15.78:0/1050663321 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dea719dc0 con 0x558dea3a21c0 2024-05-21T15:17:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f77127fc640 10 client.16074 handle_command_reply: tid=0 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 client.16074 shutdown 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 -- 172.21.15.78:0/1050663321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 msgr2=0x558dea82fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea82fd20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7724009fd0 tx=0x7f7724012d30 comp rx=0 tx=0).stop 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 -- 172.21.15.78:0/1050663321 shutdown_connections 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558dea3a21c0 0x558dea833d70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558dea3ed4d0 0x558dea7bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558dea40e1b0 0x558dea82fd20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f77127fc640 0 client.16074 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 --2- 172.21.15.78:0/1050663321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558dea3e52f0 0x558dea7be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 -- 172.21.15.78:0/1050663321 >> 172.21.15.78:0/1050663321 conn(0x558dea402300 msgr2=0x558dea7ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 -- 172.21.15.78:0/1050663321 shutdown_connections 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 1 -- 172.21.15.78:0/1050663321 wait complete. 2024-05-21T15:17:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.100+0000 7f773859a740 20 client.16074 trim_cache size 0 max 16384 2024-05-21T15:17:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:17:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:17:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:17:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 -- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f77180a2cf0 msgr2=0x7f77180a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f77180a2cf0 0x7f77180a5200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f771c02aa00 tx=0x7f771c031510 comp rx=0 tx=0).stop 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 -- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 msgr2=0x7f772c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 0x7f772c19ab70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f771400e9f0 tx=0x7f771400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 -- 172.21.15.78:0/463532413 shutdown_connections 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f77180a2cf0 0x7f77180a5200 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f772c107ed0 0x7f772c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f772c107a30 0x7f772c19ab70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 --2- 172.21.15.78:0/463532413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c102190 0x7f772c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 -- 172.21.15.78:0/463532413 >> 172.21.15.78:0/463532413 conn(0x7f772c0fd210 msgr2=0x7f772c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 -- 172.21.15.78:0/463532413 shutdown_connections 2024-05-21T15:17:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:17:48.101+0000 7f7731614640 1 -- 172.21.15.78:0/463532413 wait complete. 2024-05-21T15:17:48.114 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gettimeofday... yes 2024-05-21T15:17:48.178 INFO:tasks.workunit.client.0.smithi078.stdout:checking for mkdir... yes 2024-05-21T15:17:48.244 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strerror... yes 2024-05-21T15:17:48.312 INFO:tasks.workunit.client.0.smithi078.stdout:checking for open64... yes 2024-05-21T15:17:48.375 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stat64... yes 2024-05-21T15:17:48.439 INFO:tasks.workunit.client.0.smithi078.stdout:checking for fseeko64... yes 2024-05-21T15:17:48.501 INFO:tasks.workunit.client.0.smithi078.stdout:checking for lrand48_r... yes 2024-05-21T15:17:48.563 INFO:tasks.workunit.client.0.smithi078.stdout:checking for srand48_r... yes 2024-05-21T15:17:48.588 INFO:tasks.workunit.client.0.smithi078.stdout:configure: creating ./config.status 2024-05-21T15:17:48.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:48 smithi078 ceph-mon[31997]: pgmap v54: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-05-21T15:17:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:48 smithi084 ceph-mon[36334]: pgmap v54: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-05-21T15:17:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:48 smithi172 ceph-mon[36402]: pgmap v54: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-05-21T15:17:48.689 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating Makefile 2024-05-21T15:17:48.702 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating config.h 2024-05-21T15:17:48.719 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: executing depfiles commands 2024-05-21T15:17:48.767 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: executing default commands 2024-05-21T15:17:48.818 INFO:tasks.workunit.client.0.smithi078.stderr:+ make 2024-05-21T15:17:48.826 INFO:tasks.workunit.client.0.smithi078.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-05-21T15:17:48.828 INFO:tasks.workunit.client.0.smithi078.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-05-21T15:17:48.829 INFO:tasks.workunit.client.0.smithi078.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-05-21T15:17:48.829 INFO:tasks.workunit.client.0.smithi078.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-05-21T15:17:48.829 INFO:tasks.workunit.client.0.smithi078.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-05-21T15:17:48.830 INFO:tasks.workunit.client.0.smithi078.stderr: any GNU archive site. 2024-05-21T15:17:48.833 INFO:tasks.workunit.client.0.smithi078.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-05-21T15:17:48.834 INFO:tasks.workunit.client.0.smithi078.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-05-21T15:17:48.834 INFO:tasks.workunit.client.0.smithi078.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-05-21T15:17:48.834 INFO:tasks.workunit.client.0.smithi078.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-05-21T15:17:48.834 INFO:tasks.workunit.client.0.smithi078.stderr: You might want to install the `Automake' and `Perl' packages. 2024-05-21T15:17:48.834 INFO:tasks.workunit.client.0.smithi078.stderr: Grab them from any GNU archive site. 2024-05-21T15:17:48.900 INFO:tasks.workunit.client.0.smithi078.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-05-21T15:17:49.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:17:49.242 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:17:49.289 INFO:tasks.workunit.client.0.smithi078.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-05-21T15:17:49.289 INFO:tasks.workunit.client.0.smithi078.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-05-21T15:17:49.289 INFO:tasks.workunit.client.0.smithi078.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-05-21T15:17:49.290 INFO:tasks.workunit.client.0.smithi078.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-05-21T15:17:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 -- 172.21.15.78:0/393537619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c101b30 msgr2=0x7f031c101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 --2- 172.21.15.78:0/393537619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c101b30 0x7f031c101f40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0308009900 tx=0x7f030802a410 comp rx=0 tx=0).stop 2024-05-21T15:17:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 -- 172.21.15.78:0/393537619 shutdown_connections 2024-05-21T15:17:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 --2- 172.21.15.78:0/393537619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031c10c370 0x7f031c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 --2- 172.21.15.78:0/393537619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c101b30 0x7f031c101f40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 --2- 172.21.15.78:0/393537619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c107a10 0x7f031c1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 -- 172.21.15.78:0/393537619 >> 172.21.15.78:0/393537619 conn(0x7f031c077130 msgr2=0x7f031c077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 -- 172.21.15.78:0/393537619 shutdown_connections 2024-05-21T15:17:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.355+0000 7f0323008640 1 -- 172.21.15.78:0/393537619 wait complete. 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.356+0000 7f0323008640 1 Processor -- start 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.356+0000 7f0323008640 1 -- start start 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.356+0000 7f0323008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031c101b30 0x7f031c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.356+0000 7f0323008640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c107a10 0x7f031c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.356+0000 7f0323008640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c10c370 0x7f031c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.356+0000 7f0323008640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f031c111d70 con 0x7f031c10c370 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.356+0000 7f0323008640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f031c111bf0 con 0x7f031c101b30 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.356+0000 7f0323008640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f031c111ef0 con 0x7f031c107a10 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f03215a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c10c370 0x7f031c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0320da4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031c101b30 0x7f031c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f03215a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c10c370 0x7f031c19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55746/0 (socket says 172.21.15.78:55746) 2024-05-21T15:17:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f03215a5640 1 -- 172.21.15.78:0/1764988303 learned_addr learned my addr 172.21.15.78:0/1764988303 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f031bfff640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c107a10 0x7f031c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0320da4640 1 -- 172.21.15.78:0/1764988303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c107a10 msgr2=0x7f031c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0320da4640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c107a10 0x7f031c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0320da4640 1 -- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c10c370 msgr2=0x7f031c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0320da4640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c10c370 0x7f031c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0320da4640 1 -- 172.21.15.78:0/1764988303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c19d4f0 con 0x7f031c101b30 2024-05-21T15:17:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f03215a5640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c10c370 0x7f031c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0320da4640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031c101b30 0x7f031c198690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f031000e9c0 tx=0x7f031000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0319ffb640 1 -- 172.21.15.78:0/1764988303 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031000ccc0 con 0x7f031c101b30 2024-05-21T15:17:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.357+0000 7f0323008640 1 -- 172.21.15.78:0/1764988303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c19d7e0 con 0x7f031c101b30 2024-05-21T15:17:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.358+0000 7f0319ffb640 1 -- 172.21.15.78:0/1764988303 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0310005020 con 0x7f031c101b30 2024-05-21T15:17:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.358+0000 7f0319ffb640 1 -- 172.21.15.78:0/1764988303 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031001c070 con 0x7f031c101b30 2024-05-21T15:17:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.358+0000 7f0323008640 1 -- 172.21.15.78:0/1764988303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c1a5160 con 0x7f031c101b30 2024-05-21T15:17:49.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.360+0000 7f0319ffb640 1 -- 172.21.15.78:0/1764988303 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0310004020 con 0x7f031c101b30 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.360+0000 7f0329f81740 1 Processor -- start 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.360+0000 7f0329f81740 1 -- start start 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0329f81740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7c4390 0x55f9cab62b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0329f81740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0329f81740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7a36b0 0x55f9cab63600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0329f81740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9caac8d00 con 0x55f9ca79b4d0 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0329f81740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f9caa704c0 con 0x55f9ca7a36b0 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0329f81740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f9caac4cc0 con 0x55f9ca7c4390 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0320da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f03215a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7c4390 0x55f9cab62b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0320da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55752/0 (socket says 172.21.15.78:55752) 2024-05-21T15:17:49.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0320da4640 1 -- 172.21.15.78:0/1559478811 learned_addr learned my addr 172.21.15.78:0/1559478811 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0319ffb640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f030c0a2d00 0x7f030c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f031bfff640 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7a36b0 0x55f9cab63600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f03215a5640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f030c0a2d00 0x7f030c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0319ffb640 1 -- 172.21.15.78:0/1764988303 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f03100cf410 con 0x7f031c101b30 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0320da4640 1 -- 172.21.15.78:0/1559478811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7c4390 msgr2=0x55f9cab62b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0320da4640 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7c4390 0x55f9cab62b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0320da4640 1 -- 172.21.15.78:0/1559478811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7a36b0 msgr2=0x55f9cab63600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0320da4640 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7a36b0 0x55f9cab63600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f0320da4640 1 -- 172.21.15.78:0/1559478811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ca8a67d0 con 0x55f9ca79b4d0 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f031bfff640 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7a36b0 0x55f9cab63600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.361+0000 7f03215a5640 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7c4390 0x55f9cab62b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.362+0000 7f0320da4640 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab630c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f031009b5d0 tx=0x7f031000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1b529df12d6b30 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.362+0000 7f0306ffd640 1 -- 172.21.15.78:0/1559478811 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031009bfd0 con 0x55f9ca79b4d0 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.362+0000 7f03215a5640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f030c0a2d00 0x7f030c0a5210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55f9cab60740 tx=0x7f0314004ab0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.362+0000 7f0306ffd640 1 -- 172.21.15.78:0/1559478811 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f031009b8d0 con 0x55f9ca79b4d0 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.362+0000 7f0306ffd640 1 -- 172.21.15.78:0/1559478811 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310099c60 con 0x55f9ca79b4d0 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 -- 172.21.15.78:0/1559478811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 msgr2=0x55f9cab630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab630c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f031009b5d0 tx=0x7f031000efd0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 -- 172.21.15.78:0/1559478811 shutdown_connections 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7a36b0 0x55f9cab63600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab630c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 --2- 172.21.15.78:0/1559478811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7c4390 0x55f9cab62b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 -- 172.21.15.78:0/1559478811 >> 172.21.15.78:0/1559478811 conn(0x55f9ca7b84e0 msgr2=0x55f9cab5f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 -- 172.21.15.78:0/1559478811 shutdown_connections 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 -- 172.21.15.78:0/1559478811 wait complete. 2024-05-21T15:17:49.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 -- start start 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.363+0000 7f0329f81740 1 Processor -- start 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0329f81740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab6f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0329f81740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 0x55f9cab6fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0329f81740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7c4390 0x55f9cab75750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0329f81740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9cab72260 con 0x55f9ca79b4d0 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0329f81740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f9caa83b90 con 0x55f9ca7c4390 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0329f81740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f9cab723e0 con 0x55f9ca7a36b0 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0329f81740 10 client.0 _ll_register_callbacks cb 0x55f9caabfda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0320da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab6f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0320da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab6f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55768/0 (socket says 172.21.15.78:55768) 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0320da4640 1 -- 172.21.15.78:0/1902518276 learned_addr learned my addr 172.21.15.78:0/1902518276 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f03215a5640 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7c4390 0x55f9cab75750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f031bfff640 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 0x55f9cab6fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f03215a5640 1 -- 172.21.15.78:0/1902518276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 msgr2=0x55f9cab6fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f03215a5640 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 0x55f9cab6fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f03215a5640 1 -- 172.21.15.78:0/1902518276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 msgr2=0x55f9cab6f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f03215a5640 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab6f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f03215a5640 1 -- 172.21.15.78:0/1902518276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ca7dc3f0 con 0x55f9ca7c4390 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f0320da4640 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab6f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f03215a5640 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7c4390 0x55f9cab75750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03140049d0 tx=0x7f0314004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81a5d89d57fc3fd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.364+0000 7f02fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f02fbfff640 1 -- 172.21.15.78:0/1902518276 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314026070 con 0x55f9ca7c4390 2024-05-21T15:17:49.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f02fbfff640 1 -- 172.21.15.78:0/1902518276 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f0314004d20 con 0x55f9ca7c4390 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 1 -- 172.21.15.78:0/1902518276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031009b8d0 con 0x55f9ca7c4390 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f02fbfff640 1 -- 172.21.15.78:0/1902518276 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0314031050 con 0x55f9ca7c4390 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 10 client.25580 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 1 -- 172.21.15.78:0/1902518276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9ca79c0f0 con 0x55f9ca7c4390 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f02fbfff640 1 -- 172.21.15.78:0/1902518276 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f031009b8d0 con 0x55f9ca7c4390 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f02fbfff640 1 -- 172.21.15.78:0/1902518276 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f031401f050 con 0x55f9ca7c4390 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 10 client.25580 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 20 client.25580 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f9ca782f60 0x55f9cab81d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 4 client.25580 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0329f81740 1 -- 172.21.15.78:0/1902518276 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9caad6430 con 0x55f9ca782f60 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.365+0000 7f0320da4640 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f9ca782f60 0x55f9cab81d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.366+0000 7f02fbfff640 10 client.25580 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.366+0000 7f0320da4640 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f9ca782f60 0x55f9cab81d50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ede3b3e048f316b server_cookie=916b372c663c1912 in_seq=0 out_seq=0 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f02fbfff640 1 -- 172.21.15.78:0/1902518276 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9caad6430 con 0x55f9ca782f60 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f02fbfff640 10 client.25580 handle_command_reply: tid=0 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 client.25580 shutdown 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 -- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7c4390 msgr2=0x55f9cab75750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7c4390 0x55f9cab75750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03140049d0 tx=0x7f0314004a40 comp rx=0 tx=0).stop 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 -- 172.21.15.78:0/1902518276 shutdown_connections 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f9ca782f60 0x55f9cab81d50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca7c4390 0x55f9cab75750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 0x55f9cab6fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f02fbfff640 0 client.25580 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 --2- 172.21.15.78:0/1902518276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca79b4d0 0x55f9cab6f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 -- 172.21.15.78:0/1902518276 >> 172.21.15.78:0/1902518276 conn(0x55f9ca7b84e0 msgr2=0x55f9cab634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 -- 172.21.15.78:0/1902518276 shutdown_connections 2024-05-21T15:17:49.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 1 -- 172.21.15.78:0/1902518276 wait complete. 2024-05-21T15:17:49.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.367+0000 7f0329f81740 20 client.25580 trim_cache size 0 max 16384 2024-05-21T15:17:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0329f81740 1 Processor -- start 2024-05-21T15:17:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0329f81740 1 -- start start 2024-05-21T15:17:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0329f81740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca782f60 0x55f9cab75650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0329f81740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab68d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0329f81740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca7a36b0 0x55f9cab692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0329f81740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ac850 con 0x55f9ca7a36b0 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0329f81740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ae390 con 0x55f9ca79b4d0 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0329f81740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1acf20 con 0x55f9ca782f60 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f03215a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca7a36b0 0x55f9cab692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f03215a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca7a36b0 0x55f9cab692b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55778/0 (socket says 172.21.15.78:55778) 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f03215a5640 1 -- 172.21.15.78:0/2926732310 learned_addr learned my addr 172.21.15.78:0/2926732310 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f0320da4640 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca782f60 0x55f9cab75650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f031bfff640 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab68d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f03215a5640 1 -- 172.21.15.78:0/2926732310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca782f60 msgr2=0x55f9cab75650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f03215a5640 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca782f60 0x55f9cab75650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f03215a5640 1 -- 172.21.15.78:0/2926732310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 msgr2=0x55f9cab68d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f03215a5640 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab68d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f03215a5640 1 -- 172.21.15.78:0/2926732310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9cab822b0 con 0x55f9ca7a36b0 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.385+0000 7f031bfff640 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab68d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.386+0000 7f03215a5640 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca7a36b0 0x55f9cab692b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0314012d20 tx=0x7f0314022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6bec07e6fd1fc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.386+0000 7f03067fc640 1 -- 172.21.15.78:0/2926732310 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314026040 con 0x55f9ca7a36b0 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.386+0000 7f03067fc640 1 -- 172.21.15.78:0/2926732310 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0314021c50 con 0x55f9ca7a36b0 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.386+0000 7f03067fc640 1 -- 172.21.15.78:0/2926732310 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314005260 con 0x55f9ca7a36b0 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 -- 172.21.15.78:0/2926732310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca7a36b0 msgr2=0x55f9cab692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca7a36b0 0x55f9cab692b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0314012d20 tx=0x7f0314022000 comp rx=0 tx=0).stop 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 -- 172.21.15.78:0/2926732310 shutdown_connections 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab68d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca7a36b0 0x55f9cab692b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 --2- 172.21.15.78:0/2926732310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca782f60 0x55f9cab75650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 -- 172.21.15.78:0/2926732310 >> 172.21.15.78:0/2926732310 conn(0x55f9ca7b84e0 msgr2=0x55f9cab6a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 -- 172.21.15.78:0/2926732310 shutdown_connections 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 -- 172.21.15.78:0/2926732310 wait complete. 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 -- start start 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.387+0000 7f0329f81740 1 Processor -- start 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0329f81740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca782f60 0x55f9cab5e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0329f81740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab5e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0329f81740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 0x55f9cab7a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0329f81740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1adcc0 con 0x55f9ca782f60 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0329f81740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1ad5f0 con 0x55f9ca79b4d0 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0329f81740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1aa830 con 0x55f9ca7a36b0 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0329f81740 10 client.0 _ll_register_callbacks cb 0x7f031400f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca782f60 0x55f9cab5e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca782f60 0x55f9cab5e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55782/0 (socket says 172.21.15.78:55782) 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 -- 172.21.15.78:0/1068779760 learned_addr learned my addr 172.21.15.78:0/1068779760 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f031bfff640 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab5e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f03215a5640 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 0x55f9cab7a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 -- 172.21.15.78:0/1068779760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 msgr2=0x55f9cab7a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 0x55f9cab7a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 -- 172.21.15.78:0/1068779760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 msgr2=0x55f9cab5e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab5e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 -- 172.21.15.78:0/1068779760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9cab7ac10 con 0x55f9ca782f60 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0320da4640 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca782f60 0x55f9cab5e170 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0310099b80 tx=0x7f0310004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d7a028354c2a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f02fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f02fbfff640 1 -- 172.21.15.78:0/1068779760 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031009b8d0 con 0x55f9ca782f60 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f02fbfff640 1 -- 172.21.15.78:0/1068779760 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f03100ccd80 con 0x55f9ca782f60 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.388+0000 7f0329f81740 1 -- 172.21.15.78:0/1068779760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0314021c50 con 0x55f9ca782f60 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f02fbfff640 1 -- 172.21.15.78:0/1068779760 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f03100a4540 con 0x55f9ca782f60 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f0329f81740 10 client.16092 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f0329f81740 1 -- 172.21.15.78:0/1068779760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9cab7bee0 con 0x55f9ca782f60 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f02fbfff640 1 -- 172.21.15.78:0/1068779760 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0314021c50 con 0x55f9ca782f60 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f02fbfff640 1 -- 172.21.15.78:0/1068779760 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f03100cf020 con 0x55f9ca782f60 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f0329f81740 10 client.16092 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f0329f81740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f0329f81740 20 client.16092 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f0329f81740 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f9ca7c4390 0x55f9cabec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f0329f81740 4 client.16092 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f0329f81740 1 -- 172.21.15.78:0/1068779760 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9caad0780 con 0x55f9ca7c4390 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.389+0000 7f031bfff640 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f9ca7c4390 0x55f9cabec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f02fbfff640 10 client.16092 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f031bfff640 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f9ca7c4390 0x55f9cabec0d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78c26b86653aac7e server_cookie=ae271cb6db02b1ba in_seq=0 out_seq=0 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f02fbfff640 1 -- 172.21.15.78:0/1068779760 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f9caad0780 con 0x55f9ca7c4390 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f02fbfff640 10 client.16092 handle_command_reply: tid=0 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 client.16092 shutdown 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 -- 172.21.15.78:0/1068779760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca782f60 msgr2=0x55f9cab5e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca782f60 0x55f9cab5e170 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0310099b80 tx=0x7f0310004460 comp rx=0 tx=0).stop 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 -- 172.21.15.78:0/1068779760 shutdown_connections 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f9ca7c4390 0x55f9cabec0d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f9ca79b4d0 0x55f9cab5e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f02fbfff640 0 client.16092 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f9ca7a36b0 0x55f9cab7a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 --2- 172.21.15.78:0/1068779760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9ca782f60 0x55f9cab5e170 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 -- 172.21.15.78:0/1068779760 >> 172.21.15.78:0/1068779760 conn(0x55f9ca7b84e0 msgr2=0x55f9cab64830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 -- 172.21.15.78:0/1068779760 shutdown_connections 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 1 -- 172.21.15.78:0/1068779760 wait complete. 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.390+0000 7f0329f81740 20 client.16092 trim_cache size 0 max 16384 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:17:49.401 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:17:49.402 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:17:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 -- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f030c0a2d00 msgr2=0x7f030c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f030c0a2d00 0x7f030c0a5210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55f9cab60740 tx=0x7f0314004ab0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 -- 172.21.15.78:0/1764988303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031c101b30 msgr2=0x7f031c198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031c101b30 0x7f031c198690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f031000e9c0 tx=0x7f031000ee80 comp rx=0 tx=0).stop 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 -- 172.21.15.78:0/1764988303 shutdown_connections 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f030c0a2d00 0x7f030c0a5210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031c10c370 0x7f031c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c107a10 0x7f031c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 --2- 172.21.15.78:0/1764988303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f031c101b30 0x7f031c198690 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 -- 172.21.15.78:0/1764988303 >> 172.21.15.78:0/1764988303 conn(0x7f031c077130 msgr2=0x7f031c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 -- 172.21.15.78:0/1764988303 shutdown_connections 2024-05-21T15:17:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.391+0000 7f0323008640 1 -- 172.21.15.78:0/1764988303 wait complete. 2024-05-21T15:17:49.405 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:17:49.405 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:17:49.414 INFO:tasks.workunit.client.0.smithi078.stdout:/bin/sh ./config.status --recheck 2024-05-21T15:17:49.464 INFO:tasks.workunit.client.0.smithi078.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-05-21T15:17:49.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 -- 172.21.15.78:0/3654945744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75041079d0 msgr2=0x7f7504101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 --2- 172.21.15.78:0/3654945744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75041079d0 0x7f7504101560 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7500009860 tx=0x7f750002a340 comp rx=0 tx=0).stop 2024-05-21T15:17:49.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 -- 172.21.15.78:0/3654945744 shutdown_connections 2024-05-21T15:17:49.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 --2- 172.21.15.78:0/3654945744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f750410c330 0x7f750410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 --2- 172.21.15.78:0/3654945744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7504101aa0 0x7f7504101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 --2- 172.21.15.78:0/3654945744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75041079d0 0x7f7504101560 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 -- 172.21.15.78:0/3654945744 >> 172.21.15.78:0/3654945744 conn(0x7f75040770c0 msgr2=0x7f7504077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 -- 172.21.15.78:0/3654945744 shutdown_connections 2024-05-21T15:17:49.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.516+0000 7f750bdea640 1 -- 172.21.15.78:0/3654945744 wait complete. 2024-05-21T15:17:49.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750bdea640 1 Processor -- start 2024-05-21T15:17:49.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750bdea640 1 -- start start 2024-05-21T15:17:49.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750bdea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7504101aa0 0x7f7504198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750bdea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75041079d0 0x7f7504198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750bdea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f750410c330 0x7f750419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750bdea640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7504111ce0 con 0x7f7504101aa0 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750bdea640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7504111b60 con 0x7f750410c330 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750bdea640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7504111e60 con 0x7f75041079d0 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f7509b86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7504101aa0 0x7f7504198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f7509b86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7504101aa0 0x7f7504198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55808/0 (socket says 172.21.15.78:55808) 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f7509b86640 1 -- 172.21.15.78:0/357353315 learned_addr learned my addr 172.21.15.78:0/357353315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.517+0000 7f750a387640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f750410c330 0x7f750419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f7509385640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75041079d0 0x7f7504198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f7509b86640 1 -- 172.21.15.78:0/357353315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75041079d0 msgr2=0x7f7504198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f7509b86640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75041079d0 0x7f7504198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f7509b86640 1 -- 172.21.15.78:0/357353315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f750410c330 msgr2=0x7f750419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f7509b86640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f750410c330 0x7f750419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f7509b86640 1 -- 172.21.15.78:0/357353315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750419d4e0 con 0x7f7504101aa0 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f7509385640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75041079d0 0x7f7504198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f7509b86640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7504101aa0 0x7f7504198680 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7500000c00 tx=0x7f7500033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f74faffd640 1 -- 172.21.15.78:0/357353315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750003f020 con 0x7f7504101aa0 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750419d770 con 0x7f7504101aa0 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f74faffd640 1 -- 172.21.15.78:0/357353315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7500032770 con 0x7f7504101aa0 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f74faffd640 1 -- 172.21.15.78:0/357353315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500037040 con 0x7f7504101aa0 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.518+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750419dcb0 con 0x7f7504101aa0 2024-05-21T15:17:49.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.519+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7504102c80 con 0x7f7504101aa0 2024-05-21T15:17:49.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.520+0000 7f74faffd640 1 -- 172.21.15.78:0/357353315 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7500042020 con 0x7f7504101aa0 2024-05-21T15:17:49.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.521+0000 7f74faffd640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74f00a2d00 0x7f74f00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.521+0000 7f74faffd640 1 -- 172.21.15.78:0/357353315 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f75000ef490 con 0x7f7504101aa0 2024-05-21T15:17:49.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.521+0000 7f7509385640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74f00a2d00 0x7f74f00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.522+0000 7f7509385640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74f00a2d00 0x7f74f00a5210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f750410d6f0 tx=0x7f74f4005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.526+0000 7f74faffd640 1 -- 172.21.15.78:0/357353315 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7500049020 con 0x7f7504101aa0 2024-05-21T15:17:49.570 INFO:tasks.workunit.client.0.smithi078.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-21T15:17:49.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.773+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7504069f50 con 0x7f7504101aa0 2024-05-21T15:17:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.774+0000 7f74faffd640 1 -- 172.21.15.78:0/357353315 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11925 (secure 0 0 0) 0x7f75000bc160 con 0x7f7504101aa0 2024-05-21T15:17:49.777 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:17:49.777 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:16:06.684612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24419,24448,24502,24526,24527]},"id":1}]} 2024-05-21T15:17:49.777 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-05-21T15:17:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74f00a2d00 msgr2=0x7f74f00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74f00a2d00 0x7f74f00a5210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f750410d6f0 tx=0x7f74f4005f90 comp rx=0 tx=0).stop 2024-05-21T15:17:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7504101aa0 msgr2=0x7f7504198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7504101aa0 0x7f7504198680 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7500000c00 tx=0x7f7500033d10 comp rx=0 tx=0).stop 2024-05-21T15:17:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 shutdown_connections 2024-05-21T15:17:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f750410c330 0x7f750419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74f00a2d00 0x7f74f00a5210 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75041079d0 0x7f7504198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 --2- 172.21.15.78:0/357353315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7504101aa0 0x7f7504198680 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 >> 172.21.15.78:0/357353315 conn(0x7f75040770c0 msgr2=0x7f7504103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 shutdown_connections 2024-05-21T15:17:49.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.776+0000 7f750bdea640 1 -- 172.21.15.78:0/357353315 wait complete. 2024-05-21T15:17:49.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:17:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 -- 172.21.15.78:0/3833072764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e41079d0 msgr2=0x7f40e4101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 --2- 172.21.15.78:0/3833072764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e41079d0 0x7f40e4101560 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009900 tx=0x7f40e002a410 comp rx=0 tx=0).stop 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 -- 172.21.15.78:0/3833072764 shutdown_connections 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 --2- 172.21.15.78:0/3833072764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40e4101aa0 0x7f40e4101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 --2- 172.21.15.78:0/3833072764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e410c330 0x7f40e410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 --2- 172.21.15.78:0/3833072764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e41079d0 0x7f40e4101560 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 -- 172.21.15.78:0/3833072764 >> 172.21.15.78:0/3833072764 conn(0x7f40e40770b0 msgr2=0x7f40e40774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 -- 172.21.15.78:0/3833072764 shutdown_connections 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.941+0000 7f40ebe06640 1 -- 172.21.15.78:0/3833072764 wait complete. 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ebe06640 1 Processor -- start 2024-05-21T15:17:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ebe06640 1 -- start start 2024-05-21T15:17:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ebe06640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e4101aa0 0x7f40e4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ebe06640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40e41079d0 0x7f40e4199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ebe06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e410c330 0x7f40e419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ebe06640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4112e30 con 0x7f40e4101aa0 2024-05-21T15:17:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ebe06640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4112cb0 con 0x7f40e41079d0 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ebe06640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4112fb0 con 0x7f40e410c330 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40e9ba2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e4101aa0 0x7f40e4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40e9ba2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e4101aa0 0x7f40e4199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55818/0 (socket says 172.21.15.78:55818) 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40e9ba2640 1 -- 172.21.15.78:0/1181093046 learned_addr learned my addr 172.21.15.78:0/1181093046 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.942+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e410c330 0x7f40e419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40e93a1640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40e41079d0 0x7f40e4199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40e9ba2640 1 -- 172.21.15.78:0/1181093046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e410c330 msgr2=0x7f40e419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e410c330 0x7f40e419dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40e9ba2640 1 -- 172.21.15.78:0/1181093046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40e41079d0 msgr2=0x7f40e4199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40e41079d0 0x7f40e4199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40e9ba2640 1 -- 172.21.15.78:0/1181093046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e419e5f0 con 0x7f40e4101aa0 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e4101aa0 0x7f40e4199790 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f40e002a930 tx=0x7f40e0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40daffd640 1 -- 172.21.15.78:0/1181093046 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e003f070 con 0x7f40e4101aa0 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40daffd640 1 -- 172.21.15.78:0/1181093046 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e0031c90 con 0x7f40e4101aa0 2024-05-21T15:17:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40ebe06640 1 -- 172.21.15.78:0/1181093046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e419e880 con 0x7f40e4101aa0 2024-05-21T15:17:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40daffd640 1 -- 172.21.15.78:0/1181093046 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0035030 con 0x7f40e4101aa0 2024-05-21T15:17:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.943+0000 7f40ebe06640 1 -- 172.21.15.78:0/1181093046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e419ed70 con 0x7f40e4101aa0 2024-05-21T15:17:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.946+0000 7f40daffd640 1 -- 172.21.15.78:0/1181093046 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f40e0042070 con 0x7f40e4101aa0 2024-05-21T15:17:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.946+0000 7f40f2d88740 1 Processor -- start 2024-05-21T15:17:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40f2d88740 1 -- start start 2024-05-21T15:17:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff253a390 0x55dff28d8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28d9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28d97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40f2d88740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dff283ec40 con 0x55dff253a390 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40f2d88740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dff27e6580 con 0x55dff25114d0 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40f2d88740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dff283ac00 con 0x55dff25196b0 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40ea3a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff253a390 0x55dff28d8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40ea3a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff253a390 0x55dff28d8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55834/0 (socket says 172.21.15.78:55834) 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40ea3a3640 1 -- 172.21.15.78:0/3344492929 learned_addr learned my addr 172.21.15.78:0/3344492929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40daffd640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40d00a2d00 0x7f40d00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40e93a1640 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28d97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28d9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40d00a2d00 0x7f40d00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40daffd640 1 -- 172.21.15.78:0/1181093046 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f40e00efbd0 con 0x7f40e4101aa0 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40ea3a3640 1 -- 172.21.15.78:0/3344492929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 msgr2=0x55dff28d97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28d97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40ea3a3640 1 -- 172.21.15.78:0/3344492929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 msgr2=0x55dff28d9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.947+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28d9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.948+0000 7f40ea3a3640 1 -- 172.21.15.78:0/3344492929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff25523f0 con 0x55dff253a390 2024-05-21T15:17:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.948+0000 7f40e93a1640 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28d97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.948+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff253a390 0x55dff28d8d20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0139c0 tx=0x7f40dc013e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d59706be079897 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.948+0000 7f40cffff640 1 -- 172.21.15.78:0/3344492929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc021030 con 0x55dff253a390 2024-05-21T15:17:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.948+0000 7f40cffff640 1 -- 172.21.15.78:0/3344492929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc011c80 con 0x55dff253a390 2024-05-21T15:17:49.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.948+0000 7f40cffff640 1 -- 172.21.15.78:0/3344492929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc012450 con 0x55dff253a390 2024-05-21T15:17:49.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 -- 172.21.15.78:0/3344492929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff253a390 msgr2=0x55dff28d8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40d00a2d00 0x7f40d00a5210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40dc021000 tx=0x7f40dc020000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff253a390 0x55dff28d8d20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0139c0 tx=0x7f40dc013e80 comp rx=0 tx=0).stop 2024-05-21T15:17:49.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 -- 172.21.15.78:0/3344492929 shutdown_connections 2024-05-21T15:17:49.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28d9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28d97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 --2- 172.21.15.78:0/3344492929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff253a390 0x55dff28d8d20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 -- 172.21.15.78:0/3344492929 >> 172.21.15.78:0/3344492929 conn(0x55dff252e4e0 msgr2=0x55dff28d54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 -- 172.21.15.78:0/3344492929 shutdown_connections 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 -- 172.21.15.78:0/3344492929 wait complete. 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 -- start start 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.949+0000 7f40f2d88740 1 Processor -- start 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28e3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28e3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff253a390 0x55dff28e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40f2d88740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dff28e5fb0 con 0x55dff25196b0 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40f2d88740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dff27f9c40 con 0x55dff25114d0 2024-05-21T15:17:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40f2d88740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dff28e6130 con 0x55dff253a390 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40f2d88740 10 client.0 _ll_register_callbacks cb 0x55dff2551be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e93a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e93a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28e3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55836/0 (socket says 172.21.15.78:55836) 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e93a1640 1 -- 172.21.15.78:0/1899227769 learned_addr learned my addr 172.21.15.78:0/1899227769 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff253a390 0x55dff28e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28e3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e93a1640 1 -- 172.21.15.78:0/1899227769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff253a390 msgr2=0x55dff28e9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e93a1640 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff253a390 0x55dff28e9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e93a1640 1 -- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 msgr2=0x55dff28e3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e93a1640 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28e3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e93a1640 1 -- 172.21.15.78:0/1899227769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff261c7d0 con 0x55dff25196b0 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.950+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28e3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40e93a1640 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28e3950 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40d4012700 tx=0x7f40d4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b4c80c96a40227 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40cd7fa640 1 -- 172.21.15.78:0/1899227769 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d400f920 con 0x55dff25196b0 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40cd7fa640 1 -- 172.21.15.78:0/1899227769 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f40d400fac0 con 0x55dff25196b0 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 1 -- 172.21.15.78:0/1899227769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40dc011c80 con 0x55dff25196b0 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40cd7fa640 1 -- 172.21.15.78:0/1899227769 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40d400fcc0 con 0x55dff25196b0 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 10 client.16110 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 1 -- 172.21.15.78:0/1899227769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dff28f2780 con 0x55dff25196b0 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40cd7fa640 1 -- 172.21.15.78:0/1899227769 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc011c80 con 0x55dff25196b0 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40cd7fa640 1 -- 172.21.15.78:0/1899227769 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40d401c420 con 0x55dff25196b0 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 10 client.16110 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 10 client.16110 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 20 client.16110 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55dff24f8f60 0x55dff28f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 4 client.16110 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.951+0000 7f40f2d88740 1 -- 172.21.15.78:0/1899227769 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff284c370 con 0x55dff24f8f60 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.952+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55dff24f8f60 0x55dff28f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.952+0000 7f40cd7fa640 10 client.16110 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.952+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55dff24f8f60 0x55dff28f5ce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=996e2ac7b9c16da server_cookie=205d2fd08a682844 in_seq=0 out_seq=0 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40cd7fa640 1 -- 172.21.15.78:0/1899227769 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dff284c370 con 0x55dff24f8f60 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40cd7fa640 10 client.16110 handle_command_reply: tid=0 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 client.16110 shutdown 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 -- 172.21.15.78:0/1899227769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 msgr2=0x55dff28e3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28e3950 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40d4012700 tx=0x7f40d4012bc0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 -- 172.21.15.78:0/1899227769 shutdown_connections 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55dff24f8f60 0x55dff28f5ce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff253a390 0x55dff28e9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28e3950 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 --2- 172.21.15.78:0/1899227769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff25114d0 0x55dff28e3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40cd7fa640 0 client.16110 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:17:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 -- 172.21.15.78:0/1899227769 >> 172.21.15.78:0/1899227769 conn(0x55dff252e4e0 msgr2=0x55dff28daa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 -- 172.21.15.78:0/1899227769 shutdown_connections 2024-05-21T15:17:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 1 -- 172.21.15.78:0/1899227769 wait complete. 2024-05-21T15:17:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.953+0000 7f40f2d88740 20 client.16110 trim_cache size 0 max 16384 2024-05-21T15:17:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40f2d88740 1 Processor -- start 2024-05-21T15:17:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40f2d88740 1 -- start start 2024-05-21T15:17:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25114d0 0x55dff28d93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40f2d88740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41aec00 con 0x55dff25114d0 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40f2d88740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41b0740 con 0x55dff253a390 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40f2d88740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41af2d0 con 0x55dff25196b0 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e9ba2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25114d0 0x55dff28d93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e9ba2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25114d0 0x55dff28d93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55844/0 (socket says 172.21.15.78:55844) 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e9ba2640 1 -- 172.21.15.78:0/3964615327 learned_addr learned my addr 172.21.15.78:0/3964615327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e93a1640 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e9ba2640 1 -- 172.21.15.78:0/3964615327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 msgr2=0x55dff28fb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28fb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e9ba2640 1 -- 172.21.15.78:0/3964615327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 msgr2=0x55dff28fb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28fb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e9ba2640 1 -- 172.21.15.78:0/3964615327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff28dde30 con 0x55dff25114d0 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40e93a1640 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.967+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25114d0 0x55dff28d93d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f40e00bbe40 tx=0x7f40e0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790fee30a57fb197 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40cf7fe640 1 -- 172.21.15.78:0/3964615327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0049040 con 0x55dff25114d0 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40cf7fe640 1 -- 172.21.15.78:0/3964615327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e00c4d00 con 0x55dff25114d0 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40cf7fe640 1 -- 172.21.15.78:0/3964615327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00c00d0 con 0x55dff25114d0 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40f2d88740 1 -- 172.21.15.78:0/3964615327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25114d0 msgr2=0x55dff28d93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40f2d88740 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25114d0 0x55dff28d93d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f40e00bbe40 tx=0x7f40e0003f90 comp rx=0 tx=0).stop 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40f2d88740 1 -- 172.21.15.78:0/3964615327 shutdown_connections 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40f2d88740 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40f2d88740 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25196b0 0x55dff28fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40f2d88740 1 --2- 172.21.15.78:0/3964615327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25114d0 0x55dff28d93d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.968+0000 7f40f2d88740 1 -- 172.21.15.78:0/3964615327 >> 172.21.15.78:0/3964615327 conn(0x55dff252e4e0 msgr2=0x55dff28d8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 -- 172.21.15.78:0/3964615327 shutdown_connections 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 -- 172.21.15.78:0/3964615327 wait complete. 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 -- start start 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 Processor -- start 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25114d0 0x55dff28eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28ebc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41b0070 con 0x55dff25196b0 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41af9a0 con 0x55dff253a390 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41acbe0 con 0x55dff25114d0 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40f2d88740 10 client.0 _ll_register_callbacks cb 0x7f40d4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.969+0000 7f40e93a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28ebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e93a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28ebc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55848/0 (socket says 172.21.15.78:55848) 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e93a1640 1 -- 172.21.15.78:0/378441064 learned_addr learned my addr 172.21.15.78:0/378441064 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40ea3a3640 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25114d0 0x55dff28eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e93a1640 1 -- 172.21.15.78:0/378441064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25114d0 msgr2=0x55dff28eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e93a1640 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25114d0 0x55dff28eb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e93a1640 1 -- 172.21.15.78:0/378441064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 msgr2=0x55dff28ec170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e93a1640 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28ec170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e93a1640 1 -- 172.21.15.78:0/378441064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff295d1b0 con 0x55dff25196b0 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40e93a1640 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28ebc30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55dff28d6240 tx=0x7f40d400ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbc11b90b6f299a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40cd7fa640 1 -- 172.21.15.78:0/378441064 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d401c420 con 0x55dff25196b0 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40cd7fa640 1 -- 172.21.15.78:0/378441064 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f40d401c5c0 con 0x55dff25196b0 2024-05-21T15:17:49.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40f2d88740 1 -- 172.21.15.78:0/378441064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e00c4d00 con 0x55dff25196b0 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40cd7fa640 1 -- 172.21.15.78:0/378441064 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40d4031050 con 0x55dff25196b0 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40f2d88740 10 client.16116 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40f2d88740 1 -- 172.21.15.78:0/378441064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dff295e380 con 0x55dff25196b0 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.970+0000 7f40cd7fa640 1 -- 172.21.15.78:0/378441064 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e00c4d00 con 0x55dff25196b0 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40cd7fa640 1 -- 172.21.15.78:0/378441064 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40d4003bd0 con 0x55dff25196b0 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40f2d88740 10 client.16116 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40f2d88740 10 client.16116 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40f2d88740 20 client.16116 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40f2d88740 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55dff24ce3a0 0x55dff2960b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40f2d88740 4 client.16116 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40f2d88740 1 -- 172.21.15.78:0/378441064 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dff28466c0 con 0x55dff24ce3a0 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55dff24ce3a0 0x55dff2960b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40cd7fa640 10 client.16116 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:17:49.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40e9ba2640 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55dff24ce3a0 0x55dff2960b40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3b8c2d4df1ecaa5 server_cookie=e2551f59fba9d850 in_seq=0 out_seq=0 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40cd7fa640 1 -- 172.21.15.78:0/378441064 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dff28466c0 con 0x55dff24ce3a0 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40cd7fa640 10 client.16116 handle_command_reply: tid=0 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40f2d88740 1 client.16116 shutdown 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.971+0000 7f40f2d88740 1 -- 172.21.15.78:0/378441064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 msgr2=0x55dff28ebc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28ebc30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55dff28d6240 tx=0x7f40d400ff50 comp rx=0 tx=0).stop 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 -- 172.21.15.78:0/378441064 shutdown_connections 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55dff24ce3a0 0x55dff2960b40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dff253a390 0x55dff28ec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dff25196b0 0x55dff28ebc30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40cd7fa640 0 client.16116 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 --2- 172.21.15.78:0/378441064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dff25114d0 0x55dff28eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 -- 172.21.15.78:0/378441064 >> 172.21.15.78:0/378441064 conn(0x55dff252e4e0 msgr2=0x55dff28d96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 -- 172.21.15.78:0/378441064 shutdown_connections 2024-05-21T15:17:49.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 1 -- 172.21.15.78:0/378441064 wait complete. 2024-05-21T15:17:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40f2d88740 20 client.16116 trim_cache size 0 max 16384 2024-05-21T15:17:49.982 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:49.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 -- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40d00a2d00 msgr2=0x7f40d00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40d00a2d00 0x7f40d00a5210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40dc021000 tx=0x7f40dc020000 comp rx=0 tx=0).stop 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 -- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e4101aa0 msgr2=0x7f40e4199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e4101aa0 0x7f40e4199790 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f40e002a930 tx=0x7f40e0003b90 comp rx=0 tx=0).stop 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 -- 172.21.15.78:0/1181093046 shutdown_connections 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40e41079d0 0x7f40e4199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40d00a2d00 0x7f40d00a5210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40e410c330 0x7f40e419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 --2- 172.21.15.78:0/1181093046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40e4101aa0 0x7f40e4199790 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 -- 172.21.15.78:0/1181093046 >> 172.21.15.78:0/1181093046 conn(0x7f40e40770b0 msgr2=0x7f40e4103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 -- 172.21.15.78:0/1181093046 shutdown_connections 2024-05-21T15:17:49.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:49.972+0000 7f40ebe06640 1 -- 172.21.15.78:0/1181093046 wait complete. 2024-05-21T15:17:49.986 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:17:50.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 -- 172.21.15.78:0/3402497546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 msgr2=0x7fa1f010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 --2- 172.21.15.78:0/3402497546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 0x7fa1f010e750 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009ca0 tx=0x7fa1e802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 -- 172.21.15.78:0/3402497546 shutdown_connections 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 --2- 172.21.15.78:0/3402497546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1f0101aa0 0x7fa1f0101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 --2- 172.21.15.78:0/3402497546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 0x7fa1f010e750 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 --2- 172.21.15.78:0/3402497546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1f01079d0 0x7fa1f0101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 -- 172.21.15.78:0/3402497546 >> 172.21.15.78:0/3402497546 conn(0x7fa1f00770b0 msgr2=0x7fa1f00774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 -- 172.21.15.78:0/3402497546 shutdown_connections 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.096+0000 7fa1f660d640 1 -- 172.21.15.78:0/3402497546 wait complete. 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f660d640 1 Processor -- start 2024-05-21T15:17:50.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f660d640 1 -- start start 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f660d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1f0101aa0 0x7fa1f0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f660d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1f01079d0 0x7fa1f0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f660d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 0x7fa1f0198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f660d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0111d00 con 0x7fa1f010c330 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f660d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0111b80 con 0x7fa1f0101aa0 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f660d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0111e80 con 0x7fa1f01079d0 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f4baa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 0x7fa1f0198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f4baa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 0x7fa1f0198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55852/0 (socket says 172.21.15.78:55852) 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1f4baa640 1 -- 172.21.15.78:0/2628310894 learned_addr learned my addr 172.21.15.78:0/2628310894 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1effff640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1f0101aa0 0x7fa1f0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.097+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1f01079d0 0x7fa1f0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1f4baa640 1 -- 172.21.15.78:0/2628310894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1f01079d0 msgr2=0x7fa1f0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1f4baa640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1f01079d0 0x7fa1f0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1f4baa640 1 -- 172.21.15.78:0/2628310894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1f0101aa0 msgr2=0x7fa1f0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1f4baa640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1f0101aa0 0x7fa1f0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1f4baa640 1 -- 172.21.15.78:0/2628310894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f01a7ae0 con 0x7fa1f010c330 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1effff640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1f0101aa0 0x7fa1f0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1f4baa640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 0x7fa1f0198eb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8031a20 tx=0x7fa1e8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1ed7fa640 1 -- 172.21.15.78:0/2628310894 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8003b70 con 0x7fa1f010c330 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1ed7fa640 1 -- 172.21.15.78:0/2628310894 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8003d10 con 0x7fa1f010c330 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1f660d640 1 -- 172.21.15.78:0/2628310894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f01a7d70 con 0x7fa1f010c330 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1ed7fa640 1 -- 172.21.15.78:0/2628310894 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e802fc40 con 0x7fa1f010c330 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.098+0000 7fa1f660d640 1 -- 172.21.15.78:0/2628310894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f01a8260 con 0x7fa1f010c330 2024-05-21T15:17:50.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.100+0000 7fa1ed7fa640 1 -- 172.21.15.78:0/2628310894 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa1e8004050 con 0x7fa1f010c330 2024-05-21T15:17:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.100+0000 7fa1fd591740 1 Processor -- start 2024-05-21T15:17:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1fd591740 1 -- start start 2024-05-21T15:17:50.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f255ad390 0x557f2594bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f2594c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f2558c6b0 0x557f2594c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1fd591740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f258b1c40 con 0x557f255844d0 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1fd591740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557f25859580 con 0x557f2558c6b0 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1fd591740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557f258adc00 con 0x557f255ad390 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f2594c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ed7fa640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa1dc0a2d00 0x7fa1dc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f2594c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55856/0 (socket says 172.21.15.78:55856) 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1effff640 1 -- 172.21.15.78:0/286628031 learned_addr learned my addr 172.21.15.78:0/286628031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f255ad390 0x557f2594bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f2558c6b0 0x557f2594c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1f4baa640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa1dc0a2d00 0x7fa1dc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ed7fa640 1 -- 172.21.15.78:0/2628310894 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa1e80f0500 con 0x7fa1f010c330 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/286628031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f2558c6b0 msgr2=0x557f2594c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f2558c6b0 0x557f2594c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/286628031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 msgr2=0x557f2594c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f2594c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.101+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/286628031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f255c53f0 con 0x557f255ad390 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.102+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f255ad390 0x557f2594bd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009730 tx=0x7fa1d80096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6cbe144ab3835a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.102+0000 7fa1f4baa640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa1dc0a2d00 0x7fa1dc0a5210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80bc920 tx=0x7fa1e808ed00 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.102+0000 7fa1e67fc640 1 -- 172.21.15.78:0/286628031 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004030 con 0x557f255ad390 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.102+0000 7fa1e67fc640 1 -- 172.21.15.78:0/286628031 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8005500 con 0x557f255ad390 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.102+0000 7fa1e67fc640 1 -- 172.21.15.78:0/286628031 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004ca0 con 0x557f255ad390 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 -- 172.21.15.78:0/286628031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f255ad390 msgr2=0x557f2594bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f255ad390 0x557f2594bd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009730 tx=0x7fa1d80096c0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 -- 172.21.15.78:0/286628031 shutdown_connections 2024-05-21T15:17:50.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f2558c6b0 0x557f2594c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f2594c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 --2- 172.21.15.78:0/286628031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f255ad390 0x557f2594bd20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 -- 172.21.15.78:0/286628031 >> 172.21.15.78:0/286628031 conn(0x557f255a14e0 msgr2=0x557f259484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 -- 172.21.15.78:0/286628031 shutdown_connections 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 -- 172.21.15.78:0/286628031 wait complete. 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 -- start start 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.103+0000 7fa1fd591740 1 Processor -- start 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 0x557f25957d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f259582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f25956230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1fd591740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557f25958fb0 con 0x557f255ad390 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1fd591740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557f2586cc40 con 0x557f255844d0 2024-05-21T15:17:50.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1fd591740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557f25959130 con 0x557f2558c6b0 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1fd591740 10 client.0 _ll_register_callbacks cb 0x557f255c4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f25956230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f25956230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55862/0 (socket says 172.21.15.78:55862) 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/1960839890 learned_addr learned my addr 172.21.15.78:0/1960839890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1effff640 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f259582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 0x557f25957d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 msgr2=0x557f259582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f259582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/1960839890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 msgr2=0x557f25957d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 0x557f25957d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/1960839890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2568f7d0 con 0x557f255ad390 2024-05-21T15:17:50.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1effff640 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f259582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:50.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f25956230 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80058c0 tx=0x7fa1d8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c1171aa2a3f89f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.104+0000 7fa1cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:50.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/1960839890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004e20 con 0x557f255ad390 2024-05-21T15:17:50.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/1960839890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa1d8019040 con 0x557f255ad390 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 1 -- 172.21.15.78:0/1960839890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8005500 con 0x557f255ad390 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/1960839890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1d80051e0 con 0x557f255ad390 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 10 client.16128 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 1 -- 172.21.15.78:0/1960839890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f259657c0 con 0x557f255ad390 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/1960839890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8005500 con 0x557f255ad390 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/1960839890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1d802c020 con 0x557f255ad390 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 10 client.16128 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 10 client.16128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:17:50.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 20 client.16128 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x557f2556bf60 0x557f25968b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 4 client.16128 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.105+0000 7fa1fd591740 1 -- 172.21.15.78:0/1960839890 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f258bf370 con 0x557f2556bf60 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.106+0000 7fa1effff640 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x557f2556bf60 0x557f25968b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.106+0000 7fa1cf7fe640 10 client.16128 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.106+0000 7fa1effff640 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x557f2556bf60 0x557f25968b70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3db13d34878e5141 server_cookie=27a22aa1f0b1e4c8 in_seq=0 out_seq=0 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/1960839890 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f258bf370 con 0x557f2556bf60 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1cf7fe640 10 client.16128 handle_command_reply: tid=0 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 client.16128 shutdown 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 -- 172.21.15.78:0/1960839890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 msgr2=0x557f25956230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f25956230 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80058c0 tx=0x7fa1d8005850 comp rx=0 tx=0).stop 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 -- 172.21.15.78:0/1960839890 shutdown_connections 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x557f2556bf60 0x557f25968b70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f25956230 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f259582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1cf7fe640 0 client.16128 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:17:50.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 --2- 172.21.15.78:0/1960839890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 0x557f25957d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 -- 172.21.15.78:0/1960839890 >> 172.21.15.78:0/1960839890 conn(0x557f255a14e0 msgr2=0x557f2594da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 -- 172.21.15.78:0/1960839890 shutdown_connections 2024-05-21T15:17:50.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 1 -- 172.21.15.78:0/1960839890 wait complete. 2024-05-21T15:17:50.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.107+0000 7fa1fd591740 20 client.16128 trim_cache size 0 max 16384 2024-05-21T15:17:50.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.120+0000 7fa1fd591740 1 Processor -- start 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1fd591740 1 -- start start 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 0x557f2594e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f25955790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f2594fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1fd591740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01b0e40 con 0x557f255ad390 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1fd591740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01b2980 con 0x557f255844d0 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1fd591740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01b1510 con 0x557f2558c6b0 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f2594fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f2594fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55870/0 (socket says 172.21.15.78:55870) 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/2840458545 learned_addr learned my addr 172.21.15.78:0/2840458545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1effff640 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f25955790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 0x557f2594e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/2840458545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 msgr2=0x557f25955790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f25955790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/2840458545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 msgr2=0x557f2594e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 0x557f2594e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/2840458545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2594d310 con 0x557f255ad390 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.121+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f2594fa80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8000c00 tx=0x7fa1d8009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9135afba72ab000 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1e5ffb640 1 -- 172.21.15.78:0/2840458545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d801f9a0 con 0x557f255ad390 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1e5ffb640 1 -- 172.21.15.78:0/2840458545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8020e50 con 0x557f255ad390 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1e5ffb640 1 -- 172.21.15.78:0/2840458545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80042d0 con 0x557f255ad390 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 -- 172.21.15.78:0/2840458545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 msgr2=0x557f2594fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f2594fa80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8000c00 tx=0x7fa1d8009c60 comp rx=0 tx=0).stop 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 -- 172.21.15.78:0/2840458545 shutdown_connections 2024-05-21T15:17:50.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255ad390 0x557f2594fa80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f25955790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 --2- 172.21.15.78:0/2840458545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255844d0 0x557f2594e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 -- 172.21.15.78:0/2840458545 >> 172.21.15.78:0/2840458545 conn(0x557f255a14e0 msgr2=0x557f2594da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 -- 172.21.15.78:0/2840458545 shutdown_connections 2024-05-21T15:17:50.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.122+0000 7fa1fd591740 1 -- 172.21.15.78:0/2840458545 wait complete. 2024-05-21T15:17:50.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 1 -- start start 2024-05-21T15:17:50.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 1 Processor -- start 2024-05-21T15:17:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f25959a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f25959f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255ad390 0x557f259c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01b22b0 con 0x557f255844d0 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01b1be0 con 0x557f255ad390 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01aee20 con 0x557f2558c6b0 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1fd591740 10 client.0 _ll_register_callbacks cb 0x7fa1e0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f25959a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f25959a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55882/0 (socket says 172.21.15.78:55882) 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.123+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/3264834562 learned_addr learned my addr 172.21.15.78:0/3264834562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1effff640 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f25959f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/3264834562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 msgr2=0x557f25959f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f25959f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/3264834562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255ad390 msgr2=0x557f259c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255ad390 0x557f259c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1ef7fe640 1 -- 172.21.15.78:0/3264834562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f259c92d0 con 0x557f255844d0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1ef7fe640 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f25959a50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8006000 tx=0x7fa1d8005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c889fa54968f20 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/3264834562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8042020 con 0x557f255844d0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/3264834562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa1d8004ea0 con 0x557f255844d0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1fd591740 1 -- 172.21.15.78:0/3264834562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8020e50 con 0x557f255844d0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/3264834562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1d804d050 con 0x557f255844d0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1fd591740 10 client.16134 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1fd591740 1 -- 172.21.15.78:0/3264834562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f259d0580 con 0x557f255844d0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.124+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/3264834562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8020e50 con 0x557f255844d0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/3264834562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1d802c040 con 0x557f255844d0 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1fd591740 10 client.16134 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1fd591740 10 client.16134 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1fd591740 20 client.16134 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:50.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1fd591740 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x557f255413a0 0x557f259d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1fd591740 4 client.16134 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1fd591740 1 -- 172.21.15.78:0/3264834562 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f258b96c0 con 0x557f255413a0 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1effff640 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x557f255413a0 0x557f259d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1cf7fe640 10 client.16134 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1effff640 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x557f255413a0 0x557f259d2c10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=775a73ed4979cdd3 server_cookie=c593277ce9c1bc49 in_seq=0 out_seq=0 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1cf7fe640 1 -- 172.21.15.78:0/3264834562 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f258b96c0 con 0x557f255413a0 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1cf7fe640 10 client.16134 handle_command_reply: tid=0 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.125+0000 7fa1fd591740 1 client.16134 shutdown 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 -- 172.21.15.78:0/3264834562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 msgr2=0x557f25959a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f25959a50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8006000 tx=0x7fa1d8005b90 comp rx=0 tx=0).stop 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 -- 172.21.15.78:0/3264834562 shutdown_connections 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x557f255413a0 0x557f259d2c10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557f255ad390 0x557f259c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1cf7fe640 0 client.16134 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:17:50.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f2558c6b0 0x557f25959f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 --2- 172.21.15.78:0/3264834562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557f255844d0 0x557f25959a50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 -- 172.21.15.78:0/3264834562 >> 172.21.15.78:0/3264834562 conn(0x557f255a14e0 msgr2=0x557f2594c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 -- 172.21.15.78:0/3264834562 shutdown_connections 2024-05-21T15:17:50.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 1 -- 172.21.15.78:0/3264834562 wait complete. 2024-05-21T15:17:50.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1fd591740 20 client.16134 trim_cache size 0 max 16384 2024-05-21T15:17:50.135 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:50.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 -- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa1dc0a2d00 msgr2=0x7fa1dc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa1dc0a2d00 0x7fa1dc0a5210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80bc920 tx=0x7fa1e808ed00 comp rx=0 tx=0).stop 2024-05-21T15:17:50.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 -- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 msgr2=0x7fa1f0198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 0x7fa1f0198eb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8031a20 tx=0x7fa1e8031a50 comp rx=0 tx=0).stop 2024-05-21T15:17:50.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 -- 172.21.15.78:0/2628310894 shutdown_connections 2024-05-21T15:17:50.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa1dc0a2d00 0x7fa1dc0a5210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1f010c330 0x7fa1f0198eb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1f01079d0 0x7fa1f0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 --2- 172.21.15.78:0/2628310894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1f0101aa0 0x7fa1f0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 -- 172.21.15.78:0/2628310894 >> 172.21.15.78:0/2628310894 conn(0x7fa1f00770b0 msgr2=0x7fa1f0103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 -- 172.21.15.78:0/2628310894 shutdown_connections 2024-05-21T15:17:50.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.126+0000 7fa1f660d640 1 -- 172.21.15.78:0/2628310894 wait complete. 2024-05-21T15:17:50.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:17:50.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 -- 172.21.15.78:0/3884547785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 msgr2=0x7fc17c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 --2- 172.21.15.78:0/3884547785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 0x7fc17c102660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009900 tx=0x7fc16c02a410 comp rx=0 tx=0).stop 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 -- 172.21.15.78:0/3884547785 shutdown_connections 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 --2- 172.21.15.78:0/3884547785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17c103210 0x7fc17c10f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 --2- 172.21.15.78:0/3884547785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc17c108f60 0x7fc17c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 --2- 172.21.15.78:0/3884547785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 0x7fc17c102660 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 -- 172.21.15.78:0/3884547785 >> 172.21.15.78:0/3884547785 conn(0x7fc17c0fe1c0 msgr2=0x7fc17c100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 -- 172.21.15.78:0/3884547785 shutdown_connections 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 -- 172.21.15.78:0/3884547785 wait complete. 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 Processor -- start 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.250+0000 7fc182b96640 1 -- start start 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc182b96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc17c103210 0x7fc17c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc182b96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 0x7fc17c19fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc182b96640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17c108f60 0x7fc17c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc182b96640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c112e00 con 0x7fc17c108ac0 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc182b96640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c112c80 con 0x7fc17c108f60 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc182b96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c112f80 con 0x7fc17c103210 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 0x7fc17c19fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 0x7fc17c19fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55898/0 (socket says 172.21.15.78:55898) 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 -- 172.21.15.78:0/3531550556 learned_addr learned my addr 172.21.15.78:0/3531550556 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc180932640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc17c103210 0x7fc17c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc181133640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17c108f60 0x7fc17c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 -- 172.21.15.78:0/3531550556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc17c103210 msgr2=0x7fc17c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc17c103210 0x7fc17c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 -- 172.21.15.78:0/3531550556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17c108f60 msgr2=0x7fc17c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17c108f60 0x7fc17c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 -- 172.21.15.78:0/3531550556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c199d90 con 0x7fc17c108ac0 2024-05-21T15:17:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc17bfff640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 0x7fc17c19fa20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc17000ead0 tx=0x7fc17000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc179ffb640 1 -- 172.21.15.78:0/3531550556 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17000cde0 con 0x7fc17c108ac0 2024-05-21T15:17:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc182b96640 1 -- 172.21.15.78:0/3531550556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17c19a020 con 0x7fc17c108ac0 2024-05-21T15:17:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc179ffb640 1 -- 172.21.15.78:0/3531550556 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc170003e50 con 0x7fc17c108ac0 2024-05-21T15:17:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.251+0000 7fc179ffb640 1 -- 172.21.15.78:0/3531550556 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700116b0 con 0x7fc17c108ac0 2024-05-21T15:17:50.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.252+0000 7fc182b96640 1 -- 172.21.15.78:0/3531550556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17c1a6280 con 0x7fc17c108ac0 2024-05-21T15:17:50.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc179ffb640 1 -- 172.21.15.78:0/3531550556 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc170003970 con 0x7fc17c108ac0 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc189a44740 1 Processor -- start 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc189a44740 1 -- start start 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc189a44740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc189a44740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082a6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc189a44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 0x5615082a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc189a44740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56150820bc40 con 0x561507ede4d0 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc189a44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615081b3580 con 0x561507ee66b0 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc189a44740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561508207c00 con 0x561507f07390 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.254+0000 7fc179ffb640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc1640a2d00 0x7fc1640a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55906/0 (socket says 172.21.15.78:55906) 2024-05-21T15:17:50.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 -- 172.21.15.78:0/211382553 learned_addr learned my addr 172.21.15.78:0/211382553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc181133640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc1640a2d00 0x7fc1640a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc181133640 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc179ffb640 1 -- 172.21.15.78:0/3531550556 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc1700cfa00 con 0x7fc17c108ac0 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc17bfff640 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 0x5615082a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 -- 172.21.15.78:0/211382553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 msgr2=0x5615082a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 -- 172.21.15.78:0/211382553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 msgr2=0x5615082a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 0x5615082a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 -- 172.21.15.78:0/211382553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561507f1f3f0 con 0x561507ede4d0 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc181133640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc1640a2d00 0x7fc1640a5210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc174011890 tx=0x7fc174009b50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc180932640 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082a6260 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc17c1037a0 tx=0x7fc16c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c9a636a85bbb74 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc162ffd640 1 -- 172.21.15.78:0/211382553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c040070 con 0x561507ede4d0 2024-05-21T15:17:50.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc162ffd640 1 -- 172.21.15.78:0/211382553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16c031970 con 0x561507ede4d0 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.255+0000 7fc162ffd640 1 -- 172.21.15.78:0/211382553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c035040 con 0x561507ede4d0 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 -- 172.21.15.78:0/211382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 msgr2=0x5615082a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082a6260 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc17c1037a0 tx=0x7fc16c005e80 comp rx=0 tx=0).stop 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 -- 172.21.15.78:0/211382553 shutdown_connections 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 0x5615082a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082a6260 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 --2- 172.21.15.78:0/211382553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 -- 172.21.15.78:0/211382553 >> 172.21.15.78:0/211382553 conn(0x561507efb4e0 msgr2=0x5615082a24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 -- 172.21.15.78:0/211382553 shutdown_connections 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.256+0000 7fc189a44740 1 -- 172.21.15.78:0/211382553 wait complete. 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 1 -- start start 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 1 Processor -- start 2024-05-21T15:17:50.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 0x5615082b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507ee66b0 0x5615082b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507f07390 0x5615082bc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615082b2fb0 con 0x561507f07390 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615081c6c40 con 0x561507ede4d0 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5615082b3130 con 0x561507ee66b0 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc189a44740 10 client.0 _ll_register_callbacks cb 0x561507f1ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc181133640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507f07390 0x5615082bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc181133640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507f07390 0x5615082bc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55918/0 (socket says 172.21.15.78:55918) 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.257+0000 7fc181133640 1 -- 172.21.15.78:0/1293511059 learned_addr learned my addr 172.21.15.78:0/1293511059 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc17bfff640 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507ee66b0 0x5615082b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc180932640 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 0x5615082b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc181133640 1 -- 172.21.15.78:0/1293511059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507ee66b0 msgr2=0x5615082b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc181133640 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507ee66b0 0x5615082b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc181133640 1 -- 172.21.15.78:0/1293511059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 msgr2=0x5615082b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc181133640 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 0x5615082b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc181133640 1 -- 172.21.15.78:0/1293511059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561507fe97d0 con 0x561507f07390 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc181133640 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507f07390 0x5615082bc490 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc17400fe90 tx=0x7fc17401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ae4f1ddee0fdbb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc15bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc15bfff640 1 -- 172.21.15.78:0/1293511059 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174024070 con 0x561507f07390 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc15bfff640 1 -- 172.21.15.78:0/1293511059 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc174003d80 con 0x561507f07390 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc189a44740 1 -- 172.21.15.78:0/1293511059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16c031970 con 0x561507f07390 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc15bfff640 1 -- 172.21.15.78:0/1293511059 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc1740044c0 con 0x561507f07390 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc189a44740 10 client.16146 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc189a44740 1 -- 172.21.15.78:0/1293511059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615082bf6e0 con 0x561507f07390 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.258+0000 7fc15bfff640 1 -- 172.21.15.78:0/1293511059 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16c031970 con 0x561507f07390 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc15bfff640 1 -- 172.21.15.78:0/1293511059 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc174004ae0 con 0x561507f07390 2024-05-21T15:17:50.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc189a44740 10 client.16146 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc189a44740 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc189a44740 20 client.16146 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc189a44740 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x561507ec5f60 0x5615082c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc189a44740 4 client.16146 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc189a44740 1 -- 172.21.15.78:0/1293511059 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561508219370 con 0x561507ec5f60 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc180932640 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x561507ec5f60 0x5615082c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc15bfff640 10 client.16146 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.259+0000 7fc180932640 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x561507ec5f60 0x5615082c2cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2eb74908cb5b6ad2 server_cookie=460af1a1d8d0cba0 in_seq=0 out_seq=0 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc15bfff640 1 -- 172.21.15.78:0/1293511059 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561508219370 con 0x561507ec5f60 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc15bfff640 10 client.16146 handle_command_reply: tid=0 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 client.16146 shutdown 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 -- 172.21.15.78:0/1293511059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507f07390 msgr2=0x5615082bc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507f07390 0x5615082bc490 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc17400fe90 tx=0x7fc17401d000 comp rx=0 tx=0).stop 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 -- 172.21.15.78:0/1293511059 shutdown_connections 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x561507ec5f60 0x5615082c2cd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507f07390 0x5615082bc490 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc15bfff640 0 client.16146 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507ee66b0 0x5615082b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 --2- 172.21.15.78:0/1293511059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 0x5615082b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 -- 172.21.15.78:0/1293511059 >> 172.21.15.78:0/1293511059 conn(0x561507efb4e0 msgr2=0x5615082a7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 -- 172.21.15.78:0/1293511059 shutdown_connections 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.260+0000 7fc189a44740 1 -- 172.21.15.78:0/1293511059 wait complete. 2024-05-21T15:17:50.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.261+0000 7fc189a44740 20 client.16146 trim_cache size 0 max 16384 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc189a44740 1 Processor -- start 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc189a44740 1 -- start start 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc189a44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 0x5615082af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc189a44740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ee66b0 0x5615082b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc189a44740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc189a44740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1aec10 con 0x561507ee66b0 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc189a44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1b0750 con 0x561507ede4d0 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc189a44740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1af2e0 con 0x561507f07390 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ee66b0 0x5615082b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ee66b0 0x5615082b1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55920/0 (socket says 172.21.15.78:55920) 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc17bfff640 1 -- 172.21.15.78:0/2162039522 learned_addr learned my addr 172.21.15.78:0/2162039522 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc180932640 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 0x5615082af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc181133640 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.274+0000 7fc17bfff640 1 -- 172.21.15.78:0/2162039522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 msgr2=0x5615082b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.275+0000 7fc17bfff640 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.275+0000 7fc17bfff640 1 -- 172.21.15.78:0/2162039522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 msgr2=0x5615082af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.275+0000 7fc17bfff640 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 0x5615082af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.275+0000 7fc17bfff640 1 -- 172.21.15.78:0/2162039522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615082b0140 con 0x561507ee66b0 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.275+0000 7fc17bfff640 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ee66b0 0x5615082b1f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc170004690 tx=0x7fc1700bc050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde552e3ddb80598 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.275+0000 7fc1627fc640 1 -- 172.21.15.78:0/2162039522 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700a0270 con 0x561507ee66b0 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.275+0000 7fc1627fc640 1 -- 172.21.15.78:0/2162039522 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1700a4b30 con 0x561507ee66b0 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.275+0000 7fc1627fc640 1 -- 172.21.15.78:0/2162039522 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170098530 con 0x561507ee66b0 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 -- 172.21.15.78:0/2162039522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ee66b0 msgr2=0x5615082b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ee66b0 0x5615082b1f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc170004690 tx=0x7fc1700bc050 comp rx=0 tx=0).stop 2024-05-21T15:17:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 -- 172.21.15.78:0/2162039522 shutdown_connections 2024-05-21T15:17:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ee66b0 0x5615082b1f50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 --2- 172.21.15.78:0/2162039522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ede4d0 0x5615082af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 -- 172.21.15.78:0/2162039522 >> 172.21.15.78:0/2162039522 conn(0x561507efb4e0 msgr2=0x5615082b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 -- 172.21.15.78:0/2162039522 shutdown_connections 2024-05-21T15:17:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 -- 172.21.15.78:0/2162039522 wait complete. 2024-05-21T15:17:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 -- start start 2024-05-21T15:17:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 Processor -- start 2024-05-21T15:17:50.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.276+0000 7fc189a44740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 0x5615082b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1b0080 con 0x561507ede4d0 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1af9b0 con 0x561507ee66b0 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1acbf0 con 0x561507f07390 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 10 client.0 _ll_register_callbacks cb 0x7fc174009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082b8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55926/0 (socket says 172.21.15.78:55926) 2024-05-21T15:17:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 -- 172.21.15.78:0/708601702 learned_addr learned my addr 172.21.15.78:0/708601702 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc17bfff640 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 0x5615082b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc181133640 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 -- 172.21.15.78:0/708601702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 msgr2=0x5615082b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 -- 172.21.15.78:0/708601702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 msgr2=0x5615082b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 0x5615082b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 -- 172.21.15.78:0/708601702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615082b4c30 con 0x561507ede4d0 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc180932640 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082b8ad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc16c000c00 tx=0x7fc16c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d265e3faf56474 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc15bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc15bfff640 1 -- 172.21.15.78:0/708601702 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c040070 con 0x561507ede4d0 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc15bfff640 1 -- 172.21.15.78:0/708601702 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc16c004040 con 0x561507ede4d0 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 1 -- 172.21.15.78:0/708601702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1700a4b30 con 0x561507ede4d0 2024-05-21T15:17:50.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc15bfff640 1 -- 172.21.15.78:0/708601702 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc16c05e050 con 0x561507ede4d0 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 10 client.16152 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.277+0000 7fc189a44740 1 -- 172.21.15.78:0/708601702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56150832b080 con 0x561507ede4d0 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc15bfff640 1 -- 172.21.15.78:0/708601702 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1700a4b30 con 0x561507ede4d0 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc15bfff640 1 -- 172.21.15.78:0/708601702 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc16c0035f0 con 0x561507ede4d0 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc189a44740 10 client.16152 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc189a44740 10 client.16152 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc189a44740 20 client.16152 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc189a44740 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x561507e9b3a0 0x56150832d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc189a44740 4 client.16152 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc189a44740 1 -- 172.21.15.78:0/708601702 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615082136c0 con 0x561507e9b3a0 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc17bfff640 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x561507e9b3a0 0x56150832d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc15bfff640 10 client.16152 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.278+0000 7fc17bfff640 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x561507e9b3a0 0x56150832d370 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe4e60cf9e36431e server_cookie=fa6de5059ffa18c0 in_seq=0 out_seq=0 2024-05-21T15:17:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc15bfff640 1 -- 172.21.15.78:0/708601702 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615082136c0 con 0x561507e9b3a0 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc15bfff640 10 client.16152 handle_command_reply: tid=0 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 client.16152 shutdown 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 -- 172.21.15.78:0/708601702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 msgr2=0x5615082b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082b8ad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc16c000c00 tx=0x7fc16c02a410 comp rx=0 tx=0).stop 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 -- 172.21.15.78:0/708601702 shutdown_connections 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x561507e9b3a0 0x56150832d370 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561507ee66b0 0x5615082b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc15bfff640 0 client.16152 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561507f07390 0x5615082b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 --2- 172.21.15.78:0/708601702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561507ede4d0 0x5615082b8ad0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 -- 172.21.15.78:0/708601702 >> 172.21.15.78:0/708601702 conn(0x561507efb4e0 msgr2=0x5615082b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 -- 172.21.15.78:0/708601702 shutdown_connections 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 1 -- 172.21.15.78:0/708601702 wait complete. 2024-05-21T15:17:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.279+0000 7fc189a44740 20 client.16152 trim_cache size 0 max 16384 2024-05-21T15:17:50.289 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:50.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 -- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc1640a2d00 msgr2=0x7fc1640a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc1640a2d00 0x7fc1640a5210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc174011890 tx=0x7fc174009b50 comp rx=0 tx=0).stop 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 -- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 msgr2=0x7fc17c19fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 0x7fc17c19fa20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc17000ead0 tx=0x7fc17000ef90 comp rx=0 tx=0).stop 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 -- 172.21.15.78:0/3531550556 shutdown_connections 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17c108f60 0x7fc17c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc1640a2d00 0x7fc1640a5210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc17c108ac0 0x7fc17c19fa20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 --2- 172.21.15.78:0/3531550556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc17c103210 0x7fc17c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 -- 172.21.15.78:0/3531550556 >> 172.21.15.78:0/3531550556 conn(0x7fc17c0fe1c0 msgr2=0x7fc17c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 -- 172.21.15.78:0/3531550556 shutdown_connections 2024-05-21T15:17:50.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.280+0000 7fc182b96640 1 -- 172.21.15.78:0/3531550556 wait complete. 2024-05-21T15:17:50.294 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:17:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 -- 172.21.15.78:0/1383080500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc101b20 msgr2=0x7f3ecc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/1383080500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc101b20 0x7f3ecc101f30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009900 tx=0x7f3eb402a410 comp rx=0 tx=0).stop 2024-05-21T15:17:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 -- 172.21.15.78:0/1383080500 shutdown_connections 2024-05-21T15:17:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/1383080500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc10c360 0x7f3ecc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/1383080500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc101b20 0x7f3ecc101f30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/1383080500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ecc107a00 0x7f3ecc1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 -- 172.21.15.78:0/1383080500 >> 172.21.15.78:0/1383080500 conn(0x7f3ecc0770c0 msgr2=0x7f3ecc077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 -- 172.21.15.78:0/1383080500 shutdown_connections 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.406+0000 7f3ed1d09640 1 -- 172.21.15.78:0/1383080500 wait complete. 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ed1d09640 1 Processor -- start 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ed1d09640 1 -- start start 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ed1d09640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc101b20 0x7f3ecc198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ed1d09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc107a00 0x7f3ecc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ed1d09640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ecc10c360 0x7f3ecc19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ed1d09640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc111d10 con 0x7f3ecc107a00 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ed1d09640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc111b90 con 0x7f3ecc10c360 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ed1d09640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc111e90 con 0x7f3ecc101b20 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc107a00 0x7f3ecc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc107a00 0x7f3ecc198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55958/0 (socket says 172.21.15.78:55958) 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 -- 172.21.15.78:0/3862783644 learned_addr learned my addr 172.21.15.78:0/3862783644 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc101b20 0x7f3ecc198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ecc10c360 0x7f3ecc19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 -- 172.21.15.78:0/3862783644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc101b20 msgr2=0x7f3ecc198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc101b20 0x7f3ecc198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 -- 172.21.15.78:0/3862783644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ecc10c360 msgr2=0x7f3ecc19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ecc10c360 0x7f3ecc19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 -- 172.21.15.78:0/3862783644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc19d4f0 con 0x7f3ecc107a00 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.407+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc107a00 0x7f3ecc198c70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3eb402a930 tx=0x7f3eb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.408+0000 7f3ec97fa640 1 -- 172.21.15.78:0/3862783644 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb403f030 con 0x7f3ecc107a00 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.408+0000 7f3ec97fa640 1 -- 172.21.15.78:0/3862783644 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eb4003ce0 con 0x7f3ecc107a00 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.408+0000 7f3ec97fa640 1 -- 172.21.15.78:0/3862783644 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4035070 con 0x7f3ecc107a00 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.408+0000 7f3ed1d09640 1 -- 172.21.15.78:0/3862783644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc19d780 con 0x7f3ecc107a00 2024-05-21T15:17:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.408+0000 7f3ed1d09640 1 -- 172.21.15.78:0/3862783644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc19dcc0 con 0x7f3ecc107a00 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.410+0000 7f3ec97fa640 1 -- 172.21.15.78:0/3862783644 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3eb4042030 con 0x7f3ecc107a00 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.410+0000 7f3ed8bb3740 1 Processor -- start 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.410+0000 7f3ed8bb3740 1 -- start start 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76964d390 0x55e7699edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e7696244d0 0x55e7699ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e769951c40 con 0x55e76964d390 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e7698f9580 con 0x55e7696244d0 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e76994dc00 con 0x55e76962c6b0 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76964d390 0x55e7699edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76964d390 0x55e7699edf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55968/0 (socket says 172.21.15.78:55968) 2024-05-21T15:17:50.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 -- 172.21.15.78:0/631814154 learned_addr learned my addr 172.21.15.78:0/631814154 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ec97fa640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3eb80a7190 0x7f3eb80a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e7696244d0 0x55e7699ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3eb80a7190 0x7f3eb80a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 -- 172.21.15.78:0/631814154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 msgr2=0x55e7699ee9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699ee9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 -- 172.21.15.78:0/631814154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e7696244d0 msgr2=0x55e7699ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e7696244d0 0x55e7699ee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ec97fa640 1 -- 172.21.15.78:0/3862783644 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3eb40ef970 con 0x7f3ecc107a00 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 -- 172.21.15.78:0/631814154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7696653f0 con 0x55e76964d390 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699ee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.411+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76964d390 0x55e7699edf30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4013a50 tx=0x7f3ec4013f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edcf513dd496b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.412+0000 7f3ec1ffb640 1 -- 172.21.15.78:0/631814154 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4013050 con 0x55e76964d390 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.412+0000 7f3ec1ffb640 1 -- 172.21.15.78:0/631814154 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec4022070 con 0x55e76964d390 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.412+0000 7f3ec1ffb640 1 -- 172.21.15.78:0/631814154 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4012650 con 0x55e76964d390 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.412+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3eb80a7190 0x7f3eb80a96a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4021000 tx=0x7f3ec401f000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.412+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/631814154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76964d390 msgr2=0x55e7699edf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.412+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76964d390 0x55e7699edf30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4013a50 tx=0x7f3ec4013f10 comp rx=0 tx=0).stop 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/631814154 shutdown_connections 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e7696244d0 0x55e7699ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699ee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/631814154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76964d390 0x55e7699edf30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/631814154 >> 172.21.15.78:0/631814154 conn(0x55e7696414e0 msgr2=0x55e7699ea6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/631814154 shutdown_connections 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/631814154 wait complete. 2024-05-21T15:17:50.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 -- start start 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.413+0000 7f3ed8bb3740 1 Processor -- start 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699fa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76962c6b0 0x55e7699f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76964d390 0x55e7699f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e7699e7fc0 con 0x55e7696244d0 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e76990cc40 con 0x55e76962c6b0 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e7699e8140 con 0x55e76964d390 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 10 client.0 _ll_register_callbacks cb 0x55e769664be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699fa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55972/0 (socket says 172.21.15.78:55972) 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 -- 172.21.15.78:0/3388048783 learned_addr learned my addr 172.21.15.78:0/3388048783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76962c6b0 0x55e7699f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76964d390 0x55e7699f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 -- 172.21.15.78:0/3388048783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76964d390 msgr2=0x55e7699f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76964d390 0x55e7699f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 -- 172.21.15.78:0/3388048783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76962c6b0 msgr2=0x55e7699f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76962c6b0 0x55e7699f67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 -- 172.21.15.78:0/3388048783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76972f7d0 con 0x55e7696244d0 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76962c6b0 0x55e7699f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699fa490 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00afd0 tx=0x7f3ebc010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbc4220fa43b74b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3eaaffd640 1 -- 172.21.15.78:0/3388048783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00fc50 con 0x55e7696244d0 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3eaaffd640 1 -- 172.21.15.78:0/3388048783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3ebc010e00 con 0x55e7696244d0 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/3388048783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec4022070 con 0x55e7696244d0 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3eaaffd640 1 -- 172.21.15.78:0/3388048783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc00fdf0 con 0x55e7696244d0 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 10 client.16164 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:50.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.414+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/3388048783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e769a05800 con 0x55e7696244d0 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3eaaffd640 1 -- 172.21.15.78:0/3388048783 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec4022070 con 0x55e7696244d0 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3eaaffd640 1 -- 172.21.15.78:0/3388048783 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ebc020620 con 0x55e7696244d0 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3ed8bb3740 10 client.16164 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3ed8bb3740 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3ed8bb3740 20 client.16164 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e76960bf60 0x55e769a08e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3ed8bb3740 4 client.16164 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/3388048783 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e76995f370 con 0x55e76960bf60 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e76960bf60 0x55e769a08e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3eaaffd640 10 client.16164 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.415+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e76960bf60 0x55e769a08e00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a632e39e15574ac server_cookie=9a5545de3f6977ab in_seq=0 out_seq=0 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3eaaffd640 1 -- 172.21.15.78:0/3388048783 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e76995f370 con 0x55e76960bf60 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3eaaffd640 10 client.16164 handle_command_reply: tid=0 2024-05-21T15:17:50.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 client.16164 shutdown 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/3388048783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 msgr2=0x55e7699fa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699fa490 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00afd0 tx=0x7f3ebc010bb0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/3388048783 shutdown_connections 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e76960bf60 0x55e769a08e00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76962c6b0 0x55e7699f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76964d390 0x55e7699f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3eaaffd640 0 client.16164 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/3388048783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699fa490 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.416+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/3388048783 >> 172.21.15.78:0/3388048783 conn(0x55e7696414e0 msgr2=0x55e7699edd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.417+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/3388048783 shutdown_connections 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.417+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/3388048783 wait complete. 2024-05-21T15:17:50.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.417+0000 7f3ed8bb3740 20 client.16164 trim_cache size 0 max 16384 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ed8bb3740 1 Processor -- start 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ed8bb3740 1 -- start start 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699f6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ac980 con 0x55e7696244d0 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ae4c0 con 0x55e76960bf60 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ad050 con 0x55e76962c6b0 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ec3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ec3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55982/0 (socket says 172.21.15.78:55982) 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.430+0000 7f3ec3fff640 1 -- 172.21.15.78:0/2628394037 learned_addr learned my addr 172.21.15.78:0/2628394037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec3fff640 1 -- 172.21.15.78:0/2628394037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 msgr2=0x55e7699f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699f09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec3fff640 1 -- 172.21.15.78:0/2628394037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 msgr2=0x55e7699f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec3fff640 1 -- 172.21.15.78:0/2628394037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7699e7b50 con 0x55e7696244d0 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699f6cd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4006010 tx=0x7f3eb408eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441b75afbd2a1fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec17fa640 1 -- 172.21.15.78:0/2628394037 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4049040 con 0x55e7696244d0 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec17fa640 1 -- 172.21.15.78:0/2628394037 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eb40c4a40 con 0x55e7696244d0 2024-05-21T15:17:50.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.431+0000 7f3ec17fa640 1 -- 172.21.15.78:0/2628394037 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb403bc80 con 0x55e7696244d0 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2628394037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 msgr2=0x55e7699f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699f6cd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4006010 tx=0x7f3eb408eef0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2628394037 shutdown_connections 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e76962c6b0 0x55e7699f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e7696244d0 0x55e7699f6cd0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2628394037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2628394037 >> 172.21.15.78:0/2628394037 conn(0x55e7696414e0 msgr2=0x55e7699f6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2628394037 shutdown_connections 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2628394037 wait complete. 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 -- start start 2024-05-21T15:17:50.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.432+0000 7f3ed8bb3740 1 Processor -- start 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699fe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7696244d0 0x55e7699fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ed8bb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76962c6b0 0x55e769a70180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1addf0 con 0x55e76962c6b0 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ad720 con 0x55e76960bf60 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ed8bb3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1aa960 con 0x55e7696244d0 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ed8bb3740 10 client.0 _ll_register_callbacks cb 0x7f3ebc00a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76962c6b0 0x55e769a70180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76962c6b0 0x55e769a70180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55986/0 (socket says 172.21.15.78:55986) 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 -- 172.21.15.78:0/2443936267 learned_addr learned my addr 172.21.15.78:0/2443936267 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ec3fff640 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7696244d0 0x55e7699fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 -- 172.21.15.78:0/2443936267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7696244d0 msgr2=0x55e7699fe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7696244d0 0x55e7699fe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 -- 172.21.15.78:0/2443936267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 msgr2=0x55e7699fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 -- 172.21.15.78:0/2443936267 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7699fee50 con 0x55e76962c6b0 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ecbfff640 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76962c6b0 0x55e769a70180 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4013a50 tx=0x7f3ec4013300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1cc3ec62967da1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3eaaffd640 1 -- 172.21.15.78:0/2443936267 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4039060 con 0x55e76962c6b0 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3eaaffd640 1 -- 172.21.15.78:0/2443936267 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3ec4011e00 con 0x55e76962c6b0 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.433+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2443936267 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eb40c4a40 con 0x55e76962c6b0 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3eaaffd640 1 -- 172.21.15.78:0/2443936267 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ec40079b0 con 0x55e76962c6b0 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ed8bb3740 10 client.16170 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:50.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2443936267 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e769a716b0 con 0x55e76962c6b0 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3eaaffd640 1 -- 172.21.15.78:0/2443936267 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eb40c4a40 con 0x55e76962c6b0 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3eaaffd640 1 -- 172.21.15.78:0/2443936267 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec401d810 con 0x55e76962c6b0 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ed8bb3740 10 client.16170 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ed8bb3740 10 client.16170 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ed8bb3740 20 client.16170 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e76964d390 0x55e769a73c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ed8bb3740 4 client.16170 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2443936267 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7699596c0 con 0x55e76964d390 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.434+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e76964d390 0x55e769a73c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3eaaffd640 10 client.16170 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ecb7fe640 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e76964d390 0x55e769a73c50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41d1da29e94d969b server_cookie=f563057f1a73618d in_seq=0 out_seq=0 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3eaaffd640 1 -- 172.21.15.78:0/2443936267 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7699596c0 con 0x55e76964d390 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3eaaffd640 10 client.16170 handle_command_reply: tid=0 2024-05-21T15:17:50.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 client.16170 shutdown 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2443936267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76962c6b0 msgr2=0x55e769a70180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76962c6b0 0x55e769a70180 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4013a50 tx=0x7f3ec4013300 comp rx=0 tx=0).stop 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2443936267 shutdown_connections 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55e76964d390 0x55e769a73c50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e76962c6b0 0x55e769a70180 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3eaaffd640 0 client.16170 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7696244d0 0x55e7699fe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 --2- 172.21.15.78:0/2443936267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e76960bf60 0x55e7699fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2443936267 >> 172.21.15.78:0/2443936267 conn(0x55e7696414e0 msgr2=0x55e7699ee450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2443936267 shutdown_connections 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 1 -- 172.21.15.78:0/2443936267 wait complete. 2024-05-21T15:17:50.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.435+0000 7f3ed8bb3740 20 client.16170 trim_cache size 0 max 16384 2024-05-21T15:17:50.445 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 -- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3eb80a7190 msgr2=0x7f3eb80a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3eb80a7190 0x7f3eb80a96a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4021000 tx=0x7f3ec401f000 comp rx=0 tx=0).stop 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 -- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc107a00 msgr2=0x7f3ecc198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc107a00 0x7f3ecc198c70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3eb402a930 tx=0x7f3eb40042f0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 -- 172.21.15.78:0/3862783644 shutdown_connections 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ecc10c360 0x7f3ecc19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3eb80a7190 0x7f3eb80a96a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ecc107a00 0x7f3ecc198c70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 --2- 172.21.15.78:0/3862783644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc101b20 0x7f3ecc198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 -- 172.21.15.78:0/3862783644 >> 172.21.15.78:0/3862783644 conn(0x7f3ecc0770c0 msgr2=0x7f3ecc103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 -- 172.21.15.78:0/3862783644 shutdown_connections 2024-05-21T15:17:50.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.436+0000 7f3ed1d09640 1 -- 172.21.15.78:0/3862783644 wait complete. 2024-05-21T15:17:50.449 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:50 smithi078 ceph-mon[31997]: pgmap v55: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-05-21T15:17:50.449 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:50 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/357353315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:17:50.450 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:17:50.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.560+0000 7f296a8e9640 1 -- 172.21.15.78:0/1207975986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 msgr2=0x7f296410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.560+0000 7f296a8e9640 1 --2- 172.21.15.78:0/1207975986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 0x7f296410f530 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f295c009ca0 tx=0x7f295c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 -- 172.21.15.78:0/1207975986 shutdown_connections 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 --2- 172.21.15.78:0/1207975986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 0x7f296410f530 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 --2- 172.21.15.78:0/1207975986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f296410a650 0x7f296410ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 --2- 172.21.15.78:0/1207975986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2964103bf0 0x7f2964104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 -- 172.21.15.78:0/1207975986 >> 172.21.15.78:0/1207975986 conn(0x7f29640fd210 msgr2=0x7f29640ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 -- 172.21.15.78:0/1207975986 shutdown_connections 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 -- 172.21.15.78:0/1207975986 wait complete. 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 Processor -- start 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 -- start start 2024-05-21T15:17:50.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964103bf0 0x7f2964199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.561+0000 7f296a8e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f296410a650 0x7f2964199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f296a8e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 0x7f296419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f296a8e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2964112df0 con 0x7f296410d040 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f296a8e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2964112c70 con 0x7f296410a650 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f296a8e9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2964112f70 con 0x7f2964103bf0 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 0x7f296419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 0x7f296419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56004/0 (socket says 172.21.15.78:56004) 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 -- 172.21.15.78:0/751163210 learned_addr learned my addr 172.21.15.78:0/751163210 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f295bfff640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f296410a650 0x7f2964199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2963fff640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964103bf0 0x7f2964199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 -- 172.21.15.78:0/751163210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964103bf0 msgr2=0x7f2964199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964103bf0 0x7f2964199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 -- 172.21.15.78:0/751163210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f296410a650 msgr2=0x7f2964199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f296410a650 0x7f2964199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 -- 172.21.15.78:0/751163210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296419e590 con 0x7f296410d040 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2968e86640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 0x7f296419df50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f295c031a20 tx=0x7f295c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2961ffb640 1 -- 172.21.15.78:0/751163210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c003c30 con 0x7f296410d040 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f296a8e9640 1 -- 172.21.15.78:0/751163210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296419e820 con 0x7f296410d040 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2961ffb640 1 -- 172.21.15.78:0/751163210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f295c003dd0 con 0x7f296410d040 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f2961ffb640 1 -- 172.21.15.78:0/751163210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c02fcf0 con 0x7f296410d040 2024-05-21T15:17:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.562+0000 7f296a8e9640 1 -- 172.21.15.78:0/751163210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296419ed70 con 0x7f296410d040 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f2961ffb640 1 -- 172.21.15.78:0/751163210 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f295c004050 con 0x7f296410d040 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f297178c740 1 Processor -- start 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f297178c740 1 -- start start 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f297178c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b303f390 0x55b1b33e2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f297178c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 0x55b1b33e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f297178c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b301e6b0 0x55b1b33e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f297178c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b1b3343c40 con 0x55b1b303f390 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f297178c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b1b32eb580 con 0x55b1b301e6b0 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f297178c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b1b333fc00 con 0x55b1b30164d0 2024-05-21T15:17:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f295bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b303f390 0x55b1b33e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f295bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b303f390 0x55b1b33e2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56020/0 (socket says 172.21.15.78:56020) 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f295bfff640 1 -- 172.21.15.78:0/2083438645 learned_addr learned my addr 172.21.15.78:0/2083438645 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.565+0000 7f2961ffb640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f29500a2d00 0x7f29500a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f2963fff640 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 0x55b1b33e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295bfff640 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b301e6b0 0x55b1b33e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f2968e86640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f29500a2d00 0x7f29500a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295bfff640 1 -- 172.21.15.78:0/2083438645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 msgr2=0x55b1b33e2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295bfff640 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 0x55b1b33e2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295bfff640 1 -- 172.21.15.78:0/2083438645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b301e6b0 msgr2=0x55b1b33e2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295bfff640 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b301e6b0 0x55b1b33e2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295bfff640 1 -- 172.21.15.78:0/2083438645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b30573f0 con 0x55b1b303f390 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f2961ffb640 1 -- 172.21.15.78:0/751163210 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f295c0f06a0 con 0x7f296410d040 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295bfff640 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b303f390 0x55b1b33e2130 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f294c00bfa0 tx=0x7f294c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116917abcf7e2f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295a7fc640 1 -- 172.21.15.78:0/2083438645 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c005020 con 0x55b1b303f390 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f2968e86640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f29500a2d00 0x7f29500a5210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f295c0b96d0 tx=0x7f295c03d040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295a7fc640 1 -- 172.21.15.78:0/2083438645 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c0056a0 con 0x55b1b303f390 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.566+0000 7f295a7fc640 1 -- 172.21.15.78:0/2083438645 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c00f6c0 con 0x55b1b303f390 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 -- 172.21.15.78:0/2083438645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b303f390 msgr2=0x55b1b33e2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b303f390 0x55b1b33e2130 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f294c00bfa0 tx=0x7f294c00b7e0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 -- 172.21.15.78:0/2083438645 shutdown_connections 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b301e6b0 0x55b1b33e2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 0x55b1b33e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 --2- 172.21.15.78:0/2083438645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b303f390 0x55b1b33e2130 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 -- 172.21.15.78:0/2083438645 >> 172.21.15.78:0/2083438645 conn(0x55b1b30334e0 msgr2=0x55b1b33de8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 -- 172.21.15.78:0/2083438645 shutdown_connections 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 -- 172.21.15.78:0/2083438645 wait complete. 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.567+0000 7f297178c740 1 -- start start 2024-05-21T15:17:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f297178c740 1 Processor -- start 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f297178c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 0x55b1b33ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f297178c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f297178c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b303f390 0x55b1b33f2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f297178c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b1b33ef480 con 0x55b1b301e6b0 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f297178c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b1b32fec40 con 0x55b1b303f390 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f297178c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b1b33ef600 con 0x55b1b30164d0 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f297178c740 10 client.0 _ll_register_callbacks cb 0x55b1b3056be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f2963fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f2963fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33eccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56024/0 (socket says 172.21.15.78:56024) 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f2963fff640 1 -- 172.21.15.78:0/252539380 learned_addr learned my addr 172.21.15.78:0/252539380 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f295bfff640 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 0x55b1b33ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f295bfff640 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b303f390 0x55b1b33f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f2963fff640 1 -- 172.21.15.78:0/252539380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 msgr2=0x55b1b33ec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f2963fff640 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 0x55b1b33ec790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f2963fff640 1 -- 172.21.15.78:0/252539380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b303f390 msgr2=0x55b1b33f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f2963fff640 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b303f390 0x55b1b33f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.568+0000 7f2963fff640 1 -- 172.21.15.78:0/252539380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b31217d0 con 0x55b1b301e6b0 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f2963fff640 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33eccd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f295400be80 tx=0x7f2954012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a1c3ef4a09aab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f29437fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f29437fe640 1 -- 172.21.15.78:0/252539380 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954005220 con 0x55b1b301e6b0 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f29437fe640 1 -- 172.21.15.78:0/252539380 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f29540053c0 con 0x55b1b301e6b0 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 1 -- 172.21.15.78:0/252539380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294c0056a0 con 0x55b1b301e6b0 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f29437fe640 1 -- 172.21.15.78:0/252539380 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f29540055c0 con 0x55b1b301e6b0 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 10 client.16182 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 1 -- 172.21.15.78:0/252539380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b1b33fbc30 con 0x55b1b301e6b0 2024-05-21T15:17:50.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f29437fe640 1 -- 172.21.15.78:0/252539380 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c0056a0 con 0x55b1b301e6b0 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f29437fe640 1 -- 172.21.15.78:0/252539380 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2954019030 con 0x55b1b301e6b0 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 10 client.16182 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 20 client.16182 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b1b2ffdf60 0x55b1b33ff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 4 client.16182 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f297178c740 1 -- 172.21.15.78:0/252539380 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1b3351370 con 0x55b1b2ffdf60 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.569+0000 7f295bfff640 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b1b2ffdf60 0x55b1b33ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.570+0000 7f295bfff640 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b1b2ffdf60 0x55b1b33ff170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881e4f9cdb826fe7 server_cookie=93b93b227cd0228b in_seq=0 out_seq=0 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.570+0000 7f29437fe640 10 client.16182 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f29437fe640 1 -- 172.21.15.78:0/252539380 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b1b3351370 con 0x55b1b2ffdf60 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f29437fe640 10 client.16182 handle_command_reply: tid=0 2024-05-21T15:17:50.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 client.16182 shutdown 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 -- 172.21.15.78:0/252539380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 msgr2=0x55b1b33eccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33eccd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f295400be80 tx=0x7f2954012820 comp rx=0 tx=0).stop 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 -- 172.21.15.78:0/252539380 shutdown_connections 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b1b2ffdf60 0x55b1b33ff170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b303f390 0x55b1b33f2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33eccd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f29437fe640 0 client.16182 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 --2- 172.21.15.78:0/252539380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b30164d0 0x55b1b33ec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.571+0000 7f297178c740 1 -- 172.21.15.78:0/252539380 >> 172.21.15.78:0/252539380 conn(0x55b1b30334e0 msgr2=0x55b1b33e1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.572+0000 7f297178c740 1 -- 172.21.15.78:0/252539380 shutdown_connections 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.572+0000 7f297178c740 1 -- 172.21.15.78:0/252539380 wait complete. 2024-05-21T15:17:50.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.572+0000 7f297178c740 20 client.16182 trim_cache size 0 max 16384 2024-05-21T15:17:50.583 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether build environment is sane... yes 2024-05-21T15:17:50.583 INFO:tasks.workunit.client.0.smithi078.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-21T15:17:50.583 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gawk... gawk 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f297178c740 1 Processor -- start 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f297178c740 1 -- start start 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f297178c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f297178c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33de910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f297178c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f297178c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29641aca00 con 0x55b1b301e6b0 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f297178c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f29641ae540 con 0x55b1b30164d0 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f297178c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f29641ad0d0 con 0x55b1b2ffdf60 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f2963fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33de910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:54536/0 (socket says 172.21.15.78:54536) 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 -- 172.21.15.78:0/2224322135 learned_addr learned my addr 172.21.15.78:0/2224322135 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 -- 172.21.15.78:0/2224322135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 msgr2=0x55b1b33de910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 --2- 172.21.15.78:0/2224322135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33de910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 -- 172.21.15.78:0/2224322135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 msgr2=0x55b1b33e62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 --2- 172.21.15.78:0/2224322135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33e62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.586+0000 7f295bfff640 1 -- 172.21.15.78:0/2224322135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b33e6830 con 0x55b1b2ffdf60 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.587+0000 7f295bfff640 1 --2- 172.21.15.78:0/2224322135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f2870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f294c00bfd0 tx=0x7f294c00bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d39e8f554f3b991 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.587+0000 7f2959ffb640 1 -- 172.21.15.78:0/2224322135 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c0047b0 con 0x55b1b2ffdf60 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.587+0000 7f2959ffb640 1 -- 172.21.15.78:0/2224322135 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c004950 con 0x55b1b2ffdf60 2024-05-21T15:17:50.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.587+0000 7f2959ffb640 1 -- 172.21.15.78:0/2224322135 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c003e50 con 0x55b1b2ffdf60 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 -- 172.21.15.78:0/2224322135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b2ffdf60 msgr2=0x55b1b33f2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 --2- 172.21.15.78:0/2224322135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f2870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f294c00bfd0 tx=0x7f294c00bee0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 -- 172.21.15.78:0/2224322135 shutdown_connections 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 --2- 172.21.15.78:0/2224322135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 --2- 172.21.15.78:0/2224322135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b301e6b0 0x55b1b33e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 --2- 172.21.15.78:0/2224322135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f2870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 -- 172.21.15.78:0/2224322135 >> 172.21.15.78:0/2224322135 conn(0x55b1b30334e0 msgr2=0x55b1b33e5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 -- 172.21.15.78:0/2224322135 shutdown_connections 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 -- 172.21.15.78:0/2224322135 wait complete. 2024-05-21T15:17:50.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 -- start start 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.588+0000 7f297178c740 1 Processor -- start 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f297178c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f297178c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f297178c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b301e6b0 0x55b1b33f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f295bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f295bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56030/0 (socket says 172.21.15.78:56030) 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f295bfff640 1 -- 172.21.15.78:0/4108466174 learned_addr learned my addr 172.21.15.78:0/4108466174 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29641ade70 con 0x55b1b2ffdf60 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f29641ad7a0 con 0x55b1b30164d0 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f29641aa9e0 con 0x55b1b301e6b0 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f297178c740 10 client.0 _ll_register_callbacks cb 0x7f295400bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:50.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f2963fff640 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f295bfff640 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b301e6b0 0x55b1b33f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f2963fff640 1 -- 172.21.15.78:0/4108466174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b301e6b0 msgr2=0x55b1b33f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f2963fff640 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b301e6b0 0x55b1b33f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f2963fff640 1 -- 172.21.15.78:0/4108466174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b2ffdf60 msgr2=0x55b1b33f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f2963fff640 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f2963fff640 1 -- 172.21.15.78:0/4108466174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b33f4f70 con 0x55b1b30164d0 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.589+0000 7f295bfff640 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f2963fff640 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33f44f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2954010f00 tx=0x7f2954012e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79639b598f4b7921 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f29437fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f29437fe640 1 -- 172.21.15.78:0/4108466174 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954030020 con 0x55b1b30164d0 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f29437fe640 1 -- 172.21.15.78:0/4108466174 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2954022c60 con 0x55b1b30164d0 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294c004950 con 0x55b1b30164d0 2024-05-21T15:17:50.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f29437fe640 1 -- 172.21.15.78:0/4108466174 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f295403b050 con 0x55b1b30164d0 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f297178c740 10 client.25670 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b1b3467190 con 0x55b1b30164d0 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.590+0000 7f29437fe640 1 -- 172.21.15.78:0/4108466174 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c004950 con 0x55b1b30164d0 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f29437fe640 1 -- 172.21.15.78:0/4108466174 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2954019070 con 0x55b1b30164d0 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f297178c740 10 client.25670 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f297178c740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f297178c740 20 client.25670 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f297178c740 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b1b303f390 0x55b1b3469940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f297178c740 4 client.25670 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1b334b6c0 con 0x55b1b303f390 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f295bfff640 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b1b303f390 0x55b1b3469940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f29437fe640 10 client.25670 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f295bfff640 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b1b303f390 0x55b1b3469940 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=decedbe04252e56e server_cookie=e38991da2b6a728c in_seq=0 out_seq=0 2024-05-21T15:17:50.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f29437fe640 1 -- 172.21.15.78:0/4108466174 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1b334b6c0 con 0x55b1b303f390 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f29437fe640 10 client.25670 handle_command_reply: tid=0 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.591+0000 7f297178c740 1 client.25670 shutdown 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 msgr2=0x55b1b33f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33f44f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2954010f00 tx=0x7f2954012e60 comp rx=0 tx=0).stop 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 shutdown_connections 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b1b303f390 0x55b1b3469940 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b1b30164d0 0x55b1b33f44f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1b301e6b0 0x55b1b33f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f29437fe640 0 client.25670 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 --2- 172.21.15.78:0/4108466174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b1b2ffdf60 0x55b1b33f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 >> 172.21.15.78:0/4108466174 conn(0x55b1b30334e0 msgr2=0x55b1b33eae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 shutdown_connections 2024-05-21T15:17:50.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 1 -- 172.21.15.78:0/4108466174 wait complete. 2024-05-21T15:17:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f297178c740 20 client.25670 trim_cache size 0 max 16384 2024-05-21T15:17:50.601 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f296a8e9640 1 -- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f29500a2d00 msgr2=0x7f29500a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f296a8e9640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f29500a2d00 0x7f29500a5210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f295c0b96d0 tx=0x7f295c03d040 comp rx=0 tx=0).stop 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f296a8e9640 1 -- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 msgr2=0x7f296419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.592+0000 7f296a8e9640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 0x7f296419df50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f295c031a20 tx=0x7f295c031a50 comp rx=0 tx=0).stop 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.593+0000 7f296a8e9640 1 -- 172.21.15.78:0/751163210 shutdown_connections 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.593+0000 7f296a8e9640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f296410a650 0x7f2964199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.593+0000 7f296a8e9640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f29500a2d00 0x7f29500a5210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.593+0000 7f296a8e9640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f296410d040 0x7f296419df50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.593+0000 7f296a8e9640 1 --2- 172.21.15.78:0/751163210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964103bf0 0x7f2964199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.593+0000 7f296a8e9640 1 -- 172.21.15.78:0/751163210 >> 172.21.15.78:0/751163210 conn(0x7f29640fd210 msgr2=0x7f296410d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.593+0000 7f296a8e9640 1 -- 172.21.15.78:0/751163210 shutdown_connections 2024-05-21T15:17:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:50.593+0000 7f296a8e9640 1 -- 172.21.15.78:0/751163210 wait complete. 2024-05-21T15:17:50.612 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether make sets $(MAKE)... yes 2024-05-21T15:17:50.640 INFO:tasks.workunit.client.0.smithi078.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-21T15:17:50.641 INFO:tasks.workunit.client.0.smithi078.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-21T15:17:50.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:50 smithi084 ceph-mon[36334]: pgmap v55: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-05-21T15:17:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:50 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/357353315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:17:50.654 INFO:tasks.workunit.client.0.smithi078.stdout:checking for style of include used by make... GNU 2024-05-21T15:17:50.659 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc... gcc 2024-05-21T15:17:50.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:50 smithi172 ceph-mon[36402]: pgmap v55: 97 pgs: 97 active+clean; 2.6 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 16 op/s 2024-05-21T15:17:50.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:50 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/357353315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:17:50.748 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether the C compiler works... yes 2024-05-21T15:17:50.748 INFO:tasks.workunit.client.0.smithi078.stdout:checking for C compiler default output file name... a.out 2024-05-21T15:17:50.791 INFO:tasks.workunit.client.0.smithi078.stdout:checking for suffix of executables... 2024-05-21T15:17:50.825 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether we are cross compiling... no 2024-05-21T15:17:50.864 INFO:tasks.workunit.client.0.smithi078.stdout:checking for suffix of object files... o 2024-05-21T15:17:50.888 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether we are using the GNU C compiler... yes 2024-05-21T15:17:50.912 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether gcc accepts -g... yes 2024-05-21T15:17:50.955 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-21T15:17:51.065 INFO:tasks.workunit.client.0.smithi078.stdout:checking dependency style of gcc... gcc3 2024-05-21T15:17:51.115 INFO:tasks.workunit.client.0.smithi078.stdout:checking how to run the C preprocessor... gcc -E 2024-05-21T15:17:51.163 INFO:tasks.workunit.client.0.smithi078.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-21T15:17:51.164 INFO:tasks.workunit.client.0.smithi078.stdout:checking for egrep... /usr/bin/grep -E 2024-05-21T15:17:51.290 INFO:tasks.workunit.client.0.smithi078.stdout:checking for ANSI C header files... yes 2024-05-21T15:17:51.325 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/types.h... yes 2024-05-21T15:17:51.364 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/stat.h... yes 2024-05-21T15:17:51.406 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdlib.h... yes 2024-05-21T15:17:51.452 INFO:tasks.workunit.client.0.smithi078.stdout:checking for string.h... yes 2024-05-21T15:17:51.492 INFO:tasks.workunit.client.0.smithi078.stdout:checking for memory.h... yes 2024-05-21T15:17:51.536 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strings.h... yes 2024-05-21T15:17:51.577 INFO:tasks.workunit.client.0.smithi078.stdout:checking for inttypes.h... yes 2024-05-21T15:17:51.607 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:17:51.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:17:51.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:17:51.622 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdint.h... yes 2024-05-21T15:17:51.670 INFO:tasks.workunit.client.0.smithi078.stdout:checking for unistd.h... yes 2024-05-21T15:17:51.718 INFO:tasks.workunit.client.0.smithi078.stdout:checking minix/config.h usability... no 2024-05-21T15:17:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.726+0000 7f748ee40640 1 -- 172.21.15.78:0/3382809567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 msgr2=0x7f7488101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.726+0000 7f748ee40640 1 --2- 172.21.15.78:0/3382809567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 0x7f7488101cb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f747c009900 tx=0x7f747c02a410 comp rx=0 tx=0).stop 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 -- 172.21.15.78:0/3382809567 shutdown_connections 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 --2- 172.21.15.78:0/3382809567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7488102860 0x7f748810e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 --2- 172.21.15.78:0/3382809567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7488108080 0x7f74881021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 --2- 172.21.15.78:0/3382809567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 0x7f7488101cb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 -- 172.21.15.78:0/3382809567 >> 172.21.15.78:0/3382809567 conn(0x7f7488076830 msgr2=0x7f7488076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 -- 172.21.15.78:0/3382809567 shutdown_connections 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 -- 172.21.15.78:0/3382809567 wait complete. 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 Processor -- start 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.727+0000 7f748ee40640 1 -- start start 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748ee40640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7488102860 0x7f748819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748ee40640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 0x7f748819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748ee40640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7488108080 0x7f7488199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748ee40640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7488111f30 con 0x7f7488107be0 2024-05-21T15:17:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748ee40640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7488111db0 con 0x7f7488102860 2024-05-21T15:17:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748ee40640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f74881120b0 con 0x7f7488108080 2024-05-21T15:17:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 0x7f748819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 0x7f748819faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56042/0 (socket says 172.21.15.78:56042) 2024-05-21T15:17:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 -- 172.21.15.78:0/2617785432 learned_addr learned my addr 172.21.15.78:0/2617785432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748cbdc640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7488102860 0x7f748819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748d3dd640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7488108080 0x7f7488199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 -- 172.21.15.78:0/2617785432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7488108080 msgr2=0x7f7488199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7488108080 0x7f7488199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 -- 172.21.15.78:0/2617785432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7488102860 msgr2=0x7f748819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7488102860 0x7f748819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 -- 172.21.15.78:0/2617785432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7488199f60 con 0x7f7488107be0 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7487fff640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 0x7f748819faf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f747400e9f0 tx=0x7f747400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7485ffb640 1 -- 172.21.15.78:0/2617785432 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747400ccc0 con 0x7f7488107be0 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7485ffb640 1 -- 172.21.15.78:0/2617785432 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74740035f0 con 0x7f7488107be0 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f748ee40640 1 -- 172.21.15.78:0/2617785432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748819a250 con 0x7f7488107be0 2024-05-21T15:17:51.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.728+0000 7f7485ffb640 1 -- 172.21.15.78:0/2617785432 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747400ce60 con 0x7f7488107be0 2024-05-21T15:17:51.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.729+0000 7f748ee40640 1 -- 172.21.15.78:0/2617785432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881a6400 con 0x7f7488107be0 2024-05-21T15:17:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.731+0000 7f7485ffb640 1 -- 172.21.15.78:0/2617785432 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f74740037a0 con 0x7f7488107be0 2024-05-21T15:17:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.731+0000 7f7495db7740 1 Processor -- start 2024-05-21T15:17:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.731+0000 7f7495db7740 1 -- start start 2024-05-21T15:17:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7495db7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8570410 0x5646e890f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7495db7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e890fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7495db7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e89100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7495db7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5646e8874990 con 0x5646e8570410 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7495db7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5646e881c5b0 con 0x5646e8547550 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7495db7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5646e8870950 con 0x5646e854f730 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7485ffb640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74780a2d00 0x7f74780a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e890fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748d3dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8570410 0x5646e890f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e890fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:60938/0 (socket says 172.21.15.78:60938) 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 -- 172.21.15.78:0/900865305 learned_addr learned my addr 172.21.15.78:0/900865305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7487fff640 1 --2- 172.21.15.78:0/900865305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e89100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748d3dd640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74780a2d00 0x7f74780a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f7485ffb640 1 -- 172.21.15.78:0/2617785432 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f74740cf9b0 con 0x7f7488107be0 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 -- 172.21.15.78:0/900865305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 msgr2=0x5646e89100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 --2- 172.21.15.78:0/900865305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e89100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 -- 172.21.15.78:0/900865305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8570410 msgr2=0x5646e890f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 --2- 172.21.15.78:0/900865305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8570410 0x5646e890f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 -- 172.21.15.78:0/900865305 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646e86507e0 con 0x5646e8547550 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748cbdc640 1 --2- 172.21.15.78:0/900865305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e890fb60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f747c009230 tx=0x7f747c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4042183a65ac05a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.732+0000 7f748d3dd640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74780a2d00 0x7f74780a5210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5646e890c790 tx=0x7f7480009600 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7472ffd640 1 -- 172.21.15.78:0/900865305 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c040070 con 0x5646e8547550 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7472ffd640 1 -- 172.21.15.78:0/900865305 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c0317b0 con 0x5646e8547550 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7472ffd640 1 -- 172.21.15.78:0/900865305 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c035040 con 0x5646e8547550 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 -- 172.21.15.78:0/900865305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 msgr2=0x5646e890fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 --2- 172.21.15.78:0/900865305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e890fb60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f747c009230 tx=0x7f747c005f10 comp rx=0 tx=0).stop 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 -- 172.21.15.78:0/900865305 shutdown_connections 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 --2- 172.21.15.78:0/900865305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e890fb60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 --2- 172.21.15.78:0/900865305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e89100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 --2- 172.21.15.78:0/900865305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8570410 0x5646e890f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 -- 172.21.15.78:0/900865305 >> 172.21.15.78:0/900865305 conn(0x5646e8564560 msgr2=0x5646e890bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 -- 172.21.15.78:0/900865305 shutdown_connections 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.733+0000 7f7495db7740 1 -- 172.21.15.78:0/900865305 wait complete. 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 1 -- start start 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 1 Processor -- start 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8547550 0x5646e891b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e891bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8570410 0x5646e8919b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5646e891c910 con 0x5646e8547550 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5646e882fb40 con 0x5646e8570410 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5646e891ca90 con 0x5646e854f730 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f7495db7740 10 client.0 _ll_register_callbacks cb 0x5646e88240f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.734+0000 7f748cbdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8547550 0x5646e891b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7487fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e891bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748cbdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8547550 0x5646e891b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56060/0 (socket says 172.21.15.78:56060) 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748cbdc640 1 -- 172.21.15.78:0/2707601656 learned_addr learned my addr 172.21.15.78:0/2707601656 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748d3dd640 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8570410 0x5646e8919b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748cbdc640 1 -- 172.21.15.78:0/2707601656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 msgr2=0x5646e891bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748cbdc640 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e891bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748cbdc640 1 -- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8570410 msgr2=0x5646e8919b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748cbdc640 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8570410 0x5646e8919b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748cbdc640 1 -- 172.21.15.78:0/2707601656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646e86b0f70 con 0x5646e8547550 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7487fff640 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e891bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748d3dd640 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8570410 0x5646e8919b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f748cbdc640 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8547550 0x5646e891b6d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f747c000c00 tx=0x7f747c032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d554b63f83c2982b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7467fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:51.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7467fff640 1 -- 172.21.15.78:0/2707601656 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c04b020 con 0x5646e8547550 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7495db7740 1 -- 172.21.15.78:0/2707601656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747c0317b0 con 0x5646e8547550 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7467fff640 1 -- 172.21.15.78:0/2707601656 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f747c005500 con 0x5646e8547550 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7467fff640 1 -- 172.21.15.78:0/2707601656 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f747c056050 con 0x5646e8547550 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7495db7740 10 client.16200 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7495db7740 1 -- 172.21.15.78:0/2707601656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646e8548170 con 0x5646e8547550 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.735+0000 7f7467fff640 1 -- 172.21.15.78:0/2707601656 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c0317b0 con 0x5646e8547550 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7467fff640 1 -- 172.21.15.78:0/2707601656 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f747c004d50 con 0x5646e8547550 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7495db7740 10 client.16200 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7495db7740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7495db7740 20 client.16200 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7495db7740 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5646e852f1f0 0x5646e892bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7495db7740 4 client.16200 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:51.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7495db7740 1 -- 172.21.15.78:0/2707601656 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646e88820c0 con 0x5646e852f1f0 2024-05-21T15:17:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7487fff640 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5646e852f1f0 0x5646e892bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7487fff640 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5646e852f1f0 0x5646e892bfa0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f6e0467c2109b15 server_cookie=9050ffc92a502aad in_seq=0 out_seq=0 2024-05-21T15:17:51.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.736+0000 7f7467fff640 10 client.16200 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7467fff640 1 -- 172.21.15.78:0/2707601656 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646e88820c0 con 0x5646e852f1f0 2024-05-21T15:17:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7467fff640 10 client.16200 handle_command_reply: tid=0 2024-05-21T15:17:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 client.16200 shutdown 2024-05-21T15:17:51.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 -- 172.21.15.78:0/2707601656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8547550 msgr2=0x5646e891b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8547550 0x5646e891b6d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f747c000c00 tx=0x7f747c032e70 comp rx=0 tx=0).stop 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 -- 172.21.15.78:0/2707601656 shutdown_connections 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5646e852f1f0 0x5646e892bfa0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8570410 0x5646e8919b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e891bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7467fff640 0 client.16200 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 --2- 172.21.15.78:0/2707601656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e8547550 0x5646e891b6d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 -- 172.21.15.78:0/2707601656 >> 172.21.15.78:0/2707601656 conn(0x5646e8564560 msgr2=0x5646e8911340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 -- 172.21.15.78:0/2707601656 shutdown_connections 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 1 -- 172.21.15.78:0/2707601656 wait complete. 2024-05-21T15:17:51.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.738+0000 7f7495db7740 20 client.16200 trim_cache size 0 max 16384 2024-05-21T15:17:51.749 INFO:tasks.workunit.client.0.smithi078.stdout:checking minix/config.h presence... no 2024-05-21T15:17:51.749 INFO:tasks.workunit.client.0.smithi078.stdout:checking for minix/config.h... no 2024-05-21T15:17:51.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7495db7740 1 Processor -- start 2024-05-21T15:17:51.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7495db7740 1 -- start start 2024-05-21T15:17:51.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7495db7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e852f1f0 0x5646e89119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7495db7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e89132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7495db7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e8913810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7495db7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74881ac950 con 0x5646e852f1f0 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7495db7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f74881ae490 con 0x5646e8547550 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7495db7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f74881ad020 con 0x5646e854f730 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f748cbdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e852f1f0 0x5646e89119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f748cbdc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e852f1f0 0x5646e89119e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56076/0 (socket says 172.21.15.78:56076) 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f748cbdc640 1 -- 172.21.15.78:0/2317588003 learned_addr learned my addr 172.21.15.78:0/2317588003 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f748d3dd640 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e8913810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.757+0000 7f7487fff640 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e89132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f748cbdc640 1 -- 172.21.15.78:0/2317588003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 msgr2=0x5646e8913810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f748cbdc640 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e8913810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f748cbdc640 1 -- 172.21.15.78:0/2317588003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 msgr2=0x5646e89132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f748cbdc640 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e89132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f748cbdc640 1 -- 172.21.15.78:0/2317588003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646e890c980 con 0x5646e852f1f0 2024-05-21T15:17:51.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f748cbdc640 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e852f1f0 0x5646e89119e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f747c02a930 tx=0x7f747c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb5aef5e3104d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f74727fc640 1 -- 172.21.15.78:0/2317588003 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c04b070 con 0x5646e852f1f0 2024-05-21T15:17:51.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f74727fc640 1 -- 172.21.15.78:0/2317588003 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c005850 con 0x5646e852f1f0 2024-05-21T15:17:51.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.758+0000 7f74727fc640 1 -- 172.21.15.78:0/2317588003 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c035070 con 0x5646e852f1f0 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 -- 172.21.15.78:0/2317588003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e852f1f0 msgr2=0x5646e89119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e852f1f0 0x5646e89119e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f747c02a930 tx=0x7f747c005e20 comp rx=0 tx=0).stop 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 -- 172.21.15.78:0/2317588003 shutdown_connections 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e89132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e854f730 0x5646e8913810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 --2- 172.21.15.78:0/2317588003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e852f1f0 0x5646e89119e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 -- 172.21.15.78:0/2317588003 >> 172.21.15.78:0/2317588003 conn(0x5646e8564560 msgr2=0x5646e89112d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 -- 172.21.15.78:0/2317588003 shutdown_connections 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 -- 172.21.15.78:0/2317588003 wait complete. 2024-05-21T15:17:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 -- start start 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.759+0000 7f7495db7740 1 Processor -- start 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7495db7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e852f1f0 0x5646e892dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7495db7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e892e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7495db7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e854f730 0x5646e892e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7495db7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74881addc0 con 0x5646e854f730 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7495db7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f74881ad6f0 con 0x5646e8547550 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7495db7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f74881aa950 con 0x5646e852f1f0 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7495db7740 10 client.0 _ll_register_callbacks cb 0x7f7480009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748d3dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e854f730 0x5646e892e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748d3dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e854f730 0x5646e892e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56090/0 (socket says 172.21.15.78:56090) 2024-05-21T15:17:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748d3dd640 1 -- 172.21.15.78:0/929691594 learned_addr learned my addr 172.21.15.78:0/929691594 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748cbdc640 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e852f1f0 0x5646e892dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7487fff640 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e892e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748d3dd640 1 -- 172.21.15.78:0/929691594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e852f1f0 msgr2=0x5646e892dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748d3dd640 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e852f1f0 0x5646e892dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748d3dd640 1 -- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 msgr2=0x5646e892e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748d3dd640 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e892e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f748d3dd640 1 -- 172.21.15.78:0/929691594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646e8992100 con 0x5646e854f730 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.760+0000 7f7487fff640 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e892e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f748d3dd640 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e854f730 0x5646e892e700 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f74800239a0 tx=0x7f7480004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f6d43285ef895b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7467fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7467fff640 1 -- 172.21.15.78:0/929691594 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480004d20 con 0x5646e854f730 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7467fff640 1 -- 172.21.15.78:0/929691594 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7480022940 con 0x5646e854f730 2024-05-21T15:17:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 1 -- 172.21.15.78:0/929691594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747c005850 con 0x5646e854f730 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7467fff640 1 -- 172.21.15.78:0/929691594 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7480022b40 con 0x5646e854f730 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 10 client.16206 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 1 -- 172.21.15.78:0/929691594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646e8993380 con 0x5646e854f730 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7467fff640 1 -- 172.21.15.78:0/929691594 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c005850 con 0x5646e854f730 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7467fff640 1 -- 172.21.15.78:0/929691594 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f748002d070 con 0x5646e854f730 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 10 client.16206 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 20 client.16206 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5646e8570410 0x5646e8995bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 4 client.16206 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.761+0000 7f7495db7740 1 -- 172.21.15.78:0/929691594 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5646e887c410 con 0x5646e8570410 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.762+0000 7f748cbdc640 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5646e8570410 0x5646e8995bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.762+0000 7f7467fff640 10 client.16206 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.762+0000 7f748cbdc640 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5646e8570410 0x5646e8995bf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d54f5605434a5f server_cookie=b8c190dc75177ff1 in_seq=0 out_seq=0 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7467fff640 1 -- 172.21.15.78:0/929691594 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5646e887c410 con 0x5646e8570410 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7467fff640 10 client.16206 handle_command_reply: tid=0 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 client.16206 shutdown 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 -- 172.21.15.78:0/929691594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e854f730 msgr2=0x5646e892e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e854f730 0x5646e892e700 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f74800239a0 tx=0x7f7480004ab0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 -- 172.21.15.78:0/929691594 shutdown_connections 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5646e8570410 0x5646e8995bf0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646e8547550 0x5646e892e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646e854f730 0x5646e892e700 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7467fff640 0 client.16206 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 --2- 172.21.15.78:0/929691594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646e852f1f0 0x5646e892dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 -- 172.21.15.78:0/929691594 >> 172.21.15.78:0/929691594 conn(0x5646e8564560 msgr2=0x5646e890f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 -- 172.21.15.78:0/929691594 shutdown_connections 2024-05-21T15:17:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.763+0000 7f7495db7740 1 -- 172.21.15.78:0/929691594 wait complete. 2024-05-21T15:17:51.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f7495db7740 20 client.16206 trim_cache size 0 max 16384 2024-05-21T15:17:51.773 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:17:51.773 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:17:51.773 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "794d2515-fc95-4ead-ac0e-e8f69867706f", 2024-05-21T15:17:51.773 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:17:51.773 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:17:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 -- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74780a2d00 msgr2=0x7f74780a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74780a2d00 0x7f74780a5210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5646e890c790 tx=0x7f7480009600 comp rx=0 tx=0).stop 2024-05-21T15:17:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 -- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 msgr2=0x7f748819faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 0x7f748819faf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f747400e9f0 tx=0x7f747400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 -- 172.21.15.78:0/2617785432 shutdown_connections 2024-05-21T15:17:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74780a2d00 0x7f74780a5210 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7488108080 0x7f7488199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7488107be0 0x7f748819faf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 --2- 172.21.15.78:0/2617785432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7488102860 0x7f748819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 -- 172.21.15.78:0/2617785432 >> 172.21.15.78:0/2617785432 conn(0x7f7488076830 msgr2=0x7f7488100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 -- 172.21.15.78:0/2617785432 shutdown_connections 2024-05-21T15:17:51.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.764+0000 7f748ee40640 1 -- 172.21.15.78:0/2617785432 wait complete. 2024-05-21T15:17:51.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:17:51.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:17:51.784 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-05-21T15:17:51.784 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc... (cached) gcc 2024-05-21T15:17:51.823 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-21T15:17:51.823 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-21T15:17:51.823 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-21T15:17:51.823 INFO:tasks.workunit.client.0.smithi078.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-21T15:17:51.868 INFO:tasks.workunit.client.0.smithi078.stdout:checking for main in -lm... yes 2024-05-21T15:17:51.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 -- 172.21.15.78:0/2105862726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 msgr2=0x7f64d4101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 --2- 172.21.15.78:0/2105862726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 0x7f64d4101eb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009900 tx=0x7f64bc02a410 comp rx=0 tx=0).stop 2024-05-21T15:17:51.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 -- 172.21.15.78:0/2105862726 shutdown_connections 2024-05-21T15:17:51.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 --2- 172.21.15.78:0/2105862726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d410c330 0x7f64d410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 --2- 172.21.15.78:0/2105862726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 0x7f64d4101eb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 --2- 172.21.15.78:0/2105862726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64d41079d0 0x7f64d4101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 -- 172.21.15.78:0/2105862726 >> 172.21.15.78:0/2105862726 conn(0x7f64d40770b0 msgr2=0x7f64d40774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 -- 172.21.15.78:0/2105862726 shutdown_connections 2024-05-21T15:17:51.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 -- 172.21.15.78:0/2105862726 wait complete. 2024-05-21T15:17:51.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.900+0000 7f64da9e0640 1 Processor -- start 2024-05-21T15:17:51.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64da9e0640 1 -- start start 2024-05-21T15:17:51.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64da9e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 0x7f64d41a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64da9e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41079d0 0x7f64d41a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64da9e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64d410c330 0x7f64d419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64da9e0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4111ce0 con 0x7f64d4101aa0 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64da9e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4111b60 con 0x7f64d410c330 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64da9e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4111e60 con 0x7f64d41079d0 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 0x7f64d41a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 0x7f64d41a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56104/0 (socket says 172.21.15.78:56104) 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 -- 172.21.15.78:0/619557032 learned_addr learned my addr 172.21.15.78:0/619557032 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64cbfff640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41079d0 0x7f64d41a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64d410c330 0x7f64d419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 -- 172.21.15.78:0/619557032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41079d0 msgr2=0x7f64d41a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41079d0 0x7f64d41a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 -- 172.21.15.78:0/619557032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64d410c330 msgr2=0x7f64d419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64d410c330 0x7f64d419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 -- 172.21.15.78:0/619557032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d419be10 con 0x7f64d4101aa0 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.901+0000 7f64d3fff640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 0x7f64d41a1570 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c970 tx=0x7f64c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.902+0000 7f64d1ffb640 1 -- 172.21.15.78:0/619557032 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4014070 con 0x7f64d4101aa0 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.902+0000 7f64da9e0640 1 -- 172.21.15.78:0/619557032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d419c100 con 0x7f64d4101aa0 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.902+0000 7f64d1ffb640 1 -- 172.21.15.78:0/619557032 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c4004440 con 0x7f64d4101aa0 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.902+0000 7f64d1ffb640 1 -- 172.21.15.78:0/619557032 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c400f450 con 0x7f64d4101aa0 2024-05-21T15:17:51.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.902+0000 7f64da9e0640 1 -- 172.21.15.78:0/619557032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d41a83c0 con 0x7f64d4101aa0 2024-05-21T15:17:51.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.905+0000 7f64d1ffb640 1 -- 172.21.15.78:0/619557032 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f64c4017070 con 0x7f64d4101aa0 2024-05-21T15:17:51.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.905+0000 7f64e1522740 1 Processor -- start 2024-05-21T15:17:51.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.905+0000 7f64e1522740 1 -- start start 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64e1522740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d3346390 0x55a0d36e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64e1522740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36e5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64e1522740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d33256b0 0x55a0d36e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64e1522740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0d364ad00 con 0x55a0d331d4d0 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64e1522740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0d35f24c0 con 0x55a0d3346390 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64e1522740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0d3646cc0 con 0x55a0d33256b0 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d1ffb640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f64c00a2d00 0x7f64c00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36e5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56118/0 (socket says 172.21.15.78:56118) 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d8f7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d3346390 0x55a0d36e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 -- 172.21.15.78:0/270885081 learned_addr learned my addr 172.21.15.78:0/270885081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64cbfff640 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d33256b0 0x55a0d36e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f64c00a2d00 0x7f64c00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d1ffb640 1 -- 172.21.15.78:0/619557032 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f64c40cf2f0 con 0x7f64d4101aa0 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 -- 172.21.15.78:0/270885081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d33256b0 msgr2=0x55a0d36e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d33256b0 0x55a0d36e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 -- 172.21.15.78:0/270885081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d3346390 msgr2=0x55a0d36e4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d3346390 0x55a0d36e4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 -- 172.21.15.78:0/270885081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d34287d0 con 0x55a0d331d4d0 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64cbfff640 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d33256b0 0x55a0d36e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d3346390 0x55a0d36e4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d3fff640 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36e5270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f64c400afd0 tx=0x7f64c4003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582ea7695d2b10f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.906+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f64c00a2d00 0x7f64c00a5210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55a0d36e1ea0 tx=0x7f64cc009600 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.907+0000 7f64ca7fc640 1 -- 172.21.15.78:0/270885081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c409b110 con 0x55a0d331d4d0 2024-05-21T15:17:51.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.907+0000 7f64ca7fc640 1 -- 172.21.15.78:0/270885081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c409beb0 con 0x55a0d331d4d0 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.907+0000 7f64ca7fc640 1 -- 172.21.15.78:0/270885081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c409b7b0 con 0x55a0d331d4d0 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.907+0000 7f64e1522740 1 -- 172.21.15.78:0/270885081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 msgr2=0x55a0d36e5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.907+0000 7f64e1522740 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36e5270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f64c400afd0 tx=0x7f64c4003920 comp rx=0 tx=0).stop 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 -- 172.21.15.78:0/270885081 shutdown_connections 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d33256b0 0x55a0d36e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36e5270 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 --2- 172.21.15.78:0/270885081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d3346390 0x55a0d36e4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 -- 172.21.15.78:0/270885081 >> 172.21.15.78:0/270885081 conn(0x55a0d333a4e0 msgr2=0x55a0d36e14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 -- 172.21.15.78:0/270885081 shutdown_connections 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 -- 172.21.15.78:0/270885081 wait complete. 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 -- start start 2024-05-21T15:17:51.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 Processor -- start 2024-05-21T15:17:51.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d36f1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3346390 0x55a0d36ef2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0d36f1fc0 con 0x55a0d331d4d0 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0d3605b90 con 0x55a0d33256b0 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.908+0000 7f64e1522740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0d36f2140 con 0x55a0d3346390 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64e1522740 10 client.0 _ll_register_callbacks cb 0x55a0d3641da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36f0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56122/0 (socket says 172.21.15.78:56122) 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 -- 172.21.15.78:0/2375943212 learned_addr learned my addr 172.21.15.78:0/2375943212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64cbfff640 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d36f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3346390 0x55a0d36ef2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 -- 172.21.15.78:0/2375943212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3346390 msgr2=0x55a0d36ef2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3346390 0x55a0d36ef2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 -- 172.21.15.78:0/2375943212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 msgr2=0x55a0d36f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d36f1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 -- 172.21.15.78:0/2375943212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d335e3f0 con 0x55a0d331d4d0 2024-05-21T15:17:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64d3fff640 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36f0e30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f64c4099a60 tx=0x7f64c409b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8def71cc0e74a61a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64b37fe640 1 -- 172.21.15.78:0/2375943212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c40a8060 con 0x55a0d331d4d0 2024-05-21T15:17:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64b37fe640 1 -- 172.21.15.78:0/2375943212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f64c40ccc60 con 0x55a0d331d4d0 2024-05-21T15:17:51.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64e1522740 1 -- 172.21.15.78:0/2375943212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c409beb0 con 0x55a0d331d4d0 2024-05-21T15:17:51.917 INFO:tasks.workunit.client.0.smithi078.stdout:checking for main in -lpthread... yes 2024-05-21T15:17:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64b37fe640 1 -- 172.21.15.78:0/2375943212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64c409c140 con 0x55a0d331d4d0 2024-05-21T15:17:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64e1522740 10 client.16218 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:51.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.909+0000 7f64e1522740 1 -- 172.21.15.78:0/2375943212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0d331e0f0 con 0x55a0d331d4d0 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64b37fe640 1 -- 172.21.15.78:0/2375943212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c409beb0 con 0x55a0d331d4d0 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64b37fe640 1 -- 172.21.15.78:0/2375943212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f64c409ba10 con 0x55a0d331d4d0 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64e1522740 10 client.16218 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64e1522740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64e1522740 20 client.16218 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64e1522740 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a0d3304f60 0x55a0d37018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64e1522740 4 client.16218 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64e1522740 1 -- 172.21.15.78:0/2375943212 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d3658430 con 0x55a0d3304f60 2024-05-21T15:17:51.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64cbfff640 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a0d3304f60 0x55a0d37018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.920 INFO:tasks.workunit.client.0.smithi078.stdout:checking for ANSI C header files... (cached) yes 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64cbfff640 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a0d3304f60 0x55a0d37018f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f82905d4b9de517 server_cookie=35806f2eb005eac4 in_seq=0 out_seq=0 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.910+0000 7f64b37fe640 10 client.16218 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.911+0000 7f64b37fe640 1 -- 172.21.15.78:0/2375943212 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0d3658430 con 0x55a0d3304f60 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.911+0000 7f64b37fe640 10 client.16218 handle_command_reply: tid=0 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.911+0000 7f64e1522740 1 client.16218 shutdown 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 -- 172.21.15.78:0/2375943212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 msgr2=0x55a0d36f0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36f0e30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f64c4099a60 tx=0x7f64c409b110 comp rx=0 tx=0).stop 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 -- 172.21.15.78:0/2375943212 shutdown_connections 2024-05-21T15:17:51.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a0d3304f60 0x55a0d37018f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d36f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64b37fe640 0 client.16218 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3346390 0x55a0d36ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 --2- 172.21.15.78:0/2375943212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d331d4d0 0x55a0d36f0e30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 -- 172.21.15.78:0/2375943212 >> 172.21.15.78:0/2375943212 conn(0x55a0d333a4e0 msgr2=0x55a0d36e6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 -- 172.21.15.78:0/2375943212 shutdown_connections 2024-05-21T15:17:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 1 -- 172.21.15.78:0/2375943212 wait complete. 2024-05-21T15:17:51.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.912+0000 7f64e1522740 20 client.16218 trim_cache size 0 max 16384 2024-05-21T15:17:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64e1522740 1 Processor -- start 2024-05-21T15:17:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64e1522740 1 -- start start 2024-05-21T15:17:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64e1522740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3304f60 0x55a0d36ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64e1522740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d331d4d0 0x55a0d36e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64e1522740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d33256b0 0x55a0d36f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64e1522740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41b0c20 con 0x55a0d33256b0 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64e1522740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41b2760 con 0x55a0d331d4d0 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64e1522740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41b12f0 con 0x55a0d3304f60 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64d8f7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d33256b0 0x55a0d36f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64d8f7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d33256b0 0x55a0d36f0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56134/0 (socket says 172.21.15.78:56134) 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64d8f7d640 1 -- 172.21.15.78:0/2047724809 learned_addr learned my addr 172.21.15.78:0/2047724809 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64cbfff640 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d331d4d0 0x55a0d36e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64d3fff640 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3304f60 0x55a0d36ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64d8f7d640 1 -- 172.21.15.78:0/2047724809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3304f60 msgr2=0x55a0d36ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.930+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3304f60 0x55a0d36ef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.931+0000 7f64d8f7d640 1 -- 172.21.15.78:0/2047724809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d331d4d0 msgr2=0x55a0d36e56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.931+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d331d4d0 0x55a0d36e56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.931+0000 7f64d8f7d640 1 -- 172.21.15.78:0/2047724809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d36e4ac0 con 0x55a0d33256b0 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.931+0000 7f64cbfff640 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d331d4d0 0x55a0d36e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.931+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d33256b0 0x55a0d36f0960 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009d90 tx=0x7f64cc009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e8ea5a2d427714 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.931+0000 7f64c9ffb640 1 -- 172.21.15.78:0/2047724809 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc004b20 con 0x55a0d33256b0 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.931+0000 7f64c9ffb640 1 -- 172.21.15.78:0/2047724809 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64cc025070 con 0x55a0d33256b0 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.931+0000 7f64c9ffb640 1 -- 172.21.15.78:0/2047724809 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc005790 con 0x55a0d33256b0 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 -- 172.21.15.78:0/2047724809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d33256b0 msgr2=0x55a0d36f0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d33256b0 0x55a0d36f0960 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009d90 tx=0x7f64cc009a00 comp rx=0 tx=0).stop 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 -- 172.21.15.78:0/2047724809 shutdown_connections 2024-05-21T15:17:51.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d331d4d0 0x55a0d36e56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d33256b0 0x55a0d36f0960 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 --2- 172.21.15.78:0/2047724809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d3304f60 0x55a0d36ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 -- 172.21.15.78:0/2047724809 >> 172.21.15.78:0/2047724809 conn(0x55a0d333a4e0 msgr2=0x55a0d36edb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 -- 172.21.15.78:0/2047724809 shutdown_connections 2024-05-21T15:17:51.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 -- 172.21.15.78:0/2047724809 wait complete. 2024-05-21T15:17:51.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.932+0000 7f64e1522740 1 -- start start 2024-05-21T15:17:51.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64e1522740 1 Processor -- start 2024-05-21T15:17:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64e1522740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d3304f60 0x55a0d3762500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64e1522740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d331d4d0 0x55a0d3762a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64e1522740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d37044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64e1522740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41b2090 con 0x55a0d3304f60 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64e1522740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41b19c0 con 0x55a0d33256b0 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64e1522740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41aec00 con 0x55a0d331d4d0 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64e1522740 10 client.0 _ll_register_callbacks cb 0x7f64cc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d3304f60 0x55a0d3762500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d3304f60 0x55a0d3762500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56148/0 (socket says 172.21.15.78:56148) 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 -- 172.21.15.78:0/3874902133 learned_addr learned my addr 172.21.15.78:0/3874902133 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64cbfff640 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d331d4d0 0x55a0d3762a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d37044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 -- 172.21.15.78:0/3874902133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d331d4d0 msgr2=0x55a0d3762a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d331d4d0 0x55a0d3762a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 -- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 msgr2=0x55a0d37044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d37044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 -- 172.21.15.78:0/3874902133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d3704aa0 con 0x55a0d3304f60 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d8f7d640 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d37044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64d3fff640 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d3304f60 0x55a0d3762500 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f64c409cd20 tx=0x7f64c4098050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f0eb3b3e063c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.933+0000 7f64b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64b37fe640 1 -- 172.21.15.78:0/3874902133 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c40a8040 con 0x55a0d3304f60 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64b37fe640 1 -- 172.21.15.78:0/3874902133 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f64c4097500 con 0x55a0d3304f60 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 1 -- 172.21.15.78:0/3874902133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64cc025070 con 0x55a0d3304f60 2024-05-21T15:17:51.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64b37fe640 1 -- 172.21.15.78:0/3874902133 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64c40ae050 con 0x55a0d3304f60 2024-05-21T15:17:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 10 client.16224 fetch_fsmap learned FSMap version 26 2024-05-21T15:17:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 1 -- 172.21.15.78:0/3874902133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0d3769b90 con 0x55a0d3304f60 2024-05-21T15:17:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64b37fe640 1 -- 172.21.15.78:0/3874902133 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64cc025070 con 0x55a0d3304f60 2024-05-21T15:17:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64b37fe640 1 -- 172.21.15.78:0/3874902133 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f64c4097700 con 0x55a0d3304f60 2024-05-21T15:17:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 10 client.16224 fetch_fsmap finished waiting for FSMap version 26 2024-05-21T15:17:51.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 20 client.16224 populate_metadata read hostname 'smithi078' 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a0d3346390 0x55a0d376c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 4 client.16224 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64e1522740 1 -- 172.21.15.78:0/3874902133 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0d36e9530 con 0x55a0d3346390 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.934+0000 7f64cbfff640 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a0d3346390 0x55a0d376c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.935+0000 7f64b37fe640 10 client.16224 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.935+0000 7f64cbfff640 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a0d3346390 0x55a0d376c0c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7443ffb467baee9 server_cookie=c84f98ec2246562d in_seq=0 out_seq=0 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64b37fe640 1 -- 172.21.15.78:0/3874902133 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a0d36e9530 con 0x55a0d3346390 2024-05-21T15:17:51.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64b37fe640 10 client.16224 handle_command_reply: tid=0 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 client.16224 shutdown 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 -- 172.21.15.78:0/3874902133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d3304f60 msgr2=0x55a0d3762500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d3304f60 0x55a0d3762500 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f64c409cd20 tx=0x7f64c4098050 comp rx=0 tx=0).stop 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 -- 172.21.15.78:0/3874902133 shutdown_connections 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a0d3346390 0x55a0d376c0c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0d33256b0 0x55a0d37044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d331d4d0 0x55a0d3762a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64b37fe640 0 client.16224 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:17:51.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 --2- 172.21.15.78:0/3874902133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0d3304f60 0x55a0d3762500 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 -- 172.21.15.78:0/3874902133 >> 172.21.15.78:0/3874902133 conn(0x55a0d333a4e0 msgr2=0x55a0d36f0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 -- 172.21.15.78:0/3874902133 shutdown_connections 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 1 -- 172.21.15.78:0/3874902133 wait complete. 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.936+0000 7f64e1522740 20 client.16224 trim_cache size 0 max 16384 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stdout: "794d2515-fc95-4ead-ac0e-e8f69867706f": { 2024-05-21T15:17:51.948 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:17:51.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "794d2515-fc95-4ead-ac0e-e8f69867706f", 2024-05-21T15:17:51.949 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:17:51.949 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:17:51.949 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:17:51.949 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:17:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 -- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f64c00a2d00 msgr2=0x7f64c00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f64c00a2d00 0x7f64c00a5210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55a0d36e1ea0 tx=0x7f64cc009600 comp rx=0 tx=0).stop 2024-05-21T15:17:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 -- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 msgr2=0x7f64d41a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:17:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 0x7f64d41a1570 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c970 tx=0x7f64c400ce30 comp rx=0 tx=0).stop 2024-05-21T15:17:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 -- 172.21.15.78:0/619557032 shutdown_connections 2024-05-21T15:17:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64d410c330 0x7f64d419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f64c00a2d00 0x7f64c00a5210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41079d0 0x7f64d41a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 --2- 172.21.15.78:0/619557032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d4101aa0 0x7f64d41a1570 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:17:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 -- 172.21.15.78:0/619557032 >> 172.21.15.78:0/619557032 conn(0x7f64d40770b0 msgr2=0x7f64d4100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:17:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 -- 172.21.15.78:0/619557032 shutdown_connections 2024-05-21T15:17:51.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:17:51.937+0000 7f64da9e0640 1 -- 172.21.15.78:0/619557032 wait complete. 2024-05-21T15:17:51.951 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-21T15:17:51.952 INFO:tasks.cephfs.filesystem:scrub status for tag:794d2515-fc95-4ead-ac0e-e8f69867706f - {'path': '/', 'tag': '794d2515-fc95-4ead-ac0e-e8f69867706f', 'options': 'recursive,force'} 2024-05-21T15:17:52.010 INFO:tasks.workunit.client.0.smithi078.stdout:checking pthread.h usability... yes 2024-05-21T15:17:52.038 INFO:tasks.workunit.client.0.smithi078.stdout:checking pthread.h presence... yes 2024-05-21T15:17:52.038 INFO:tasks.workunit.client.0.smithi078.stdout:checking for pthread.h... yes 2024-05-21T15:17:52.082 INFO:tasks.workunit.client.0.smithi078.stdout:checking fcntl.h usability... yes 2024-05-21T15:17:52.100 INFO:tasks.workunit.client.0.smithi078.stdout:checking fcntl.h presence... yes 2024-05-21T15:17:52.100 INFO:tasks.workunit.client.0.smithi078.stdout:checking for fcntl.h... yes 2024-05-21T15:17:52.149 INFO:tasks.workunit.client.0.smithi078.stdout:checking limits.h usability... yes 2024-05-21T15:17:52.166 INFO:tasks.workunit.client.0.smithi078.stdout:checking limits.h presence... yes 2024-05-21T15:17:52.166 INFO:tasks.workunit.client.0.smithi078.stdout:checking for limits.h... yes 2024-05-21T15:17:52.181 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdint.h... (cached) yes 2024-05-21T15:17:52.219 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/time.h usability... yes 2024-05-21T15:17:52.236 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/time.h presence... yes 2024-05-21T15:17:52.236 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/time.h... yes 2024-05-21T15:17:52.251 INFO:tasks.workunit.client.0.smithi078.stdout:checking for unistd.h... (cached) yes 2024-05-21T15:17:52.288 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/vfs.h usability... yes 2024-05-21T15:17:52.305 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/vfs.h presence... yes 2024-05-21T15:17:52.305 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/vfs.h... yes 2024-05-21T15:17:52.359 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/limits.h usability... no 2024-05-21T15:17:52.389 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/limits.h presence... no 2024-05-21T15:17:52.389 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/limits.h... no 2024-05-21T15:17:52.416 INFO:tasks.workunit.client.0.smithi078.stdout:checking for an ANSI C-conforming const... yes 2024-05-21T15:17:52.444 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-21T15:17:52.506 INFO:tasks.workunit.client.0.smithi078.stdout:checking for system... yes 2024-05-21T15:17:52.559 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gettimeofday... yes 2024-05-21T15:17:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:52 smithi078 ceph-mon[31997]: pgmap v56: 97 pgs: 97 active+clean; 5.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 449 KiB/s wr, 156 op/s 2024-05-21T15:17:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:52 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:17:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:52 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:52 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:17:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:52 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:52.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:52 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:17:52.616 INFO:tasks.workunit.client.0.smithi078.stdout:checking for mkdir... yes 2024-05-21T15:17:52.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:52 smithi084 ceph-mon[36334]: pgmap v56: 97 pgs: 97 active+clean; 5.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 449 KiB/s wr, 156 op/s 2024-05-21T15:17:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:52 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:17:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:52 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:52 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:17:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:52 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:52 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:17:52.672 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strerror... yes 2024-05-21T15:17:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:52 smithi172 ceph-mon[36402]: pgmap v56: 97 pgs: 97 active+clean; 5.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 449 KiB/s wr, 156 op/s 2024-05-21T15:17:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:52 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:17:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:52 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:52 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:17:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:52 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:17:52.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:52 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:17:52.728 INFO:tasks.workunit.client.0.smithi078.stdout:checking for open64... yes 2024-05-21T15:17:52.784 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stat64... yes 2024-05-21T15:17:52.840 INFO:tasks.workunit.client.0.smithi078.stdout:checking for fseeko64... yes 2024-05-21T15:17:52.900 INFO:tasks.workunit.client.0.smithi078.stdout:checking for lrand48_r... yes 2024-05-21T15:17:52.960 INFO:tasks.workunit.client.0.smithi078.stdout:checking for srand48_r... yes 2024-05-21T15:17:52.984 INFO:tasks.workunit.client.0.smithi078.stdout:configure: creating ./config.status 2024-05-21T15:17:53.046 INFO:tasks.workunit.client.0.smithi078.stdout: /bin/sh ./config.status 2024-05-21T15:17:53.095 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating Makefile 2024-05-21T15:17:53.107 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating config.h 2024-05-21T15:17:53.130 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: executing depfiles commands 2024-05-21T15:17:53.141 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: executing default commands 2024-05-21T15:17:53.170 INFO:tasks.workunit.client.0.smithi078.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-05-21T15:17:53.311 INFO:tasks.workunit.client.0.smithi078.stdout:rm -f stamp-h1 2024-05-21T15:17:53.314 INFO:tasks.workunit.client.0.smithi078.stdout:touch config.h.in 2024-05-21T15:17:53.316 INFO:tasks.workunit.client.0.smithi078.stdout:cd . && /bin/sh ./config.status config.h 2024-05-21T15:17:53.359 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating config.h 2024-05-21T15:17:53.392 INFO:tasks.workunit.client.0.smithi078.stdout:make all-am 2024-05-21T15:17:53.400 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-21T15:17:53.400 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-05-21T15:17:53.575 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-05-21T15:17:53.579 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-05-21T15:17:53.677 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-05-21T15:17:53.680 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-05-21T15:17:53.761 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-05-21T15:17:53.765 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-05-21T15:17:53.851 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-05-21T15:17:53.855 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-05-21T15:17:53.976 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-05-21T15:17:53.980 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-05-21T15:17:54.028 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-05-21T15:17:54.032 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-05-21T15:17:54.074 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-05-21T15:17:54.078 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-05-21T15:17:54.124 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-05-21T15:17:54.127 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-05-21T15:17:54.324 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-05-21T15:17:54.328 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-05-21T15:17:54.450 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-05-21T15:17:54.454 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-05-21T15:17:54.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:54 smithi078 ceph-mon[31997]: pgmap v57: 97 pgs: 97 active+clean; 5.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 448 KiB/s wr, 150 op/s 2024-05-21T15:17:54.598 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-05-21T15:17:54.602 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-05-21T15:17:54.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:54 smithi084 ceph-mon[36334]: pgmap v57: 97 pgs: 97 active+clean; 5.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 448 KiB/s wr, 150 op/s 2024-05-21T15:17:54.667 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-05-21T15:17:54.675 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-05-21T15:17:54.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:54 smithi172 ceph-mon[36402]: pgmap v57: 97 pgs: 97 active+clean; 5.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 448 KiB/s wr, 150 op/s 2024-05-21T15:17:54.740 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-05-21T15:17:54.744 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-05-21T15:17:54.847 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-05-21T15:17:54.851 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-05-21T15:17:54.878 INFO:tasks.workunit.client.0.smithi078.stderr:In file included from parser.h:22, 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr: from parser.c:25: 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c: In function ‘init_filesys’: 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr: | ^ 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~ 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:17:54.879 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: | ^ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:17:54.880 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~ 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr: | ^ 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:17:54.881 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr: | ^ 2024-05-21T15:17:54.882 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~ 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~ 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c: In function ‘init_tg_stats’: 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:17:54.883 INFO:tasks.workunit.client.0.smithi078.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: | ^ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.884 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: | ^ 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~ 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~ 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-21T15:17:54.885 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr: | ^ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~ 2024-05-21T15:17:54.886 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: | ^ 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~ 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: 36 | container_of(ptr, type, member) 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~ 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-21T15:17:54.887 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~ 2024-05-21T15:17:54.888 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-21T15:17:54.888 INFO:tasks.workunit.client.0.smithi078.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-21T15:17:54.888 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.923 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c: In function ‘get_optstr’: 2024-05-21T15:17:54.923 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-05-21T15:17:54.923 INFO:tasks.workunit.client.0.smithi078.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-05-21T15:17:54.923 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:54.923 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:200:14: note: source object declared here 2024-05-21T15:17:54.923 INFO:tasks.workunit.client.0.smithi078.stderr: 200 | char temp[BUFSIZE]; 2024-05-21T15:17:54.923 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~ 2024-05-21T15:17:54.929 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c: In function ‘size64_convert’: 2024-05-21T15:17:54.929 INFO:tasks.workunit.client.0.smithi078.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-05-21T15:17:54.929 INFO:tasks.workunit.client.0.smithi078.stderr: 119 | memcpy(unit, "\0", 3); 2024-05-21T15:17:54.930 INFO:tasks.workunit.client.0.smithi078.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-21T15:17:55.212 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-05-21T15:17:55.217 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-05-21T15:17:55.273 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-05-21T15:17:55.277 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-05-21T15:17:55.365 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-05-21T15:17:55.369 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-05-21T15:17:55.400 INFO:tasks.workunit.client.0.smithi078.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-05-21T15:17:55.405 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-05-21T15:17:55.436 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-21T15:17:55.437 INFO:tasks.workunit.client.0.smithi078.stderr:+ cd .. 2024-05-21T15:17:55.437 INFO:tasks.workunit.client.0.smithi078.stderr:+ mkdir tmp 2024-05-21T15:17:55.439 INFO:tasks.workunit.client.0.smithi078.stderr:+ cd tmp 2024-05-21T15:17:55.439 INFO:tasks.workunit.client.0.smithi078.stderr:+ for f in $mydir/*.ffsb 2024-05-21T15:17:55.439 INFO:tasks.workunit.client.0.smithi078.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-05-21T15:17:55.440 INFO:tasks.workunit.client.0.smithi078.stdout:FFSB version 6.0-RC2 started 2024-05-21T15:17:55.440 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:17:55.442 INFO:tasks.workunit.client.0.smithi078.stdout:benchmark time = 300 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout:ThreadGroup 0 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout:================ 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: num_threads = 32 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: read_random = off 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: read_size = 0 (0B) 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: read_blocksize = 0 (0B) 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: read_skip = off 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: read_skipsize = 0 (0B) 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: write_random = on 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: write_size = 5242880 (5MB) 2024-05-21T15:17:55.443 INFO:tasks.workunit.client.0.smithi078.stdout: fsync_file = 0 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: write_blocksize = 4096 (4KB) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: wait time = 0 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: op weights 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: read = 0 (0.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: readall = 0 (0.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: write = 1 (100.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: create = 0 (0.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: append = 0 (0.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: delete = 0 (0.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: metaop = 0 (0.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: createdir = 0 (0.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: stat = 0 (0.00%) 2024-05-21T15:17:55.444 INFO:tasks.workunit.client.0.smithi078.stdout: writeall = 0 (0.00%) 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: writeall_fsync = 0 (0.00%) 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: open_close = 0 (0.00%) 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: write_fsync = 0 (0.00%) 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: create_fsync = 0 (0.00%) 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: append_fsync = 0 (0.00%) 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout:FileSystem . 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout:========== 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: num_dirs = 0 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: starting files = 128 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: min file size = 104857600 (100MB) 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: max file size = 104857600 (100MB) 2024-05-21T15:17:55.445 INFO:tasks.workunit.client.0.smithi078.stdout: directio = off 2024-05-21T15:17:55.446 INFO:tasks.workunit.client.0.smithi078.stdout: alignedio = on 2024-05-21T15:17:55.446 INFO:tasks.workunit.client.0.smithi078.stdout: bufferedio = off 2024-05-21T15:17:55.446 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:17:55.446 INFO:tasks.workunit.client.0.smithi078.stdout: aging is off 2024-05-21T15:17:55.446 INFO:tasks.workunit.client.0.smithi078.stdout: current utilization = 0.00% 2024-05-21T15:17:55.446 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:17:55.446 INFO:tasks.workunit.client.0.smithi078.stdout:checking existing fs: . 2024-05-21T15:17:55.446 INFO:tasks.workunit.client.0.smithi078.stdout:recreating new fileset 2024-05-21T15:17:55.447 INFO:tasks.workunit.client.0.smithi078.stderr:opendir: No such file or directory 2024-05-21T15:17:56.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:56 smithi078 ceph-mon[31997]: pgmap v58: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 598 KiB/s wr, 183 op/s 2024-05-21T15:17:56.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:56 smithi084 ceph-mon[36334]: pgmap v58: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 598 KiB/s wr, 183 op/s 2024-05-21T15:17:56.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:56 smithi172 ceph-mon[36402]: pgmap v58: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 598 KiB/s wr, 183 op/s 2024-05-21T15:17:58.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:58 smithi078 ceph-mon[31997]: pgmap v59: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 488 KiB/s wr, 172 op/s 2024-05-21T15:17:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:58 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:17:58.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:58 smithi084 ceph-mon[36334]: pgmap v59: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 488 KiB/s wr, 172 op/s 2024-05-21T15:17:58.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:58 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:17:58.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:58 smithi172 ceph-mon[36402]: pgmap v59: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 488 KiB/s wr, 172 op/s 2024-05-21T15:17:58.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:58 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:17:59.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:17:59 smithi078 ceph-mon[31997]: pgmap v60: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 488 KiB/s wr, 172 op/s 2024-05-21T15:17:59.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:17:59 smithi084 ceph-mon[36334]: pgmap v60: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 488 KiB/s wr, 172 op/s 2024-05-21T15:17:59.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:17:59 smithi172 ceph-mon[36402]: pgmap v60: 97 pgs: 97 active+clean; 7.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 488 KiB/s wr, 172 op/s 2024-05-21T15:18:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 -- 172.21.15.78:0/1723376438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8105990 msgr2=0x7f44d8105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 --2- 172.21.15.78:0/1723376438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8105990 0x7f44d8105dc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009900 tx=0x7f44c802a410 comp rx=0 tx=0).stop 2024-05-21T15:18:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 -- 172.21.15.78:0/1723376438 shutdown_connections 2024-05-21T15:18:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 --2- 172.21.15.78:0/1723376438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d8106450 0x7f44d810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 --2- 172.21.15.78:0/1723376438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8105990 0x7f44d8105dc0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 --2- 172.21.15.78:0/1723376438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44d8101400 0x7f44d8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 -- 172.21.15.78:0/1723376438 >> 172.21.15.78:0/1723376438 conn(0x7f44d8076860 msgr2=0x7f44d8076ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 -- 172.21.15.78:0/1723376438 shutdown_connections 2024-05-21T15:18:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.256+0000 7f44df873640 1 -- 172.21.15.78:0/1723376438 wait complete. 2024-05-21T15:18:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.257+0000 7f44df873640 1 Processor -- start 2024-05-21T15:18:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.257+0000 7f44df873640 1 -- start start 2024-05-21T15:18:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44df873640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8101400 0x7f44d819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44df873640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44d8105990 0x7f44d819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44df873640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d8106450 0x7f44d81a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44df873640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44d8111e30 con 0x7f44d8101400 2024-05-21T15:18:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44df873640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f44d8111cb0 con 0x7f44d8106450 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44df873640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44d8111fb0 con 0x7f44d8105990 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8101400 0x7f44d819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8101400 0x7f44d819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42374/0 (socket says 172.21.15.78:42374) 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 -- 172.21.15.78:0/830346373 learned_addr learned my addr 172.21.15.78:0/830346373 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 -- 172.21.15.78:0/830346373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44d8105990 msgr2=0x7f44d819acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44d8105990 0x7f44d819acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 -- 172.21.15.78:0/830346373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d8106450 msgr2=0x7f44d81a5810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d8106450 0x7f44d81a5810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 -- 172.21.15.78:0/830346373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d819b550 con 0x7f44d8101400 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.258+0000 7f44dd60f640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8101400 0x7f44d819a790 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f44d400b4a0 tx=0x7f44d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.259+0000 7f44ce7fc640 1 -- 172.21.15.78:0/830346373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4014070 con 0x7f44d8101400 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.259+0000 7f44ce7fc640 1 -- 172.21.15.78:0/830346373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d4010af0 con 0x7f44d8101400 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.259+0000 7f44df873640 1 -- 172.21.15.78:0/830346373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d81a5e10 con 0x7f44d8101400 2024-05-21T15:18:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.259+0000 7f44ce7fc640 1 -- 172.21.15.78:0/830346373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4017d10 con 0x7f44d8101400 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.259+0000 7f44df873640 1 -- 172.21.15.78:0/830346373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d81a62a0 con 0x7f44d8101400 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44e67f8740 1 Processor -- start 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44e67f8740 1 -- start start 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44ce7fc640 1 -- 172.21.15.78:0/830346373 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f44d4017070 con 0x7f44d8101400 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f15e1b0 0x55e93f4fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f4fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f4fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44e67f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e93f462340 con 0x55e93f1352f0 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44e67f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e93f40a290 con 0x55e93f13d4d0 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.265+0000 7f44e67f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e93f45e300 con 0x55e93f15e1b0 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.266+0000 7f44dd60f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f4fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.266+0000 7f44dd60f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f4fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42386/0 (socket says 172.21.15.78:42386) 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.266+0000 7f44dd60f640 1 -- 172.21.15.78:0/2110931277 learned_addr learned my addr 172.21.15.78:0/2110931277 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.266+0000 7f44dd60f640 1 -- 172.21.15.78:0/2110931277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f15e1b0 msgr2=0x55e93f4fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.266+0000 7f44dd60f640 1 --2- 172.21.15.78:0/2110931277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f15e1b0 0x55e93f4fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.266+0000 7f44dd60f640 1 -- 172.21.15.78:0/2110931277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 msgr2=0x55e93f4fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.266+0000 7f44dd60f640 1 --2- 172.21.15.78:0/2110931277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f4fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.266+0000 7f44dd60f640 1 -- 172.21.15.78:0/2110931277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93f42e8e0 con 0x55e93f1352f0 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.267+0000 7f44dd60f640 1 --2- 172.21.15.78:0/2110931277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f4fd5c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f44d40cbf20 tx=0x7f44d40d4db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b624c3d576f38ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.267+0000 7f44ce7fc640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f44c40a2d70 0x7f44c40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.267+0000 7f44c37fe640 1 -- 172.21.15.78:0/2110931277 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40d6d60 con 0x55e93f1352f0 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.267+0000 7f44c37fe640 1 -- 172.21.15.78:0/2110931277 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d40db070 con 0x55e93f1352f0 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.267+0000 7f44c37fe640 1 -- 172.21.15.78:0/2110931277 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40c2b10 con 0x55e93f1352f0 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.268+0000 7f44dde10640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f44c40a2d70 0x7f44c40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.268+0000 7f44ce7fc640 1 -- 172.21.15.78:0/830346373 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f44d40ce7b0 con 0x7f44d8101400 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 -- 172.21.15.78:0/2110931277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 msgr2=0x55e93f4fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44dde10640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f44c40a2d70 0x7f44c40a5280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55e93f4fa1f0 tx=0x7f44d0009250 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2110931277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f4fd5c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f44d40cbf20 tx=0x7f44d40d4db0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 -- 172.21.15.78:0/2110931277 shutdown_connections 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2110931277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f4fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2110931277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f4fd5c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2110931277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f15e1b0 0x55e93f4fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 -- 172.21.15.78:0/2110931277 >> 172.21.15.78:0/2110931277 conn(0x55e93f152300 msgr2=0x55e93f4f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 -- 172.21.15.78:0/2110931277 shutdown_connections 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.269+0000 7f44e67f8740 1 -- 172.21.15.78:0/2110931277 wait complete. 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.270+0000 7f44e67f8740 1 -- start start 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.270+0000 7f44e67f8740 1 Processor -- start 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f507770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f507cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f15e1b0 0x55e93f50d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44e67f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e93f50a1b0 con 0x55e93f15e1b0 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44dde10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f15e1b0 0x55e93f50d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44dde10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f15e1b0 0x55e93f50d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42400/0 (socket says 172.21.15.78:42400) 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44dde10640 1 -- 172.21.15.78:0/2910895246 learned_addr learned my addr 172.21.15.78:0/2910895246 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e93f41d790 con 0x55e93f13d4d0 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.272+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e93f50a330 con 0x55e93f1352f0 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dce0e640 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f507cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44e67f8740 10 client.0 _ll_register_callbacks cb 0x55e93f458a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dd60f640 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f507770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dde10640 1 -- 172.21.15.78:0/2910895246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 msgr2=0x55e93f507770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dde10640 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f507770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dde10640 1 -- 172.21.15.78:0/2910895246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 msgr2=0x55e93f507cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dde10640 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f507cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dde10640 1 -- 172.21.15.78:0/2910895246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93f176210 con 0x55e93f15e1b0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dd60f640 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f507770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44dde10640 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f15e1b0 0x55e93f50d770 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f44d0004aa0 tx=0x7f44d0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef168c4da653fe48 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.273+0000 7f44c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.274+0000 7f44c0ff9640 1 -- 172.21.15.78:0/2910895246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0018d90 con 0x55e93f15e1b0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.274+0000 7f44c0ff9640 1 -- 172.21.15.78:0/2910895246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f44d0023480 con 0x55e93f15e1b0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.274+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44d40db070 con 0x55e93f15e1b0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.274+0000 7f44c0ff9640 1 -- 172.21.15.78:0/2910895246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44d0023680 con 0x55e93f15e1b0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.274+0000 7f44e67f8740 10 client.16236 fetch_fsmap learned FSMap version 27 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.274+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e93f242620 con 0x55e93f15e1b0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.275+0000 7f44c0ff9640 1 -- 172.21.15.78:0/2910895246 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d0023d00 con 0x55e93f15e1b0 2024-05-21T15:18:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.275+0000 7f44c0ff9640 1 -- 172.21.15.78:0/2910895246 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f44d001ab00 con 0x55e93f15e1b0 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.275+0000 7f44e67f8740 10 client.16236 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.275+0000 7f44e67f8740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.275+0000 7f44e67f8740 20 client.16236 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.275+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e93f11cd80 0x55e93f519a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.275+0000 7f44e67f8740 4 client.16236 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.275+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f44d40db070 con 0x55e93f11cd80 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.276+0000 7f44dd60f640 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e93f11cd80 0x55e93f519a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.279+0000 7f44c0ff9640 10 client.16236 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.279+0000 7f44dd60f640 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e93f11cd80 0x55e93f519a10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47172f496a214a3 server_cookie=91dcb345998ac3c0 in_seq=0 out_seq=0 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44c0ff9640 1 -- 172.21.15.78:0/2910895246 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f44d40db070 con 0x55e93f11cd80 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44c0ff9640 10 client.16236 handle_command_reply: tid=0 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44e67f8740 1 client.16236 shutdown 2024-05-21T15:18:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f15e1b0 msgr2=0x55e93f50d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f15e1b0 0x55e93f50d770 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f44d0004aa0 tx=0x7f44d0004b40 comp rx=0 tx=0).stop 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 shutdown_connections 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e93f11cd80 0x55e93f519a10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f507cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f15e1b0 0x55e93f50d770 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.280+0000 7f44e67f8740 1 --2- 172.21.15.78:0/2910895246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f507770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.281+0000 7f44c0ff9640 0 client.16236 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.281+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 >> 172.21.15.78:0/2910895246 conn(0x55e93f152300 msgr2=0x55e93f4fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.281+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 shutdown_connections 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.281+0000 7f44e67f8740 1 -- 172.21.15.78:0/2910895246 wait complete. 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.281+0000 7f44e67f8740 20 client.16236 trim_cache size 0 max 16384 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44e67f8740 1 Processor -- start 2024-05-21T15:18:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44e67f8740 1 -- start start 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f11cd80 0x55e93f51f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f51f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f13d4d0 0x55e93f513440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44e67f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44d81aeea0 con 0x55e93f1352f0 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44e67f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f44d81b0a10 con 0x55e93f11cd80 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44e67f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44d81af570 con 0x55e93f13d4d0 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44dce0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f51f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44dce0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f51f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42414/0 (socket says 172.21.15.78:42414) 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.305+0000 7f44dce0e640 1 -- 172.21.15.78:0/397237090 learned_addr learned my addr 172.21.15.78:0/397237090 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dd60f640 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f11cd80 0x55e93f51f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dde10640 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f13d4d0 0x55e93f513440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dce0e640 1 -- 172.21.15.78:0/397237090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f13d4d0 msgr2=0x55e93f513440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dce0e640 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f13d4d0 0x55e93f513440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dce0e640 1 -- 172.21.15.78:0/397237090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f11cd80 msgr2=0x55e93f51f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dce0e640 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f11cd80 0x55e93f51f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dce0e640 1 -- 172.21.15.78:0/397237090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93f516570 con 0x55e93f1352f0 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dde10640 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f13d4d0 0x55e93f513440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44dce0e640 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f51f750 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f44d8074450 tx=0x7f44c8005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f575b8ca31bbab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44c2ffd640 1 -- 172.21.15.78:0/397237090 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8040070 con 0x55e93f1352f0 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44c2ffd640 1 -- 172.21.15.78:0/397237090 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44c80317b0 con 0x55e93f1352f0 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.306+0000 7f44c2ffd640 1 -- 172.21.15.78:0/397237090 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8035040 con 0x55e93f1352f0 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 -- 172.21.15.78:0/397237090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 msgr2=0x55e93f51f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f51f750 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f44d8074450 tx=0x7f44c8005f30 comp rx=0 tx=0).stop 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 -- 172.21.15.78:0/397237090 shutdown_connections 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f13d4d0 0x55e93f513440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f1352f0 0x55e93f51f750 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 --2- 172.21.15.78:0/397237090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f11cd80 0x55e93f51f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 -- 172.21.15.78:0/397237090 >> 172.21.15.78:0/397237090 conn(0x55e93f152300 msgr2=0x55e93f505e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 -- 172.21.15.78:0/397237090 shutdown_connections 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.307+0000 7f44e67f8740 1 -- 172.21.15.78:0/397237090 wait complete. 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 1 -- start start 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 1 Processor -- start 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f11cd80 0x55e93f513a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f50eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f50f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44d81b0310 con 0x55e93f11cd80 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f44d81afc40 con 0x55e93f13d4d0 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44d81ace80 con 0x55e93f1352f0 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.308+0000 7f44e67f8740 10 client.0 _ll_register_callbacks cb 0x7f44d000a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f11cd80 0x55e93f513a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f11cd80 0x55e93f513a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42428/0 (socket says 172.21.15.78:42428) 2024-05-21T15:18:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 -- 172.21.15.78:0/3389175690 learned_addr learned my addr 172.21.15.78:0/3389175690 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dce0e640 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f50eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 -- 172.21.15.78:0/3389175690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 msgr2=0x55e93f50eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f50eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 -- 172.21.15.78:0/3389175690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 msgr2=0x55e93f50f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f50f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 -- 172.21.15.78:0/3389175690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93f50fa90 con 0x55e93f11cd80 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44dd60f640 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f11cd80 0x55e93f513a50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f44d40c4fd0 tx=0x7f44d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5651dc17574b046f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44c0ff9640 1 -- 172.21.15.78:0/3389175690 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d401e070 con 0x55e93f11cd80 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44c0ff9640 1 -- 172.21.15.78:0/3389175690 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f44d4099080 con 0x55e93f11cd80 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.309+0000 7f44e67f8740 1 -- 172.21.15.78:0/3389175690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44c80317b0 con 0x55e93f11cd80 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44c0ff9640 1 -- 172.21.15.78:0/3389175690 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44d40c6b20 con 0x55e93f11cd80 2024-05-21T15:18:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44e67f8740 10 client.16242 fetch_fsmap learned FSMap version 27 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44e67f8740 1 -- 172.21.15.78:0/3389175690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e93f581ca0 con 0x55e93f11cd80 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44c0ff9640 1 -- 172.21.15.78:0/3389175690 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44c80317b0 con 0x55e93f11cd80 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44c0ff9640 1 -- 172.21.15.78:0/3389175690 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f44d40cc120 con 0x55e93f11cd80 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44e67f8740 10 client.16242 fetch_fsmap finished waiting for FSMap version 27 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44e67f8740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44e67f8740 20 client.16242 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44e67f8740 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e93f15e1b0 0x55e93f583fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44e67f8740 4 client.16242 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.310+0000 7f44e67f8740 1 -- 172.21.15.78:0/3389175690 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e93f46fa70 con 0x55e93f15e1b0 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.311+0000 7f44dce0e640 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e93f15e1b0 0x55e93f583fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.312+0000 7f44c0ff9640 10 client.16242 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.312+0000 7f44dce0e640 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e93f15e1b0 0x55e93f583fa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d0922fab19ebc6 server_cookie=63eed9d59c84e6f7 in_seq=0 out_seq=0 2024-05-21T15:18:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.441+0000 7f44c0ff9640 1 -- 172.21.15.78:0/3389175690 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e93f46fa70 con 0x55e93f15e1b0 2024-05-21T15:18:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.441+0000 7f44c0ff9640 10 client.16242 handle_command_reply: tid=0 2024-05-21T15:18:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.441+0000 7f44e67f8740 1 client.16242 shutdown 2024-05-21T15:18:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.441+0000 7f44e67f8740 1 -- 172.21.15.78:0/3389175690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f11cd80 msgr2=0x55e93f513a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.441+0000 7f44e67f8740 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f11cd80 0x55e93f513a50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f44d40c4fd0 tx=0x7f44d40046c0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 1 -- 172.21.15.78:0/3389175690 shutdown_connections 2024-05-21T15:18:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e93f15e1b0 0x55e93f583fa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e93f13d4d0 0x55e93f50f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e93f1352f0 0x55e93f50eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 1 --2- 172.21.15.78:0/3389175690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e93f11cd80 0x55e93f513a50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 1 -- 172.21.15.78:0/3389175690 >> 172.21.15.78:0/3389175690 conn(0x55e93f152300 msgr2=0x55e93f500cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44c0ff9640 0 client.16242 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 1 -- 172.21.15.78:0/3389175690 shutdown_connections 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 1 -- 172.21.15.78:0/3389175690 wait complete. 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.442+0000 7f44e67f8740 20 client.16242 trim_cache size 0 max 16384 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:18:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:18:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.443+0000 7f44df873640 1 -- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f44c40a2d70 msgr2=0x7f44c40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.443+0000 7f44df873640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f44c40a2d70 0x7f44c40a5280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55e93f4fa1f0 tx=0x7f44d0009250 comp rx=0 tx=0).stop 2024-05-21T15:18:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.443+0000 7f44df873640 1 -- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8101400 msgr2=0x7f44d819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.443+0000 7f44df873640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8101400 0x7f44d819a790 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f44d400b4a0 tx=0x7f44d400b960 comp rx=0 tx=0).stop 2024-05-21T15:18:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.443+0000 7f44df873640 1 -- 172.21.15.78:0/830346373 shutdown_connections 2024-05-21T15:18:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.443+0000 7f44df873640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d8106450 0x7f44d81a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.444+0000 7f44df873640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f44c40a2d70 0x7f44c40a5280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.444+0000 7f44df873640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44d8105990 0x7f44d819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.444+0000 7f44df873640 1 --2- 172.21.15.78:0/830346373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d8101400 0x7f44d819a790 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.444+0000 7f44df873640 1 -- 172.21.15.78:0/830346373 >> 172.21.15.78:0/830346373 conn(0x7f44d8076860 msgr2=0x7f44d80ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.444+0000 7f44df873640 1 -- 172.21.15.78:0/830346373 shutdown_connections 2024-05-21T15:18:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:01.444+0000 7f44df873640 1 -- 172.21.15.78:0/830346373 wait complete. 2024-05-21T15:18:02.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:01 smithi078 ceph-mon[31997]: pgmap v61: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 149 MiB/s wr, 285 op/s 2024-05-21T15:18:02.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:01 smithi084 ceph-mon[36334]: pgmap v61: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 149 MiB/s wr, 285 op/s 2024-05-21T15:18:02.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:01 smithi172 ceph-mon[36402]: pgmap v61: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 149 MiB/s wr, 285 op/s 2024-05-21T15:18:04.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:03 smithi172 ceph-mon[36402]: pgmap v62: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 148 MiB/s wr, 145 op/s 2024-05-21T15:18:04.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:03 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:04.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:03 smithi078 ceph-mon[31997]: pgmap v62: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 148 MiB/s wr, 145 op/s 2024-05-21T15:18:04.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:03 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:03 smithi084 ceph-mon[36334]: pgmap v62: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 148 MiB/s wr, 145 op/s 2024-05-21T15:18:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:03 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:05.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:05 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:05.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:05 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:05.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:05 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:06.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:06 smithi078 ceph-mon[31997]: pgmap v63: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 418 MiB/s wr, 282 op/s 2024-05-21T15:18:06.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:06 smithi084 ceph-mon[36334]: pgmap v63: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 418 MiB/s wr, 282 op/s 2024-05-21T15:18:06.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:06 smithi172 ceph-mon[36402]: pgmap v63: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 418 MiB/s wr, 282 op/s 2024-05-21T15:18:08.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:08 smithi084 ceph-mon[36334]: pgmap v64: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 418 MiB/s wr, 249 op/s 2024-05-21T15:18:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:08 smithi172 ceph-mon[36402]: pgmap v64: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 418 MiB/s wr, 249 op/s 2024-05-21T15:18:08.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:08 smithi078 ceph-mon[31997]: pgmap v64: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 418 MiB/s wr, 249 op/s 2024-05-21T15:18:10.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:10 smithi172 ceph-mon[36402]: pgmap v65: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 418 MiB/s wr, 249 op/s 2024-05-21T15:18:10.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:10 smithi078 ceph-mon[31997]: pgmap v65: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 418 MiB/s wr, 249 op/s 2024-05-21T15:18:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:10 smithi084 ceph-mon[36334]: pgmap v65: 97 pgs: 97 active+clean; 4.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 418 MiB/s wr, 249 op/s 2024-05-21T15:18:12.021 INFO:tasks.workunit.client.0.smithi078.stdout:fs setup took 16 secs 2024-05-21T15:18:12.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:12 smithi078 ceph-mon[31997]: pgmap v66: 97 pgs: 97 active+clean; 8.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 695 MiB/s wr, 355 op/s 2024-05-21T15:18:12.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:12 smithi084 ceph-mon[36334]: pgmap v66: 97 pgs: 97 active+clean; 8.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 695 MiB/s wr, 355 op/s 2024-05-21T15:18:12.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:12 smithi172 ceph-mon[36402]: pgmap v66: 97 pgs: 97 active+clean; 8.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 695 MiB/s wr, 355 op/s 2024-05-21T15:18:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.592+0000 7fccd4b8f640 1 -- 172.21.15.78:0/4248968665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 msgr2=0x7fccd010e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.592+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/4248968665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 0x7fccd010e4c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009ca0 tx=0x7fccc802a8e0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.592+0000 7fccd4b8f640 1 -- 172.21.15.78:0/4248968665 shutdown_connections 2024-05-21T15:18:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.592+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/4248968665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd0107ed0 0x7fccd0101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.592+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/4248968665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 0x7fccd010e4c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.592+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/4248968665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccd0107a30 0x7fccd01015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.592+0000 7fccd4b8f640 1 -- 172.21.15.78:0/4248968665 >> 172.21.15.78:0/4248968665 conn(0x7fccd0077030 msgr2=0x7fccd0077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.593+0000 7fccd4b8f640 1 -- 172.21.15.78:0/4248968665 shutdown_connections 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.593+0000 7fccd4b8f640 1 -- 172.21.15.78:0/4248968665 wait complete. 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.594+0000 7fccd4b8f640 1 Processor -- start 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.594+0000 7fccd4b8f640 1 -- start start 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.594+0000 7fccd4b8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 0x7fccd0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.594+0000 7fccd4b8f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd0107a30 0x7fccd0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.594+0000 7fccd4b8f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccd0107ed0 0x7fccd019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.594+0000 7fccd4b8f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0111e70 con 0x7fccd0102190 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.594+0000 7fccd4b8f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0111cf0 con 0x7fccd0107a30 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.594+0000 7fccd4b8f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fccd0111ff0 con 0x7fccd0107ed0 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.595+0000 7fccce59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 0x7fccd0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.595+0000 7fccce59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 0x7fccd0199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48110/0 (socket says 172.21.15.78:48110) 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.595+0000 7fccce59c640 1 -- 172.21.15.78:0/2343986813 learned_addr learned my addr 172.21.15.78:0/2343986813 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.595+0000 7fccce59c640 1 -- 172.21.15.78:0/2343986813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccd0107ed0 msgr2=0x7fccd019e020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.595+0000 7fccce59c640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccd0107ed0 0x7fccd019e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.595+0000 7fccce59c640 1 -- 172.21.15.78:0/2343986813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd0107a30 msgr2=0x7fccd0199d60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.595+0000 7fccce59c640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd0107a30 0x7fccd0199d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.595+0000 7fccce59c640 1 -- 172.21.15.78:0/2343986813 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd019e5e0 con 0x7fccd0102190 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.596+0000 7fccce59c640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 0x7fccd0199800 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c8f0 tx=0x7fccc000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.596+0000 7fccc7fff640 1 -- 172.21.15.78:0/2343986813 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0007bb0 con 0x7fccd0102190 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.596+0000 7fccd4b8f640 1 -- 172.21.15.78:0/2343986813 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd019e8d0 con 0x7fccd0102190 2024-05-21T15:18:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.596+0000 7fccc7fff640 1 -- 172.21.15.78:0/2343986813 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc0007d50 con 0x7fccd0102190 2024-05-21T15:18:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.596+0000 7fccc7fff640 1 -- 172.21.15.78:0/2343986813 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0018820 con 0x7fccd0102190 2024-05-21T15:18:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.596+0000 7fccd4b8f640 1 -- 172.21.15.78:0/2343986813 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd01a62d0 con 0x7fccd0102190 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.600+0000 7fccc7fff640 1 -- 172.21.15.78:0/2343986813 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fccc0018a00 con 0x7fccd0102190 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.600+0000 7fccdba37740 1 Processor -- start 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccdba37740 1 -- start start 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccdba37740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f523e1b0 0x55c0f55dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccdba37740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 0x55c0f55dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccdba37740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f521d4d0 0x55c0f55ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccdba37740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c0f5542340 con 0x55c0f521d4d0 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccdba37740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c0f54ea290 con 0x55c0f52152f0 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccdba37740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c0f553e300 con 0x55c0f523e1b0 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccc5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f521d4d0 0x55c0f55ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccc5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f521d4d0 0x55c0f55ddb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48112/0 (socket says 172.21.15.78:48112) 2024-05-21T15:18:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.601+0000 7fccc5d9b640 1 -- 172.21.15.78:0/4228280040 learned_addr learned my addr 172.21.15.78:0/4228280040 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc7fff640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fccbc0a2cf0 0x7fccbc0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccced9d640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fccbc0a2cf0 0x7fccbc0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc5d9b640 1 -- 172.21.15.78:0/4228280040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f523e1b0 msgr2=0x55c0f55dd080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc5d9b640 1 --2- 172.21.15.78:0/4228280040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f523e1b0 0x55c0f55dd080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc5d9b640 1 -- 172.21.15.78:0/4228280040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 msgr2=0x55c0f55dd5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc5d9b640 1 --2- 172.21.15.78:0/4228280040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 0x55c0f55dd5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc5d9b640 1 -- 172.21.15.78:0/4228280040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f550e8e0 con 0x55c0f521d4d0 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc7fff640 1 -- 172.21.15.78:0/2343986813 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fccc00cf630 con 0x7fccd0102190 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc5d9b640 1 --2- 172.21.15.78:0/4228280040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f521d4d0 0x55c0f55ddb00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fccb80096f0 tx=0x7fccb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957b7951e4c86c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.602+0000 7fccc4d99640 1 -- 172.21.15.78:0/4228280040 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8015020 con 0x55c0f521d4d0 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.603+0000 7fccc4d99640 1 -- 172.21.15.78:0/4228280040 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb800bdb0 con 0x55c0f521d4d0 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.603+0000 7fccced9d640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fccbc0a2cf0 0x7fccbc0a5200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fccc8031af0 tx=0x7fccc8031000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.603+0000 7fccc4d99640 1 -- 172.21.15.78:0/4228280040 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8010380 con 0x55c0f521d4d0 2024-05-21T15:18:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.604+0000 7fccdba37740 1 -- 172.21.15.78:0/4228280040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f521d4d0 msgr2=0x55c0f55ddb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.604+0000 7fccdba37740 1 --2- 172.21.15.78:0/4228280040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f521d4d0 0x55c0f55ddb00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fccb80096f0 tx=0x7fccb8009bb0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.604+0000 7fccdba37740 1 -- 172.21.15.78:0/4228280040 shutdown_connections 2024-05-21T15:18:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.604+0000 7fccdba37740 1 --2- 172.21.15.78:0/4228280040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 0x55c0f55dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.604+0000 7fccdba37740 1 --2- 172.21.15.78:0/4228280040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f521d4d0 0x55c0f55ddb00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.604+0000 7fccdba37740 1 --2- 172.21.15.78:0/4228280040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f523e1b0 0x55c0f55dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.604+0000 7fccdba37740 1 -- 172.21.15.78:0/4228280040 >> 172.21.15.78:0/4228280040 conn(0x55c0f5232300 msgr2=0x55c0f55d9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.604+0000 7fccdba37740 1 -- 172.21.15.78:0/4228280040 shutdown_connections 2024-05-21T15:18:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.605+0000 7fccdba37740 1 -- 172.21.15.78:0/4228280040 wait complete. 2024-05-21T15:18:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.605+0000 7fccdba37740 1 -- start start 2024-05-21T15:18:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.606+0000 7fccdba37740 1 Processor -- start 2024-05-21T15:18:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.606+0000 7fccdba37740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 0x55c0f55e9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.606+0000 7fccdba37740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 0x55c0f55e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.606+0000 7fccdba37740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f523e1b0 0x55c0f55e7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.606+0000 7fccdba37740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c0f55ea1b0 con 0x55c0f523e1b0 2024-05-21T15:18:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.606+0000 7fccdba37740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c0f54fd790 con 0x55c0f52152f0 2024-05-21T15:18:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.606+0000 7fccdba37740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c0f55ea330 con 0x55c0f521d4d0 2024-05-21T15:18:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.606+0000 7fccdba37740 10 client.0 _ll_register_callbacks cb 0x55c0f5538a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f523e1b0 0x55c0f55e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f523e1b0 0x55c0f55e7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48124/0 (socket says 172.21.15.78:48124) 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 -- 172.21.15.78:0/4053942574 learned_addr learned my addr 172.21.15.78:0/4053942574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccce59c640 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 0x55c0f55e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 -- 172.21.15.78:0/4053942574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 msgr2=0x55c0f55e9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 0x55c0f55e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 -- 172.21.15.78:0/4053942574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 msgr2=0x55c0f55e9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 0x55c0f55e9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 -- 172.21.15.78:0/4053942574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f5256210 con 0x55c0f523e1b0 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccced9d640 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f523e1b0 0x55c0f55e7640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009ca0 tx=0x7fccc802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd40a02e06a94abc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.607+0000 7fccb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.608+0000 7fccb5ffb640 1 -- 172.21.15.78:0/4053942574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc803f070 con 0x55c0f523e1b0 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.608+0000 7fccb5ffb640 1 -- 172.21.15.78:0/4053942574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fccc80047a0 con 0x55c0f523e1b0 2024-05-21T15:18:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.608+0000 7fccdba37740 1 -- 172.21.15.78:0/4053942574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb800bdb0 con 0x55c0f523e1b0 2024-05-21T15:18:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.608+0000 7fccb5ffb640 1 -- 172.21.15.78:0/4053942574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccc80305e0 con 0x55c0f523e1b0 2024-05-21T15:18:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.608+0000 7fccdba37740 10 client.16254 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.608+0000 7fccdba37740 1 -- 172.21.15.78:0/4053942574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0f5322620 con 0x55c0f523e1b0 2024-05-21T15:18:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.609+0000 7fccb5ffb640 1 -- 172.21.15.78:0/4053942574 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb800bdb0 con 0x55c0f523e1b0 2024-05-21T15:18:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.610+0000 7fccb5ffb640 1 -- 172.21.15.78:0/4053942574 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fccc8046760 con 0x55c0f523e1b0 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.610+0000 7fccdba37740 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.610+0000 7fccdba37740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.610+0000 7fccdba37740 20 client.16254 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.610+0000 7fccdba37740 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c0f51fcd80 0x55c0f55fdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.610+0000 7fccdba37740 4 client.16254 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.610+0000 7fccdba37740 1 -- 172.21.15.78:0/4053942574 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0f554fa70 con 0x55c0f51fcd80 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.611+0000 7fccce59c640 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c0f51fcd80 0x55c0f55fdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.613+0000 7fccb5ffb640 10 client.16254 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.613+0000 7fccce59c640 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c0f51fcd80 0x55c0f55fdf80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c092359c7f65df server_cookie=74be51e522ed9b30 in_seq=0 out_seq=0 2024-05-21T15:18:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.614+0000 7fccb5ffb640 1 -- 172.21.15.78:0/4053942574 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0f554fa70 con 0x55c0f51fcd80 2024-05-21T15:18:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.614+0000 7fccb5ffb640 10 client.16254 handle_command_reply: tid=0 2024-05-21T15:18:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.614+0000 7fccdba37740 1 client.16254 shutdown 2024-05-21T15:18:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.614+0000 7fccdba37740 1 -- 172.21.15.78:0/4053942574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f523e1b0 msgr2=0x55c0f55e7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.614+0000 7fccdba37740 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f523e1b0 0x55c0f55e7640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009ca0 tx=0x7fccc802a8e0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 1 -- 172.21.15.78:0/4053942574 shutdown_connections 2024-05-21T15:18:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c0f51fcd80 0x55c0f55fdf80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f523e1b0 0x55c0f55e7640 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 0x55c0f55e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccb5ffb640 0 client.16254 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 1 --2- 172.21.15.78:0/4053942574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f52152f0 0x55c0f55e9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 1 -- 172.21.15.78:0/4053942574 >> 172.21.15.78:0/4053942574 conn(0x55c0f5232300 msgr2=0x55c0f55deda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 1 -- 172.21.15.78:0/4053942574 shutdown_connections 2024-05-21T15:18:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 1 -- 172.21.15.78:0/4053942574 wait complete. 2024-05-21T15:18:14.620 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.615+0000 7fccdba37740 20 client.16254 trim_cache size 0 max 16384 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccdba37740 1 Processor -- start 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccdba37740 1 -- start start 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccdba37740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f51fcd80 0x55c0f55e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccdba37740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f52152f0 0x55c0f55e1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccdba37740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f521d4d0 0x55c0f55decc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccdba37740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01aca40 con 0x55c0f51fcd80 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccdba37740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01ae670 con 0x55c0f521d4d0 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccdba37740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01ad110 con 0x55c0f52152f0 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccce59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f51fcd80 0x55c0f55e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccce59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f51fcd80 0x55c0f55e0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48132/0 (socket says 172.21.15.78:48132) 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.637+0000 7fccce59c640 1 -- 172.21.15.78:0/1069865791 learned_addr learned my addr 172.21.15.78:0/1069865791 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.638+0000 7fccced9d640 1 --2- 172.21.15.78:0/1069865791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f521d4d0 0x55c0f55decc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.638+0000 7fccce59c640 1 -- 172.21.15.78:0/1069865791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f52152f0 msgr2=0x55c0f55e1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.638+0000 7fccce59c640 1 --2- 172.21.15.78:0/1069865791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f52152f0 0x55c0f55e1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.638+0000 7fccce59c640 1 -- 172.21.15.78:0/1069865791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f521d4d0 msgr2=0x55c0f55decc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.638+0000 7fccce59c640 1 --2- 172.21.15.78:0/1069865791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f521d4d0 0x55c0f55decc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.638+0000 7fccce59c640 1 -- 172.21.15.78:0/1069865791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f55d4a60 con 0x55c0f51fcd80 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.639+0000 7fccce59c640 1 --2- 172.21.15.78:0/1069865791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f51fcd80 0x55c0f55e0de0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fccc00ccdd0 tx=0x7fccc000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71abd08c4763eaf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.639+0000 7fccb7fff640 1 -- 172.21.15.78:0/1069865791 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc00b8020 con 0x55c0f51fcd80 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.639+0000 7fccb7fff640 1 -- 172.21.15.78:0/1069865791 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc00a4760 con 0x55c0f51fcd80 2024-05-21T15:18:14.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.639+0000 7fccb7fff640 1 -- 172.21.15.78:0/1069865791 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0098160 con 0x55c0f51fcd80 2024-05-21T15:18:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 -- 172.21.15.78:0/1069865791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f51fcd80 msgr2=0x55c0f55e0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 --2- 172.21.15.78:0/1069865791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f51fcd80 0x55c0f55e0de0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fccc00ccdd0 tx=0x7fccc000fec0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 -- 172.21.15.78:0/1069865791 shutdown_connections 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 --2- 172.21.15.78:0/1069865791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f521d4d0 0x55c0f55decc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 --2- 172.21.15.78:0/1069865791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f52152f0 0x55c0f55e1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 --2- 172.21.15.78:0/1069865791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f51fcd80 0x55c0f55e0de0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 -- 172.21.15.78:0/1069865791 >> 172.21.15.78:0/1069865791 conn(0x55c0f5232300 msgr2=0x55c0f55e5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 -- 172.21.15.78:0/1069865791 shutdown_connections 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.640+0000 7fccdba37740 1 -- 172.21.15.78:0/1069865791 wait complete. 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.641+0000 7fccdba37740 1 -- start start 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.641+0000 7fccdba37740 1 Processor -- start 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.641+0000 7fccdba37740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f51fcd80 0x55c0f55efb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccdba37740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f52152f0 0x55c0f55ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccdba37740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 0x55c0f55ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccdba37740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01adee0 con 0x55c0f52152f0 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccdba37740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01ad7e0 con 0x55c0f51fcd80 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccdba37740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fccd01035f0 con 0x55c0f521d4d0 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccdba37740 10 client.0 _ll_register_callbacks cb 0x7fccc8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccc5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f52152f0 0x55c0f55ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccc5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f52152f0 0x55c0f55ecf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48134/0 (socket says 172.21.15.78:48134) 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccc5d9b640 1 -- 172.21.15.78:0/1913686875 learned_addr learned my addr 172.21.15.78:0/1913686875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccced9d640 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 0x55c0f55ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccc5d9b640 1 -- 172.21.15.78:0/1913686875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 msgr2=0x55c0f55ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccc5d9b640 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 0x55c0f55ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccc5d9b640 1 -- 172.21.15.78:0/1913686875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f51fcd80 msgr2=0x55c0f55efb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccc5d9b640 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f51fcd80 0x55c0f55efb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.642+0000 7fccc5d9b640 1 -- 172.21.15.78:0/1913686875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f55edbd0 con 0x55c0f52152f0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.643+0000 7fccc5d9b640 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f52152f0 0x55c0f55ecf00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009640 tx=0x7fccb8006ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0859d9a0648b27c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.643+0000 7fccb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.643+0000 7fccb5ffb640 1 -- 172.21.15.78:0/1913686875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb800bde0 con 0x55c0f52152f0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.643+0000 7fccb5ffb640 1 -- 172.21.15.78:0/1913686875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fccb8025480 con 0x55c0f52152f0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.643+0000 7fccdba37740 1 -- 172.21.15.78:0/1913686875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc00a4760 con 0x55c0f52152f0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.643+0000 7fccb5ffb640 1 -- 172.21.15.78:0/1913686875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccb80256c0 con 0x55c0f52152f0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.643+0000 7fccdba37740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.643+0000 7fccdba37740 1 -- 172.21.15.78:0/1913686875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0f56662a0 con 0x55c0f52152f0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.644+0000 7fccb5ffb640 1 -- 172.21.15.78:0/1913686875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc00a4760 con 0x55c0f52152f0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.644+0000 7fccb5ffb640 1 -- 172.21.15.78:0/1913686875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fccb801ab50 con 0x55c0f52152f0 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.644+0000 7fccdba37740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.644+0000 7fccdba37740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.644+0000 7fccdba37740 20 client.16260 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.644+0000 7fccdba37740 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c0f523e1b0 0x55c0f5668860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.644+0000 7fccdba37740 4 client.16260 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:18:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.644+0000 7fccdba37740 1 -- 172.21.15.78:0/1913686875 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0f5549dc0 con 0x55c0f523e1b0 2024-05-21T15:18:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.645+0000 7fccce59c640 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c0f523e1b0 0x55c0f5668860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.649+0000 7fccb5ffb640 10 client.16260 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:14.649+0000 7fccce59c640 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c0f523e1b0 0x55c0f5668860 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e0d6c5aae5c05b server_cookie=90dba68245b6500 in_seq=0 out_seq=0 2024-05-21T15:18:14.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:14 smithi078 ceph-mon[31997]: pgmap v67: 97 pgs: 97 active+clean; 8.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 547 MiB/s wr, 242 op/s 2024-05-21T15:18:14.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:14 smithi084 ceph-mon[36334]: pgmap v67: 97 pgs: 97 active+clean; 8.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 547 MiB/s wr, 242 op/s 2024-05-21T15:18:14.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:14 smithi172 ceph-mon[36402]: pgmap v67: 97 pgs: 97 active+clean; 8.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 547 MiB/s wr, 242 op/s 2024-05-21T15:18:15.538 INFO:tasks.workunit.client.0.smithi078.stdout:Syncing()...3 sec 2024-05-21T15:18:15.539 INFO:tasks.workunit.client.0.smithi078.stdout:Starting Actual Benchmark At: Tue May 21 15:18:15 2024 2024-05-21T15:18:15.539 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-05-21T15:18:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.540+0000 7fccb5ffb640 1 -- 172.21.15.78:0/1913686875 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0f5549dc0 con 0x55c0f523e1b0 2024-05-21T15:18:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.540+0000 7fccb5ffb640 10 client.16260 handle_command_reply: tid=0 2024-05-21T15:18:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.547+0000 7fccdba37740 1 client.16260 shutdown 2024-05-21T15:18:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.583+0000 7fccdba37740 1 -- 172.21.15.78:0/1913686875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f52152f0 msgr2=0x55c0f55ecf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.583+0000 7fccdba37740 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f52152f0 0x55c0f55ecf00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009640 tx=0x7fccb8006ed0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 1 -- 172.21.15.78:0/1913686875 shutdown_connections 2024-05-21T15:18:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c0f523e1b0 0x55c0f5668860 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c0f521d4d0 0x55c0f55ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c0f52152f0 0x55c0f55ecf00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 1 --2- 172.21.15.78:0/1913686875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0f51fcd80 0x55c0f55efb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 1 -- 172.21.15.78:0/1913686875 >> 172.21.15.78:0/1913686875 conn(0x55c0f5232300 msgr2=0x55c0f55decc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccb5ffb640 0 client.16260 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 1 -- 172.21.15.78:0/1913686875 shutdown_connections 2024-05-21T15:18:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 1 -- 172.21.15.78:0/1913686875 wait complete. 2024-05-21T15:18:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.612+0000 7fccdba37740 20 client.16260 trim_cache size 0 max 16384 2024-05-21T15:18:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:18:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:18:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:18:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:18:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.638+0000 7fccd4b8f640 1 -- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fccbc0a2cf0 msgr2=0x7fccbc0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.638+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fccbc0a2cf0 0x7fccbc0a5200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fccc8031af0 tx=0x7fccc8031000 comp rx=0 tx=0).stop 2024-05-21T15:18:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.639+0000 7fccd4b8f640 1 -- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 msgr2=0x7fccd0199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.639+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 0x7fccd0199800 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c8f0 tx=0x7fccc000cdb0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.640+0000 7fccd4b8f640 1 -- 172.21.15.78:0/2343986813 shutdown_connections 2024-05-21T15:18:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.640+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fccd0107a30 0x7fccd0199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.640+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fccbc0a2cf0 0x7fccbc0a5200 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.640+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccd0107ed0 0x7fccd019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.640+0000 7fccd4b8f640 1 --2- 172.21.15.78:0/2343986813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccd0102190 0x7fccd0199800 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.640+0000 7fccd4b8f640 1 -- 172.21.15.78:0/2343986813 >> 172.21.15.78:0/2343986813 conn(0x7fccd0077030 msgr2=0x7fccd0108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.640+0000 7fccd4b8f640 1 -- 172.21.15.78:0/2343986813 shutdown_connections 2024-05-21T15:18:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:15.642+0000 7fccd4b8f640 1 -- 172.21.15.78:0/2343986813 wait complete. 2024-05-21T15:18:16.854 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:16 smithi078 ceph-mon[31997]: pgmap v68: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 865 MiB/s wr, 340 op/s 2024-05-21T15:18:16.854 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:16 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:18:16.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:16 smithi084 ceph-mon[36334]: pgmap v68: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 865 MiB/s wr, 340 op/s 2024-05-21T15:18:16.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:16 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:18:16.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:16 smithi172 ceph-mon[36402]: pgmap v68: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 865 MiB/s wr, 340 op/s 2024-05-21T15:18:16.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:16 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:18:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:18 smithi078 ceph-mon[31997]: pgmap v69: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 596 MiB/s wr, 203 op/s 2024-05-21T15:18:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:18 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:18 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:18 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:18 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:18 smithi084 ceph-mon[36334]: pgmap v69: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 596 MiB/s wr, 203 op/s 2024-05-21T15:18:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:18 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:18 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:18 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:18 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:18 smithi172 ceph-mon[36402]: pgmap v69: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 596 MiB/s wr, 203 op/s 2024-05-21T15:18:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:18 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:18 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:18 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:18 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:19.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:19 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:18:19.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:19 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:18:19.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:19 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:19 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:18:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:19 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:18:19.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:19 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:19.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:19 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:18:19.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:19 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:18:19.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:19 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:18:20.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:20 smithi078 ceph-mon[31997]: pgmap v70: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 596 MiB/s wr, 203 op/s 2024-05-21T15:18:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:20 smithi084 ceph-mon[36334]: pgmap v70: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 596 MiB/s wr, 203 op/s 2024-05-21T15:18:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:20 smithi172 ceph-mon[36402]: pgmap v70: 97 pgs: 97 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 596 MiB/s wr, 203 op/s 2024-05-21T15:18:21.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:18:21.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:18:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 -- 172.21.15.78:0/3966369799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 msgr2=0x7f1d04101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 --2- 172.21.15.78:0/3966369799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 0x7f1d04101eb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009900 tx=0x7f1cec02a410 comp rx=0 tx=0).stop 2024-05-21T15:18:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 -- 172.21.15.78:0/3966369799 shutdown_connections 2024-05-21T15:18:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 --2- 172.21.15.78:0/3966369799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d0410c330 0x7f1d0410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 --2- 172.21.15.78:0/3966369799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 0x7f1d04101eb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 --2- 172.21.15.78:0/3966369799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d041079d0 0x7f1d04101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 -- 172.21.15.78:0/3966369799 >> 172.21.15.78:0/3966369799 conn(0x7f1d040770b0 msgr2=0x7f1d040774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 -- 172.21.15.78:0/3966369799 shutdown_connections 2024-05-21T15:18:22.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.078+0000 7f1d09008640 1 -- 172.21.15.78:0/3966369799 wait complete. 2024-05-21T15:18:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d09008640 1 Processor -- start 2024-05-21T15:18:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d09008640 1 -- start start 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d09008640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 0x7f1d0419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d09008640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d041079d0 0x7f1d0419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d09008640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d0410c330 0x7f1d0419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d09008640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04111d30 con 0x7f1d04101aa0 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d09008640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04111bb0 con 0x7f1d0410c330 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d09008640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04111eb0 con 0x7f1d041079d0 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d02d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 0x7f1d0419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d0359e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d0410c330 0x7f1d0419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d02d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 0x7f1d0419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49882/0 (socket says 172.21.15.78:49882) 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d02d9d640 1 -- 172.21.15.78:0/3497836238 learned_addr learned my addr 172.21.15.78:0/3497836238 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d0359e640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d0410c330 0x7f1d0419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:53472/0 (socket says 172.21.15.78:53472) 2024-05-21T15:18:22.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d041079d0 0x7f1d0419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d02d9d640 1 -- 172.21.15.78:0/3497836238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d041079d0 msgr2=0x7f1d0419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d041079d0 0x7f1d0419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d02d9d640 1 -- 172.21.15.78:0/3497836238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d0410c330 msgr2=0x7f1d0419b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d0410c330 0x7f1d0419b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d02d9d640 1 -- 172.21.15.78:0/3497836238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d041a5970 con 0x7f1d04101aa0 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.079+0000 7f1d0359e640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d0410c330 0x7f1d0419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.080+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 0x7f1d0419a630 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e9f0 tx=0x7f1cf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.080+0000 7f1d00d99640 1 -- 172.21.15.78:0/3497836238 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400ccc0 con 0x7f1d04101aa0 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.080+0000 7f1d09008640 1 -- 172.21.15.78:0/3497836238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d041a5c60 con 0x7f1d04101aa0 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.080+0000 7f1d00d99640 1 -- 172.21.15.78:0/3497836238 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf4005500 con 0x7f1d04101aa0 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.080+0000 7f1d00d99640 1 -- 172.21.15.78:0/3497836238 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4010690 con 0x7f1d04101aa0 2024-05-21T15:18:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.080+0000 7f1d09008640 1 -- 172.21.15.78:0/3497836238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d041a6130 con 0x7f1d04101aa0 2024-05-21T15:18:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.082+0000 7f1d00d99640 1 -- 172.21.15.78:0/3497836238 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1cf40046c0 con 0x7f1d04101aa0 2024-05-21T15:18:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.082+0000 7f1d0ff82740 1 Processor -- start 2024-05-21T15:18:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0ff82740 1 -- start start 2024-05-21T15:18:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d00d99640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1cf00a2d00 0x7f1cf00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2c0390 0x55f50d65ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d65f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d65f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f50d5c4d00 con 0x55f50d2c0390 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f50d56c4c0 con 0x55f50d29f6b0 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f50d5c0cc0 con 0x55f50d2974d0 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0359e640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1cf00a2d00 0x7f1cf00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d02d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2c0390 0x55f50d65ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1cfbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d65f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d00d99640 1 -- 172.21.15.78:0/3497836238 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f1cf40cf820 con 0x7f1d04101aa0 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d02d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2c0390 0x55f50d65ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49886/0 (socket says 172.21.15.78:49886) 2024-05-21T15:18:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d02d9d640 1 -- 172.21.15.78:0/2865183785 learned_addr learned my addr 172.21.15.78:0/2865183785 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.083+0000 7f1d0359e640 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d65f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1d02d9d640 1 -- 172.21.15.78:0/2865183785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 msgr2=0x55f50d65f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d65f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1d02d9d640 1 -- 172.21.15.78:0/2865183785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 msgr2=0x55f50d65f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d65f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1d02d9d640 1 -- 172.21.15.78:0/2865183785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50d3a27d0 con 0x55f50d2c0390 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d65f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1d0359e640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1cf00a2d00 0x7f1cf00a5210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009fd0 tx=0x7f1cfc009e80 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2c0390 0x55f50d65ed30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4098140 tx=0x7f1cf401c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4dc954e282cd941 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1cf97fa640 1 -- 172.21.15.78:0/2865183785 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf409b820 con 0x55f50d2c0390 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1cf97fa640 1 -- 172.21.15.78:0/2865183785 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf409c3e0 con 0x55f50d2c0390 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.084+0000 7f1cf97fa640 1 -- 172.21.15.78:0/2865183785 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40ae820 con 0x55f50d2c0390 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 -- 172.21.15.78:0/2865183785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2c0390 msgr2=0x55f50d65ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2c0390 0x55f50d65ed30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4098140 tx=0x7f1cf401c590 comp rx=0 tx=0).stop 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 -- 172.21.15.78:0/2865183785 shutdown_connections 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d65f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d65f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/2865183785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2c0390 0x55f50d65ed30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 -- 172.21.15.78:0/2865183785 >> 172.21.15.78:0/2865183785 conn(0x55f50d2b44e0 msgr2=0x55f50d65b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 -- 172.21.15.78:0/2865183785 shutdown_connections 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 -- 172.21.15.78:0/2865183785 wait complete. 2024-05-21T15:18:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 -- start start 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.085+0000 7f1d0ff82740 1 Processor -- start 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d66ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d66b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2c0390 0x55f50d6692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f50d66bfc0 con 0x55f50d2974d0 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f50d57fb90 con 0x55f50d29f6b0 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f50d66c140 con 0x55f50d2c0390 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0ff82740 10 client.0 _ll_register_callbacks cb 0x55f50d5bbda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d66ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d66ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49888/0 (socket says 172.21.15.78:49888) 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 -- 172.21.15.78:0/3749707981 learned_addr learned my addr 172.21.15.78:0/3749707981 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0359e640 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2c0390 0x55f50d6692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d66b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 -- 172.21.15.78:0/3749707981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2c0390 msgr2=0x55f50d6692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2c0390 0x55f50d6692b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 -- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 msgr2=0x55f50d66b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d66b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 -- 172.21.15.78:0/3749707981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50d2d83f0 con 0x55f50d2974d0 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d66b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d66ae30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40cd510 tx=0x7f1cf409bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3026c331c11a47b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1cea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1cea7fc640 1 -- 172.21.15.78:0/3749707981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40aea00 con 0x55f50d2974d0 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1cea7fc640 1 -- 172.21.15.78:0/3749707981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f1cf409bf40 con 0x55f50d2974d0 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.086+0000 7f1d0ff82740 1 -- 172.21.15.78:0/3749707981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf409c3e0 con 0x55f50d2974d0 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1cea7fc640 1 -- 172.21.15.78:0/3749707981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1cf40aebe0 con 0x55f50d2974d0 2024-05-21T15:18:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1d0ff82740 10 client.16272 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1d0ff82740 1 -- 172.21.15.78:0/3749707981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f50d2980f0 con 0x55f50d2974d0 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1cea7fc640 1 -- 172.21.15.78:0/3749707981 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf409c3e0 con 0x55f50d2974d0 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1cea7fc640 1 -- 172.21.15.78:0/3749707981 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f1cf40a44e0 con 0x55f50d2974d0 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1d0ff82740 10 client.16272 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1d0ff82740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1d0ff82740 20 client.16272 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f50d27ef60 0x55f50d67b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1d0ff82740 4 client.16272 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1d0ff82740 1 -- 172.21.15.78:0/3749707981 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f50d5d2430 con 0x55f50d27ef60 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.087+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f50d27ef60 0x55f50d67b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1cea7fc640 10 client.16272 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f50d27ef60 0x55f50d67b9b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48486905f2d4b91 server_cookie=9e0cabcfab9787ca in_seq=0 out_seq=0 2024-05-21T15:18:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1cea7fc640 1 -- 172.21.15.78:0/3749707981 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f50d5d2430 con 0x55f50d27ef60 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1cea7fc640 10 client.16272 handle_command_reply: tid=0 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1d0ff82740 1 client.16272 shutdown 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1d0ff82740 1 -- 172.21.15.78:0/3749707981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 msgr2=0x55f50d66ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d66ae30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40cd510 tx=0x7f1cf409bce0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1d0ff82740 1 -- 172.21.15.78:0/3749707981 shutdown_connections 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f50d27ef60 0x55f50d67b9b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d66b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2c0390 0x55f50d6692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1cea7fc640 0 client.16272 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.088+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/3749707981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d66ae30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.089+0000 7f1d0ff82740 1 -- 172.21.15.78:0/3749707981 >> 172.21.15.78:0/3749707981 conn(0x55f50d2b44e0 msgr2=0x55f50d660a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.089+0000 7f1d0ff82740 1 -- 172.21.15.78:0/3749707981 shutdown_connections 2024-05-21T15:18:22.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.089+0000 7f1d0ff82740 1 -- 172.21.15.78:0/3749707981 wait complete. 2024-05-21T15:18:22.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.089+0000 7f1d0ff82740 20 client.16272 trim_cache size 0 max 16384 2024-05-21T15:18:22.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.106+0000 7f1d0ff82740 1 Processor -- start 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.106+0000 7f1d0ff82740 1 -- start start 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.106+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d27ef60 0x55f50d661200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.106+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d662ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d662ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041aeb70 con 0x55f50d2974d0 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041b07a0 con 0x55f50d29f6b0 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041af240 con 0x55f50d27ef60 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d662ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d662ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49896/0 (socket says 172.21.15.78:49896) 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 -- 172.21.15.78:0/1260917783 learned_addr learned my addr 172.21.15.78:0/1260917783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1d0359e640 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d662ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d27ef60 0x55f50d661200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 -- 172.21.15.78:0/1260917783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d27ef60 msgr2=0x55f50d661200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d27ef60 0x55f50d661200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 -- 172.21.15.78:0/1260917783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 msgr2=0x55f50d662ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d662ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 -- 172.21.15.78:0/1260917783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50d66c0f0 con 0x55f50d2974d0 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1d0359e640 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d662ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d662ab0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1d0406b480 tx=0x7f1cec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4c272d5a136239 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cf8ff9640 1 -- 172.21.15.78:0/1260917783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec035040 con 0x55f50d2974d0 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cf8ff9640 1 -- 172.21.15.78:0/1260917783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cec04a070 con 0x55f50d2974d0 2024-05-21T15:18:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.107+0000 7f1cf8ff9640 1 -- 172.21.15.78:0/1260917783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec004920 con 0x55f50d2974d0 2024-05-21T15:18:22.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1260917783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 msgr2=0x55f50d662ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d662ab0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1d0406b480 tx=0x7f1cec000bc0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1260917783 shutdown_connections 2024-05-21T15:18:22.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d29f6b0 0x55f50d662ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d2974d0 0x55f50d662ab0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1260917783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d27ef60 0x55f50d661200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1260917783 >> 172.21.15.78:0/1260917783 conn(0x55f50d2b44e0 msgr2=0x55f50d65eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1260917783 shutdown_connections 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.108+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1260917783 wait complete. 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 1 -- start start 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 1 Processor -- start 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d27ef60 0x55f50d67d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d67dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d29f6b0 0x55f50d67e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041b0010 con 0x55f50d29f6b0 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041af910 con 0x55f50d27ef60 2024-05-21T15:18:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041acb50 con 0x55f50d2974d0 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0ff82740 10 client.0 _ll_register_callbacks cb 0x7f1cfc0139d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d29f6b0 0x55f50d67e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d29f6b0 0x55f50d67e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49910/0 (socket says 172.21.15.78:49910) 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0359e640 1 -- 172.21.15.78:0/1479136226 learned_addr learned my addr 172.21.15.78:0/1479136226 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d67dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d27ef60 0x55f50d67d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0359e640 1 -- 172.21.15.78:0/1479136226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 msgr2=0x55f50d67dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0359e640 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d67dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0359e640 1 -- 172.21.15.78:0/1479136226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d27ef60 msgr2=0x55f50d67d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0359e640 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d27ef60 0x55f50d67d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1d0359e640 1 -- 172.21.15.78:0/1479136226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50d6e1bb0 con 0x55f50d29f6b0 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.109+0000 7f1cfbfff640 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d67dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0359e640 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d29f6b0 0x55f50d67e1b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc005fd0 tx=0x7f1cfc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23761745ae0bf243 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1cea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1cea7fc640 1 -- 172.21.15.78:0/1479136226 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc021610 con 0x55f50d29f6b0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1cea7fc640 1 -- 172.21.15.78:0/1479136226 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f1cfc01c040 con 0x55f50d29f6b0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1479136226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cec04a070 con 0x55f50d29f6b0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1cea7fc640 1 -- 172.21.15.78:0/1479136226 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1cfc0217b0 con 0x55f50d29f6b0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 10 client.16278 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1479136226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f50d6e2e00 con 0x55f50d29f6b0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1cea7fc640 1 -- 172.21.15.78:0/1479136226 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cec04a070 con 0x55f50d29f6b0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1cea7fc640 1 -- 172.21.15.78:0/1479136226 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f1cfc020070 con 0x55f50d29f6b0 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 20 client.16278 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f50d2c0390 0x55f50d6e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 4 client.16278 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.110+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1479136226 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f50d5cc780 con 0x55f50d2c0390 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f50d2c0390 0x55f50d6e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1d02d9d640 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f50d2c0390 0x55f50d6e56b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e6142290d2e55e server_cookie=57cc8caf0d1f983e in_seq=0 out_seq=0 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1cea7fc640 10 client.16278 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1cea7fc640 1 -- 172.21.15.78:0/1479136226 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f50d5cc780 con 0x55f50d2c0390 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1cea7fc640 10 client.16278 handle_command_reply: tid=0 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1d0ff82740 1 client.16278 shutdown 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1479136226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d29f6b0 msgr2=0x55f50d67e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d29f6b0 0x55f50d67e1b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc005fd0 tx=0x7f1cfc01d000 comp rx=0 tx=0).stop 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1479136226 shutdown_connections 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.111+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f50d2c0390 0x55f50d6e56b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f50d29f6b0 0x55f50d67e1b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f50d2974d0 0x55f50d67dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d0ff82740 1 --2- 172.21.15.78:0/1479136226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f50d27ef60 0x55f50d67d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1cea7fc640 0 client.16278 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1479136226 >> 172.21.15.78:0/1479136226 conn(0x55f50d2b44e0 msgr2=0x55f50d660f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1479136226 shutdown_connections 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d0ff82740 1 -- 172.21.15.78:0/1479136226 wait complete. 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d0ff82740 20 client.16278 trim_cache size 0 max 16384 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:18:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:18:22.122 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:18:22.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 -- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1cf00a2d00 msgr2=0x7f1cf00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1cf00a2d00 0x7f1cf00a5210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009fd0 tx=0x7f1cfc009e80 comp rx=0 tx=0).stop 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 -- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 msgr2=0x7f1d0419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 0x7f1d0419a630 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e9f0 tx=0x7f1cf400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 -- 172.21.15.78:0/3497836238 shutdown_connections 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d0410c330 0x7f1d0419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1cf00a2d00 0x7f1cf00a5210 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d041079d0 0x7f1d0419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 --2- 172.21.15.78:0/3497836238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d04101aa0 0x7f1d0419a630 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 -- 172.21.15.78:0/3497836238 >> 172.21.15.78:0/3497836238 conn(0x7f1d040770b0 msgr2=0x7f1d04103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 -- 172.21.15.78:0/3497836238 shutdown_connections 2024-05-21T15:18:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.112+0000 7f1d09008640 1 -- 172.21.15.78:0/3497836238 wait complete. 2024-05-21T15:18:22.126 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:18:22.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:18:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.235+0000 7fe6f296c640 1 -- 172.21.15.78:0/657698538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 msgr2=0x7fe6ec101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.235+0000 7fe6f296c640 1 --2- 172.21.15.78:0/657698538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 0x7fe6ec101eb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009900 tx=0x7fe6e002a410 comp rx=0 tx=0).stop 2024-05-21T15:18:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.235+0000 7fe6f296c640 1 -- 172.21.15.78:0/657698538 shutdown_connections 2024-05-21T15:18:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.235+0000 7fe6f296c640 1 --2- 172.21.15.78:0/657698538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec10c330 0x7fe6ec10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.236+0000 7fe6f296c640 1 --2- 172.21.15.78:0/657698538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 0x7fe6ec101eb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.236+0000 7fe6f296c640 1 --2- 172.21.15.78:0/657698538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6ec1079d0 0x7fe6ec101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.236+0000 7fe6f296c640 1 -- 172.21.15.78:0/657698538 >> 172.21.15.78:0/657698538 conn(0x7fe6ec0770c0 msgr2=0x7fe6ec077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.236+0000 7fe6f296c640 1 -- 172.21.15.78:0/657698538 shutdown_connections 2024-05-21T15:18:22.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.236+0000 7fe6f296c640 1 -- 172.21.15.78:0/657698538 wait complete. 2024-05-21T15:18:22.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.236+0000 7fe6f296c640 1 Processor -- start 2024-05-21T15:18:22.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.236+0000 7fe6f296c640 1 -- start start 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6f296c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 0x7fe6ec198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6f296c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6ec1079d0 0x7fe6ec198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6f296c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec10c330 0x7fe6ec19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6f296c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec111ce0 con 0x7fe6ec101aa0 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6f296c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec111b60 con 0x7fe6ec1079d0 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6f296c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec111e60 con 0x7fe6ec10c330 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 0x7fe6ec198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6eb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6ec1079d0 0x7fe6ec198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 0x7fe6ec198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49932/0 (socket says 172.21.15.78:49932) 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 -- 172.21.15.78:0/2055328649 learned_addr learned my addr 172.21.15.78:0/2055328649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6f0f09640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec10c330 0x7fe6ec19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 -- 172.21.15.78:0/2055328649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec10c330 msgr2=0x7fe6ec19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec10c330 0x7fe6ec19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 -- 172.21.15.78:0/2055328649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6ec1079d0 msgr2=0x7fe6ec198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6ec1079d0 0x7fe6ec198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 -- 172.21.15.78:0/2055328649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec19d560 con 0x7fe6ec101aa0 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6f0f09640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec10c330 0x7fe6ec19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6eb7fe640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6ec1079d0 0x7fe6ec198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6ebfff640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 0x7fe6ec198700 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e9a0 tx=0x7fe6dc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.237+0000 7fe6e97fa640 1 -- 172.21.15.78:0/2055328649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00cd80 con 0x7fe6ec101aa0 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.238+0000 7fe6e97fa640 1 -- 172.21.15.78:0/2055328649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6dc0035f0 con 0x7fe6ec101aa0 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.238+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec19d850 con 0x7fe6ec101aa0 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.238+0000 7fe6e97fa640 1 -- 172.21.15.78:0/2055328649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc010670 con 0x7fe6ec101aa0 2024-05-21T15:18:22.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.238+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec1a51d0 con 0x7fe6ec101aa0 2024-05-21T15:18:22.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.239+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ec102c80 con 0x7fe6ec101aa0 2024-05-21T15:18:22.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.241+0000 7fe6e97fa640 1 -- 172.21.15.78:0/2055328649 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe6dc0037a0 con 0x7fe6ec101aa0 2024-05-21T15:18:22.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.242+0000 7fe6e97fa640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe6d40a2d00 0x7fe6d40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.242+0000 7fe6eb7fe640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe6d40a2d00 0x7fe6d40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.242+0000 7fe6e97fa640 1 -- 172.21.15.78:0/2055328649 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe6dc0cf8e0 con 0x7fe6ec101aa0 2024-05-21T15:18:22.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.242+0000 7fe6eb7fe640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe6d40a2d00 0x7fe6d40a5210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0005940 tx=0x7fe6e00058d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.245+0000 7fe6e97fa640 1 -- 172.21.15.78:0/2055328649 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe6dc09c530 con 0x7fe6ec101aa0 2024-05-21T15:18:22.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.469+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6ec069f50 con 0x7fe6ec101aa0 2024-05-21T15:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.470+0000 7fe6e97fa640 1 -- 172.21.15.78:0/2055328649 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11933 (secure 0 0 0) 0x7fe6dc09be30 con 0x7fe6ec101aa0 2024-05-21T15:18:22.473 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:18:22.474 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:18:03.344495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-05-21T15:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe6d40a2d00 msgr2=0x7fe6d40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe6d40a2d00 0x7fe6d40a5210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0005940 tx=0x7fe6e00058d0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 msgr2=0x7fe6ec198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 0x7fe6ec198700 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e9a0 tx=0x7fe6dc00ee60 comp rx=0 tx=0).stop 2024-05-21T15:18:22.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 shutdown_connections 2024-05-21T15:18:22.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6ec1079d0 0x7fe6ec198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe6d40a2d00 0x7fe6d40a5210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec10c330 0x7fe6ec19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 --2- 172.21.15.78:0/2055328649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6ec101aa0 0x7fe6ec198700 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 >> 172.21.15.78:0/2055328649 conn(0x7fe6ec0770c0 msgr2=0x7fe6ec103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 shutdown_connections 2024-05-21T15:18:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.472+0000 7fe6f296c640 1 -- 172.21.15.78:0/2055328649 wait complete. 2024-05-21T15:18:22.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:18:22.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:22 smithi078 ceph-mon[31997]: pgmap v71: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 706 MiB/s wr, 9.37k op/s 2024-05-21T15:18:22.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 -- 172.21.15.78:0/497570522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0001079d0 msgr2=0x7fd000101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 --2- 172.21.15.78:0/497570522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0001079d0 0x7fd000101560 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009900 tx=0x7fcffc02a410 comp rx=0 tx=0).stop 2024-05-21T15:18:22.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 -- 172.21.15.78:0/497570522 shutdown_connections 2024-05-21T15:18:22.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 --2- 172.21.15.78:0/497570522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd00010c330 0x7fd00010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 --2- 172.21.15.78:0/497570522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd000101aa0 0x7fd000101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 --2- 172.21.15.78:0/497570522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0001079d0 0x7fd000101560 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 -- 172.21.15.78:0/497570522 >> 172.21.15.78:0/497570522 conn(0x7fd0000770b0 msgr2=0x7fd0000774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 -- 172.21.15.78:0/497570522 shutdown_connections 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.598+0000 7fd0085aa640 1 -- 172.21.15.78:0/497570522 wait complete. 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd0085aa640 1 Processor -- start 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd0085aa640 1 -- start start 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd0085aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd000101aa0 0x7fd0001986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd0085aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0001079d0 0x7fd000198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd0085aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd00010c330 0x7fd00019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd0085aa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd000111ce0 con 0x7fd000101aa0 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd0085aa640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd000111b60 con 0x7fd0001079d0 2024-05-21T15:18:22.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd0085aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd000111e60 con 0x7fd00010c330 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd006346640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd000101aa0 0x7fd0001986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd006346640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd000101aa0 0x7fd0001986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49956/0 (socket says 172.21.15.78:49956) 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd006346640 1 -- 172.21.15.78:0/769092268 learned_addr learned my addr 172.21.15.78:0/769092268 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.599+0000 7fd005b45640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0001079d0 0x7fd000198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd006b47640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd00010c330 0x7fd00019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd006346640 1 -- 172.21.15.78:0/769092268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd00010c330 msgr2=0x7fd00019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd006346640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd00010c330 0x7fd00019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd006346640 1 -- 172.21.15.78:0/769092268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0001079d0 msgr2=0x7fd000198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd006346640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0001079d0 0x7fd000198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd006346640 1 -- 172.21.15.78:0/769092268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00019d500 con 0x7fd000101aa0 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd006346640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd000101aa0 0x7fd0001986a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005940 tx=0x7fcffc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fcff77fe640 1 -- 172.21.15.78:0/769092268 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc03f030 con 0x7fd000101aa0 2024-05-21T15:18:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fcff77fe640 1 -- 172.21.15.78:0/769092268 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcffc0044c0 con 0x7fd000101aa0 2024-05-21T15:18:22.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd0085aa640 1 -- 172.21.15.78:0/769092268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00019d790 con 0x7fd000101aa0 2024-05-21T15:18:22.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fcff77fe640 1 -- 172.21.15.78:0/769092268 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc035070 con 0x7fd000101aa0 2024-05-21T15:18:22.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.600+0000 7fd0085aa640 1 -- 172.21.15.78:0/769092268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00019dc80 con 0x7fd000101aa0 2024-05-21T15:18:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fcff77fe640 1 -- 172.21.15.78:0/769092268 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcffc042030 con 0x7fd000101aa0 2024-05-21T15:18:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd00f451740 1 Processor -- start 2024-05-21T15:18:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd00f451740 1 -- start start 2024-05-21T15:18:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd00f451740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd00f451740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab1b4d0 0x56290aee3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd00f451740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aee37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd00f451740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56290ae48c40 con 0x56290ab1b4d0 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd00f451740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56290adf0580 con 0x56290ab236b0 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd00f451740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56290ae44c00 con 0x56290ab44390 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd006346640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab1b4d0 0x56290aee3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd006346640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab1b4d0 0x56290aee3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49964/0 (socket says 172.21.15.78:49964) 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd006346640 1 -- 172.21.15.78:0/1250236556 learned_addr learned my addr 172.21.15.78:0/1250236556 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.603+0000 7fd006b47640 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fcff77fe640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcfec0a2d00 0x7fcfec0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd005b45640 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aee37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006b47640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcfec0a2d00 0x7fcfec0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fcff77fe640 1 -- 172.21.15.78:0/769092268 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcffc0ef960 con 0x7fd000101aa0 2024-05-21T15:18:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006b47640 1 -- 172.21.15.78:0/1250236556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 msgr2=0x56290aee37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006b47640 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aee37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006b47640 1 -- 172.21.15.78:0/1250236556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab1b4d0 msgr2=0x56290aee3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006b47640 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab1b4d0 0x56290aee3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006b47640 1 -- 172.21.15.78:0/1250236556 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56290ab5c3f0 con 0x56290ab44390 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006346640 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab1b4d0 0x56290aee3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006b47640 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee2d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcff8005fd0 tx=0x7fcff8012e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aeacf04afd0262e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fd006b47640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcfec0a2d00 0x7fcfec0a5210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcff801b000 tx=0x7fcff801f000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fcff4bfe640 1 -- 172.21.15.78:0/1250236556 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8003970 con 0x56290ab44390 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fcff4bfe640 1 -- 172.21.15.78:0/1250236556 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff801e820 con 0x56290ab44390 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.604+0000 7fcff4bfe640 1 -- 172.21.15.78:0/1250236556 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8004250 con 0x56290ab44390 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 -- 172.21.15.78:0/1250236556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 msgr2=0x56290aee2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee2d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcff8005fd0 tx=0x7fcff8012e40 comp rx=0 tx=0).stop 2024-05-21T15:18:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 -- 172.21.15.78:0/1250236556 shutdown_connections 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aee37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab1b4d0 0x56290aee3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 --2- 172.21.15.78:0/1250236556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee2d20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 -- 172.21.15.78:0/1250236556 >> 172.21.15.78:0/1250236556 conn(0x56290ab384e0 msgr2=0x56290aedf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 -- 172.21.15.78:0/1250236556 shutdown_connections 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.605+0000 7fd00f451740 1 -- 172.21.15.78:0/1250236556 wait complete. 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 1 -- start start 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 1 Processor -- start 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab1b4d0 0x56290aeed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aeed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aef3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56290aeeffb0 con 0x56290ab44390 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56290ae03c40 con 0x56290ab236b0 2024-05-21T15:18:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56290aef0130 con 0x56290ab1b4d0 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd00f451740 10 client.0 _ll_register_callbacks cb 0x56290ab5bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd006b47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aef3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd005b45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aeed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd006b47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aef3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49968/0 (socket says 172.21.15.78:49968) 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd006b47640 1 -- 172.21.15.78:0/2558480222 learned_addr learned my addr 172.21.15.78:0/2558480222 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.606+0000 7fd006346640 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab1b4d0 0x56290aeed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd006b47640 1 -- 172.21.15.78:0/2558480222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab1b4d0 msgr2=0x56290aeed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd006b47640 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab1b4d0 0x56290aeed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd006b47640 1 -- 172.21.15.78:0/2558480222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 msgr2=0x56290aeed890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd006b47640 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aeed890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd006b47640 1 -- 172.21.15.78:0/2558480222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290ac267d0 con 0x56290ab44390 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd005b45640 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aeed890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd006b47640 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aef3370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcff801ef50 tx=0x7fcff8012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac75daaa291acae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fcfe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/2558480222 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff802d020 con 0x56290ab44390 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/2558480222 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcff8003e00 con 0x56290ab44390 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd00f451740 1 -- 172.21.15.78:0/2558480222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff801e820 con 0x56290ab44390 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/2558480222 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcff8038050 con 0x56290ab44390 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd00f451740 10 client.16296 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fd00f451740 1 -- 172.21.15.78:0/2558480222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56290aefc600 con 0x56290ab44390 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.607+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/2558480222 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff801e820 con 0x56290ab44390 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/2558480222 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fcff801e070 con 0x56290ab44390 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fd00f451740 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fd00f451740 10 client.16296 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fd00f451740 20 client.16296 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fd00f451740 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x56290ab02f60 0x56290aeffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fd00f451740 4 client.16296 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fd00f451740 1 -- 172.21.15.78:0/2558480222 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56290ae56370 con 0x56290ab02f60 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fd006346640 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x56290ab02f60 0x56290aeffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fcfe9ffb640 10 client.16296 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.608+0000 7fd006346640 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x56290ab02f60 0x56290aeffad0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3ab255843950454f server_cookie=f384e0c9e3742451 in_seq=0 out_seq=0 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/2558480222 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56290ae56370 con 0x56290ab02f60 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fcfe9ffb640 10 client.16296 handle_command_reply: tid=0 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 client.16296 shutdown 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 -- 172.21.15.78:0/2558480222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 msgr2=0x56290aef3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aef3370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcff801ef50 tx=0x7fcff8012630 comp rx=0 tx=0).stop 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 -- 172.21.15.78:0/2558480222 shutdown_connections 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x56290ab02f60 0x56290aeffad0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab236b0 0x56290aeed890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aef3370 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 --2- 172.21.15.78:0/2558480222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab1b4d0 0x56290aeed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fcfe9ffb640 0 client.16296 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 -- 172.21.15.78:0/2558480222 >> 172.21.15.78:0/2558480222 conn(0x56290ab384e0 msgr2=0x56290aee4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 -- 172.21.15.78:0/2558480222 shutdown_connections 2024-05-21T15:18:22.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 1 -- 172.21.15.78:0/2558480222 wait complete. 2024-05-21T15:18:22.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.609+0000 7fd00f451740 20 client.16296 trim_cache size 0 max 16384 2024-05-21T15:18:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd00f451740 1 Processor -- start 2024-05-21T15:18:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd00f451740 1 -- start start 2024-05-21T15:18:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd00f451740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290aef3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd00f451740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab236b0 0x56290aeef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd00f451740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aeebff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd00f451740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001ac8b0 con 0x56290ab44390 2024-05-21T15:18:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd00f451740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001ae4e0 con 0x56290ab1b4d0 2024-05-21T15:18:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd00f451740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001acf80 con 0x56290ab236b0 2024-05-21T15:18:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd005b45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab236b0 0x56290aeef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd006b47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aeebff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd005b45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab236b0 0x56290aeef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33570/0 (socket says 172.21.15.78:33570) 2024-05-21T15:18:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd005b45640 1 -- 172.21.15.78:0/300326246 learned_addr learned my addr 172.21.15.78:0/300326246 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd006346640 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290aef3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd006b47640 1 -- 172.21.15.78:0/300326246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab236b0 msgr2=0x56290aeef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd006b47640 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab236b0 0x56290aeef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd006b47640 1 -- 172.21.15.78:0/300326246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 msgr2=0x56290aef3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd006b47640 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290aef3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd006b47640 1 -- 172.21.15.78:0/300326246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56290aee2ab0 con 0x56290ab44390 2024-05-21T15:18:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.624+0000 7fd006346640 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290aef3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:18:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.625+0000 7fd006b47640 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aeebff0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009560 tx=0x7fcff8012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fe66f4923c3fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.625+0000 7fcfebfff640 1 -- 172.21.15.78:0/300326246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff802d070 con 0x56290ab44390 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.625+0000 7fcfebfff640 1 -- 172.21.15.78:0/300326246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff801e820 con 0x56290ab44390 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.625+0000 7fcfebfff640 1 -- 172.21.15.78:0/300326246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8003c00 con 0x56290ab44390 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- 172.21.15.78:0/300326246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 msgr2=0x56290aeebff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aeebff0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009560 tx=0x7fcff8012520 comp rx=0 tx=0).stop 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- 172.21.15.78:0/300326246 shutdown_connections 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab44390 0x56290aeebff0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab236b0 0x56290aeef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 --2- 172.21.15.78:0/300326246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290aef3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- 172.21.15.78:0/300326246 >> 172.21.15.78:0/300326246 conn(0x56290ab384e0 msgr2=0x56290aeeb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- 172.21.15.78:0/300326246 shutdown_connections 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- 172.21.15.78:0/300326246 wait complete. 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- start start 2024-05-21T15:18:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 Processor -- start 2024-05-21T15:18:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290af641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab236b0 0x56290af646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001add50 con 0x56290ab236b0 2024-05-21T15:18:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001ad650 con 0x56290ab1b4d0 2024-05-21T15:18:22.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0001aa890 con 0x56290ab44390 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.626+0000 7fd00f451740 10 client.0 _ll_register_callbacks cb 0x7fcffc0b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab236b0 0x56290af646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd006b47640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd006346640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290af641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab236b0 0x56290af646e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49980/0 (socket says 172.21.15.78:49980) 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd006b47640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33576/0 (socket says 172.21.15.78:33576) 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 -- 172.21.15.78:0/726392929 learned_addr learned my addr 172.21.15.78:0/726392929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 -- 172.21.15.78:0/726392929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 msgr2=0x56290aee6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 -- 172.21.15.78:0/726392929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 msgr2=0x56290af641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290af641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 -- 172.21.15.78:0/726392929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290aee67b0 con 0x56290ab236b0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd006346640 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290af641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd005b45640 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab236b0 0x56290af646e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcff0012cb0 tx=0x7fcff000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa404b431d1bc631 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fcfe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/726392929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0023060 con 0x56290ab236b0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/726392929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcff0004250 con 0x56290ab236b0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd00f451740 1 -- 172.21.15.78:0/726392929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff801e820 con 0x56290ab236b0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/726392929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcff0012e60 con 0x56290ab236b0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd00f451740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.627+0000 7fd00f451740 1 -- 172.21.15.78:0/726392929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56290af6b890 con 0x56290ab236b0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/726392929 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff801e820 con 0x56290ab236b0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/726392929 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fcff0010430 con 0x56290ab236b0 2024-05-21T15:18:22.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fd00f451740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fd00f451740 10 client.16302 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fd00f451740 20 client.16302 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fd00f451740 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x56290aad83a0 0x56290af6e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fd00f451740 4 client.16302 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fd00f451740 1 -- 172.21.15.78:0/726392929 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56290ae506c0 con 0x56290aad83a0 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fd006346640 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x56290aad83a0 0x56290af6e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fcfe9ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fd006346640 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x56290aad83a0 0x56290af6e0b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=336e50c8513e010 server_cookie=af03934beb6b38ae in_seq=0 out_seq=0 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fcfe9ffb640 1 -- 172.21.15.78:0/726392929 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56290ae506c0 con 0x56290aad83a0 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.628+0000 7fcfe9ffb640 10 client.16302 handle_command_reply: tid=0 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 client.16302 shutdown 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 -- 172.21.15.78:0/726392929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab236b0 msgr2=0x56290af646e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab236b0 0x56290af646e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcff0012cb0 tx=0x7fcff000f790 comp rx=0 tx=0).stop 2024-05-21T15:18:22.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 -- 172.21.15.78:0/726392929 shutdown_connections 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x56290aad83a0 0x56290af6e0b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56290ab44390 0x56290aee6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56290ab236b0 0x56290af646e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fcfe9ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 --2- 172.21.15.78:0/726392929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56290ab1b4d0 0x56290af641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 -- 172.21.15.78:0/726392929 >> 172.21.15.78:0/726392929 conn(0x56290ab384e0 msgr2=0x56290aeec070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 -- 172.21.15.78:0/726392929 shutdown_connections 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 1 -- 172.21.15.78:0/726392929 wait complete. 2024-05-21T15:18:22.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd00f451740 20 client.16302 trim_cache size 0 max 16384 2024-05-21T15:18:22.643 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:22.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 -- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcfec0a2d00 msgr2=0x7fcfec0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcfec0a2d00 0x7fcfec0a5210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcff801b000 tx=0x7fcff801f000 comp rx=0 tx=0).stop 2024-05-21T15:18:22.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 -- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd000101aa0 msgr2=0x7fd0001986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd000101aa0 0x7fd0001986a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005940 tx=0x7fcffc0040c0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 -- 172.21.15.78:0/769092268 shutdown_connections 2024-05-21T15:18:22.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0001079d0 0x7fd000198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcfec0a2d00 0x7fcfec0a5210 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd00010c330 0x7fd00019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 --2- 172.21.15.78:0/769092268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd000101aa0 0x7fd0001986a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 -- 172.21.15.78:0/769092268 >> 172.21.15.78:0/769092268 conn(0x7fd0000770b0 msgr2=0x7fd000103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.629+0000 7fd0085aa640 1 -- 172.21.15.78:0/769092268 shutdown_connections 2024-05-21T15:18:22.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.630+0000 7fd0085aa640 1 -- 172.21.15.78:0/769092268 wait complete. 2024-05-21T15:18:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:22 smithi084 ceph-mon[36334]: pgmap v71: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 706 MiB/s wr, 9.37k op/s 2024-05-21T15:18:22.647 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:18:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:22 smithi172 ceph-mon[36402]: pgmap v71: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 706 MiB/s wr, 9.37k op/s 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.758+0000 7f66f7d16640 1 -- 172.21.15.78:0/1211486327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f01063e0 msgr2=0x7f66f01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.758+0000 7f66f7d16640 1 --2- 172.21.15.78:0/1211486327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f01063e0 0x7f66f01067f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009900 tx=0x7f66e002a410 comp rx=0 tx=0).stop 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 -- 172.21.15.78:0/1211486327 shutdown_connections 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 --2- 172.21.15.78:0/1211486327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66f0106e10 0x7f66f01072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 --2- 172.21.15.78:0/1211486327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f01063e0 0x7f66f01067f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 --2- 172.21.15.78:0/1211486327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f0101990 0x7f66f0101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 -- 172.21.15.78:0/1211486327 >> 172.21.15.78:0/1211486327 conn(0x7f66f00770b0 msgr2=0x7f66f00774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 -- 172.21.15.78:0/1211486327 shutdown_connections 2024-05-21T15:18:22.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 -- 172.21.15.78:0/1211486327 wait complete. 2024-05-21T15:18:22.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 Processor -- start 2024-05-21T15:18:22.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.759+0000 7f66f7d16640 1 -- start start 2024-05-21T15:18:22.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f7d16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f0101990 0x7f66f019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f7d16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66f01063e0 0x7f66f01a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f7d16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f0106e10 0x7f66f01a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f7d16640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01193e0 con 0x7f66f0101990 2024-05-21T15:18:22.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f7d16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0119260 con 0x7f66f01063e0 2024-05-21T15:18:22.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f7d16640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0119560 con 0x7f66f0106e10 2024-05-21T15:18:22.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f0101990 0x7f66f019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f0101990 0x7f66f019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49998/0 (socket says 172.21.15.78:49998) 2024-05-21T15:18:22.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 -- 172.21.15.78:0/4014157768 learned_addr learned my addr 172.21.15.78:0/4014157768 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f52b1640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66f01063e0 0x7f66f01a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f62b3640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f0106e10 0x7f66f01a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 -- 172.21.15.78:0/4014157768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f0106e10 msgr2=0x7f66f01a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f0106e10 0x7f66f01a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 -- 172.21.15.78:0/4014157768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66f01063e0 msgr2=0x7f66f01a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66f01063e0 0x7f66f01a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 -- 172.21.15.78:0/4014157768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f01a4c00 con 0x7f66f0101990 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f0101990 0x7f66f019fda0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00ea60 tx=0x7f66ec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66e6ffd640 1 -- 172.21.15.78:0/4014157768 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00cde0 con 0x7f66f0101990 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66f7d16640 1 -- 172.21.15.78:0/4014157768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f01a4ef0 con 0x7f66f0101990 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66e6ffd640 1 -- 172.21.15.78:0/4014157768 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ec0035f0 con 0x7f66f0101990 2024-05-21T15:18:22.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.760+0000 7f66e6ffd640 1 -- 172.21.15.78:0/4014157768 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec010670 con 0x7f66f0101990 2024-05-21T15:18:22.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.761+0000 7f66f7d16640 1 -- 172.21.15.78:0/4014157768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f01ac870 con 0x7f66f0101990 2024-05-21T15:18:22.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.763+0000 7f66e6ffd640 1 -- 172.21.15.78:0/4014157768 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f66ec0037a0 con 0x7f66f0101990 2024-05-21T15:18:22.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.763+0000 7f66febb7740 1 Processor -- start 2024-05-21T15:18:22.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.763+0000 7f66febb7740 1 -- start start 2024-05-21T15:18:22.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66febb7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba7bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66febb7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6b44d0 0x5609dba7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66febb7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dba7c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66febb7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609db9e1c40 con 0x5609db6bc6b0 2024-05-21T15:18:22.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66febb7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5609db989580 con 0x5609db6dd390 2024-05-21T15:18:22.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66febb7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5609db9ddc00 con 0x5609db6b44d0 2024-05-21T15:18:22.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66e6ffd640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f66dc0a2d00 0x7f66dc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f5ab2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6b44d0 0x5609dba7c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f52b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dba7c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f5ab2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6b44d0 0x5609dba7c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33612/0 (socket says 172.21.15.78:33612) 2024-05-21T15:18:22.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f52b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dba7c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50004/0 (socket says 172.21.15.78:50004) 2024-05-21T15:18:22.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f5ab2640 1 -- 172.21.15.78:0/3280862625 learned_addr learned my addr 172.21.15.78:0/3280862625 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f62b3640 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba7bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f62b3640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f66dc0a2d00 0x7f66dc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66e6ffd640 1 -- 172.21.15.78:0/4014157768 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f66ec0cf760 con 0x7f66f0101990 2024-05-21T15:18:22.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f5ab2640 1 -- 172.21.15.78:0/3280862625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 msgr2=0x5609dba7bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba7bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f5ab2640 1 -- 172.21.15.78:0/3280862625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 msgr2=0x5609dba7c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dba7c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f5ab2640 1 -- 172.21.15.78:0/3280862625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609db6f53f0 con 0x5609db6b44d0 2024-05-21T15:18:22.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.764+0000 7f66f62b3640 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba7bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.765+0000 7f66f62b3640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f66dc0a2d00 0x7f66dc0a5210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5609dba78e90 tx=0x7f66e8009500 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.765+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6b44d0 0x5609dba7c260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0a4360 tx=0x7f66ec003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=835422ac84129d9e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.765+0000 7f66dbfff640 1 -- 172.21.15.78:0/3280862625 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec09bc20 con 0x5609db6b44d0 2024-05-21T15:18:22.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.765+0000 7f66dbfff640 1 -- 172.21.15.78:0/3280862625 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ec09ffd0 con 0x5609db6b44d0 2024-05-21T15:18:22.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.765+0000 7f66dbfff640 1 -- 172.21.15.78:0/3280862625 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0c7600 con 0x5609db6b44d0 2024-05-21T15:18:22.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 -- 172.21.15.78:0/3280862625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6b44d0 msgr2=0x5609dba7c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6b44d0 0x5609dba7c260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0a4360 tx=0x7f66ec003f30 comp rx=0 tx=0).stop 2024-05-21T15:18:22.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 -- 172.21.15.78:0/3280862625 shutdown_connections 2024-05-21T15:18:22.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dba7c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6b44d0 0x5609dba7c260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 --2- 172.21.15.78:0/3280862625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba7bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 -- 172.21.15.78:0/3280862625 >> 172.21.15.78:0/3280862625 conn(0x5609db6d14e0 msgr2=0x5609dba784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 -- 172.21.15.78:0/3280862625 shutdown_connections 2024-05-21T15:18:22.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 -- 172.21.15.78:0/3280862625 wait complete. 2024-05-21T15:18:22.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 -- start start 2024-05-21T15:18:22.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.766+0000 7f66febb7740 1 Processor -- start 2024-05-21T15:18:22.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66febb7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba87f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66febb7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6bc6b0 0x5609dba88480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66febb7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba86330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66febb7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609dba88fb0 con 0x5609db6b44d0 2024-05-21T15:18:22.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66febb7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5609db99cc40 con 0x5609db6dd390 2024-05-21T15:18:22.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66febb7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5609dba89130 con 0x5609db6bc6b0 2024-05-21T15:18:22.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66febb7740 10 client.0 _ll_register_callbacks cb 0x5609db6f4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:22.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba87f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba87f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50012/0 (socket says 172.21.15.78:50012) 2024-05-21T15:18:22.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f62b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba86330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 -- 172.21.15.78:0/3283736431 learned_addr learned my addr 172.21.15.78:0/3283736431 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f52b1640 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6bc6b0 0x5609dba88480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 -- 172.21.15.78:0/3283736431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6bc6b0 msgr2=0x5609dba88480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6bc6b0 0x5609dba88480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 -- 172.21.15.78:0/3283736431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 msgr2=0x5609dba86330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba86330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 -- 172.21.15.78:0/3283736431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609db7bf7d0 con 0x5609db6b44d0 2024-05-21T15:18:22.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f62b3640 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba86330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba87f40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00c760 tx=0x7f66ec0cce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434a20b3622a585c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.767+0000 7f66d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:22.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66d97fa640 1 -- 172.21.15.78:0/3283736431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec09bc20 con 0x5609db6b44d0 2024-05-21T15:18:22.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66d97fa640 1 -- 172.21.15.78:0/3283736431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f66ec098290 con 0x5609db6b44d0 2024-05-21T15:18:22.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 1 -- 172.21.15.78:0/3283736431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec09ffd0 con 0x5609db6b44d0 2024-05-21T15:18:22.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66d97fa640 1 -- 172.21.15.78:0/3283736431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66ec09b070 con 0x5609db6b44d0 2024-05-21T15:18:22.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 10 client.16314 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:22.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 1 -- 172.21.15.78:0/3283736431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609dba957e0 con 0x5609db6b44d0 2024-05-21T15:18:22.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66d97fa640 1 -- 172.21.15.78:0/3283736431 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66ec09ffd0 con 0x5609db6b44d0 2024-05-21T15:18:22.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66d97fa640 1 -- 172.21.15.78:0/3283736431 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f66ec09aae0 con 0x5609db6b44d0 2024-05-21T15:18:22.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 10 client.16314 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:22.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 10 client.16314 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:18:22.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 20 client.16314 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:22.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5609db69bf60 0x5609dba98b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 4 client.16314 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.768+0000 7f66febb7740 1 -- 172.21.15.78:0/3283736431 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609db9ef370 con 0x5609db69bf60 2024-05-21T15:18:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.769+0000 7f66f52b1640 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5609db69bf60 0x5609dba98b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.769+0000 7f66d97fa640 10 client.16314 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:18:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.769+0000 7f66f52b1640 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5609db69bf60 0x5609dba98b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4e3b6a5804991c48 server_cookie=6b7fb6006d0e9428 in_seq=0 out_seq=0 2024-05-21T15:18:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66d97fa640 1 -- 172.21.15.78:0/3283736431 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609db9ef370 con 0x5609db69bf60 2024-05-21T15:18:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66d97fa640 10 client.16314 handle_command_reply: tid=0 2024-05-21T15:18:22.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 client.16314 shutdown 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 -- 172.21.15.78:0/3283736431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 msgr2=0x5609dba87f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba87f40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00c760 tx=0x7f66ec0cce30 comp rx=0 tx=0).stop 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 -- 172.21.15.78:0/3283736431 shutdown_connections 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5609db69bf60 0x5609dba98b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6dd390 0x5609dba86330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66d97fa640 0 client.16314 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db6bc6b0 0x5609dba88480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 --2- 172.21.15.78:0/3283736431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba87f40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 -- 172.21.15.78:0/3283736431 >> 172.21.15.78:0/3283736431 conn(0x5609db6d14e0 msgr2=0x5609dba7da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 -- 172.21.15.78:0/3283736431 shutdown_connections 2024-05-21T15:18:22.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 1 -- 172.21.15.78:0/3283736431 wait complete. 2024-05-21T15:18:22.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.770+0000 7f66febb7740 20 client.16314 trim_cache size 0 max 16384 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.784+0000 7f66febb7740 1 Processor -- start 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.784+0000 7f66febb7740 1 -- start start 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.784+0000 7f66febb7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba7e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.784+0000 7f66febb7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba9bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.784+0000 7f66febb7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6bc6b0 0x5609dba9c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.784+0000 7f66febb7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01b52f0 con 0x5609db6b44d0 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.784+0000 7f66febb7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01b6f20 con 0x5609db6bc6b0 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.784+0000 7f66febb7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01b59c0 con 0x5609db69bf60 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba9bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba9bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50018/0 (socket says 172.21.15.78:50018) 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 -- 172.21.15.78:0/1153118851 learned_addr learned my addr 172.21.15.78:0/1153118851 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba7e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f62b3640 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6bc6b0 0x5609dba9c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 -- 172.21.15.78:0/1153118851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 msgr2=0x5609dba7e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba7e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 -- 172.21.15.78:0/1153118851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6bc6b0 msgr2=0x5609dba9c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6bc6b0 0x5609dba9c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 -- 172.21.15.78:0/1153118851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dba7ccb0 con 0x5609db6b44d0 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba7e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66f52b1640 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba9bb70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000c00 tx=0x7f66e0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e38ea0f5c3c04d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66db7fe640 1 -- 172.21.15.78:0/1153118851 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0035040 con 0x5609db6b44d0 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66db7fe640 1 -- 172.21.15.78:0/1153118851 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e004b070 con 0x5609db6b44d0 2024-05-21T15:18:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.785+0000 7f66db7fe640 1 -- 172.21.15.78:0/1153118851 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0004900 con 0x5609db6b44d0 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 -- 172.21.15.78:0/1153118851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 msgr2=0x5609dba9bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba9bb70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000c00 tx=0x7f66e0009560 comp rx=0 tx=0).stop 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 -- 172.21.15.78:0/1153118851 shutdown_connections 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6bc6b0 0x5609dba9c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6b44d0 0x5609dba9bb70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 --2- 172.21.15.78:0/1153118851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba7e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 -- 172.21.15.78:0/1153118851 >> 172.21.15.78:0/1153118851 conn(0x5609db6d14e0 msgr2=0x5609dba7d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 -- 172.21.15.78:0/1153118851 shutdown_connections 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 -- 172.21.15.78:0/1153118851 wait complete. 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.786+0000 7f66febb7740 1 -- start start 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66febb7740 1 Processor -- start 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66febb7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba9aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66febb7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6b44d0 0x5609dba9b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66febb7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dbaf8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66febb7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01b6790 con 0x5609db6bc6b0 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66febb7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01b6090 con 0x5609db6b44d0 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66febb7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01b32d0 con 0x5609db69bf60 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66febb7740 10 client.0 _ll_register_callbacks cb 0x7f66ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66f62b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dbaf8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66f62b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dbaf8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50020/0 (socket says 172.21.15.78:50020) 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66f62b3640 1 -- 172.21.15.78:0/918716850 learned_addr learned my addr 172.21.15.78:0/918716850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66f52b1640 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6b44d0 0x5609dba9b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.787+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba9aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66f52b1640 1 -- 172.21.15.78:0/918716850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 msgr2=0x5609dba9aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66f52b1640 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba9aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66f52b1640 1 -- 172.21.15.78:0/918716850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 msgr2=0x5609dbaf8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66f52b1640 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dbaf8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66f52b1640 1 -- 172.21.15.78:0/918716850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dbaf9220 con 0x5609db6b44d0 2024-05-21T15:18:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66f62b3640 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dbaf8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66f52b1640 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6b44d0 0x5609dba9b530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66e0048040 tx=0x7f66e0046b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1b4546fe581a47 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66d97fa640 1 -- 172.21.15.78:0/918716850 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0035040 con 0x5609db6b44d0 2024-05-21T15:18:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66febb7740 1 -- 172.21.15.78:0/918716850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e004b070 con 0x5609db6b44d0 2024-05-21T15:18:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66d97fa640 1 -- 172.21.15.78:0/918716850 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f66e0005240 con 0x5609db6b44d0 2024-05-21T15:18:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66d97fa640 1 -- 172.21.15.78:0/918716850 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66e0005440 con 0x5609db6b44d0 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66febb7740 10 client.25769 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.788+0000 7f66febb7740 1 -- 172.21.15.78:0/918716850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609dbb00520 con 0x5609db6b44d0 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66d97fa640 1 -- 172.21.15.78:0/918716850 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e004b070 con 0x5609db6b44d0 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66d97fa640 1 -- 172.21.15.78:0/918716850 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f66e00317e0 con 0x5609db6b44d0 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66febb7740 10 client.25769 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66febb7740 10 client.25769 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66febb7740 20 client.25769 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66febb7740 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5609db6dd390 0x5609dbb02c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66febb7740 4 client.25769 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66febb7740 1 -- 172.21.15.78:0/918716850 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609db9e96c0 con 0x5609db6dd390 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5609db6dd390 0x5609dbb02c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66d97fa640 10 client.25769 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.789+0000 7f66f5ab2640 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5609db6dd390 0x5609dbb02c00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90bd8107cfd0c453 server_cookie=3aea56cc72287c58 in_seq=0 out_seq=0 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66d97fa640 1 -- 172.21.15.78:0/918716850 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609db9e96c0 con 0x5609db6dd390 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66d97fa640 10 client.25769 handle_command_reply: tid=0 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 client.25769 shutdown 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 -- 172.21.15.78:0/918716850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6b44d0 msgr2=0x5609dba9b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6b44d0 0x5609dba9b530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66e0048040 tx=0x7f66e0046b30 comp rx=0 tx=0).stop 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 -- 172.21.15.78:0/918716850 shutdown_connections 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5609db6dd390 0x5609dbb02c00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5609db6b44d0 0x5609dba9b530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609db6bc6b0 0x5609dbaf8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 --2- 172.21.15.78:0/918716850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609db69bf60 0x5609dba9aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66d97fa640 0 client.25769 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:18:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 -- 172.21.15.78:0/918716850 >> 172.21.15.78:0/918716850 conn(0x5609db6d14e0 msgr2=0x5609dba84c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 -- 172.21.15.78:0/918716850 shutdown_connections 2024-05-21T15:18:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 1 -- 172.21.15.78:0/918716850 wait complete. 2024-05-21T15:18:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66febb7740 20 client.25769 trim_cache size 0 max 16384 2024-05-21T15:18:22.805 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:22.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 -- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f66dc0a2d00 msgr2=0x7f66dc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f66dc0a2d00 0x7f66dc0a5210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x5609dba78e90 tx=0x7f66e8009500 comp rx=0 tx=0).stop 2024-05-21T15:18:22.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 -- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f0101990 msgr2=0x7f66f019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f0101990 0x7f66f019fda0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00ea60 tx=0x7f66ec00ef20 comp rx=0 tx=0).stop 2024-05-21T15:18:22.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 -- 172.21.15.78:0/4014157768 shutdown_connections 2024-05-21T15:18:22.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66f01063e0 0x7f66f01a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f66dc0a2d00 0x7f66dc0a5210 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f0106e10 0x7f66f01a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.790+0000 7f66f7d16640 1 --2- 172.21.15.78:0/4014157768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66f0101990 0x7f66f019fda0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.791+0000 7f66f7d16640 1 -- 172.21.15.78:0/4014157768 >> 172.21.15.78:0/4014157768 conn(0x7f66f00770b0 msgr2=0x7f66f0103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.791+0000 7f66f7d16640 1 -- 172.21.15.78:0/4014157768 shutdown_connections 2024-05-21T15:18:22.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.791+0000 7f66f7d16640 1 -- 172.21.15.78:0/4014157768 wait complete. 2024-05-21T15:18:22.810 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:18:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 -- 172.21.15.78:0/1472448281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70107be0 msgr2=0x7fae70101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 --2- 172.21.15.78:0/1472448281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70107be0 0x7fae70101cb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae60009860 tx=0x7fae6002a340 comp rx=0 tx=0).stop 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 -- 172.21.15.78:0/1472448281 shutdown_connections 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 --2- 172.21.15.78:0/1472448281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70102860 0x7fae7010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 --2- 172.21.15.78:0/1472448281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae70108080 0x7fae701021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 --2- 172.21.15.78:0/1472448281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70107be0 0x7fae70101cb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 -- 172.21.15.78:0/1472448281 >> 172.21.15.78:0/1472448281 conn(0x7fae70076830 msgr2=0x7fae70076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 -- 172.21.15.78:0/1472448281 shutdown_connections 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 -- 172.21.15.78:0/1472448281 wait complete. 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.922+0000 7fae75206640 1 Processor -- start 2024-05-21T15:18:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae75206640 1 -- start start 2024-05-21T15:18:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae75206640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae70102860 0x7fae7019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae75206640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70107be0 0x7fae7019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae75206640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70108080 0x7fae70198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae75206640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae70111f30 con 0x7fae70102860 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae75206640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fae70111db0 con 0x7fae70108080 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae75206640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae701120b0 con 0x7fae70107be0 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae70102860 0x7fae7019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae70102860 0x7fae7019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50048/0 (socket says 172.21.15.78:50048) 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae6ed9d640 1 -- 172.21.15.78:0/1511966000 learned_addr learned my addr 172.21.15.78:0/1511966000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70108080 0x7fae70198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae6e59c640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70107be0 0x7fae7019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae6ed9d640 1 -- 172.21.15.78:0/1511966000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70107be0 msgr2=0x7fae7019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.923+0000 7fae6ed9d640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70107be0 0x7fae7019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae6ed9d640 1 -- 172.21.15.78:0/1511966000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70108080 msgr2=0x7fae70198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae6ed9d640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70108080 0x7fae70198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae6ed9d640 1 -- 172.21.15.78:0/1511966000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae70198e40 con 0x7fae70102860 2024-05-21T15:18:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70108080 0x7fae70198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae6e59c640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70107be0 0x7fae7019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae6ed9d640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae70102860 0x7fae7019e590 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fae6002a860 tx=0x7fae60030430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae57fff640 1 -- 172.21.15.78:0/1511966000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60041020 con 0x7fae70102860 2024-05-21T15:18:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae57fff640 1 -- 172.21.15.78:0/1511966000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae60033de0 con 0x7fae70102860 2024-05-21T15:18:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae75206640 1 -- 172.21.15.78:0/1511966000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae70199070 con 0x7fae70102860 2024-05-21T15:18:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae57fff640 1 -- 172.21.15.78:0/1511966000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60037040 con 0x7fae70102860 2024-05-21T15:18:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.924+0000 7fae75206640 1 -- 172.21.15.78:0/1511966000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae70199560 con 0x7fae70102860 2024-05-21T15:18:22.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae57fff640 1 -- 172.21.15.78:0/1511966000 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fae60044020 con 0x7fae70102860 2024-05-21T15:18:22.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae7c188740 1 Processor -- start 2024-05-21T15:18:22.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae7c188740 1 -- start start 2024-05-21T15:18:22.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae7c188740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae7c188740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48e5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae7c188740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae7c188740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5d484ac40 con 0x55e5d451d4d0 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae7c188740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e5d47f2580 con 0x55e5d45256b0 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae7c188740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e5d4846c00 con 0x55e5d4546390 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae6e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48e4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33674/0 (socket says 172.21.15.78:33674) 2024-05-21T15:18:22.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae6f59e640 1 -- 172.21.15.78:0/1337153314 learned_addr learned my addr 172.21.15.78:0/1337153314 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.927+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50060/0 (socket says 172.21.15.78:50060) 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae6f59e640 1 -- 172.21.15.78:0/1337153314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 msgr2=0x55e5d48e57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1337153314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48e57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae6f59e640 1 -- 172.21.15.78:0/1337153314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 msgr2=0x55e5d48e5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1337153314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48e5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae6f59e640 1 -- 172.21.15.78:0/1337153314 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d455e3f0 con 0x55e5d4546390 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae6e59c640 1 --2- 172.21.15.78:0/1337153314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48e57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae57fff640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fae580a2d00 0x7fae580a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae6ed9d640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fae580a2d00 0x7fae580a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1337153314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48e4d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae680117e0 tx=0x7fae68011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e2980e954f816c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae57fff640 1 -- 172.21.15.78:0/1511966000 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fae600f18a0 con 0x7fae70102860 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae557fa640 1 -- 172.21.15.78:0/1337153314 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68011310 con 0x55e5d4546390 2024-05-21T15:18:22.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae557fa640 1 -- 172.21.15.78:0/1337153314 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae68003e50 con 0x55e5d4546390 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.928+0000 7fae557fa640 1 -- 172.21.15.78:0/1337153314 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68004560 con 0x55e5d4546390 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 -- 172.21.15.78:0/1337153314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 msgr2=0x55e5d48e4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae6ed9d640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fae580a2d00 0x7fae580a5210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae600ef150 tx=0x7fae600f7550 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 --2- 172.21.15.78:0/1337153314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48e4d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae680117e0 tx=0x7fae68011ca0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 -- 172.21.15.78:0/1337153314 shutdown_connections 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 --2- 172.21.15.78:0/1337153314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48e57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 --2- 172.21.15.78:0/1337153314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48e5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 --2- 172.21.15.78:0/1337153314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48e4d20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 -- 172.21.15.78:0/1337153314 >> 172.21.15.78:0/1337153314 conn(0x55e5d453a4e0 msgr2=0x55e5d48e14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 -- 172.21.15.78:0/1337153314 shutdown_connections 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 -- 172.21.15.78:0/1337153314 wait complete. 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 -- start start 2024-05-21T15:18:22.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.929+0000 7fae7c188740 1 Processor -- start 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae7c188740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae7c188740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48ef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae7c188740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae7c188740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5d48f1fb0 con 0x55e5d451d4d0 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae7c188740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e5d4805c40 con 0x55e5d45256b0 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae7c188740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e5d48f2130 con 0x55e5d4546390 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae7c188740 10 client.0 _ll_register_callbacks cb 0x55e5d455dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50074/0 (socket says 172.21.15.78:50074) 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 -- 172.21.15.78:0/724706789 learned_addr learned my addr 172.21.15.78:0/724706789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6e59c640 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 -- 172.21.15.78:0/724706789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 msgr2=0x55e5d48f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 -- 172.21.15.78:0/724706789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 msgr2=0x55e5d48ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48ef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 -- 172.21.15.78:0/724706789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d46287d0 con 0x55e5d451d4d0 2024-05-21T15:18:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae6f59e640 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48ef3e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fae68004180 tx=0x7fae68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d786ba083e6248 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae52ff5640 1 -- 172.21.15.78:0/724706789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68020070 con 0x55e5d451d4d0 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae52ff5640 1 -- 172.21.15.78:0/724706789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fae68004a40 con 0x55e5d451d4d0 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.930+0000 7fae7c188740 1 -- 172.21.15.78:0/724706789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae68003e50 con 0x55e5d451d4d0 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae52ff5640 1 -- 172.21.15.78:0/724706789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae68031050 con 0x55e5d451d4d0 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae7c188740 10 client.16332 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae7c188740 1 -- 172.21.15.78:0/724706789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5d48fe770 con 0x55e5d451d4d0 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae52ff5640 1 -- 172.21.15.78:0/724706789 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae68003e50 con 0x55e5d451d4d0 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae52ff5640 1 -- 172.21.15.78:0/724706789 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fae68018020 con 0x55e5d451d4d0 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae7c188740 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae7c188740 10 client.16332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae7c188740 20 client.16332 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:22.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae7c188740 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55e5d4504f60 0x55e5d4901b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae7c188740 4 client.16332 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae7c188740 1 -- 172.21.15.78:0/724706789 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5d4858370 con 0x55e5d4504f60 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.931+0000 7fae6e59c640 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55e5d4504f60 0x55e5d4901b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.932+0000 7fae52ff5640 10 client.16332 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.932+0000 7fae6e59c640 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55e5d4504f60 0x55e5d4901b30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cb4de6cda1b7833 server_cookie=4bc7293151a79123 in_seq=0 out_seq=0 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.932+0000 7fae52ff5640 1 -- 172.21.15.78:0/724706789 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5d4858370 con 0x55e5d4504f60 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae52ff5640 10 client.16332 handle_command_reply: tid=0 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 client.16332 shutdown 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 -- 172.21.15.78:0/724706789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 msgr2=0x55e5d48ef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48ef3e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fae68004180 tx=0x7fae68004270 comp rx=0 tx=0).stop 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 -- 172.21.15.78:0/724706789 shutdown_connections 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55e5d4504f60 0x55e5d4901b30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d45256b0 0x55e5d48ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d4546390 0x55e5d48f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae52ff5640 0 client.16332 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:18:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 --2- 172.21.15.78:0/724706789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48ef3e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 -- 172.21.15.78:0/724706789 >> 172.21.15.78:0/724706789 conn(0x55e5d453a4e0 msgr2=0x55e5d48e6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 -- 172.21.15.78:0/724706789 shutdown_connections 2024-05-21T15:18:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 1 -- 172.21.15.78:0/724706789 wait complete. 2024-05-21T15:18:22.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.933+0000 7fae7c188740 20 client.16332 trim_cache size 0 max 16384 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae7c188740 1 Processor -- start 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae7c188740 1 -- start start 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae7c188740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 0x55e5d48e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae7c188740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d451d4d0 0x55e5d48f5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae7c188740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d45256b0 0x55e5d4907160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae7c188740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae701acb00 con 0x55e5d45256b0 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae7c188740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fae701ae730 con 0x55e5d4504f60 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae7c188740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae701ad1d0 con 0x55e5d451d4d0 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d45256b0 0x55e5d4907160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d45256b0 0x55e5d4907160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50078/0 (socket says 172.21.15.78:50078) 2024-05-21T15:18:22.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 -- 172.21.15.78:0/474161070 learned_addr learned my addr 172.21.15.78:0/474161070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 0x55e5d48e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6e59c640 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d451d4d0 0x55e5d48f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 -- 172.21.15.78:0/474161070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d451d4d0 msgr2=0x55e5d48f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d451d4d0 0x55e5d48f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 -- 172.21.15.78:0/474161070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 msgr2=0x55e5d48e99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 0x55e5d48e99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.947+0000 7fae6f59e640 1 -- 172.21.15.78:0/474161070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d48ed720 con 0x55e5d45256b0 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.948+0000 7fae6f59e640 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d45256b0 0x55e5d4907160 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fae68005d70 tx=0x7fae68011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54be684b6e6a25d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.948+0000 7fae54ff9640 1 -- 172.21.15.78:0/474161070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68020040 con 0x55e5d45256b0 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.948+0000 7fae54ff9640 1 -- 172.21.15.78:0/474161070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae68024040 con 0x55e5d45256b0 2024-05-21T15:18:22.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.948+0000 7fae54ff9640 1 -- 172.21.15.78:0/474161070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68023040 con 0x55e5d45256b0 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.948+0000 7fae7c188740 1 -- 172.21.15.78:0/474161070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d45256b0 msgr2=0x55e5d4907160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.948+0000 7fae7c188740 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d45256b0 0x55e5d4907160 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fae68005d70 tx=0x7fae68011ca0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 -- 172.21.15.78:0/474161070 shutdown_connections 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d45256b0 0x55e5d4907160 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d451d4d0 0x55e5d48f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 --2- 172.21.15.78:0/474161070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 0x55e5d48e99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 -- 172.21.15.78:0/474161070 >> 172.21.15.78:0/474161070 conn(0x55e5d453a4e0 msgr2=0x55e5d48e0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 -- 172.21.15.78:0/474161070 shutdown_connections 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 -- 172.21.15.78:0/474161070 wait complete. 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 -- start start 2024-05-21T15:18:22.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 Processor -- start 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 0x55e5d48f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d45256b0 0x55e5d4967d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae701adfa0 con 0x55e5d451d4d0 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fae701ad8a0 con 0x55e5d4504f60 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae701aaae0 con 0x55e5d45256b0 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae7c188740 10 client.0 _ll_register_callbacks cb 0x7fae600f7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae6e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae6e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50092/0 (socket says 172.21.15.78:50092) 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae6e59c640 1 -- 172.21.15.78:0/1676013170 learned_addr learned my addr 172.21.15.78:0/1676013170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.949+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 0x55e5d48f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d45256b0 0x55e5d4967d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae6e59c640 1 -- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d45256b0 msgr2=0x55e5d4967d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae6e59c640 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d45256b0 0x55e5d4967d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae6e59c640 1 -- 172.21.15.78:0/1676013170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 msgr2=0x55e5d48f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae6e59c640 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 0x55e5d48f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae6e59c640 1 -- 172.21.15.78:0/1676013170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d4968240 con 0x55e5d451d4d0 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d45256b0 0x55e5d4967d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae6e59c640 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48f71e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fae701084b0 tx=0x7fae64010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19ed44a7e8e3002 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae52ff5640 1 -- 172.21.15.78:0/1676013170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6400fbc0 con 0x55e5d451d4d0 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae52ff5640 1 -- 172.21.15.78:0/1676013170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fae6400fd60 con 0x55e5d451d4d0 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae7c188740 1 -- 172.21.15.78:0/1676013170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae68024040 con 0x55e5d451d4d0 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae52ff5640 1 -- 172.21.15.78:0/1676013170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae6402f050 con 0x55e5d451d4d0 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae7c188740 10 client.16338 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae7c188740 1 -- 172.21.15.78:0/1676013170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5d49694b0 con 0x55e5d451d4d0 2024-05-21T15:18:22.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.950+0000 7fae52ff5640 1 -- 172.21.15.78:0/1676013170 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae68024040 con 0x55e5d451d4d0 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae52ff5640 1 -- 172.21.15.78:0/1676013170 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fae6401e5d0 con 0x55e5d451d4d0 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 10 client.16338 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 10 client.16338 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 20 client.16338 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55e5d4546390 0x55e5d496bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 4 client.16338 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 1 -- 172.21.15.78:0/1676013170 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5d48526c0 con 0x55e5d4546390 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55e5d4546390 0x55e5d496bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae52ff5640 10 client.16338 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae6f59e640 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55e5d4546390 0x55e5d496bcf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de42310f9e40d319 server_cookie=c8932299453fa974 in_seq=0 out_seq=0 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae52ff5640 1 -- 172.21.15.78:0/1676013170 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5d48526c0 con 0x55e5d4546390 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae52ff5640 10 client.16338 handle_command_reply: tid=0 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 1 client.16338 shutdown 2024-05-21T15:18:22.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 1 -- 172.21.15.78:0/1676013170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 msgr2=0x55e5d48f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.951+0000 7fae7c188740 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48f71e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fae701084b0 tx=0x7fae64010e40 comp rx=0 tx=0).stop 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 1 -- 172.21.15.78:0/1676013170 shutdown_connections 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55e5d4546390 0x55e5d496bcf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5d45256b0 0x55e5d4967d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae52ff5640 0 client.16338 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5d451d4d0 0x55e5d48f71e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 1 --2- 172.21.15.78:0/1676013170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5d4504f60 0x55e5d48f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 1 -- 172.21.15.78:0/1676013170 >> 172.21.15.78:0/1676013170 conn(0x55e5d453a4e0 msgr2=0x55e5d48e4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 1 -- 172.21.15.78:0/1676013170 shutdown_connections 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 1 -- 172.21.15.78:0/1676013170 wait complete. 2024-05-21T15:18:22.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae7c188740 20 client.16338 trim_cache size 0 max 16384 2024-05-21T15:18:22.962 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:22.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 -- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fae580a2d00 msgr2=0x7fae580a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fae580a2d00 0x7fae580a5210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae600ef150 tx=0x7fae600f7550 comp rx=0 tx=0).stop 2024-05-21T15:18:22.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 -- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae70102860 msgr2=0x7fae7019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae70102860 0x7fae7019e590 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fae6002a860 tx=0x7fae60030430 comp rx=0 tx=0).stop 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 -- 172.21.15.78:0/1511966000 shutdown_connections 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae70108080 0x7fae70198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fae580a2d00 0x7fae580a5210 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70107be0 0x7fae7019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 --2- 172.21.15.78:0/1511966000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae70102860 0x7fae7019e590 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 -- 172.21.15.78:0/1511966000 >> 172.21.15.78:0/1511966000 conn(0x7fae70076830 msgr2=0x7fae701032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 -- 172.21.15.78:0/1511966000 shutdown_connections 2024-05-21T15:18:22.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:22.952+0000 7fae75206640 1 -- 172.21.15.78:0/1511966000 wait complete. 2024-05-21T15:18:22.966 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:18:23.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 -- 172.21.15.78:0/319748609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd104101aa0 msgr2=0x7fd104101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 --2- 172.21.15.78:0/319748609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd104101aa0 0x7fd104101eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009900 tx=0x7fd0f802a410 comp rx=0 tx=0).stop 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 -- 172.21.15.78:0/319748609 shutdown_connections 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 --2- 172.21.15.78:0/319748609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd104101aa0 0x7fd104101eb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 --2- 172.21.15.78:0/319748609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd10410c330 0x7fd10410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 --2- 172.21.15.78:0/319748609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd1041079d0 0x7fd104101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 -- 172.21.15.78:0/319748609 >> 172.21.15.78:0/319748609 conn(0x7fd1040770b0 msgr2=0x7fd1040774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 -- 172.21.15.78:0/319748609 shutdown_connections 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.079+0000 7fd10a97e640 1 -- 172.21.15.78:0/319748609 wait complete. 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd10a97e640 1 Processor -- start 2024-05-21T15:18:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd10a97e640 1 -- start start 2024-05-21T15:18:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd10a97e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd104101aa0 0x7fd10419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd10a97e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1041079d0 0x7fd10419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd10a97e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd10410c330 0x7fd10419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd10a97e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd104111d30 con 0x7fd104101aa0 2024-05-21T15:18:23.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd10a97e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd104111bb0 con 0x7fd10410c330 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd10a97e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd104111eb0 con 0x7fd1041079d0 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd103fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd104101aa0 0x7fd10419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd103fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd104101aa0 0x7fd10419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50104/0 (socket says 172.21.15.78:50104) 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd103fff640 1 -- 172.21.15.78:0/619353044 learned_addr learned my addr 172.21.15.78:0/619353044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd1037fe640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1041079d0 0x7fd10419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.080+0000 7fd108f1b640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd10410c330 0x7fd10419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd103fff640 1 -- 172.21.15.78:0/619353044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1041079d0 msgr2=0x7fd10419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd103fff640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1041079d0 0x7fd10419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd103fff640 1 -- 172.21.15.78:0/619353044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd10410c330 msgr2=0x7fd10419b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd103fff640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd10410c330 0x7fd10419b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd103fff640 1 -- 172.21.15.78:0/619353044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1041a5920 con 0x7fd104101aa0 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd103fff640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd104101aa0 0x7fd10419a610 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400e9c0 tx=0x7fd0f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd1017fa640 1 -- 172.21.15.78:0/619353044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400cd80 con 0x7fd104101aa0 2024-05-21T15:18:23.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd1017fa640 1 -- 172.21.15.78:0/619353044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4005500 con 0x7fd104101aa0 2024-05-21T15:18:23.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd10a97e640 1 -- 172.21.15.78:0/619353044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1041a5c10 con 0x7fd104101aa0 2024-05-21T15:18:23.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd1017fa640 1 -- 172.21.15.78:0/619353044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4010690 con 0x7fd104101aa0 2024-05-21T15:18:23.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.081+0000 7fd10a97e640 1 -- 172.21.15.78:0/619353044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a6130 con 0x7fd104101aa0 2024-05-21T15:18:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.083+0000 7fd1017fa640 1 -- 172.21.15.78:0/619353044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd0f40046c0 con 0x7fd104101aa0 2024-05-21T15:18:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd11182e740 1 Processor -- start 2024-05-21T15:18:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd11182e740 1 -- start start 2024-05-21T15:18:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd11182e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c979665390 0x55c979a03d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd11182e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97963c4d0 0x55c979a04260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd11182e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9796446b0 0x55c979a047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd11182e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c979969c40 con 0x55c979665390 2024-05-21T15:18:23.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd11182e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c979911580 con 0x55c9796446b0 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd11182e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c979965c00 con 0x55c97963c4d0 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd108f1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c979665390 0x55c979a03d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd108f1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c979665390 0x55c979a03d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50108/0 (socket says 172.21.15.78:50108) 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd108f1b640 1 -- 172.21.15.78:0/234356194 learned_addr learned my addr 172.21.15.78:0/234356194 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd103fff640 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97963c4d0 0x55c979a04260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd1037fe640 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9796446b0 0x55c979a047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.084+0000 7fd1017fa640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd0ec0a2d00 0x7fd0ec0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd108f1b640 1 -- 172.21.15.78:0/234356194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97963c4d0 msgr2=0x55c979a04260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd108f1b640 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97963c4d0 0x55c979a04260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd108f1b640 1 -- 172.21.15.78:0/234356194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9796446b0 msgr2=0x55c979a047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd108f1b640 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9796446b0 0x55c979a047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd108f1b640 1 -- 172.21.15.78:0/234356194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97967d3f0 con 0x55c979665390 2024-05-21T15:18:23.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd1017fa640 1 -- 172.21.15.78:0/619353044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd0f40cf850 con 0x7fd104101aa0 2024-05-21T15:18:23.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd108f1b640 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c979665390 0x55c979a03d20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009f40 tx=0x7fd0fc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6ca6e190d1b864 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd108f1b640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd0ec0a2d00 0x7fd0ec0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd0ea7fc640 1 -- 172.21.15.78:0/234356194 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc022070 con 0x55c979665390 2024-05-21T15:18:23.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd0ea7fc640 1 -- 172.21.15.78:0/234356194 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0fc0113e0 con 0x55c979665390 2024-05-21T15:18:23.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.085+0000 7fd0ea7fc640 1 -- 172.21.15.78:0/234356194 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc016070 con 0x55c979665390 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 -- 172.21.15.78:0/234356194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c979665390 msgr2=0x55c979a03d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd108f1b640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd0ec0a2d00 0x7fd0ec0a5210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc01f000 tx=0x7fd0fc022000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c979665390 0x55c979a03d20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009f40 tx=0x7fd0fc010800 comp rx=0 tx=0).stop 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 -- 172.21.15.78:0/234356194 shutdown_connections 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c9796446b0 0x55c979a047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c97963c4d0 0x55c979a04260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 --2- 172.21.15.78:0/234356194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c979665390 0x55c979a03d20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 -- 172.21.15.78:0/234356194 >> 172.21.15.78:0/234356194 conn(0x55c9796594e0 msgr2=0x55c979a004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 -- 172.21.15.78:0/234356194 shutdown_connections 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 -- 172.21.15.78:0/234356194 wait complete. 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.086+0000 7fd11182e740 1 -- start start 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd11182e740 1 Processor -- start 2024-05-21T15:18:23.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd11182e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97963c4d0 0x55c979a0e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd11182e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9796446b0 0x55c979a0e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd11182e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c979665390 0x55c979a144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd11182e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c979a10fb0 con 0x55c9796446b0 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd11182e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c979924c40 con 0x55c97963c4d0 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd11182e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c979a11130 con 0x55c979665390 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd11182e740 10 client.0 _ll_register_callbacks cb 0x55c97967cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9796446b0 0x55c979a0e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9796446b0 0x55c979a0e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50110/0 (socket says 172.21.15.78:50110) 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd1037fe640 1 -- 172.21.15.78:0/1546545549 learned_addr learned my addr 172.21.15.78:0/1546545549 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd108f1b640 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c979665390 0x55c979a144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd103fff640 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97963c4d0 0x55c979a0e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd1037fe640 1 -- 172.21.15.78:0/1546545549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c979665390 msgr2=0x55c979a144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd1037fe640 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c979665390 0x55c979a144a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd1037fe640 1 -- 172.21.15.78:0/1546545549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97963c4d0 msgr2=0x55c979a0e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd1037fe640 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97963c4d0 0x55c979a0e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.087+0000 7fd1037fe640 1 -- 172.21.15.78:0/1546545549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9797477d0 con 0x55c9796446b0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd1037fe640 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9796446b0 0x55c979a0e900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd10406b480 tx=0x7fd0f80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280b4ad6da9011e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd0e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd0e37fe640 1 -- 172.21.15.78:0/1546545549 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8035040 con 0x55c9796446b0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd0e37fe640 1 -- 172.21.15.78:0/1546545549 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd0f8005450 con 0x55c9796446b0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 1 -- 172.21.15.78:0/1546545549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0fc0113e0 con 0x55c9796446b0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd0e37fe640 1 -- 172.21.15.78:0/1546545549 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0f8005650 con 0x55c9796446b0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 10 client.16350 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 1 -- 172.21.15.78:0/1546545549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c979a1d790 con 0x55c9796446b0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd0e37fe640 1 -- 172.21.15.78:0/1546545549 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0fc0113e0 con 0x55c9796446b0 2024-05-21T15:18:23.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd0e37fe640 1 -- 172.21.15.78:0/1546545549 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fd0f8003960 con 0x55c9796446b0 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 10 client.16350 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 10 client.16350 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 20 client.16350 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55c979623f60 0x55c979a20c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 4 client.16350 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.088+0000 7fd11182e740 1 -- 172.21.15.78:0/1546545549 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c979977370 con 0x55c979623f60 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.089+0000 7fd103fff640 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55c979623f60 0x55c979a20c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.089+0000 7fd0e37fe640 10 client.16350 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:18:23.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.089+0000 7fd103fff640 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55c979623f60 0x55c979a20c80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b0c5725a22a0062 server_cookie=2852ef6c187e25e1 in_seq=0 out_seq=0 2024-05-21T15:18:23.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd0e37fe640 1 -- 172.21.15.78:0/1546545549 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c979977370 con 0x55c979623f60 2024-05-21T15:18:23.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd0e37fe640 10 client.16350 handle_command_reply: tid=0 2024-05-21T15:18:23.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 client.16350 shutdown 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 -- 172.21.15.78:0/1546545549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9796446b0 msgr2=0x55c979a0e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9796446b0 0x55c979a0e900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd10406b480 tx=0x7fd0f80043b0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 -- 172.21.15.78:0/1546545549 shutdown_connections 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55c979623f60 0x55c979a20c80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c979665390 0x55c979a144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c9796446b0 0x55c979a0e900 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd0e37fe640 0 client.16350 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 --2- 172.21.15.78:0/1546545549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c97963c4d0 0x55c979a0e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 -- 172.21.15.78:0/1546545549 >> 172.21.15.78:0/1546545549 conn(0x55c9796594e0 msgr2=0x55c979a05a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 -- 172.21.15.78:0/1546545549 shutdown_connections 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 1 -- 172.21.15.78:0/1546545549 wait complete. 2024-05-21T15:18:23.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.090+0000 7fd11182e740 20 client.16350 trim_cache size 0 max 16384 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd11182e740 1 Processor -- start 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd11182e740 1 -- start start 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd11182e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd11182e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c979a14360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd11182e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 0x55c979a079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd11182e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041aecd0 con 0x55c97963c4d0 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd11182e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041b0900 con 0x55c979623f60 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd11182e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041af3a0 con 0x55c9796446b0 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c979a14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c979a14360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50120/0 (socket says 172.21.15.78:50120) 2024-05-21T15:18:23.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd1037fe640 1 -- 172.21.15.78:0/1304128418 learned_addr learned my addr 172.21.15.78:0/1304128418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd108f1b640 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 0x55c979a079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd103fff640 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd1037fe640 1 -- 172.21.15.78:0/1304128418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 msgr2=0x55c979a079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd1037fe640 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 0x55c979a079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd1037fe640 1 -- 172.21.15.78:0/1304128418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 msgr2=0x55c979a005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd1037fe640 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd1037fe640 1 -- 172.21.15.78:0/1304128418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c979a0d4b0 con 0x55c97963c4d0 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.104+0000 7fd103fff640 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.105+0000 7fd1037fe640 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c979a14360 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd0f802a930 tx=0x7fd0f8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425f98b21ac68422 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.105+0000 7fd0e9ffb640 1 -- 172.21.15.78:0/1304128418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8035030 con 0x55c97963c4d0 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.105+0000 7fd0e9ffb640 1 -- 172.21.15.78:0/1304128418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f8041070 con 0x55c97963c4d0 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.105+0000 7fd0e9ffb640 1 -- 172.21.15.78:0/1304128418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8052070 con 0x55c97963c4d0 2024-05-21T15:18:23.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.105+0000 7fd11182e740 1 -- 172.21.15.78:0/1304128418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 msgr2=0x55c979a14360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.105+0000 7fd11182e740 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c979a14360 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd0f802a930 tx=0x7fd0f8005d90 comp rx=0 tx=0).stop 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 -- 172.21.15.78:0/1304128418 shutdown_connections 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 0x55c979a079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c979a14360 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 --2- 172.21.15.78:0/1304128418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 -- 172.21.15.78:0/1304128418 >> 172.21.15.78:0/1304128418 conn(0x55c9796594e0 msgr2=0x55c9799ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 -- 172.21.15.78:0/1304128418 shutdown_connections 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 -- 172.21.15.78:0/1304128418 wait complete. 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 -- start start 2024-05-21T15:18:23.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 Processor -- start 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.106+0000 7fd11182e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a1aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd11182e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c9799ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd11182e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 0x55c9799ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd11182e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041b0170 con 0x55c97963c4d0 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd11182e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041afa70 con 0x55c979623f60 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd11182e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041accb0 con 0x55c9796446b0 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd11182e740 10 client.0 _ll_register_callbacks cb 0x7fd0f8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c9799ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c9799ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50124/0 (socket says 172.21.15.78:50124) 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 -- 172.21.15.78:0/4054694940 learned_addr learned my addr 172.21.15.78:0/4054694940 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd108f1b640 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 0x55c9799ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd103fff640 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a1aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 -- 172.21.15.78:0/4054694940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 msgr2=0x55c9799ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 0x55c9799ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 -- 172.21.15.78:0/4054694940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 msgr2=0x55c979a1aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a1aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 -- 172.21.15.78:0/4054694940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c979a00100 con 0x55c97963c4d0 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd1037fe640 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c9799ff680 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8031550 tx=0x7fd0f802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327daad0e6bd0a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd0e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd0e37fe640 1 -- 172.21.15.78:0/4054694940 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8052070 con 0x55c97963c4d0 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd0e37fe640 1 -- 172.21.15.78:0/4054694940 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd0f8031d80 con 0x55c97963c4d0 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd11182e740 1 -- 172.21.15.78:0/4054694940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f8041070 con 0x55c97963c4d0 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd0e37fe640 1 -- 172.21.15.78:0/4054694940 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0f8064050 con 0x55c97963c4d0 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.107+0000 7fd11182e740 10 client.16356 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd11182e740 1 -- 172.21.15.78:0/4054694940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c979a18fe0 con 0x55c97963c4d0 2024-05-21T15:18:23.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd0e37fe640 1 -- 172.21.15.78:0/4054694940 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f8041070 con 0x55c97963c4d0 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd0e37fe640 1 -- 172.21.15.78:0/4054694940 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fd0f8004c40 con 0x55c97963c4d0 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd11182e740 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd11182e740 10 client.16356 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd11182e740 20 client.16356 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd11182e740 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55c979665390 0x55c979a8f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd11182e740 4 client.16356 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd11182e740 1 -- 172.21.15.78:0/4054694940 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9799716c0 con 0x55c979665390 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd103fff640 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55c979665390 0x55c979a8f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd0e37fe640 10 client.16356 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.108+0000 7fd103fff640 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55c979665390 0x55c979a8f650 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2aae04f262d634ad server_cookie=609d391861ab7448 in_seq=0 out_seq=0 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd0e37fe640 1 -- 172.21.15.78:0/4054694940 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9799716c0 con 0x55c979665390 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd0e37fe640 10 client.16356 handle_command_reply: tid=0 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 client.16356 shutdown 2024-05-21T15:18:23.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 -- 172.21.15.78:0/4054694940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 msgr2=0x55c9799ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c9799ff680 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8031550 tx=0x7fd0f802ab90 comp rx=0 tx=0).stop 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 -- 172.21.15.78:0/4054694940 shutdown_connections 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55c979665390 0x55c979a8f650 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9796446b0 0x55c9799ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c97963c4d0 0x55c9799ff680 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd0e37fe640 0 client.16356 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 --2- 172.21.15.78:0/4054694940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c979623f60 0x55c979a1aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 -- 172.21.15.78:0/4054694940 >> 172.21.15.78:0/4054694940 conn(0x55c9796594e0 msgr2=0x55c979a03ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 -- 172.21.15.78:0/4054694940 shutdown_connections 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 1 -- 172.21.15.78:0/4054694940 wait complete. 2024-05-21T15:18:23.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd11182e740 20 client.16356 trim_cache size 0 max 16384 2024-05-21T15:18:23.119 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:23.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd10a97e640 1 -- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd0ec0a2d00 msgr2=0x7fd0ec0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd10a97e640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd0ec0a2d00 0x7fd0ec0a5210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc01f000 tx=0x7fd0fc022000 comp rx=0 tx=0).stop 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd10a97e640 1 -- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd104101aa0 msgr2=0x7fd10419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.109+0000 7fd10a97e640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd104101aa0 0x7fd10419a610 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400e9c0 tx=0x7fd0f400ee80 comp rx=0 tx=0).stop 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.110+0000 7fd10a97e640 1 -- 172.21.15.78:0/619353044 shutdown_connections 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.110+0000 7fd10a97e640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd10410c330 0x7fd10419b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.110+0000 7fd10a97e640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd0ec0a2d00 0x7fd0ec0a5210 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.110+0000 7fd10a97e640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1041079d0 0x7fd10419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.110+0000 7fd10a97e640 1 --2- 172.21.15.78:0/619353044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd104101aa0 0x7fd10419a610 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.110+0000 7fd10a97e640 1 -- 172.21.15.78:0/619353044 >> 172.21.15.78:0/619353044 conn(0x7fd1040770b0 msgr2=0x7fd104103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.110+0000 7fd10a97e640 1 -- 172.21.15.78:0/619353044 shutdown_connections 2024-05-21T15:18:23.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.110+0000 7fd10a97e640 1 -- 172.21.15.78:0/619353044 wait complete. 2024-05-21T15:18:23.124 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:18:23.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 -- 172.21.15.78:0/3372220962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8041079d0 msgr2=0x7ff804101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 --2- 172.21.15.78:0/3372220962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8041079d0 0x7ff804101560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009860 tx=0x7ff7f402a340 comp rx=0 tx=0).stop 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 -- 172.21.15.78:0/3372220962 shutdown_connections 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 --2- 172.21.15.78:0/3372220962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff80410c330 0x7ff80410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 --2- 172.21.15.78:0/3372220962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff804101aa0 0x7ff804101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 --2- 172.21.15.78:0/3372220962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8041079d0 0x7ff804101560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 -- 172.21.15.78:0/3372220962 >> 172.21.15.78:0/3372220962 conn(0x7ff8040770b0 msgr2=0x7ff8040774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 -- 172.21.15.78:0/3372220962 shutdown_connections 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.235+0000 7ff80938b640 1 -- 172.21.15.78:0/3372220962 wait complete. 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff80938b640 1 Processor -- start 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff80938b640 1 -- start start 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff80938b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff804101aa0 0x7ff80419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff80938b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8041079d0 0x7ff80419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff80938b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff80410c330 0x7ff80419b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff80938b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff804111ce0 con 0x7ff8041079d0 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff80938b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff804111b60 con 0x7ff80410c330 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff80938b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff804111e60 con 0x7ff804101aa0 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff7fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8041079d0 0x7ff80419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff802ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff804101aa0 0x7ff80419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff7fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8041079d0 0x7ff80419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50128/0 (socket says 172.21.15.78:50128) 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.236+0000 7ff7fbfff640 1 -- 172.21.15.78:0/3087203245 learned_addr learned my addr 172.21.15.78:0/3087203245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff8037fe640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff80410c330 0x7ff80419b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff7fbfff640 1 -- 172.21.15.78:0/3087203245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff804101aa0 msgr2=0x7ff80419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff804101aa0 0x7ff80419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff7fbfff640 1 -- 172.21.15.78:0/3087203245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff80410c330 msgr2=0x7ff80419b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff80410c330 0x7ff80419b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff7fbfff640 1 -- 172.21.15.78:0/3087203245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8041a58d0 con 0x7ff8041079d0 2024-05-21T15:18:23.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff802ffd640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff804101aa0 0x7ff80419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8041079d0 0x7ff80419ab00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e980 tx=0x7ff7ec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff800ff9640 1 -- 172.21.15.78:0/3087203245 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00cd80 con 0x7ff8041079d0 2024-05-21T15:18:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff80938b640 1 -- 172.21.15.78:0/3087203245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8041a5bc0 con 0x7ff8041079d0 2024-05-21T15:18:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff800ff9640 1 -- 172.21.15.78:0/3087203245 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ec010910 con 0x7ff8041079d0 2024-05-21T15:18:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff800ff9640 1 -- 172.21.15.78:0/3087203245 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec018a00 con 0x7ff8041079d0 2024-05-21T15:18:23.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.237+0000 7ff80938b640 1 -- 172.21.15.78:0/3087203245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8041a6110 con 0x7ff8041079d0 2024-05-21T15:18:23.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.240+0000 7ff800ff9640 1 -- 172.21.15.78:0/3087203245 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff7ec01f020 con 0x7ff8041079d0 2024-05-21T15:18:23.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.240+0000 7ff810231740 1 Processor -- start 2024-05-21T15:18:23.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.240+0000 7ff810231740 1 -- start start 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff810231740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634a7390 0x555e63845d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff810231740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e6347e4d0 0x555e63846260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff810231740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e634866b0 0x555e638467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff810231740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555e637abc40 con 0x555e634a7390 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff810231740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555e63753580 con 0x555e6347e4d0 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff810231740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e637a7c00 con 0x555e634866b0 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634a7390 0x555e63845d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634a7390 0x555e63845d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50134/0 (socket says 172.21.15.78:50134) 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 -- 172.21.15.78:0/1657675530 learned_addr learned my addr 172.21.15.78:0/1657675530 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e634866b0 0x555e638467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff802ffd640 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e6347e4d0 0x555e63846260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff800ff9640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff7f00a2d00 0x7ff7f00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 -- 172.21.15.78:0/1657675530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e634866b0 msgr2=0x555e638467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e634866b0 0x555e638467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 -- 172.21.15.78:0/1657675530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e6347e4d0 msgr2=0x555e63846260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e6347e4d0 0x555e63846260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 -- 172.21.15.78:0/1657675530 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e634bf3f0 con 0x555e634a7390 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e634866b0 0x555e638467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff7f00a2d00 0x7ff7f00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff800ff9640 1 -- 172.21.15.78:0/3087203245 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7ff7ec0cf5b0 con 0x7ff8041079d0 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.241+0000 7ff8037fe640 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634a7390 0x555e63845d20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0114c0 tx=0x7ff7fc0118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae125df6b7670c7f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.242+0000 7ff7f97fa640 1 -- 172.21.15.78:0/1657675530 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc017070 con 0x555e634a7390 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.242+0000 7ff7f97fa640 1 -- 172.21.15.78:0/1657675530 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7fc00fbc0 con 0x555e634a7390 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.242+0000 7ff7f97fa640 1 -- 172.21.15.78:0/1657675530 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc01bac0 con 0x555e634a7390 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- 172.21.15.78:0/1657675530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634a7390 msgr2=0x555e63845d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff8037fe640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff7f00a2d00 0x7ff7f00a5210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc022000 tx=0x7ff7fc023000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634a7390 0x555e63845d20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0114c0 tx=0x7ff7fc0118c0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- 172.21.15.78:0/1657675530 shutdown_connections 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e6347e4d0 0x555e63846260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e634866b0 0x555e638467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 --2- 172.21.15.78:0/1657675530 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634a7390 0x555e63845d20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- 172.21.15.78:0/1657675530 >> 172.21.15.78:0/1657675530 conn(0x555e6349b4e0 msgr2=0x555e638424d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- 172.21.15.78:0/1657675530 shutdown_connections 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- 172.21.15.78:0/1657675530 wait complete. 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- start start 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 Processor -- start 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e63850370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e634a7390 0x555e63856430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555e63852fb0 con 0x555e634866b0 2024-05-21T15:18:23.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555e63766c40 con 0x555e634a7390 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.243+0000 7ff810231740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e63853130 con 0x555e6347e4d0 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff810231740 10 client.0 _ll_register_callbacks cb 0x555e634bebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50148/0 (socket says 172.21.15.78:50148) 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 -- 172.21.15.78:0/3497851277 learned_addr learned my addr 172.21.15.78:0/3497851277 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff802ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e63850370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff8037fe640 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e634a7390 0x555e63856430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 -- 172.21.15.78:0/3497851277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 msgr2=0x555e63850370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e63850370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 -- 172.21.15.78:0/3497851277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e634a7390 msgr2=0x555e63856430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e634a7390 0x555e63856430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 -- 172.21.15.78:0/3497851277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e635897d0 con 0x555e634866b0 2024-05-21T15:18:23.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff802ffd640 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e63850370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638508b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0cd2a0 tx=0x7ff7ec009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1df0d9f0e7e0a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/3497851277 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec09b3d0 con 0x555e634866b0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/3497851277 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7ff7ec0980e0 con 0x555e634866b0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff810231740 1 -- 172.21.15.78:0/3497851277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7fc00fbc0 con 0x555e634866b0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/3497851277 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7ec09c170 con 0x555e634866b0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff810231740 10 client.16368 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.244+0000 7ff810231740 1 -- 172.21.15.78:0/3497851277 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555e6385f740 con 0x555e634866b0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/3497851277 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7fc00fbc0 con 0x555e634866b0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/3497851277 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7ff7ec0a4250 con 0x555e634866b0 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff810231740 10 client.16368 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff810231740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff810231740 20 client.16368 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff810231740 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555e63465f60 0x555e63862b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff810231740 4 client.16368 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:23.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff810231740 1 -- 172.21.15.78:0/3497851277 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e637b9370 con 0x555e63465f60 2024-05-21T15:18:23.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.245+0000 7ff802ffd640 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555e63465f60 0x555e63862b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff7ea7fc640 10 client.16368 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:23.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff802ffd640 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555e63465f60 0x555e63862b60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f113d487d580d0bc server_cookie=343749e762cd8c47 in_seq=0 out_seq=0 2024-05-21T15:18:23.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/3497851277 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e637b9370 con 0x555e63465f60 2024-05-21T15:18:23.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff7ea7fc640 10 client.16368 handle_command_reply: tid=0 2024-05-21T15:18:23.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 client.16368 shutdown 2024-05-21T15:18:23.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 -- 172.21.15.78:0/3497851277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 msgr2=0x555e638508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638508b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0cd2a0 tx=0x7ff7ec009560 comp rx=0 tx=0).stop 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 -- 172.21.15.78:0/3497851277 shutdown_connections 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555e63465f60 0x555e63862b60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e634a7390 0x555e63856430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638508b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff7ea7fc640 0 client.16368 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 --2- 172.21.15.78:0/3497851277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e63850370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.246+0000 7ff810231740 1 -- 172.21.15.78:0/3497851277 >> 172.21.15.78:0/3497851277 conn(0x555e6349b4e0 msgr2=0x555e63847a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.247+0000 7ff810231740 1 -- 172.21.15.78:0/3497851277 shutdown_connections 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.247+0000 7ff810231740 1 -- 172.21.15.78:0/3497851277 wait complete. 2024-05-21T15:18:23.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.247+0000 7ff810231740 20 client.16368 trim_cache size 0 max 16384 2024-05-21T15:18:23.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.260+0000 7ff810231740 1 Processor -- start 2024-05-21T15:18:23.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff810231740 1 -- start start 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff810231740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e638563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff810231740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e638522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff810231740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff810231740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041aeca0 con 0x555e634866b0 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff810231740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041b08d0 con 0x555e63465f60 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff810231740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041af370 con 0x555e6347e4d0 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50154/0 (socket says 172.21.15.78:50154) 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff8037fe640 1 -- 172.21.15.78:0/1795800725 learned_addr learned my addr 172.21.15.78:0/1795800725 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff802ffd640 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e638563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e638522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff802ffd640 1 -- 172.21.15.78:0/1795800725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 msgr2=0x555e638522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff802ffd640 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e638522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff802ffd640 1 -- 172.21.15.78:0/1795800725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 msgr2=0x555e638477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff802ffd640 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff802ffd640 1 -- 172.21.15.78:0/1795800725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e638456b0 con 0x555e63465f60 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.261+0000 7ff8037fe640 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.262+0000 7ff802ffd640 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e638563a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4035040 tx=0x7ff7f4030460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a498cd02c516403f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.262+0000 7ff7f8ff9640 1 -- 172.21.15.78:0/1795800725 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f404e070 con 0x555e63465f60 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.262+0000 7ff7f8ff9640 1 -- 172.21.15.78:0/1795800725 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f40035f0 con 0x555e63465f60 2024-05-21T15:18:23.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.262+0000 7ff7f8ff9640 1 -- 172.21.15.78:0/1795800725 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4037040 con 0x555e63465f60 2024-05-21T15:18:23.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- 172.21.15.78:0/1795800725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 msgr2=0x555e638563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e638563a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4035040 tx=0x7ff7f4030460 comp rx=0 tx=0).stop 2024-05-21T15:18:23.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- 172.21.15.78:0/1795800725 shutdown_connections 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e638522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 --2- 172.21.15.78:0/1795800725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e638563a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- 172.21.15.78:0/1795800725 >> 172.21.15.78:0/1795800725 conn(0x555e6349b4e0 msgr2=0x555e63845ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- 172.21.15.78:0/1795800725 shutdown_connections 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- 172.21.15.78:0/1795800725 wait complete. 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- start start 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 Processor -- start 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e63857f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e638584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041b0140 con 0x555e634866b0 2024-05-21T15:18:23.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041afa40 con 0x555e63465f60 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8041acc80 con 0x555e6347e4d0 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.263+0000 7ff810231740 10 client.0 _ll_register_callbacks cb 0x7ff7ec00e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50158/0 (socket says 172.21.15.78:50158) 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 -- 172.21.15.78:0/214835816 learned_addr learned my addr 172.21.15.78:0/214835816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff7fbfff640 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e638584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff802ffd640 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e63857f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 -- 172.21.15.78:0/214835816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 msgr2=0x555e638584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e638584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 -- 172.21.15.78:0/214835816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 msgr2=0x555e63857f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e63857f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 -- 172.21.15.78:0/214835816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e638c9400 con 0x555e634866b0 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff8037fe640 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638c8ec0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011dc0 tx=0x7ff7fc003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162458db000cd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:23.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff7ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/214835816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc037060 con 0x555e634866b0 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/214835816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7ff7fc01b8b0 con 0x555e634866b0 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff810231740 1 -- 172.21.15.78:0/214835816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f40035f0 con 0x555e634866b0 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/214835816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7fc017040 con 0x555e634866b0 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff810231740 10 client.16374 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.264+0000 7ff810231740 1 -- 172.21.15.78:0/214835816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555e638ca530 con 0x555e634866b0 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/214835816 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f40035f0 con 0x555e634866b0 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/214835816 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7ff7fc0325a0 con 0x555e634866b0 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff810231740 10 client.16374 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff810231740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff810231740 20 client.16374 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff810231740 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555e634a7390 0x555e638ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:23.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff810231740 4 client.16374 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff810231740 1 -- 172.21.15.78:0/214835816 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e637b36c0 con 0x555e634a7390 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff802ffd640 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555e634a7390 0x555e638ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff7ea7fc640 10 client.16374 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff802ffd640 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555e634a7390 0x555e638ccb80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745e56e8e9c019f6 server_cookie=152dc1110905d94a in_seq=0 out_seq=0 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.265+0000 7ff7ea7fc640 1 -- 172.21.15.78:0/214835816 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e637b36c0 con 0x555e634a7390 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff7ea7fc640 10 client.16374 handle_command_reply: tid=0 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 client.16374 shutdown 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 -- 172.21.15.78:0/214835816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 msgr2=0x555e638c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638c8ec0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc011dc0 tx=0x7ff7fc003a00 comp rx=0 tx=0).stop 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 -- 172.21.15.78:0/214835816 shutdown_connections 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555e634a7390 0x555e638ccb80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e634866b0 0x555e638c8ec0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e6347e4d0 0x555e638584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff7ea7fc640 0 client.16374 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 --2- 172.21.15.78:0/214835816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e63465f60 0x555e63857f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 -- 172.21.15.78:0/214835816 >> 172.21.15.78:0/214835816 conn(0x555e6349b4e0 msgr2=0x555e63851fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 -- 172.21.15.78:0/214835816 shutdown_connections 2024-05-21T15:18:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 1 -- 172.21.15.78:0/214835816 wait complete. 2024-05-21T15:18:23.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff810231740 20 client.16374 trim_cache size 0 max 16384 2024-05-21T15:18:23.275 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 -- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff7f00a2d00 msgr2=0x7ff7f00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff7f00a2d00 0x7ff7f00a5210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc022000 tx=0x7ff7fc023000 comp rx=0 tx=0).stop 2024-05-21T15:18:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 -- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8041079d0 msgr2=0x7ff80419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8041079d0 0x7ff80419ab00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e980 tx=0x7ff7ec00ee40 comp rx=0 tx=0).stop 2024-05-21T15:18:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 -- 172.21.15.78:0/3087203245 shutdown_connections 2024-05-21T15:18:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff80410c330 0x7ff80419b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff7f00a2d00 0x7ff7f00a5210 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff8041079d0 0x7ff80419ab00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 --2- 172.21.15.78:0/3087203245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff804101aa0 0x7ff80419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 -- 172.21.15.78:0/3087203245 >> 172.21.15.78:0/3087203245 conn(0x7ff8040770b0 msgr2=0x7ff804103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.266+0000 7ff80938b640 1 -- 172.21.15.78:0/3087203245 shutdown_connections 2024-05-21T15:18:23.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:23.267+0000 7ff80938b640 1 -- 172.21.15.78:0/3087203245 wait complete. 2024-05-21T15:18:23.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:23 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2055328649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:18:23.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:23 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2055328649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:18:23.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:23 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2055328649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:18:24.281 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:18:24.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:18:24.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:18:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 -- 172.21.15.78:0/767563211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc1079d0 msgr2=0x7fe4dc101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 --2- 172.21.15.78:0/767563211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc1079d0 0x7fe4dc101560 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009900 tx=0x7fe4cc02a410 comp rx=0 tx=0).stop 2024-05-21T15:18:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 -- 172.21.15.78:0/767563211 shutdown_connections 2024-05-21T15:18:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 --2- 172.21.15.78:0/767563211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4dc101aa0 0x7fe4dc101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 --2- 172.21.15.78:0/767563211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc10c330 0x7fe4dc10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 --2- 172.21.15.78:0/767563211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc1079d0 0x7fe4dc101560 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 -- 172.21.15.78:0/767563211 >> 172.21.15.78:0/767563211 conn(0x7fe4dc0770b0 msgr2=0x7fe4dc0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 -- 172.21.15.78:0/767563211 shutdown_connections 2024-05-21T15:18:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.397+0000 7fe4e1902640 1 -- 172.21.15.78:0/767563211 wait complete. 2024-05-21T15:18:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4e1902640 1 Processor -- start 2024-05-21T15:18:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4e1902640 1 -- start start 2024-05-21T15:18:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4e1902640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4dc101aa0 0x7fe4dc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4e1902640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1079d0 0x7fe4dc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4e1902640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc10c330 0x7fe4dc19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4e1902640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc111ce0 con 0x7fe4dc10c330 2024-05-21T15:18:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4e1902640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc111b60 con 0x7fe4dc101aa0 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4e1902640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc111e60 con 0x7fe4dc1079d0 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc10c330 0x7fe4dc19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4dc101aa0 0x7fe4dc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc10c330 0x7fe4dc19b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50180/0 (socket says 172.21.15.78:50180) 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4dc101aa0 0x7fe4dc19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:53814/0 (socket says 172.21.15.78:53814) 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4db7fe640 1 -- 172.21.15.78:0/3333455925 learned_addr learned my addr 172.21.15.78:0/3333455925 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.398+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1079d0 0x7fe4dc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4db7fe640 1 -- 172.21.15.78:0/3333455925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1079d0 msgr2=0x7fe4dc19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1079d0 0x7fe4dc19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4db7fe640 1 -- 172.21.15.78:0/3333455925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4dc101aa0 msgr2=0x7fe4dc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4dc101aa0 0x7fe4dc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4db7fe640 1 -- 172.21.15.78:0/3333455925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc1a5910 con 0x7fe4dc10c330 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4daffd640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4dc101aa0 0x7fe4dc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc10c330 0x7fe4dc19b080 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400eaa0 tx=0x7fe4d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4e0900640 1 -- 172.21.15.78:0/3333455925 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400cdb0 con 0x7fe4dc10c330 2024-05-21T15:18:24.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4e1902640 1 -- 172.21.15.78:0/3333455925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc1a5c00 con 0x7fe4dc10c330 2024-05-21T15:18:24.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4e0900640 1 -- 172.21.15.78:0/3333455925 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4005020 con 0x7fe4dc10c330 2024-05-21T15:18:24.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4e0900640 1 -- 172.21.15.78:0/3333455925 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40116b0 con 0x7fe4dc10c330 2024-05-21T15:18:24.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.399+0000 7fe4e1902640 1 -- 172.21.15.78:0/3333455925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a6100 con 0x7fe4dc10c330 2024-05-21T15:18:24.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.401+0000 7fe4e0900640 1 -- 172.21.15.78:0/3333455925 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe4d4004b20 con 0x7fe4dc10c330 2024-05-21T15:18:24.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e87ae740 1 Processor -- start 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e87ae740 1 -- start start 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e0900640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4c40a2d00 0x7fe4c40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be92c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 0x5647be92c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 0x5647be92cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4c40a2d00 0x7fe4c40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e0900640 1 -- 172.21.15.78:0/3333455925 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4d40cf620 con 0x7fe4dc10c330 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647be891990 con 0x5647be58d410 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5647be8395b0 con 0x5647be564550 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5647be88d950 con 0x5647be56c730 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be92c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be92c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50194/0 (socket says 172.21.15.78:50194) 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4daffd640 1 -- 172.21.15.78:0/2884052311 learned_addr learned my addr 172.21.15.78:0/2884052311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.402+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 0x5647be92c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 0x5647be92cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4daffd640 1 -- 172.21.15.78:0/2884052311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 msgr2=0x5647be92cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4daffd640 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 0x5647be92cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4daffd640 1 -- 172.21.15.78:0/2884052311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 msgr2=0x5647be92c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4daffd640 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 0x5647be92c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4daffd640 1 -- 172.21.15.78:0/2884052311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647be66d7e0 con 0x5647be58d410 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4c40a2d00 0x7fe4c40a5210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc10d840 tx=0x7fe4d0013880 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4daffd640 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be92c490 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009230 tx=0x7fe4cc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d12e222112f7223 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4c1ffb640 1 -- 172.21.15.78:0/2884052311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc035040 con 0x5647be58d410 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4c1ffb640 1 -- 172.21.15.78:0/2884052311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc0317b0 con 0x5647be58d410 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.403+0000 7fe4c1ffb640 1 -- 172.21.15.78:0/2884052311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0047a0 con 0x5647be58d410 2024-05-21T15:18:24.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2884052311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 msgr2=0x5647be92c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be92c490 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009230 tx=0x7fe4cc005d50 comp rx=0 tx=0).stop 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2884052311 shutdown_connections 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 0x5647be92c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 0x5647be92cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2884052311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be92c490 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2884052311 >> 172.21.15.78:0/2884052311 conn(0x5647be581560 msgr2=0x5647be928dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2884052311 shutdown_connections 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2884052311 wait complete. 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.404+0000 7fe4e87ae740 1 -- start start 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4e87ae740 1 Processor -- start 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be93b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be56c730 0x5647be93b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be58d410 0x5647be941200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647be93dd20 con 0x5647be56c730 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5647be84cb40 con 0x5647be58d410 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5647be93dea0 con 0x5647be564550 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4e87ae740 10 client.0 _ll_register_callbacks cb 0x5647be8410f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4da7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be56c730 0x5647be93b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4da7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be56c730 0x5647be93b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50210/0 (socket says 172.21.15.78:50210) 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4da7fc640 1 -- 172.21.15.78:0/2510997937 learned_addr learned my addr 172.21.15.78:0/2510997937 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4daffd640 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be93b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be58d410 0x5647be941200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4da7fc640 1 -- 172.21.15.78:0/2510997937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 msgr2=0x5647be93b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be93b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4da7fc640 1 -- 172.21.15.78:0/2510997937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be58d410 msgr2=0x5647be941200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be58d410 0x5647be941200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4da7fc640 1 -- 172.21.15.78:0/2510997937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647be6cdf70 con 0x5647be56c730 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.405+0000 7fe4daffd640 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be93b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be56c730 0x5647be93b5a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000f860 tx=0x7fe4d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae880b371113231d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4baffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4baffd640 1 -- 172.21.15.78:0/2510997937 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0010bb0 con 0x5647be56c730 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4baffd640 1 -- 172.21.15.78:0/2510997937 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4d0010d50 con 0x5647be56c730 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2510997937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4cc0317b0 con 0x5647be56c730 2024-05-21T15:18:24.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4baffd640 1 -- 172.21.15.78:0/2510997937 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0028400 con 0x5647be56c730 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 10 client.16386 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2510997937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5647be565170 con 0x5647be56c730 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4baffd640 1 -- 172.21.15.78:0/2510997937 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc0317b0 con 0x5647be56c730 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4baffd640 1 -- 172.21.15.78:0/2510997937 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fe4d001f070 con 0x5647be56c730 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 10 client.16386 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 20 client.16386 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5647be54c1f0 0x5647be94d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 4 client.16386 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.406+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2510997937 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647be89f0c0 con 0x5647be54c1f0 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.407+0000 7fe4daffd640 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5647be54c1f0 0x5647be94d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.407+0000 7fe4baffd640 10 client.16386 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.407+0000 7fe4daffd640 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5647be54c1f0 0x5647be94d500 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ccc8e70e041503 server_cookie=55039ea36d3ae390 in_seq=0 out_seq=0 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4baffd640 1 -- 172.21.15.78:0/2510997937 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647be89f0c0 con 0x5647be54c1f0 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4baffd640 10 client.16386 handle_command_reply: tid=0 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 client.16386 shutdown 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2510997937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be56c730 msgr2=0x5647be93b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be56c730 0x5647be93b5a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000f860 tx=0x7fe4d0004730 comp rx=0 tx=0).stop 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2510997937 shutdown_connections 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5647be54c1f0 0x5647be94d500 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be58d410 0x5647be941200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be56c730 0x5647be93b5a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4baffd640 0 client.16386 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/2510997937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be93b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2510997937 >> 172.21.15.78:0/2510997937 conn(0x5647be581560 msgr2=0x5647be92e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2510997937 shutdown_connections 2024-05-21T15:18:24.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 1 -- 172.21.15.78:0/2510997937 wait complete. 2024-05-21T15:18:24.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.408+0000 7fe4e87ae740 20 client.16386 trim_cache size 0 max 16384 2024-05-21T15:18:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.426+0000 7fe4e87ae740 1 Processor -- start 2024-05-21T15:18:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.426+0000 7fe4e87ae740 1 -- start start 2024-05-21T15:18:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 0x5647be927e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 0x5647be934860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be934da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ac8d0 con 0x5647be58d410 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ae500 con 0x5647be564550 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1acfa0 con 0x5647be56c730 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be934da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be934da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50224/0 (socket says 172.21.15.78:50224) 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 -- 172.21.15.78:0/1946243400 learned_addr learned my addr 172.21.15.78:0/1946243400 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4daffd640 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 0x5647be927e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 0x5647be934860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 -- 172.21.15.78:0/1946243400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 msgr2=0x5647be934860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 0x5647be934860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 -- 172.21.15.78:0/1946243400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 msgr2=0x5647be927e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 0x5647be927e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 -- 172.21.15.78:0/1946243400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647be948240 con 0x5647be58d410 2024-05-21T15:18:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be934da0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4098560 tx=0x7fe4d409ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd896720b9b3590 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4c17fa640 1 -- 172.21.15.78:0/1946243400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d401f040 con 0x5647be58d410 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4c17fa640 1 -- 172.21.15.78:0/1946243400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d40b4070 con 0x5647be58d410 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.427+0000 7fe4c17fa640 1 -- 172.21.15.78:0/1946243400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40a4520 con 0x5647be58d410 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 -- 172.21.15.78:0/1946243400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 msgr2=0x5647be934da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be934da0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4098560 tx=0x7fe4d409ff50 comp rx=0 tx=0).stop 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 -- 172.21.15.78:0/1946243400 shutdown_connections 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be934da0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be56c730 0x5647be934860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/1946243400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be564550 0x5647be927e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 -- 172.21.15.78:0/1946243400 >> 172.21.15.78:0/1946243400 conn(0x5647be581560 msgr2=0x5647be928c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 -- 172.21.15.78:0/1946243400 shutdown_connections 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.428+0000 7fe4e87ae740 1 -- 172.21.15.78:0/1946243400 wait complete. 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 1 -- start start 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 1 Processor -- start 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be9475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be56c730 0x5647be947af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be940760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1add70 con 0x5647be58d410 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ad670 con 0x5647be56c730 2024-05-21T15:18:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1aa8b0 con 0x5647be564550 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4e87ae740 10 client.0 _ll_register_callbacks cb 0x7fe4d000f4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be940760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4da7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be56c730 0x5647be947af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4da7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be56c730 0x5647be947af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:53852/0 (socket says 172.21.15.78:53852) 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be9475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.429+0000 7fe4da7fc640 1 -- 172.21.15.78:0/3754837420 learned_addr learned my addr 172.21.15.78:0/3754837420 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4da7fc640 1 -- 172.21.15.78:0/3754837420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 msgr2=0x5647be9475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be9475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4da7fc640 1 -- 172.21.15.78:0/3754837420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 msgr2=0x5647be940760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be940760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4da7fc640 1 -- 172.21.15.78:0/3754837420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647be940e50 con 0x5647be56c730 2024-05-21T15:18:24.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4db7fe640 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be940760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4da7fc640 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be56c730 0x5647be947af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0012040 tx=0x7fe4d00206b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193ac3b35d21c7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4baffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4baffd640 1 -- 172.21.15.78:0/3754837420 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0034030 con 0x5647be56c730 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4e87ae740 1 -- 172.21.15.78:0/3754837420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d40b4070 con 0x5647be56c730 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4baffd640 1 -- 172.21.15.78:0/3754837420 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4d0009040 con 0x5647be56c730 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4baffd640 1 -- 172.21.15.78:0/3754837420 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0010a10 con 0x5647be56c730 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4e87ae740 10 client.25838 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4e87ae740 1 -- 172.21.15.78:0/3754837420 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5647be945b30 con 0x5647be56c730 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.430+0000 7fe4baffd640 1 -- 172.21.15.78:0/3754837420 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d40b4070 con 0x5647be56c730 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4baffd640 1 -- 172.21.15.78:0/3754837420 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fe4d00288f0 con 0x5647be56c730 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4e87ae740 10 client.25838 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4e87ae740 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4e87ae740 20 client.25838 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5647be521630 0x5647be9b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4e87ae740 4 client.25838 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:18:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4e87ae740 1 -- 172.21.15.78:0/3754837420 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5647be899410 con 0x5647be521630 2024-05-21T15:18:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4daffd640 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5647be521630 0x5647be9b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4baffd640 10 client.25838 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.431+0000 7fe4daffd640 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5647be521630 0x5647be9b7b70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9798545ac5355a59 server_cookie=b733e13a8142d385 in_seq=0 out_seq=0 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4baffd640 1 -- 172.21.15.78:0/3754837420 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647be899410 con 0x5647be521630 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4baffd640 10 client.25838 handle_command_reply: tid=0 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 client.25838 shutdown 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 -- 172.21.15.78:0/3754837420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be56c730 msgr2=0x5647be947af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be56c730 0x5647be947af0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0012040 tx=0x7fe4d00206b0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 -- 172.21.15.78:0/3754837420 shutdown_connections 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5647be521630 0x5647be9b7b70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5647be56c730 0x5647be947af0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647be58d410 0x5647be940760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4baffd640 0 client.25838 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 --2- 172.21.15.78:0/3754837420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647be564550 0x5647be9475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 -- 172.21.15.78:0/3754837420 >> 172.21.15.78:0/3754837420 conn(0x5647be581560 msgr2=0x5647be92c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 -- 172.21.15.78:0/3754837420 shutdown_connections 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 1 -- 172.21.15.78:0/3754837420 wait complete. 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.432+0000 7fe4e87ae740 20 client.25838 trim_cache size 0 max 16384 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:18:24.442 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "30ac17e5-5ee7-4a42-9afb-57f7539015ab", 2024-05-21T15:18:24.443 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:18:24.443 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:18:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 -- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4c40a2d00 msgr2=0x7fe4c40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4c40a2d00 0x7fe4c40a5210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc10d840 tx=0x7fe4d0013880 comp rx=0 tx=0).stop 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 -- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc10c330 msgr2=0x7fe4dc19b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc10c330 0x7fe4dc19b080 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400eaa0 tx=0x7fe4d400ef60 comp rx=0 tx=0).stop 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 -- 172.21.15.78:0/3333455925 shutdown_connections 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4c40a2d00 0x7fe4c40a5210 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4dc10c330 0x7fe4dc19b080 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc1079d0 0x7fe4dc19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 --2- 172.21.15.78:0/3333455925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4dc101aa0 0x7fe4dc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 -- 172.21.15.78:0/3333455925 >> 172.21.15.78:0/3333455925 conn(0x7fe4dc0770b0 msgr2=0x7fe4dc103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 -- 172.21.15.78:0/3333455925 shutdown_connections 2024-05-21T15:18:24.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.433+0000 7fe4e1902640 1 -- 172.21.15.78:0/3333455925 wait complete. 2024-05-21T15:18:24.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:18:24.449 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 -- 172.21.15.78:0/2556180480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c88101aa0 msgr2=0x7f5c88101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/2556180480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c88101aa0 0x7f5c88101eb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009900 tx=0x7f5c7002a410 comp rx=0 tx=0).stop 2024-05-21T15:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 -- 172.21.15.78:0/2556180480 shutdown_connections 2024-05-21T15:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/2556180480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c8810c330 0x7f5c8810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/2556180480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c88101aa0 0x7f5c88101eb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/2556180480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c881079d0 0x7f5c88101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 -- 172.21.15.78:0/2556180480 >> 172.21.15.78:0/2556180480 conn(0x7f5c880770b0 msgr2=0x7f5c880774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 -- 172.21.15.78:0/2556180480 shutdown_connections 2024-05-21T15:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 -- 172.21.15.78:0/2556180480 wait complete. 2024-05-21T15:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 Processor -- start 2024-05-21T15:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.563+0000 7f5c8d72a640 1 -- start start 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c8d72a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c88101aa0 0x7f5c88199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c8d72a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c881079d0 0x7f5c88199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c8d72a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c8810c330 0x7f5c8819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c8d72a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c88111d30 con 0x7f5c881079d0 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c8d72a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5c88111bb0 con 0x7f5c88101aa0 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c8d72a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c88111eb0 con 0x7f5c8810c330 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c881079d0 0x7f5c88199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c881079d0 0x7f5c88199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50254/0 (socket says 172.21.15.78:50254) 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 -- 172.21.15.78:0/1862719041 learned_addr learned my addr 172.21.15.78:0/1862719041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c877fe640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c8810c330 0x7f5c8819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c88101aa0 0x7f5c88199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 -- 172.21.15.78:0/1862719041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c8810c330 msgr2=0x7f5c8819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c8810c330 0x7f5c8819e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 -- 172.21.15.78:0/1862719041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c88101aa0 msgr2=0x7f5c88199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c88101aa0 0x7f5c88199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 -- 172.21.15.78:0/1862719041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8819e5e0 con 0x7f5c881079d0 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c877fe640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c8810c330 0x7f5c8819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c881079d0 0x7f5c88199d60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009230 tx=0x7f5c70003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c84ff9640 1 -- 172.21.15.78:0/1862719041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7003f070 con 0x7f5c881079d0 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c8d72a640 1 -- 172.21.15.78:0/1862719041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8819e870 con 0x7f5c881079d0 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c84ff9640 1 -- 172.21.15.78:0/1862719041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c70031c90 con 0x7f5c881079d0 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c84ff9640 1 -- 172.21.15.78:0/1862719041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70035030 con 0x7f5c881079d0 2024-05-21T15:18:24.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.564+0000 7f5c8d72a640 1 -- 172.21.15.78:0/1862719041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8819edb0 con 0x7f5c881079d0 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.567+0000 7f5c945cb740 1 Processor -- start 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.567+0000 7f5c84ff9640 1 -- 172.21.15.78:0/1862719041 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5c70042070 con 0x7f5c881079d0 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c945cb740 1 -- start start 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4d9390 0x55e33d877d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b04d0 0x55e33d878270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 0x55e33d8787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c945cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e33d7ddd00 con 0x55e33d4d9390 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c945cb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e33d7854c0 con 0x55e33d4b04d0 2024-05-21T15:18:24.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c945cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e33d7d9cc0 con 0x55e33d4b86b0 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4d9390 0x55e33d877d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4d9390 0x55e33d877d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50260/0 (socket says 172.21.15.78:50260) 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c877fe640 1 -- 172.21.15.78:0/1538525359 learned_addr learned my addr 172.21.15.78:0/1538525359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b04d0 0x55e33d878270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 0x55e33d8787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c877fe640 1 -- 172.21.15.78:0/1538525359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 msgr2=0x55e33d8787b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c877fe640 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 0x55e33d8787b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c84ff9640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5c740a2d00 0x7f5c740a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c877fe640 1 -- 172.21.15.78:0/1538525359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b04d0 msgr2=0x55e33d878270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c877fe640 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b04d0 0x55e33d878270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c877fe640 1 -- 172.21.15.78:0/1538525359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33d5bb7d0 con 0x55e33d4d9390 2024-05-21T15:18:24.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.568+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b04d0 0x55e33d878270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.569+0000 7f5c84ff9640 1 -- 172.21.15.78:0/1862719041 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f5c700efcd0 con 0x7f5c881079d0 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.569+0000 7f5c877fe640 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4d9390 0x55e33d877d30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5c80007f40 tx=0x7f5c8000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c848935f088a74a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.569+0000 7f5c877fe640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5c740a2d00 0x7f5c740a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.569+0000 7f5c7d7fa640 1 -- 172.21.15.78:0/1538525359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80014070 con 0x55e33d4d9390 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.569+0000 7f5c7d7fa640 1 -- 172.21.15.78:0/1538525359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c80010ca0 con 0x55e33d4d9390 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.569+0000 7f5c7d7fa640 1 -- 172.21.15.78:0/1538525359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c800188e0 con 0x55e33d4d9390 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 -- 172.21.15.78:0/1538525359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4d9390 msgr2=0x55e33d877d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c877fe640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5c740a2d00 0x7f5c740a5210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000f010 tx=0x7f5c8000f3c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4d9390 0x55e33d877d30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5c80007f40 tx=0x7f5c8000e7e0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 -- 172.21.15.78:0/1538525359 shutdown_connections 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b04d0 0x55e33d878270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 0x55e33d8787b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 --2- 172.21.15.78:0/1538525359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4d9390 0x55e33d877d30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 -- 172.21.15.78:0/1538525359 >> 172.21.15.78:0/1538525359 conn(0x55e33d4cd4e0 msgr2=0x55e33d8744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 -- 172.21.15.78:0/1538525359 shutdown_connections 2024-05-21T15:18:24.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 -- 172.21.15.78:0/1538525359 wait complete. 2024-05-21T15:18:24.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 -- start start 2024-05-21T15:18:24.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.570+0000 7f5c945cb740 1 Processor -- start 2024-05-21T15:18:24.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d883ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 0x55e33d884420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 0x55e33d882420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c945cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e33d884fc0 con 0x55e33d4b04d0 2024-05-21T15:18:24.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c945cb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e33d798b90 con 0x55e33d4d9390 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c945cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e33d885140 con 0x55e33d4b86b0 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d883ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c945cb740 10 client.0 _ll_register_callbacks cb 0x55e33d7d4da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d883ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50268/0 (socket says 172.21.15.78:50268) 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 -- 172.21.15.78:0/3586788710 learned_addr learned my addr 172.21.15.78:0/3586788710 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 0x55e33d884420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c877fe640 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 0x55e33d882420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 -- 172.21.15.78:0/3586788710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 msgr2=0x55e33d884420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 0x55e33d884420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 -- 172.21.15.78:0/3586788710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 msgr2=0x55e33d882420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 0x55e33d882420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 -- 172.21.15.78:0/3586788710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33d4f13f0 con 0x55e33d4b04d0 2024-05-21T15:18:24.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d883ee0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5c78012700 tx=0x7f5c78012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4db3db1d8322d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.571+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/3586788710 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78011c70 con 0x55e33d4b04d0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 1 -- 172.21.15.78:0/3586788710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c80010ca0 con 0x55e33d4b04d0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/3586788710 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f5c78011e10 con 0x55e33d4b04d0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/3586788710 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c78005020 con 0x55e33d4b04d0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 10 client.16404 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 1 -- 172.21.15.78:0/3586788710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e33d4b10f0 con 0x55e33d4b04d0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/3586788710 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c80010ca0 con 0x55e33d4b04d0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/3586788710 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f5c7801c470 con 0x55e33d4b04d0 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 10 client.16404 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 20 client.16404 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e33d497f60 0x55e33d894d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 4 client.16404 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c945cb740 1 -- 172.21.15.78:0/3586788710 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e33d7eb430 con 0x55e33d497f60 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.572+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e33d497f60 0x55e33d894d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.573+0000 7f5c6e7fc640 10 client.16404 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.573+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e33d497f60 0x55e33d894d40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c26b6391a7019479 server_cookie=246b3879e4e52846 in_seq=0 out_seq=0 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.573+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/3586788710 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e33d7eb430 con 0x55e33d497f60 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.573+0000 7f5c6e7fc640 10 client.16404 handle_command_reply: tid=0 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.573+0000 7f5c945cb740 1 client.16404 shutdown 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 -- 172.21.15.78:0/3586788710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 msgr2=0x55e33d883ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d883ee0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5c78012700 tx=0x7f5c78012bc0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 -- 172.21.15.78:0/3586788710 shutdown_connections 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e33d497f60 0x55e33d894d40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 0x55e33d882420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b86b0 0x55e33d884420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c6e7fc640 0 client.16404 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 --2- 172.21.15.78:0/3586788710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d883ee0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 -- 172.21.15.78:0/3586788710 >> 172.21.15.78:0/3586788710 conn(0x55e33d4cd4e0 msgr2=0x55e33d879a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 -- 172.21.15.78:0/3586788710 shutdown_connections 2024-05-21T15:18:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 1 -- 172.21.15.78:0/3586788710 wait complete. 2024-05-21T15:18:24.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.574+0000 7f5c945cb740 20 client.16404 trim_cache size 0 max 16384 2024-05-21T15:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.592+0000 7f5c945cb740 1 Processor -- start 2024-05-21T15:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.592+0000 7f5c945cb740 1 -- start start 2024-05-21T15:18:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.592+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b04d0 0x55e33d8783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.592+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b86b0 0x55e33d89a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.592+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 0x55e33d89aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.592+0000 7f5c945cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c881aebf0 con 0x55e33d4b86b0 2024-05-21T15:18:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.592+0000 7f5c945cb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5c881b0820 con 0x55e33d4d9390 2024-05-21T15:18:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.592+0000 7f5c945cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c881af2c0 con 0x55e33d4b04d0 2024-05-21T15:18:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b86b0 0x55e33d89a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b86b0 0x55e33d89a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50270/0 (socket says 172.21.15.78:50270) 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 -- 172.21.15.78:0/1188415431 learned_addr learned my addr 172.21.15.78:0/1188415431 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c877fe640 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 0x55e33d89aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b04d0 0x55e33d8783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 -- 172.21.15.78:0/1188415431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b04d0 msgr2=0x55e33d8783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b04d0 0x55e33d8783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 -- 172.21.15.78:0/1188415431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 msgr2=0x55e33d89aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 0x55e33d89aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 -- 172.21.15.78:0/1188415431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e33d877ac0 con 0x55e33d4b86b0 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b04d0 0x55e33d8783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b86b0 0x55e33d89a500 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5c700f0030 tx=0x7f5c700dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccccddae08ab80dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7cff9640 1 -- 172.21.15.78:0/1188415431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70049040 con 0x55e33d4b86b0 2024-05-21T15:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7cff9640 1 -- 172.21.15.78:0/1188415431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c700f1040 con 0x55e33d4b86b0 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.593+0000 7f5c7cff9640 1 -- 172.21.15.78:0/1188415431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c700c0540 con 0x55e33d4b86b0 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 -- 172.21.15.78:0/1188415431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b86b0 msgr2=0x55e33d89a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b86b0 0x55e33d89a500 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5c700f0030 tx=0x7f5c700dc040 comp rx=0 tx=0).stop 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 -- 172.21.15.78:0/1188415431 shutdown_connections 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4d9390 0x55e33d89aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b86b0 0x55e33d89a500 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 --2- 172.21.15.78:0/1188415431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4b04d0 0x55e33d8783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 -- 172.21.15.78:0/1188415431 >> 172.21.15.78:0/1188415431 conn(0x55e33d4cd4e0 msgr2=0x55e33d877cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 -- 172.21.15.78:0/1188415431 shutdown_connections 2024-05-21T15:18:24.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.594+0000 7f5c945cb740 1 -- 172.21.15.78:0/1188415431 wait complete. 2024-05-21T15:18:24.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:24 smithi078 ceph-mon[31997]: pgmap v72: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 428 MiB/s wr, 9.26k op/s 2024-05-21T15:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 1 -- start start 2024-05-21T15:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 1 Processor -- start 2024-05-21T15:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d88e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b86b0 0x55e33d88edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4d9390 0x55e33d88bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c881b0090 con 0x55e33d4b04d0 2024-05-21T15:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5c881af990 con 0x55e33d4b86b0 2024-05-21T15:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c881acbd0 con 0x55e33d4d9390 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c945cb740 10 client.0 _ll_register_callbacks cb 0x7f5c78009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d88e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d88e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50280/0 (socket says 172.21.15.78:50280) 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c86ffd640 1 -- 172.21.15.78:0/137073388 learned_addr learned my addr 172.21.15.78:0/137073388 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b86b0 0x55e33d88edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c877fe640 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4d9390 0x55e33d88bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c86ffd640 1 -- 172.21.15.78:0/137073388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4d9390 msgr2=0x55e33d88bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4d9390 0x55e33d88bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c86ffd640 1 -- 172.21.15.78:0/137073388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b86b0 msgr2=0x55e33d88edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b86b0 0x55e33d88edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.595+0000 7f5c86ffd640 1 -- 172.21.15.78:0/137073388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33d88c440 con 0x55e33d4b04d0 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c86ffd640 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d88e890 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5c7801ca50 tx=0x7f5c78009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f68895392a0aec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:24.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/137073388 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78028070 con 0x55e33d4b04d0 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 1 -- 172.21.15.78:0/137073388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c700f1040 con 0x55e33d4b04d0 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/137073388 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f5c780051c0 con 0x55e33d4b04d0 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/137073388 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c7801cce0 con 0x55e33d4b04d0 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 10 client.16410 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 1 -- 172.21.15.78:0/137073388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e33d8fc980 con 0x55e33d4b04d0 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/137073388 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c700f1040 con 0x55e33d4b04d0 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/137073388 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f5c7800f460 con 0x55e33d4b04d0 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 10 client.16410 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 20 client.16410 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e33d46d3a0 0x55e33d8ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 4 client.16410 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:18:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.596+0000 7f5c945cb740 1 -- 172.21.15.78:0/137073388 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e33d7e5780 con 0x55e33d46d3a0 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e33d46d3a0 0x55e33d8ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c6e7fc640 10 client.16410 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c7ffff640 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e33d46d3a0 0x55e33d8ff1c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e169f9e0660803 server_cookie=10abdaf742ba44cc in_seq=0 out_seq=0 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c6e7fc640 1 -- 172.21.15.78:0/137073388 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55e33d7e5780 con 0x55e33d46d3a0 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c6e7fc640 10 client.16410 handle_command_reply: tid=0 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 client.16410 shutdown 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 -- 172.21.15.78:0/137073388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 msgr2=0x55e33d88e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d88e890 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5c7801ca50 tx=0x7f5c78009cf0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 -- 172.21.15.78:0/137073388 shutdown_connections 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e33d46d3a0 0x55e33d8ff1c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e33d4b86b0 0x55e33d88edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e33d4d9390 0x55e33d88bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c6e7fc640 0 client.16410 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 --2- 172.21.15.78:0/137073388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e33d4b04d0 0x55e33d88e890 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 -- 172.21.15.78:0/137073388 >> 172.21.15.78:0/137073388 conn(0x55e33d4cd4e0 msgr2=0x55e33d884210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 -- 172.21.15.78:0/137073388 shutdown_connections 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 1 -- 172.21.15.78:0/137073388 wait complete. 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.597+0000 7f5c945cb740 20 client.16410 trim_cache size 0 max 16384 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (1 inodes in the stack)", 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stdout: "30ac17e5-5ee7-4a42-9afb-57f7539015ab": { 2024-05-21T15:18:24.607 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:18:24.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "30ac17e5-5ee7-4a42-9afb-57f7539015ab", 2024-05-21T15:18:24.608 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:18:24.608 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:18:24.608 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:18:24.608 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 -- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5c740a2d00 msgr2=0x7f5c740a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5c740a2d00 0x7f5c740a5210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000f010 tx=0x7f5c8000f3c0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 -- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c881079d0 msgr2=0x7f5c88199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c881079d0 0x7f5c88199d60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009230 tx=0x7f5c70003b90 comp rx=0 tx=0).stop 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 -- 172.21.15.78:0/1862719041 shutdown_connections 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5c740a2d00 0x7f5c740a5210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c8810c330 0x7f5c8819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c881079d0 0x7f5c88199d60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 --2- 172.21.15.78:0/1862719041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c88101aa0 0x7f5c88199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 -- 172.21.15.78:0/1862719041 >> 172.21.15.78:0/1862719041 conn(0x7f5c880770b0 msgr2=0x7f5c88068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:24.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 -- 172.21.15.78:0/1862719041 shutdown_connections 2024-05-21T15:18:24.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:24.598+0000 7f5c8d72a640 1 -- 172.21.15.78:0/1862719041 wait complete. 2024-05-21T15:18:24.612 INFO:tasks.cephfs.filesystem:scrub status for tag:30ac17e5-5ee7-4a42-9afb-57f7539015ab - {'path': '/', 'tag': '30ac17e5-5ee7-4a42-9afb-57f7539015ab', 'options': 'recursive,force'} 2024-05-21T15:18:24.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:24 smithi084 ceph-mon[36334]: pgmap v72: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 428 MiB/s wr, 9.26k op/s 2024-05-21T15:18:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:24 smithi172 ceph-mon[36402]: pgmap v72: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 428 MiB/s wr, 9.26k op/s 2024-05-21T15:18:25.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:25 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:18:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:25 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:25.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:25 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:18:25.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:25 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:18:25.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:25 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:25.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:25 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:18:25.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:25 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:18:25.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:25 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:25.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:25 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:18:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:26 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:18:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:26 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:26 smithi084 ceph-mon[36334]: pgmap v73: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 458 MiB/s wr, 13.89k op/s 2024-05-21T15:18:26.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:26 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:18:26.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:26 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:26.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:26 smithi172 ceph-mon[36402]: pgmap v73: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 458 MiB/s wr, 13.89k op/s 2024-05-21T15:18:26.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:26 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:18:26.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:26 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:26.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:26 smithi078 ceph-mon[31997]: pgmap v73: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 458 MiB/s wr, 13.89k op/s 2024-05-21T15:18:28.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:28 smithi084 ceph-mon[36334]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 146 MiB/s wr, 14.83k op/s 2024-05-21T15:18:28.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:28 smithi172 ceph-mon[36402]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 146 MiB/s wr, 14.83k op/s 2024-05-21T15:18:28.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:28 smithi078 ceph-mon[31997]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 146 MiB/s wr, 14.83k op/s 2024-05-21T15:18:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.879+0000 7f2b5c907640 1 -- 172.21.15.78:0/512447414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b58107a10 msgr2=0x7f2b581015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.879+0000 7f2b5c907640 1 --2- 172.21.15.78:0/512447414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b58107a10 0x7f2b581015c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009900 tx=0x7f2b4802a440 comp rx=0 tx=0).stop 2024-05-21T15:18:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.879+0000 7f2b5c907640 1 -- 172.21.15.78:0/512447414 shutdown_connections 2024-05-21T15:18:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.879+0000 7f2b5c907640 1 --2- 172.21.15.78:0/512447414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b58102170 0x7f2b5810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.879+0000 7f2b5c907640 1 --2- 172.21.15.78:0/512447414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b58107eb0 0x7f2b58101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.879+0000 7f2b5c907640 1 --2- 172.21.15.78:0/512447414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b58107a10 0x7f2b581015c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.879+0000 7f2b5c907640 1 -- 172.21.15.78:0/512447414 >> 172.21.15.78:0/512447414 conn(0x7f2b58077160 msgr2=0x7f2b580775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.880+0000 7f2b5c907640 1 -- 172.21.15.78:0/512447414 shutdown_connections 2024-05-21T15:18:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.880+0000 7f2b5c907640 1 -- 172.21.15.78:0/512447414 wait complete. 2024-05-21T15:18:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.881+0000 7f2b5c907640 1 Processor -- start 2024-05-21T15:18:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.881+0000 7f2b5c907640 1 -- start start 2024-05-21T15:18:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.881+0000 7f2b5c907640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b58102170 0x7f2b5819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.881+0000 7f2b5c907640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b58107a10 0x7f2b5819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.881+0000 7f2b5c907640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b58107eb0 0x7f2b58197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.882+0000 7f2b5c907640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58111e90 con 0x7f2b58107eb0 2024-05-21T15:18:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.882+0000 7f2b5c907640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58111d10 con 0x7f2b58102170 2024-05-21T15:18:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.882+0000 7f2b5c907640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58112010 con 0x7f2b58107a10 2024-05-21T15:18:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.882+0000 7f2b55d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b58107a10 0x7f2b5819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.882+0000 7f2b55d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b58107a10 0x7f2b5819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:39938/0 (socket says 172.21.15.78:39938) 2024-05-21T15:18:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.882+0000 7f2b55d9b640 1 -- 172.21.15.78:0/3308439923 learned_addr learned my addr 172.21.15.78:0/3308439923 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.883+0000 7f2b5659c640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b58102170 0x7f2b5819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.883+0000 7f2b56d9d640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b58107eb0 0x7f2b58197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.884+0000 7f2b55d9b640 1 -- 172.21.15.78:0/3308439923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b58102170 msgr2=0x7f2b5819c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.884+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b58102170 0x7f2b5819c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.885+0000 7f2b55d9b640 1 -- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b58107eb0 msgr2=0x7f2b58197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.885+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b58107eb0 0x7f2b58197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.885+0000 7f2b55d9b640 1 -- 172.21.15.78:0/3308439923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b58198080 con 0x7f2b58107a10 2024-05-21T15:18:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.886+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b58107a10 0x7f2b5819cbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00b450 tx=0x7f2b4c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.887+0000 7f2b477fe640 1 -- 172.21.15.78:0/3308439923 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c015020 con 0x7f2b58107a10 2024-05-21T15:18:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.887+0000 7f2b477fe640 1 -- 172.21.15.78:0/3308439923 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c011d70 con 0x7f2b58107a10 2024-05-21T15:18:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.887+0000 7f2b477fe640 1 -- 172.21.15.78:0/3308439923 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c00f340 con 0x7f2b58107a10 2024-05-21T15:18:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.887+0000 7f2b5c907640 1 -- 172.21.15.78:0/3308439923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b58198370 con 0x7f2b58107a10 2024-05-21T15:18:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.887+0000 7f2b5c907640 1 -- 172.21.15.78:0/3308439923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b581a2390 con 0x7f2b58107a10 2024-05-21T15:18:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.894+0000 7f2b637a8740 1 Processor -- start 2024-05-21T15:18:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.895+0000 7f2b637a8740 1 -- start start 2024-05-21T15:18:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.895+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a56a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.895+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1a22f0 0x55724a56a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.895+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1aa4d0 0x55724a56ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.895+0000 7f2b637a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55724a4cf340 con 0x55724a1aa4d0 2024-05-21T15:18:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b637a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55724a477290 con 0x55724a1cb1b0 2024-05-21T15:18:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b637a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55724a4cb300 con 0x55724a1a22f0 2024-05-21T15:18:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a56a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a56a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49782/0 (socket says 172.21.15.78:49782) 2024-05-21T15:18:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 -- 172.21.15.78:0/1679550442 learned_addr learned my addr 172.21.15.78:0/1679550442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 -- 172.21.15.78:0/1679550442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1a22f0 msgr2=0x55724a56a5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:18:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 --2- 172.21.15.78:0/1679550442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1a22f0 0x55724a56a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 -- 172.21.15.78:0/1679550442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1aa4d0 msgr2=0x55724a56ab00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:18:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 --2- 172.21.15.78:0/1679550442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1aa4d0 0x55724a56ab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 -- 172.21.15.78:0/1679550442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724a49b8e0 con 0x55724a1cb1b0 2024-05-21T15:18:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.896+0000 7f2b56d9d640 1 --2- 172.21.15.78:0/1679550442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a56a080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000a400 tx=0x7f2b50009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c884086b8698402 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.898+0000 7f2b477fe640 1 -- 172.21.15.78:0/3308439923 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2b4c018020 con 0x7f2b58107a10 2024-05-21T15:18:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.898+0000 7f2b44ff9640 1 -- 172.21.15.78:0/1679550442 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50013790 con 0x55724a1cb1b0 2024-05-21T15:18:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.898+0000 7f2b44ff9640 1 -- 172.21.15.78:0/1679550442 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b50019070 con 0x55724a1cb1b0 2024-05-21T15:18:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.899+0000 7f2b477fe640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2b400a2e40 0x7f2b400a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b477fe640 1 -- 172.21.15.78:0/3308439923 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2b4c0cec70 con 0x7f2b58107a10 2024-05-21T15:18:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b637a8740 1 -- 172.21.15.78:0/1679550442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 msgr2=0x55724a56a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b637a8740 1 --2- 172.21.15.78:0/1679550442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a56a080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000a400 tx=0x7f2b50009650 comp rx=0 tx=0).stop 2024-05-21T15:18:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b637a8740 1 -- 172.21.15.78:0/1679550442 shutdown_connections 2024-05-21T15:18:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b637a8740 1 --2- 172.21.15.78:0/1679550442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1aa4d0 0x55724a56ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b637a8740 1 --2- 172.21.15.78:0/1679550442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1a22f0 0x55724a56a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b637a8740 1 --2- 172.21.15.78:0/1679550442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a56a080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b637a8740 1 -- 172.21.15.78:0/1679550442 >> 172.21.15.78:0/1679550442 conn(0x55724a1bf300 msgr2=0x55724a566830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.900+0000 7f2b5659c640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2b400a2e40 0x7f2b400a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.903+0000 7f2b637a8740 1 -- 172.21.15.78:0/1679550442 shutdown_connections 2024-05-21T15:18:28.910 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.903+0000 7f2b637a8740 1 -- 172.21.15.78:0/1679550442 wait complete. 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.904+0000 7f2b637a8740 1 -- start start 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.904+0000 7f2b5659c640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2b400a2e40 0x7f2b400a5350 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b4802aa30 tx=0x7f2b48000bc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.904+0000 7f2b637a8740 1 Processor -- start 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.904+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1aa4d0 0x55724a574c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a57a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b637a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55724a577200 con 0x55724a1a22f0 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b637a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55724a48a790 con 0x55724a1cb1b0 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b637a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55724a577380 con 0x55724a1aa4d0 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b56d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a57a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b56d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a57a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49788/0 (socket says 172.21.15.78:49788) 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b56d9d640 1 -- 172.21.15.78:0/2297319109 learned_addr learned my addr 172.21.15.78:0/2297319109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b637a8740 10 client.0 _ll_register_callbacks cb 0x55724a4c5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.905+0000 7f2b56d9d640 1 -- 172.21.15.78:0/2297319109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1aa4d0 msgr2=0x55724a574c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.906+0000 7f2b56d9d640 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1aa4d0 0x55724a574c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.906+0000 7f2b56d9d640 1 -- 172.21.15.78:0/2297319109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 msgr2=0x55724a5746e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.906+0000 7f2b56d9d640 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5746e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.906+0000 7f2b56d9d640 1 -- 172.21.15.78:0/2297319109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724a1e3210 con 0x55724a1cb1b0 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.906+0000 7f2b56d9d640 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a57a6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b50014af0 tx=0x7f2b50014650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9febebf5f4291f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.906+0000 7f2b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.907+0000 7f2b3dffb640 1 -- 172.21.15.78:0/2297319109 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b500139a0 con 0x55724a1cb1b0 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.907+0000 7f2b637a8740 1 -- 172.21.15.78:0/2297319109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b50019070 con 0x55724a1cb1b0 2024-05-21T15:18:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.907+0000 7f2b3dffb640 1 -- 172.21.15.78:0/2297319109 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2b50013b40 con 0x55724a1cb1b0 2024-05-21T15:18:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.908+0000 7f2b3dffb640 1 -- 172.21.15.78:0/2297319109 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b50013d40 con 0x55724a1cb1b0 2024-05-21T15:18:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.909+0000 7f2b637a8740 10 client.25865 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.909+0000 7f2b637a8740 1 -- 172.21.15.78:0/2297319109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55724a2af620 con 0x55724a1cb1b0 2024-05-21T15:18:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.910+0000 7f2b3dffb640 1 -- 172.21.15.78:0/2297319109 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b50019070 con 0x55724a1cb1b0 2024-05-21T15:18:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.910+0000 7f2b3dffb640 1 -- 172.21.15.78:0/2297319109 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f2b5001e410 con 0x55724a1cb1b0 2024-05-21T15:18:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.910+0000 7f2b637a8740 10 client.25865 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.910+0000 7f2b637a8740 10 client.25865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.910+0000 7f2b637a8740 20 client.25865 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.910+0000 7f2b637a8740 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55724a189d80 0x55724a586e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.910+0000 7f2b637a8740 4 client.25865 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.910+0000 7f2b637a8740 1 -- 172.21.15.78:0/2297319109 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55724a4dca70 con 0x55724a189d80 2024-05-21T15:18:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.912+0000 7f2b5659c640 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55724a189d80 0x55724a586e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.913+0000 7f2b5659c640 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55724a189d80 0x55724a586e20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e9322e8819bbed server_cookie=ecac0ca0f435ce71 in_seq=0 out_seq=0 2024-05-21T15:18:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.913+0000 7f2b3dffb640 10 client.25865 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.915+0000 7f2b3dffb640 1 -- 172.21.15.78:0/2297319109 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55724a4dca70 con 0x55724a189d80 2024-05-21T15:18:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.915+0000 7f2b3dffb640 10 client.25865 handle_command_reply: tid=0 2024-05-21T15:18:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.916+0000 7f2b637a8740 1 client.25865 shutdown 2024-05-21T15:18:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.922+0000 7f2b637a8740 1 -- 172.21.15.78:0/2297319109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 msgr2=0x55724a57a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.922+0000 7f2b637a8740 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a57a6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b50014af0 tx=0x7f2b50014650 comp rx=0 tx=0).stop 2024-05-21T15:18:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.924+0000 7f2b637a8740 1 -- 172.21.15.78:0/2297319109 shutdown_connections 2024-05-21T15:18:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.924+0000 7f2b637a8740 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55724a189d80 0x55724a586e20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.924+0000 7f2b637a8740 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1cb1b0 0x55724a57a6d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.924+0000 7f2b637a8740 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1aa4d0 0x55724a574c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.924+0000 7f2b637a8740 1 --2- 172.21.15.78:0/2297319109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.924+0000 7f2b637a8740 1 -- 172.21.15.78:0/2297319109 >> 172.21.15.78:0/2297319109 conn(0x55724a1bf300 msgr2=0x55724a569eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.924+0000 7f2b3dffb640 0 client.25865 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.927+0000 7f2b637a8740 1 -- 172.21.15.78:0/2297319109 shutdown_connections 2024-05-21T15:18:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.927+0000 7f2b637a8740 1 -- 172.21.15.78:0/2297319109 wait complete. 2024-05-21T15:18:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.927+0000 7f2b637a8740 20 client.25865 trim_cache size 0 max 16384 2024-05-21T15:18:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.961+0000 7f2b637a8740 1 Processor -- start 2024-05-21T15:18:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.962+0000 7f2b637a8740 1 -- start start 2024-05-21T15:18:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.962+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1a22f0 0x55724a574df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.962+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a58c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.962+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1cb1b0 0x55724a58cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.962+0000 7f2b637a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581bdca0 con 0x55724a1cb1b0 2024-05-21T15:18:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.962+0000 7f2b637a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581bf8d0 con 0x55724a1aa4d0 2024-05-21T15:18:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.962+0000 7f2b637a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581be370 con 0x55724a1a22f0 2024-05-21T15:18:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.962+0000 7f2b55d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a58c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.963+0000 7f2b55d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a58c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49790/0 (socket says 172.21.15.78:49790) 2024-05-21T15:18:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.963+0000 7f2b55d9b640 1 -- 172.21.15.78:0/353491918 learned_addr learned my addr 172.21.15.78:0/353491918 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.963+0000 7f2b55d9b640 1 -- 172.21.15.78:0/353491918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1a22f0 msgr2=0x55724a574df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.963+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/353491918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1a22f0 0x55724a574df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.963+0000 7f2b55d9b640 1 -- 172.21.15.78:0/353491918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1cb1b0 msgr2=0x55724a58cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.963+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/353491918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1cb1b0 0x55724a58cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.963+0000 7f2b55d9b640 1 -- 172.21.15.78:0/353491918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724a56e7a0 con 0x55724a1aa4d0 2024-05-21T15:18:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.964+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/353491918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a58c620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c09b1d0 tx=0x7f2b4c09b130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e43d32b54acec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.964+0000 7f2b3ffff640 1 -- 172.21.15.78:0/353491918 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c09b240 con 0x55724a1aa4d0 2024-05-21T15:18:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.964+0000 7f2b3ffff640 1 -- 172.21.15.78:0/353491918 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c09f5f0 con 0x55724a1aa4d0 2024-05-21T15:18:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.965+0000 7f2b637a8740 1 -- 172.21.15.78:0/353491918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 msgr2=0x55724a58c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.965+0000 7f2b637a8740 1 --2- 172.21.15.78:0/353491918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a58c620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c09b1d0 tx=0x7f2b4c09b130 comp rx=0 tx=0).stop 2024-05-21T15:18:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.966+0000 7f2b637a8740 1 -- 172.21.15.78:0/353491918 shutdown_connections 2024-05-21T15:18:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.966+0000 7f2b637a8740 1 --2- 172.21.15.78:0/353491918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a58c620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.966+0000 7f2b637a8740 1 --2- 172.21.15.78:0/353491918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1cb1b0 0x55724a58cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.966+0000 7f2b637a8740 1 --2- 172.21.15.78:0/353491918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1a22f0 0x55724a574df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.966+0000 7f2b637a8740 1 -- 172.21.15.78:0/353491918 >> 172.21.15.78:0/353491918 conn(0x55724a1bf300 msgr2=0x55724a5746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.967+0000 7f2b637a8740 1 -- 172.21.15.78:0/353491918 shutdown_connections 2024-05-21T15:18:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.967+0000 7f2b637a8740 1 -- 172.21.15.78:0/353491918 wait complete. 2024-05-21T15:18:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 1 -- start start 2024-05-21T15:18:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 1 Processor -- start 2024-05-21T15:18:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a5e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1cb1b0 0x55724a5e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581bf140 con 0x55724a1a22f0 2024-05-21T15:18:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581bea40 con 0x55724a1aa4d0 2024-05-21T15:18:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581bbc80 con 0x55724a1cb1b0 2024-05-21T15:18:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.968+0000 7f2b637a8740 10 client.0 _ll_register_callbacks cb 0x7f2b50009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.969+0000 7f2b5659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.969+0000 7f2b5659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5e74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50966/0 (socket says 172.21.15.78:50966) 2024-05-21T15:18:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.969+0000 7f2b5659c640 1 -- 172.21.15.78:0/653893774 learned_addr learned my addr 172.21.15.78:0/653893774 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.970+0000 7f2b5659c640 1 -- 172.21.15.78:0/653893774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1cb1b0 msgr2=0x55724a5e7f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.970+0000 7f2b5659c640 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1cb1b0 0x55724a5e7f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.970+0000 7f2b5659c640 1 -- 172.21.15.78:0/653893774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 msgr2=0x55724a5e7a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.970+0000 7f2b5659c640 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a5e7a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.970+0000 7f2b5659c640 1 -- 172.21.15.78:0/653893774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724a5fe280 con 0x55724a1a22f0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.970+0000 7f2b5659c640 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5e74e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2b48004420 tx=0x7f2b48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a2b3e38b16bc64 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.970+0000 7f2b3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.971+0000 7f2b3dffb640 1 -- 172.21.15.78:0/653893774 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48035040 con 0x55724a1a22f0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.971+0000 7f2b637a8740 1 -- 172.21.15.78:0/653893774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b4c09f5f0 con 0x55724a1a22f0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.972+0000 7f2b3dffb640 1 -- 172.21.15.78:0/653893774 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2b48043040 con 0x55724a1a22f0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.972+0000 7f2b3dffb640 1 -- 172.21.15.78:0/653893774 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b4805d050 con 0x55724a1a22f0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.972+0000 7f2b637a8740 10 client.16416 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.972+0000 7f2b637a8740 1 -- 172.21.15.78:0/653893774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55724a5ff540 con 0x55724a1a22f0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.973+0000 7f2b3dffb640 1 -- 172.21.15.78:0/653893774 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c09f5f0 con 0x55724a1a22f0 2024-05-21T15:18:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.973+0000 7f2b3dffb640 1 -- 172.21.15.78:0/653893774 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f2b48060020 con 0x55724a1a22f0 2024-05-21T15:18:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.973+0000 7f2b637a8740 10 client.16416 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.973+0000 7f2b637a8740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.973+0000 7f2b637a8740 20 client.16416 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.973+0000 7f2b637a8740 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55724a15f1c0 0x55724a601e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.973+0000 7f2b637a8740 4 client.16416 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:18:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.973+0000 7f2b637a8740 1 -- 172.21.15.78:0/653893774 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55724a4d6dc0 con 0x55724a15f1c0 2024-05-21T15:18:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.974+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55724a15f1c0 0x55724a601e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.975+0000 7f2b3dffb640 10 client.16416 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:28.975+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55724a15f1c0 0x55724a601e50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1bd5704a4462e7d server_cookie=a430be2e676b38e7 in_seq=0 out_seq=0 2024-05-21T15:18:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:30 smithi084 ceph-mon[36334]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 146 MiB/s wr, 14.83k op/s 2024-05-21T15:18:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:30 smithi172 ceph-mon[36402]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 146 MiB/s wr, 14.83k op/s 2024-05-21T15:18:30.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:30 smithi078 ceph-mon[31997]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 146 MiB/s wr, 14.83k op/s 2024-05-21T15:18:32.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:32 smithi084 ceph-mon[36334]: pgmap v76: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 198 MiB/s wr, 22.69k op/s 2024-05-21T15:18:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:32 smithi172 ceph-mon[36402]: pgmap v76: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 198 MiB/s wr, 22.69k op/s 2024-05-21T15:18:32.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:32 smithi078 ceph-mon[31997]: pgmap v76: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 198 MiB/s wr, 22.69k op/s 2024-05-21T15:18:34.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:34 smithi084 ceph-mon[36334]: pgmap v77: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 89 MiB/s wr, 13.53k op/s 2024-05-21T15:18:34.649 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:34 smithi078 ceph-mon[31997]: pgmap v77: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 89 MiB/s wr, 13.53k op/s 2024-05-21T15:18:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:34 smithi172 ceph-mon[36402]: pgmap v77: 97 pgs: 97 active+clean; 14 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 89 MiB/s wr, 13.53k op/s 2024-05-21T15:18:36.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:36 smithi084 ceph-mon[36334]: pgmap v78: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 135 MiB/s wr, 20.74k op/s 2024-05-21T15:18:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:36 smithi172 ceph-mon[36402]: pgmap v78: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 135 MiB/s wr, 20.74k op/s 2024-05-21T15:18:36.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:36 smithi078 ceph-mon[31997]: pgmap v78: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 135 MiB/s wr, 20.74k op/s 2024-05-21T15:18:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.053+0000 7f2b3dffb640 1 -- 172.21.15.78:0/653893774 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55724a4d6dc0 con 0x55724a15f1c0 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.053+0000 7f2b3dffb640 10 client.16416 handle_command_reply: tid=0 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.053+0000 7f2b637a8740 1 client.16416 shutdown 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 -- 172.21.15.78:0/653893774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 msgr2=0x55724a5e74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5e74e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2b48004420 tx=0x7f2b48009260 comp rx=0 tx=0).stop 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 -- 172.21.15.78:0/653893774 shutdown_connections 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55724a15f1c0 0x55724a601e50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55724a1aa4d0 0x55724a5e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55724a1cb1b0 0x55724a5e7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 --2- 172.21.15.78:0/653893774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55724a1a22f0 0x55724a5e74e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 -- 172.21.15.78:0/653893774 >> 172.21.15.78:0/653893774 conn(0x55724a1bf300 msgr2=0x55724a575b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b3dffb640 0 client.16416 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 -- 172.21.15.78:0/653893774 shutdown_connections 2024-05-21T15:18:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 1 -- 172.21.15.78:0/653893774 wait complete. 2024-05-21T15:18:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.054+0000 7f2b637a8740 20 client.16416 trim_cache size 0 max 16384 2024-05-21T15:18:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:18:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:18:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:18:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:18:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.055+0000 7f2b5c907640 1 -- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2b400a2e40 msgr2=0x7f2b400a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.055+0000 7f2b5c907640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2b400a2e40 0x7f2b400a5350 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b4802aa30 tx=0x7f2b48000bc0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.055+0000 7f2b5c907640 1 -- 172.21.15.78:0/3308439923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b58107a10 msgr2=0x7f2b5819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.055+0000 7f2b5c907640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b58107a10 0x7f2b5819cbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00b450 tx=0x7f2b4c00b910 comp rx=0 tx=0).stop 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.056+0000 7f2b5c907640 1 -- 172.21.15.78:0/3308439923 shutdown_connections 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.056+0000 7f2b5c907640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2b400a2e40 0x7f2b400a5350 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.056+0000 7f2b5c907640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b58107eb0 0x7f2b58197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.056+0000 7f2b5c907640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b58107a10 0x7f2b5819cbe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.056+0000 7f2b5c907640 1 --2- 172.21.15.78:0/3308439923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b58102170 0x7f2b5819c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.056+0000 7f2b5c907640 1 -- 172.21.15.78:0/3308439923 >> 172.21.15.78:0/3308439923 conn(0x7f2b58077160 msgr2=0x7f2b58100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.056+0000 7f2b5c907640 1 -- 172.21.15.78:0/3308439923 shutdown_connections 2024-05-21T15:18:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:37.056+0000 7f2b5c907640 1 -- 172.21.15.78:0/3308439923 wait complete. 2024-05-21T15:18:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:38 smithi172 ceph-mon[36402]: pgmap v79: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 110 MiB/s wr, 16.91k op/s 2024-05-21T15:18:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:38 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:18:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:38 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:18:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:38 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:18:38.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:38 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:38.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:38 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:38.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:38 smithi078 ceph-mon[31997]: pgmap v79: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 110 MiB/s wr, 16.91k op/s 2024-05-21T15:18:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:38 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:18:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:38 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:18:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:38 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:18:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:38 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:38 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:38.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:38 smithi084 ceph-mon[36334]: pgmap v79: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 110 MiB/s wr, 16.91k op/s 2024-05-21T15:18:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:38 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:18:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:38 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:18:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:38 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:18:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:38 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:38 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:39.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:39 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:39.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:39 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:39.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:39 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:39.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:39 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:39.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:39 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:39.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:39 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:18:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:40 smithi172 ceph-mon[36402]: pgmap v80: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 104 MiB/s wr, 15.87k op/s 2024-05-21T15:18:40.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:40 smithi078 ceph-mon[31997]: pgmap v80: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 104 MiB/s wr, 15.87k op/s 2024-05-21T15:18:40.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:40 smithi084 ceph-mon[36334]: pgmap v80: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 104 MiB/s wr, 15.87k op/s 2024-05-21T15:18:42.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:42 smithi078 ceph-mon[31997]: pgmap v81: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 151 MiB/s wr, 23.08k op/s 2024-05-21T15:18:42.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:42 smithi084 ceph-mon[36334]: pgmap v81: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 151 MiB/s wr, 23.08k op/s 2024-05-21T15:18:42.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:42 smithi172 ceph-mon[36402]: pgmap v81: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 151 MiB/s wr, 23.08k op/s 2024-05-21T15:18:44.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:44 smithi078 ceph-mon[31997]: pgmap v82: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 99 MiB/s wr, 15.23k op/s 2024-05-21T15:18:44.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:44 smithi084 ceph-mon[36334]: pgmap v82: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 99 MiB/s wr, 15.23k op/s 2024-05-21T15:18:44.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:44 smithi172 ceph-mon[36402]: pgmap v82: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 99 MiB/s wr, 15.23k op/s 2024-05-21T15:18:46.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:46 smithi078 ceph-mon[31997]: pgmap v83: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 126 MiB/s wr, 19.07k op/s 2024-05-21T15:18:46.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:46 smithi084 ceph-mon[36334]: pgmap v83: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 126 MiB/s wr, 19.07k op/s 2024-05-21T15:18:46.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:46 smithi172 ceph-mon[36402]: pgmap v83: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 126 MiB/s wr, 19.07k op/s 2024-05-21T15:18:47.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:47 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:18:47.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:47 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:18:47.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:47 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:18:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:47 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:18:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:47 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:18:47.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:47 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:18:47.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:47 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:18:47.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:47 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:18:47.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:47 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:18:48.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:48 smithi078 ceph-mon[31997]: pgmap v84: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 123 MiB/s wr, 18.02k op/s 2024-05-21T15:18:48.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:48 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:18:48.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:48 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:18:48.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:48 smithi084 ceph-mon[36334]: pgmap v84: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 123 MiB/s wr, 18.02k op/s 2024-05-21T15:18:48.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:48 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:18:48.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:48 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:18:48.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:48 smithi172 ceph-mon[36402]: pgmap v84: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 123 MiB/s wr, 18.02k op/s 2024-05-21T15:18:48.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:48 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:18:48.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:48 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.225+0000 7f134633e640 1 -- 172.21.15.78:0/1756532601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1340108fd0 msgr2=0x7f1340102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.225+0000 7f134633e640 1 --2- 172.21.15.78:0/1756532601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1340108fd0 0x7f1340102c10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1334009900 tx=0x7f133402a3f0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.225+0000 7f134633e640 1 -- 172.21.15.78:0/1756532601 shutdown_connections 2024-05-21T15:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.225+0000 7f134633e640 1 --2- 172.21.15.78:0/1756532601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1340103280 0x7f134010f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.225+0000 7f134633e640 1 --2- 172.21.15.78:0/1756532601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1340108fd0 0x7f1340102c10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.225+0000 7f134633e640 1 --2- 172.21.15.78:0/1756532601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1340108b30 0x7f13401026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.225+0000 7f134633e640 1 -- 172.21.15.78:0/1756532601 >> 172.21.15.78:0/1756532601 conn(0x7f1340078220 msgr2=0x7f1340078660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.226+0000 7f134633e640 1 -- 172.21.15.78:0/1756532601 shutdown_connections 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.226+0000 7f134633e640 1 -- 172.21.15.78:0/1756532601 wait complete. 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.226+0000 7f134633e640 1 Processor -- start 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.226+0000 7f134633e640 1 -- start start 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.227+0000 7f134633e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1340103280 0x7f134010aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.227+0000 7f134633e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1340108b30 0x7f134010afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.227+0000 7f134633e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1340108fd0 0x7f13401aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.227+0000 7f133ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1340103280 0x7f134010aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.227+0000 7f133ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1340103280 0x7f134010aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52256/0 (socket says 172.21.15.78:52256) 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.227+0000 7f133ffff640 1 -- 172.21.15.78:0/1364613105 learned_addr learned my addr 172.21.15.78:0/1364613105 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f133f7fe640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1340108b30 0x7f134010afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f13448db640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1340108fd0 0x7f13401aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.227+0000 7f134633e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1340075b70 con 0x7f1340103280 2024-05-21T15:18:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13400759f0 con 0x7f1340108fd0 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1340075cf0 con 0x7f1340108b30 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f133ffff640 1 -- 172.21.15.78:0/1364613105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1340108b30 msgr2=0x7f134010afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f133ffff640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1340108b30 0x7f134010afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f133ffff640 1 -- 172.21.15.78:0/1364613105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1340108fd0 msgr2=0x7f13401aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f133ffff640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1340108fd0 0x7f13401aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f133ffff640 1 -- 172.21.15.78:0/1364613105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13401abf80 con 0x7f1340103280 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.228+0000 7f13448db640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1340108fd0 0x7f13401aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.229+0000 7f133ffff640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1340103280 0x7f134010aa80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f133000e9c0 tx=0x7f133000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.229+0000 7f133d7fa640 1 -- 172.21.15.78:0/1364613105 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133000cd20 con 0x7f1340103280 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.229+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13401ac270 con 0x7f1340103280 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.229+0000 7f133d7fa640 1 -- 172.21.15.78:0/1364613105 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1330005500 con 0x7f1340103280 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.229+0000 7f133d7fa640 1 -- 172.21.15.78:0/1364613105 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330010690 con 0x7f1340103280 2024-05-21T15:18:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.229+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13401ac810 con 0x7f1340103280 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.232+0000 7f133d7fa640 1 -- 172.21.15.78:0/1364613105 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1330010830 con 0x7f1340103280 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.232+0000 7f134d1e0740 1 Processor -- start 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.232+0000 7f134d1e0740 1 -- start start 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be04e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be04e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be04eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f134d1e0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5640bdfb3340 con 0x5640bdc8e4d0 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f134d1e0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5640bdf5b290 con 0x5640bdcaf1b0 2024-05-21T15:18:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f134d1e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5640bdfaf300 con 0x5640bdc862f0 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f13448db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be04e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f13448db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be04e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44932/0 (socket says 172.21.15.78:44932) 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f13448db640 1 -- 172.21.15.78:0/247246790 learned_addr learned my addr 172.21.15.78:0/247246790 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f133ffff640 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be04e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f133d7fa640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13280a2cf0 0x7f13280a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.233+0000 7f133f7fe640 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be04eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f133d7fa640 1 -- 172.21.15.78:0/1364613105 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f13300cf7f0 con 0x7f1340103280 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f133ffff640 1 -- 172.21.15.78:0/247246790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 msgr2=0x5640be04e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f133ffff640 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be04e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f133ffff640 1 -- 172.21.15.78:0/247246790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 msgr2=0x5640be04eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f133ffff640 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be04eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f133ffff640 1 -- 172.21.15.78:0/247246790 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640bdf7f8e0 con 0x5640bdc862f0 2024-05-21T15:18:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f13448db640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13280a2cf0 0x7f13280a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f133ffff640 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be04e5c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f13300ccee0 tx=0x7f1330009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1bc10055c92d9b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f13267fc640 1 -- 172.21.15.78:0/247246790 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133009b610 con 0x5640bdc862f0 2024-05-21T15:18:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.234+0000 7f13448db640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13280a2cf0 0x7f13280a5200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5640be04b1f0 tx=0x7f1338004a40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.235+0000 7f13267fc640 1 -- 172.21.15.78:0/247246790 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f133009c3b0 con 0x5640bdc862f0 2024-05-21T15:18:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.235+0000 7f13267fc640 1 -- 172.21.15.78:0/247246790 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300046c0 con 0x5640bdc862f0 2024-05-21T15:18:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 -- 172.21.15.78:0/247246790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 msgr2=0x5640be04e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be04e5c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f13300ccee0 tx=0x7f1330009700 comp rx=0 tx=0).stop 2024-05-21T15:18:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 -- 172.21.15.78:0/247246790 shutdown_connections 2024-05-21T15:18:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be04eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be04e5c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 --2- 172.21.15.78:0/247246790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be04e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 -- 172.21.15.78:0/247246790 >> 172.21.15.78:0/247246790 conn(0x5640bdca3300 msgr2=0x5640be04a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 -- 172.21.15.78:0/247246790 shutdown_connections 2024-05-21T15:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 -- 172.21.15.78:0/247246790 wait complete. 2024-05-21T15:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.236+0000 7f134d1e0740 1 -- start start 2024-05-21T15:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.237+0000 7f134d1e0740 1 Processor -- start 2024-05-21T15:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.237+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be05a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.237+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be05a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.237+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be058620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f134d1e0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5640be05b150 con 0x5640bdc8e4d0 2024-05-21T15:18:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f134d1e0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5640bdf6e790 con 0x5640bdcaf1b0 2024-05-21T15:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f134d1e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5640be05b2d0 con 0x5640bdc862f0 2024-05-21T15:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be05a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be05a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f134d1e0740 10 client.0 _ll_register_callbacks cb 0x5640bdfa9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be05a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52282/0 (socket says 172.21.15.78:52282) 2024-05-21T15:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133f7fe640 1 -- 172.21.15.78:0/151877187 learned_addr learned my addr 172.21.15.78:0/151877187 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f13448db640 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be058620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133f7fe640 1 -- 172.21.15.78:0/151877187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 msgr2=0x5640be05a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133f7fe640 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be05a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133f7fe640 1 -- 172.21.15.78:0/151877187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 msgr2=0x5640be058620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133f7fe640 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be058620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.238+0000 7f133f7fe640 1 -- 172.21.15.78:0/151877187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640bdcc7210 con 0x5640bdc8e4d0 2024-05-21T15:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.239+0000 7f133f7fe640 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be05a750 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1334009230 tx=0x7f1334005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9c42e131657a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.239+0000 7f131f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.239+0000 7f131f7fe640 1 -- 172.21.15.78:0/151877187 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334035040 con 0x5640bdc8e4d0 2024-05-21T15:18:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.239+0000 7f131f7fe640 1 -- 172.21.15.78:0/151877187 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f1334004040 con 0x5640bdc8e4d0 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.239+0000 7f134d1e0740 1 -- 172.21.15.78:0/151877187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f133009c3b0 con 0x5640bdc8e4d0 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.240+0000 7f131f7fe640 1 -- 172.21.15.78:0/151877187 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13340319f0 con 0x5640bdc8e4d0 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.240+0000 7f134d1e0740 10 client.16428 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.240+0000 7f134d1e0740 1 -- 172.21.15.78:0/151877187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640bdd93620 con 0x5640bdc8e4d0 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.240+0000 7f131f7fe640 1 -- 172.21.15.78:0/151877187 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f133009c3b0 con 0x5640bdc8e4d0 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.241+0000 7f131f7fe640 1 -- 172.21.15.78:0/151877187 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f1334004980 con 0x5640bdc8e4d0 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.241+0000 7f134d1e0740 10 client.16428 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.241+0000 7f134d1e0740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.241+0000 7f134d1e0740 20 client.16428 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.241+0000 7f134d1e0740 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5640bdc6dd80 0x5640be06aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.241+0000 7f134d1e0740 4 client.16428 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.241+0000 7f134d1e0740 1 -- 172.21.15.78:0/151877187 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640bdfc0a70 con 0x5640bdc6dd80 2024-05-21T15:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.241+0000 7f133ffff640 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5640bdc6dd80 0x5640be06aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.242+0000 7f133ffff640 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5640bdc6dd80 0x5640be06aa10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ee6e82657b2ceb server_cookie=5ee8fa6d277fd41b in_seq=0 out_seq=0 2024-05-21T15:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.243+0000 7f131f7fe640 10 client.16428 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f131f7fe640 1 -- 172.21.15.78:0/151877187 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5640bdfc0a70 con 0x5640bdc6dd80 2024-05-21T15:18:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f131f7fe640 10 client.16428 handle_command_reply: tid=0 2024-05-21T15:18:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 client.16428 shutdown 2024-05-21T15:18:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 -- 172.21.15.78:0/151877187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 msgr2=0x5640be05a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be05a750 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1334009230 tx=0x7f1334005f20 comp rx=0 tx=0).stop 2024-05-21T15:18:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 -- 172.21.15.78:0/151877187 shutdown_connections 2024-05-21T15:18:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5640bdc6dd80 0x5640be06aa10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdcaf1b0 0x5640be058620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f131f7fe640 0 client.16428 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be05a750 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 --2- 172.21.15.78:0/151877187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be05a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 -- 172.21.15.78:0/151877187 >> 172.21.15.78:0/151877187 conn(0x5640bdca3300 msgr2=0x5640be04fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 -- 172.21.15.78:0/151877187 shutdown_connections 2024-05-21T15:18:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 1 -- 172.21.15.78:0/151877187 wait complete. 2024-05-21T15:18:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.244+0000 7f134d1e0740 20 client.16428 trim_cache size 0 max 16384 2024-05-21T15:18:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.268+0000 7f134d1e0740 1 Processor -- start 2024-05-21T15:18:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.268+0000 7f134d1e0740 1 -- start start 2024-05-21T15:18:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.268+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc6dd80 0x5640be04a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.268+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be060550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.268+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be060a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f134d1e0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13401b5410 con 0x5640bdc8e4d0 2024-05-21T15:18:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f134d1e0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13401b7040 con 0x5640bdc6dd80 2024-05-21T15:18:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f134d1e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13401b5ae0 con 0x5640bdc862f0 2024-05-21T15:18:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f13448db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be060a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f133ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc6dd80 0x5640be04a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f13448db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be060a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52286/0 (socket says 172.21.15.78:52286) 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f133f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be060550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f13448db640 1 -- 172.21.15.78:0/2183480655 learned_addr learned my addr 172.21.15.78:0/2183480655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f133ffff640 1 -- 172.21.15.78:0/2183480655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 msgr2=0x5640be060550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f133ffff640 1 --2- 172.21.15.78:0/2183480655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be060550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f133ffff640 1 -- 172.21.15.78:0/2183480655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 msgr2=0x5640be060a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f133ffff640 1 --2- 172.21.15.78:0/2183480655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be060a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f133ffff640 1 -- 172.21.15.78:0/2183480655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640be0590c0 con 0x5640bdc6dd80 2024-05-21T15:18:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.269+0000 7f133ffff640 1 --2- 172.21.15.78:0/2183480655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc6dd80 0x5640be04a860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f133009be70 tx=0x7f1330004b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6d643b4fea300f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.270+0000 7f1325ffb640 1 -- 172.21.15.78:0/2183480655 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133009c3b0 con 0x5640bdc6dd80 2024-05-21T15:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.270+0000 7f1325ffb640 1 -- 172.21.15.78:0/2183480655 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1330084070 con 0x5640bdc6dd80 2024-05-21T15:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.270+0000 7f1325ffb640 1 -- 172.21.15.78:0/2183480655 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300046c0 con 0x5640bdc6dd80 2024-05-21T15:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 -- 172.21.15.78:0/2183480655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc6dd80 msgr2=0x5640be04a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2183480655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc6dd80 0x5640be04a860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f133009be70 tx=0x7f1330004b30 comp rx=0 tx=0).stop 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 -- 172.21.15.78:0/2183480655 shutdown_connections 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2183480655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc8e4d0 0x5640be060a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2183480655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc862f0 0x5640be060550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2183480655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc6dd80 0x5640be04a860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 -- 172.21.15.78:0/2183480655 >> 172.21.15.78:0/2183480655 conn(0x5640bdca3300 msgr2=0x5640be048de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 -- 172.21.15.78:0/2183480655 shutdown_connections 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.271+0000 7f134d1e0740 1 -- 172.21.15.78:0/2183480655 wait complete. 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.272+0000 7f134d1e0740 1 -- start start 2024-05-21T15:18:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.272+0000 7f134d1e0740 1 Processor -- start 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.272+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc6dd80 0x5640be060550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc862f0 0x5640be060a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f134d1e0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc8e4d0 0x5640be05ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f134d1e0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13401b68b0 con 0x5640bdc862f0 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f134d1e0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13401b61b0 con 0x5640bdc8e4d0 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f134d1e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13401b33f0 con 0x5640bdc6dd80 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f134d1e0740 10 client.0 _ll_register_callbacks cb 0x7f1334009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc862f0 0x5640be060a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc862f0 0x5640be060a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52298/0 (socket says 172.21.15.78:52298) 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133f7fe640 1 -- 172.21.15.78:0/2827562833 learned_addr learned my addr 172.21.15.78:0/2827562833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133ffff640 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc6dd80 0x5640be060550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f13448db640 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc8e4d0 0x5640be05ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133f7fe640 1 -- 172.21.15.78:0/2827562833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc6dd80 msgr2=0x5640be060550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133f7fe640 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc6dd80 0x5640be060550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133f7fe640 1 -- 172.21.15.78:0/2827562833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc8e4d0 msgr2=0x5640be05ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133f7fe640 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc8e4d0 0x5640be05ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.273+0000 7f133f7fe640 1 -- 172.21.15.78:0/2827562833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640be05c1d0 con 0x5640bdc862f0 2024-05-21T15:18:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f133ffff640 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc6dd80 0x5640be060550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f133f7fe640 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc862f0 0x5640be060a90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x5640be011ca0 tx=0x7f1334000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf9c06aada9c5e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f131f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f131f7fe640 1 -- 172.21.15.78:0/2827562833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334035070 con 0x5640bdc862f0 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f131f7fe640 1 -- 172.21.15.78:0/2827562833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f13340035f0 con 0x5640bdc862f0 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f134d1e0740 1 -- 172.21.15.78:0/2827562833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1330084070 con 0x5640bdc862f0 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f131f7fe640 1 -- 172.21.15.78:0/2827562833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1334052040 con 0x5640bdc862f0 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f134d1e0740 10 client.16434 fetch_fsmap learned FSMap version 29 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.274+0000 7f134d1e0740 1 -- 172.21.15.78:0/2827562833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640be0d28e0 con 0x5640bdc862f0 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f131f7fe640 1 -- 172.21.15.78:0/2827562833 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1334005360 con 0x5640bdc862f0 2024-05-21T15:18:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f131f7fe640 1 -- 172.21.15.78:0/2827562833 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f1334057a40 con 0x5640bdc862f0 2024-05-21T15:18:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f134d1e0740 10 client.16434 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T15:18:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f134d1e0740 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f134d1e0740 20 client.16434 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5640bdcaf1b0 0x5640be0d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f134d1e0740 4 client.16434 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:18:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f134d1e0740 1 -- 172.21.15.78:0/2827562833 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1330084070 con 0x5640bdcaf1b0 2024-05-21T15:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.275+0000 7f133ffff640 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5640bdcaf1b0 0x5640be0d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.276+0000 7f131f7fe640 10 client.16434 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.276+0000 7f133ffff640 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5640bdcaf1b0 0x5640be0d5130 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3f2c7dca096e20 server_cookie=d521b1ced9bdbdf2 in_seq=0 out_seq=0 2024-05-21T15:18:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f131f7fe640 1 -- 172.21.15.78:0/2827562833 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1330084070 con 0x5640bdcaf1b0 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f131f7fe640 10 client.16434 handle_command_reply: tid=0 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 client.16434 shutdown 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 -- 172.21.15.78:0/2827562833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc862f0 msgr2=0x5640be060a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc862f0 0x5640be060a90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x5640be011ca0 tx=0x7f1334000bc0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 -- 172.21.15.78:0/2827562833 shutdown_connections 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5640bdcaf1b0 0x5640be0d5130 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5640bdc8e4d0 0x5640be05ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5640bdc862f0 0x5640be060a90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 --2- 172.21.15.78:0/2827562833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640bdc6dd80 0x5640be060550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 -- 172.21.15.78:0/2827562833 >> 172.21.15.78:0/2827562833 conn(0x5640bdca3300 msgr2=0x5640be050290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f131f7fe640 0 client.16434 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 -- 172.21.15.78:0/2827562833 shutdown_connections 2024-05-21T15:18:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.298+0000 7f134d1e0740 1 -- 172.21.15.78:0/2827562833 wait complete. 2024-05-21T15:18:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.299+0000 7f134d1e0740 20 client.16434 trim_cache size 0 max 16384 2024-05-21T15:18:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:18:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:18:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:18:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:18:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.299+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13280a2cf0 msgr2=0x7f13280a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.299+0000 7f134633e640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13280a2cf0 0x7f13280a5200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5640be04b1f0 tx=0x7f1338004a40 comp rx=0 tx=0).stop 2024-05-21T15:18:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.299+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1340103280 msgr2=0x7f134010aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.299+0000 7f134633e640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1340103280 0x7f134010aa80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f133000e9c0 tx=0x7f133000ee80 comp rx=0 tx=0).stop 2024-05-21T15:18:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.300+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 shutdown_connections 2024-05-21T15:18:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.300+0000 7f134633e640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1340108fd0 0x7f13401aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.300+0000 7f134633e640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13280a2cf0 0x7f13280a5200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.300+0000 7f134633e640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1340108b30 0x7f134010afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.300+0000 7f134633e640 1 --2- 172.21.15.78:0/1364613105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1340103280 0x7f134010aa80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.300+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 >> 172.21.15.78:0/1364613105 conn(0x7f1340078220 msgr2=0x7f1340101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.300+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 shutdown_connections 2024-05-21T15:18:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:18:50.300+0000 7f134633e640 1 -- 172.21.15.78:0/1364613105 wait complete. 2024-05-21T15:18:50.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:50 smithi078 ceph-mon[31997]: pgmap v85: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 118 MiB/s wr, 17.21k op/s 2024-05-21T15:18:50.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:50 smithi084 ceph-mon[36334]: pgmap v85: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 118 MiB/s wr, 17.21k op/s 2024-05-21T15:18:50.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:50 smithi172 ceph-mon[36402]: pgmap v85: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 118 MiB/s wr, 17.21k op/s 2024-05-21T15:18:51.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:51 smithi078 ceph-mon[31997]: pgmap v86: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 136 MiB/s wr, 20.35k op/s 2024-05-21T15:18:51.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:51 smithi084 ceph-mon[36334]: pgmap v86: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 136 MiB/s wr, 20.35k op/s 2024-05-21T15:18:51.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:51 smithi172 ceph-mon[36402]: pgmap v86: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 136 MiB/s wr, 20.35k op/s 2024-05-21T15:18:54.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:53 smithi078 ceph-mon[31997]: pgmap v87: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 89 MiB/s wr, 13.13k op/s 2024-05-21T15:18:54.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:53 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:54.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:53 smithi084 ceph-mon[36334]: pgmap v87: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 89 MiB/s wr, 13.13k op/s 2024-05-21T15:18:54.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:53 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:54.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:53 smithi172 ceph-mon[36402]: pgmap v87: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 89 MiB/s wr, 13.13k op/s 2024-05-21T15:18:54.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:53 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:54.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:18:54.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:18:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/403222896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3810c330 msgr2=0x7fcd3810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/403222896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3810c330 0x7fcd3810e750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009ca0 tx=0x7fcd3002a8f0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/403222896 shutdown_connections 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/403222896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd3810c330 0x7fcd3810e750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/403222896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd38101aa0 0x7fcd38101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/403222896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd381079d0 0x7fcd38101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/403222896 >> 172.21.15.78:0/403222896 conn(0x7fcd380770b0 msgr2=0x7fcd380774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/403222896 shutdown_connections 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.813+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/403222896 wait complete. 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.814+0000 7fcd3f2a6640 1 Processor -- start 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.814+0000 7fcd3f2a6640 1 -- start start 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.815+0000 7fcd3f2a6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd38101aa0 0x7fcd381972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.815+0000 7fcd3f2a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd381079d0 0x7fcd38197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.815+0000 7fcd3f2a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3810c330 0x7fcd3819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.815+0000 7fcd3f2a6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38110c20 con 0x7fcd3810c330 2024-05-21T15:18:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.815+0000 7fcd3f2a6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38110aa0 con 0x7fcd38101aa0 2024-05-21T15:18:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.815+0000 7fcd3f2a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38110da0 con 0x7fcd381079d0 2024-05-21T15:18:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.816+0000 7fcd3d042640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd38101aa0 0x7fcd381972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.816+0000 7fcd3c841640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd381079d0 0x7fcd38197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.816+0000 7fcd3d042640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd38101aa0 0x7fcd381972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44970/0 (socket says 172.21.15.78:44970) 2024-05-21T15:18:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.816+0000 7fcd3d042640 1 -- 172.21.15.78:0/2011839629 learned_addr learned my addr 172.21.15.78:0/2011839629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.816+0000 7fcd3d843640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3810c330 0x7fcd3819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.817+0000 7fcd3d843640 1 -- 172.21.15.78:0/2011839629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd381079d0 msgr2=0x7fcd38197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.817+0000 7fcd3d843640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd381079d0 0x7fcd38197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.817+0000 7fcd3d843640 1 -- 172.21.15.78:0/2011839629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd38101aa0 msgr2=0x7fcd381972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.817+0000 7fcd3d843640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd38101aa0 0x7fcd381972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.817+0000 7fcd3d843640 1 -- 172.21.15.78:0/2011839629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3819c140 con 0x7fcd3810c330 2024-05-21T15:18:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.817+0000 7fcd3d843640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3810c330 0x7fcd3819bb00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcd3002ae10 tx=0x7fcd30031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:54.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.821+0000 7fcd2e7fc640 1 -- 172.21.15.78:0/2011839629 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3003fc20 con 0x7fcd3810c330 2024-05-21T15:18:54.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.821+0000 7fcd2e7fc640 1 -- 172.21.15.78:0/2011839629 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd3002d070 con 0x7fcd3810c330 2024-05-21T15:18:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.821+0000 7fcd2e7fc640 1 -- 172.21.15.78:0/2011839629 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd300309e0 con 0x7fcd3810c330 2024-05-21T15:18:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.822+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/2011839629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3819c3d0 con 0x7fcd3810c330 2024-05-21T15:18:54.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.822+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/2011839629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3819c8a0 con 0x7fcd3810c330 2024-05-21T15:18:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.826+0000 7fcd2e7fc640 1 -- 172.21.15.78:0/2011839629 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcd3002fbb0 con 0x7fcd3810c330 2024-05-21T15:18:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.827+0000 7fcd4621f740 1 Processor -- start 2024-05-21T15:18:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.828+0000 7fcd2e7fc640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcd240a2e30 0x7fcd240a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.828+0000 7fcd4621f740 1 -- start start 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.828+0000 7fcd2e7fc640 1 -- 172.21.15.78:0/2011839629 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcd300f4b20 con 0x7fcd3810c330 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e85416d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 0x560e85417270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e854177b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3c841640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcd240a2e30 0x7fcd240a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd4621f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e8537cd00 con 0x560e85078390 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd4621f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560e853244c0 con 0x560e850576b0 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd4621f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560e85378cc0 con 0x560e8504f4d0 2024-05-21T15:18:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d843640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e854177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d843640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e854177b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44976/0 (socket says 172.21.15.78:44976) 2024-05-21T15:18:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d843640 1 -- 172.21.15.78:0/1531404232 learned_addr learned my addr 172.21.15.78:0/1531404232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d042640 1 --2- 172.21.15.78:0/1531404232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e85416d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d843640 1 -- 172.21.15.78:0/1531404232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 msgr2=0x560e85417270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d843640 1 --2- 172.21.15.78:0/1531404232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 0x560e85417270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d843640 1 -- 172.21.15.78:0/1531404232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 msgr2=0x560e85416d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d843640 1 --2- 172.21.15.78:0/1531404232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e85416d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.829+0000 7fcd3d843640 1 -- 172.21.15.78:0/1531404232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e8515a7d0 con 0x560e850576b0 2024-05-21T15:18:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.830+0000 7fcd3d843640 1 --2- 172.21.15.78:0/1531404232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e854177b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd300bd550 tx=0x7fcd300c17f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a78cce2b7c5287d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.830+0000 7fcd3c841640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcd240a2e30 0x7fcd240a5340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcd38100f70 tx=0x7fcd34009830 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.833+0000 7fcd237fe640 1 -- 172.21.15.78:0/1531404232 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd300c54a0 con 0x560e850576b0 2024-05-21T15:18:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.833+0000 7fcd237fe640 1 -- 172.21.15.78:0/1531404232 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd30036040 con 0x560e850576b0 2024-05-21T15:18:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.833+0000 7fcd237fe640 1 -- 172.21.15.78:0/1531404232 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd300bd760 con 0x560e850576b0 2024-05-21T15:18:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.835+0000 7fcd4621f740 1 -- 172.21.15.78:0/1531404232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 msgr2=0x560e854177b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.835+0000 7fcd4621f740 1 --2- 172.21.15.78:0/1531404232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e854177b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd300bd550 tx=0x7fcd300c17f0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.837+0000 7fcd4621f740 1 -- 172.21.15.78:0/1531404232 shutdown_connections 2024-05-21T15:18:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.837+0000 7fcd4621f740 1 --2- 172.21.15.78:0/1531404232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e854177b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.837+0000 7fcd4621f740 1 --2- 172.21.15.78:0/1531404232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 0x560e85417270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.837+0000 7fcd4621f740 1 --2- 172.21.15.78:0/1531404232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e85416d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.837+0000 7fcd4621f740 1 -- 172.21.15.78:0/1531404232 >> 172.21.15.78:0/1531404232 conn(0x560e8506c4e0 msgr2=0x560e854134e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.838+0000 7fcd4621f740 1 -- 172.21.15.78:0/1531404232 shutdown_connections 2024-05-21T15:18:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.839+0000 7fcd4621f740 1 -- 172.21.15.78:0/1531404232 wait complete. 2024-05-21T15:18:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.840+0000 7fcd4621f740 1 -- start start 2024-05-21T15:18:54.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.840+0000 7fcd4621f740 1 Processor -- start 2024-05-21T15:18:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8504f4d0 0x560e85422ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e850576b0 0x560e85423410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e85078390 0x560e854212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd4621f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560e85423ea0 con 0x560e8504f4d0 2024-05-21T15:18:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd4621f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560e85337b90 con 0x560e85078390 2024-05-21T15:18:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd4621f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560e85424020 con 0x560e850576b0 2024-05-21T15:18:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd3d843640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e85078390 0x560e854212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd3d843640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e85078390 0x560e854212e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44980/0 (socket says 172.21.15.78:44980) 2024-05-21T15:18:54.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.841+0000 7fcd3d843640 1 -- 172.21.15.78:0/3382596405 learned_addr learned my addr 172.21.15.78:0/3382596405 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:54.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.842+0000 7fcd4621f740 10 client.0 _ll_register_callbacks cb 0x560e85373da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:54.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.842+0000 7fcd3d042640 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8504f4d0 0x560e85422ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.842+0000 7fcd3c841640 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e850576b0 0x560e85423410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.844+0000 7fcd3d042640 1 -- 172.21.15.78:0/3382596405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e850576b0 msgr2=0x560e85423410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.844+0000 7fcd3d042640 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e850576b0 0x560e85423410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.844+0000 7fcd3d042640 1 -- 172.21.15.78:0/3382596405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e85078390 msgr2=0x560e854212e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.844+0000 7fcd3d042640 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e85078390 0x560e854212e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.844+0000 7fcd3d042640 1 -- 172.21.15.78:0/3382596405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e850903f0 con 0x560e8504f4d0 2024-05-21T15:18:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.844+0000 7fcd3d042640 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8504f4d0 0x560e85422ed0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcd280058c0 tx=0x7fcd28012aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc16d369cccbcce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.844+0000 7fcd20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.845+0000 7fcd20ff9640 1 -- 172.21.15.78:0/3382596405 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28004050 con 0x560e8504f4d0 2024-05-21T15:18:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.845+0000 7fcd20ff9640 1 -- 172.21.15.78:0/3382596405 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcd280041f0 con 0x560e8504f4d0 2024-05-21T15:18:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.847+0000 7fcd4621f740 1 -- 172.21.15.78:0/3382596405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd30036040 con 0x560e8504f4d0 2024-05-21T15:18:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.848+0000 7fcd20ff9640 1 -- 172.21.15.78:0/3382596405 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd280043f0 con 0x560e8504f4d0 2024-05-21T15:18:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.848+0000 7fcd4621f740 10 client.16446 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.849+0000 7fcd4621f740 1 -- 172.21.15.78:0/3382596405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e850500f0 con 0x560e8504f4d0 2024-05-21T15:18:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.850+0000 7fcd20ff9640 1 -- 172.21.15.78:0/3382596405 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd30036040 con 0x560e8504f4d0 2024-05-21T15:18:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.850+0000 7fcd20ff9640 1 -- 172.21.15.78:0/3382596405 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcd28003ab0 con 0x560e8504f4d0 2024-05-21T15:18:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.851+0000 7fcd4621f740 10 client.16446 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.851+0000 7fcd4621f740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.851+0000 7fcd4621f740 20 client.16446 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.852+0000 7fcd4621f740 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560e85036f60 0x560e85433d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.852+0000 7fcd4621f740 4 client.16446 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.852+0000 7fcd4621f740 1 -- 172.21.15.78:0/3382596405 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e8538a430 con 0x560e85036f60 2024-05-21T15:18:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.852+0000 7fcd3c841640 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560e85036f60 0x560e85433d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.853+0000 7fcd3c841640 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560e85036f60 0x560e85433d50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0eb2686d8c5e45 server_cookie=ca1480eae7e8e4c in_seq=0 out_seq=0 2024-05-21T15:18:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.853+0000 7fcd20ff9640 10 client.16446 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.854+0000 7fcd20ff9640 1 -- 172.21.15.78:0/3382596405 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e8538a430 con 0x560e85036f60 2024-05-21T15:18:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.854+0000 7fcd20ff9640 10 client.16446 handle_command_reply: tid=0 2024-05-21T15:18:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.854+0000 7fcd4621f740 1 client.16446 shutdown 2024-05-21T15:18:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.856+0000 7fcd4621f740 1 -- 172.21.15.78:0/3382596405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8504f4d0 msgr2=0x560e85422ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.856+0000 7fcd4621f740 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8504f4d0 0x560e85422ed0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcd280058c0 tx=0x7fcd28012aa0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd4621f740 1 -- 172.21.15.78:0/3382596405 shutdown_connections 2024-05-21T15:18:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd4621f740 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560e85036f60 0x560e85433d50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd4621f740 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e85078390 0x560e854212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd4621f740 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e850576b0 0x560e85423410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd4621f740 1 --2- 172.21.15.78:0/3382596405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8504f4d0 0x560e85422ed0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd4621f740 1 -- 172.21.15.78:0/3382596405 >> 172.21.15.78:0/3382596405 conn(0x560e8506c4e0 msgr2=0x560e85418a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd20ff9640 0 client.16446 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:54.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd4621f740 1 -- 172.21.15.78:0/3382596405 shutdown_connections 2024-05-21T15:18:54.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.857+0000 7fcd4621f740 1 -- 172.21.15.78:0/3382596405 wait complete. 2024-05-21T15:18:54.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.858+0000 7fcd4621f740 20 client.16446 trim_cache size 0 max 16384 2024-05-21T15:18:54.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.892+0000 7fcd4621f740 1 Processor -- start 2024-05-21T15:18:54.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.892+0000 7fcd4621f740 1 -- start start 2024-05-21T15:18:54.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.892+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 0x560e8541cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e85421280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e854130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd4621f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ac590 con 0x560e85078390 2024-05-21T15:18:54.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd4621f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ae1f0 con 0x560e850576b0 2024-05-21T15:18:54.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd4621f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381acc60 con 0x560e8504f4d0 2024-05-21T15:18:54.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd3c841640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e85421280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd3d042640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 0x560e8541cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd3c841640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e85421280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44988/0 (socket says 172.21.15.78:44988) 2024-05-21T15:18:54.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd3c841640 1 -- 172.21.15.78:0/2981828808 learned_addr learned my addr 172.21.15.78:0/2981828808 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:54.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.893+0000 7fcd3d843640 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e854130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.894+0000 7fcd3c841640 1 -- 172.21.15.78:0/2981828808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 msgr2=0x560e8541cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.894+0000 7fcd3c841640 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 0x560e8541cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.894+0000 7fcd3c841640 1 -- 172.21.15.78:0/2981828808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 msgr2=0x560e854130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.894+0000 7fcd3c841640 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e854130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.894+0000 7fcd3c841640 1 -- 172.21.15.78:0/2981828808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e8540e2d0 con 0x560e850576b0 2024-05-21T15:18:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.895+0000 7fcd3d843640 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e854130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:18:54.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.895+0000 7fcd3c841640 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e85421280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd34013d30 tx=0x7fcd34016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9165a8198c6a048e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:54.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.896+0000 7fcd22ffd640 1 -- 172.21.15.78:0/2981828808 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3401fe30 con 0x560e850576b0 2024-05-21T15:18:54.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.896+0000 7fcd22ffd640 1 -- 172.21.15.78:0/2981828808 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd34009db0 con 0x560e850576b0 2024-05-21T15:18:54.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.896+0000 7fcd22ffd640 1 -- 172.21.15.78:0/2981828808 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd340035f0 con 0x560e850576b0 2024-05-21T15:18:54.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 -- 172.21.15.78:0/2981828808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 msgr2=0x560e85421280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e85421280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd34013d30 tx=0x7fcd34016000 comp rx=0 tx=0).stop 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 -- 172.21.15.78:0/2981828808 shutdown_connections 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e850576b0 0x560e85421280 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd34013d30 tx=0x7fcd34016000 comp rx=0 tx=0).stop 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e85078390 0x560e854130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 --2- 172.21.15.78:0/2981828808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e8504f4d0 0x560e8541cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 -- 172.21.15.78:0/2981828808 >> 172.21.15.78:0/2981828808 conn(0x560e8506c4e0 msgr2=0x560e85420b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 -- 172.21.15.78:0/2981828808 shutdown_connections 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.897+0000 7fcd4621f740 1 -- 172.21.15.78:0/2981828808 wait complete. 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.898+0000 7fcd4621f740 1 -- start start 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.898+0000 7fcd4621f740 1 Processor -- start 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e8504f4d0 0x560e85429860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e85078390 0x560e85425bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd4621f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8500c3a0 0x560e85426100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd4621f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ada60 con 0x560e8500c3a0 2024-05-21T15:18:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd4621f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ad360 con 0x560e8504f4d0 2024-05-21T15:18:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd4621f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381aa570 con 0x560e85078390 2024-05-21T15:18:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd4621f740 10 client.0 _ll_register_callbacks cb 0x7fcd280096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd3d843640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8500c3a0 0x560e85426100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd3d843640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8500c3a0 0x560e85426100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52348/0 (socket says 172.21.15.78:52348) 2024-05-21T15:18:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd3d843640 1 -- 172.21.15.78:0/4053618764 learned_addr learned my addr 172.21.15.78:0/4053618764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd3d042640 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e8504f4d0 0x560e85429860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.899+0000 7fcd3c841640 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e85078390 0x560e85425bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.900+0000 7fcd3d843640 1 -- 172.21.15.78:0/4053618764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e85078390 msgr2=0x560e85425bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.900+0000 7fcd3d843640 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e85078390 0x560e85425bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.900+0000 7fcd3d843640 1 -- 172.21.15.78:0/4053618764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e8504f4d0 msgr2=0x560e85429860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.900+0000 7fcd3d843640 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e8504f4d0 0x560e85429860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.900+0000 7fcd3d843640 1 -- 172.21.15.78:0/4053618764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e85426840 con 0x560e8500c3a0 2024-05-21T15:18:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.901+0000 7fcd3d843640 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8500c3a0 0x560e85426100 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcd30004af0 tx=0x7fcd300c0a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ad37bc759af887 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.901+0000 7fcd20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:54.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.901+0000 7fcd20ff9640 1 -- 172.21.15.78:0/4053618764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3003fdc0 con 0x560e8500c3a0 2024-05-21T15:18:54.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.901+0000 7fcd4621f740 1 -- 172.21.15.78:0/4053618764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd34009db0 con 0x560e8500c3a0 2024-05-21T15:18:54.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.902+0000 7fcd20ff9640 1 -- 172.21.15.78:0/4053618764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcd300bd760 con 0x560e8500c3a0 2024-05-21T15:18:54.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.902+0000 7fcd20ff9640 1 -- 172.21.15.78:0/4053618764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd30004b40 con 0x560e8500c3a0 2024-05-21T15:18:54.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.906+0000 7fcd4621f740 10 client.16452 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.906+0000 7fcd4621f740 1 -- 172.21.15.78:0/4053618764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e8549afc0 con 0x560e8500c3a0 2024-05-21T15:18:54.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.907+0000 7fcd20ff9640 1 -- 172.21.15.78:0/4053618764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd34009db0 con 0x560e8500c3a0 2024-05-21T15:18:54.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.910+0000 7fcd20ff9640 1 -- 172.21.15.78:0/4053618764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcd300db070 con 0x560e8500c3a0 2024-05-21T15:18:54.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.910+0000 7fcd4621f740 10 client.16452 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:54.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.910+0000 7fcd4621f740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:54.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.910+0000 7fcd4621f740 20 client.16452 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:54.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.910+0000 7fcd4621f740 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560e8549b870 0x560e8549dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:54.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.910+0000 7fcd4621f740 4 client.16452 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:18:54.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.910+0000 7fcd4621f740 1 -- 172.21.15.78:0/4053618764 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e85384780 con 0x560e8549b870 2024-05-21T15:18:54.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.911+0000 7fcd3d042640 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560e8549b870 0x560e8549dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.915+0000 7fcd3d042640 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560e8549b870 0x560e8549dc90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc8d731022f069d server_cookie=bd643e5f8ae7f736 in_seq=0 out_seq=0 2024-05-21T15:18:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.916+0000 7fcd20ff9640 10 client.16452 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.916+0000 7fcd20ff9640 1 -- 172.21.15.78:0/4053618764 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e85384780 con 0x560e8549b870 2024-05-21T15:18:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.916+0000 7fcd20ff9640 10 client.16452 handle_command_reply: tid=0 2024-05-21T15:18:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.916+0000 7fcd4621f740 1 client.16452 shutdown 2024-05-21T15:18:54.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.943+0000 7fcd4621f740 1 -- 172.21.15.78:0/4053618764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8500c3a0 msgr2=0x560e85426100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.943+0000 7fcd4621f740 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8500c3a0 0x560e85426100 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcd30004af0 tx=0x7fcd300c0a50 comp rx=0 tx=0).stop 2024-05-21T15:18:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.948+0000 7fcd4621f740 1 -- 172.21.15.78:0/4053618764 shutdown_connections 2024-05-21T15:18:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.948+0000 7fcd4621f740 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560e8549b870 0x560e8549dc90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.948+0000 7fcd4621f740 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560e8500c3a0 0x560e85426100 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.948+0000 7fcd4621f740 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e85078390 0x560e85425bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.948+0000 7fcd4621f740 1 --2- 172.21.15.78:0/4053618764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e8504f4d0 0x560e85429860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.948+0000 7fcd4621f740 1 -- 172.21.15.78:0/4053618764 >> 172.21.15.78:0/4053618764 conn(0x560e8506c4e0 msgr2=0x560e8541a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:54.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.948+0000 7fcd20ff9640 0 client.16452 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:54.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.953+0000 7fcd4621f740 1 -- 172.21.15.78:0/4053618764 shutdown_connections 2024-05-21T15:18:54.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.954+0000 7fcd4621f740 1 -- 172.21.15.78:0/4053618764 wait complete. 2024-05-21T15:18:54.961 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:18:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:18:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:18:54.961 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:18:54.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.954+0000 7fcd4621f740 20 client.16452 trim_cache size 0 max 16384 2024-05-21T15:18:54.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.960+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcd240a2e30 msgr2=0x7fcd240a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.960+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcd240a2e30 0x7fcd240a5340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcd38100f70 tx=0x7fcd34009830 comp rx=0 tx=0).stop 2024-05-21T15:18:54.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.960+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3810c330 msgr2=0x7fcd3819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.960+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3810c330 0x7fcd3819bb00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcd3002ae10 tx=0x7fcd30031990 comp rx=0 tx=0).stop 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.961+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/2011839629 shutdown_connections 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.961+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcd240a2e30 0x7fcd240a5340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.961+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3810c330 0x7fcd3819bb00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.962+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd381079d0 0x7fcd38197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.962+0000 7fcd3f2a6640 1 --2- 172.21.15.78:0/2011839629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd38101aa0 0x7fcd381972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.962+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/2011839629 >> 172.21.15.78:0/2011839629 conn(0x7fcd380770b0 msgr2=0x7fcd38103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.962+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/2011839629 shutdown_connections 2024-05-21T15:18:54.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:54.962+0000 7fcd3f2a6640 1 -- 172.21.15.78:0/2011839629 wait complete. 2024-05-21T15:18:54.989 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:18:54.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:18:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.213+0000 7ff862738640 1 -- 172.21.15.78:0/949458228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff85c107d90 msgr2=0x7ff85c101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.213+0000 7ff862738640 1 --2- 172.21.15.78:0/949458228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff85c107d90 0x7ff85c101010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009900 tx=0x7ff84c02a490 comp rx=0 tx=0).stop 2024-05-21T15:18:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.214+0000 7ff862738640 1 -- 172.21.15.78:0/949458228 shutdown_connections 2024-05-21T15:18:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.214+0000 7ff862738640 1 --2- 172.21.15.78:0/949458228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff85c107d90 0x7ff85c101010 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.214+0000 7ff862738640 1 --2- 172.21.15.78:0/949458228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff85c106220 0x7ff85c106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.214+0000 7ff862738640 1 --2- 172.21.15.78:0/949458228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff85c107980 0x7ff85c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.214+0000 7ff862738640 1 -- 172.21.15.78:0/949458228 >> 172.21.15.78:0/949458228 conn(0x7ff85c0fc7c0 msgr2=0x7ff85c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.216+0000 7ff862738640 1 -- 172.21.15.78:0/949458228 shutdown_connections 2024-05-21T15:18:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.216+0000 7ff862738640 1 -- 172.21.15.78:0/949458228 wait complete. 2024-05-21T15:18:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.217+0000 7ff862738640 1 Processor -- start 2024-05-21T15:18:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.217+0000 7ff862738640 1 -- start start 2024-05-21T15:18:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.217+0000 7ff862738640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff85c106220 0x7ff85c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.217+0000 7ff862738640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff85c107980 0x7ff85c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.217+0000 7ff862738640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff85c107d90 0x7ff85c1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff862738640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c119470 con 0x7ff85c106220 2024-05-21T15:18:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff862738640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1192f0 con 0x7ff85c107980 2024-05-21T15:18:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff862738640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1195f0 con 0x7ff85c107d90 2024-05-21T15:18:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff860cd5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff85c107d90 0x7ff85c1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff85c107980 0x7ff85c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff860cd5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff85c107d90 0x7ff85c1a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33160/0 (socket says 172.21.15.78:33160) 2024-05-21T15:18:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff860cd5640 1 -- 172.21.15.78:0/1026964100 learned_addr learned my addr 172.21.15.78:0/1026964100 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff860cd5640 1 -- 172.21.15.78:0/1026964100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff85c107980 msgr2=0x7ff85c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff860cd5640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff85c107980 0x7ff85c1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff860cd5640 1 -- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff85c106220 msgr2=0x7ff85c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff860cd5640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff85c106220 0x7ff85c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.218+0000 7ff860cd5640 1 -- 172.21.15.78:0/1026964100 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c1a4c10 con 0x7ff85c107d90 2024-05-21T15:18:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.219+0000 7ff860cd5640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff85c107d90 0x7ff85c1a45d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff85400eb10 tx=0x7ff85400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.219+0000 7ff8597fa640 1 -- 172.21.15.78:0/1026964100 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854004b20 con 0x7ff85c107d90 2024-05-21T15:18:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.219+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c1a4f00 con 0x7ff85c107d90 2024-05-21T15:18:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.220+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c1ac880 con 0x7ff85c107d90 2024-05-21T15:18:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.220+0000 7ff8597fa640 1 -- 172.21.15.78:0/1026964100 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff854004cc0 con 0x7ff85c107d90 2024-05-21T15:18:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.220+0000 7ff8597fa640 1 -- 172.21.15.78:0/1026964100 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854010770 con 0x7ff85c107d90 2024-05-21T15:18:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.225+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff848005290 con 0x7ff85c107d90 2024-05-21T15:18:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.225+0000 7ff8597fa640 1 -- 172.21.15.78:0/1026964100 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff854005000 con 0x7ff85c107d90 2024-05-21T15:18:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.227+0000 7ff8597fa640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8440a2e40 0x7ff8440a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.227+0000 7ff85bfff640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8440a2e40 0x7ff8440a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.227+0000 7ff8597fa640 1 -- 172.21.15.78:0/1026964100 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7ff8540cfc70 con 0x7ff85c107d90 2024-05-21T15:18:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.228+0000 7ff85bfff640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8440a2e40 0x7ff8440a5350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff85c068b20 tx=0x7ff850006c90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.237+0000 7ff8597fa640 1 -- 172.21.15.78:0/1026964100 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff85409c7d0 con 0x7ff85c107d90 2024-05-21T15:18:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.691+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff848005850 con 0x7ff85c107d90 2024-05-21T15:18:55.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.693+0000 7ff8597fa640 1 -- 172.21.15.78:0/1026964100 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11934 (secure 0 0 0) 0x7ff85409c0d0 con 0x7ff85c107d90 2024-05-21T15:18:55.697 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:18:55.697 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:18:52.751930+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24419,24448,24502,24526,24527]},"id":1}]} 2024-05-21T15:18:55.697 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 30 2024-05-21T15:18:55.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.699+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8440a2e40 msgr2=0x7ff8440a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:55.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.699+0000 7ff862738640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8440a2e40 0x7ff8440a5350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff85c068b20 tx=0x7ff850006c90 comp rx=0 tx=0).stop 2024-05-21T15:18:55.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.700+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff85c107d90 msgr2=0x7ff85c1a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:55.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.700+0000 7ff862738640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff85c107d90 0x7ff85c1a45d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff85400eb10 tx=0x7ff85400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.702+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 shutdown_connections 2024-05-21T15:18:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.702+0000 7ff862738640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff85c107980 0x7ff85c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.702+0000 7ff862738640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8440a2e40 0x7ff8440a5350 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.702+0000 7ff862738640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff85c107d90 0x7ff85c1a45d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.702+0000 7ff862738640 1 --2- 172.21.15.78:0/1026964100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff85c106220 0x7ff85c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.702+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 >> 172.21.15.78:0/1026964100 conn(0x7ff85c0fc7c0 msgr2=0x7ff85c0fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.705+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 shutdown_connections 2024-05-21T15:18:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.706+0000 7ff862738640 1 -- 172.21.15.78:0/1026964100 wait complete. 2024-05-21T15:18:55.735 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:18:55.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.945+0000 7f9b32c7a640 1 -- 172.21.15.78:0/2527528959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c109710 msgr2=0x7f9b2c1159c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:55.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.945+0000 7f9b32c7a640 1 --2- 172.21.15.78:0/2527528959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c109710 0x7f9b2c1159c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009ca0 tx=0x7f9b2402a8f0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.945+0000 7f9b32c7a640 1 -- 172.21.15.78:0/2527528959 shutdown_connections 2024-05-21T15:18:55.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.945+0000 7f9b32c7a640 1 --2- 172.21.15.78:0/2527528959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2c10f3d0 0x7f9b2c1090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.945+0000 7f9b32c7a640 1 --2- 172.21.15.78:0/2527528959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c109710 0x7f9b2c1159c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.945+0000 7f9b32c7a640 1 --2- 172.21.15.78:0/2527528959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b2c10ef30 0x7f9b2c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.945+0000 7f9b32c7a640 1 -- 172.21.15.78:0/2527528959 >> 172.21.15.78:0/2527528959 conn(0x7f9b2c074f00 msgr2=0x7f9b2c075310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:55.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.946+0000 7f9b32c7a640 1 -- 172.21.15.78:0/2527528959 shutdown_connections 2024-05-21T15:18:55.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.947+0000 7f9b32c7a640 1 -- 172.21.15.78:0/2527528959 wait complete. 2024-05-21T15:18:55.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.948+0000 7f9b32c7a640 1 Processor -- start 2024-05-21T15:18:55.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.948+0000 7f9b32c7a640 1 -- start start 2024-05-21T15:18:55.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.948+0000 7f9b32c7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2c10ef30 0x7f9b2c082220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.949+0000 7f9b32c7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c10f3d0 0x7f9b2c07b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.949+0000 7f9b32c7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b2c07b810 0x7f9b2c07bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.949+0000 7f9b32c7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1193d0 con 0x7f9b2c07b810 2024-05-21T15:18:55.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.949+0000 7f9b32c7a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c119250 con 0x7f9b2c10ef30 2024-05-21T15:18:55.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.949+0000 7f9b32c7a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c119550 con 0x7f9b2c10f3d0 2024-05-21T15:18:55.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.951+0000 7f9b31217640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b2c07b810 0x7f9b2c07bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.951+0000 7f9b31217640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b2c07b810 0x7f9b2c07bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52376/0 (socket says 172.21.15.78:52376) 2024-05-21T15:18:55.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.951+0000 7f9b31217640 1 -- 172.21.15.78:0/1812470653 learned_addr learned my addr 172.21.15.78:0/1812470653 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:55.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.951+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c10f3d0 0x7f9b2c07b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.952+0000 7f9b30a16640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2c10ef30 0x7f9b2c082220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.952+0000 7f9b31217640 1 -- 172.21.15.78:0/1812470653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c10f3d0 msgr2=0x7f9b2c07b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:55.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.952+0000 7f9b31217640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c10f3d0 0x7f9b2c07b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.952+0000 7f9b31217640 1 -- 172.21.15.78:0/1812470653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2c10ef30 msgr2=0x7f9b2c082220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:55.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.952+0000 7f9b31217640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2c10ef30 0x7f9b2c082220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.952+0000 7f9b31217640 1 -- 172.21.15.78:0/1812470653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c07fc80 con 0x7f9b2c07b810 2024-05-21T15:18:55.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.956+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c10f3d0 0x7f9b2c07b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.959+0000 7f9b31217640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b2c07b810 0x7f9b2c07bc20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9b240045e0 tx=0x7f9b24004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.960+0000 7f9b29ffb640 1 -- 172.21.15.78:0/1812470653 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24030a00 con 0x7f9b2c07b810 2024-05-21T15:18:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.960+0000 7f9b29ffb640 1 -- 172.21.15.78:0/1812470653 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2403f070 con 0x7f9b2c07b810 2024-05-21T15:18:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.960+0000 7f9b32c7a640 1 -- 172.21.15.78:0/1812470653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c07ff10 con 0x7f9b2c07b810 2024-05-21T15:18:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.960+0000 7f9b32c7a640 1 -- 172.21.15.78:0/1812470653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c080460 con 0x7f9b2c07b810 2024-05-21T15:18:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.964+0000 7f9b39c04740 1 Processor -- start 2024-05-21T15:18:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.965+0000 7f9b29ffb640 1 -- 172.21.15.78:0/1812470653 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24031b20 con 0x7f9b2c07b810 2024-05-21T15:18:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.965+0000 7f9b39c04740 1 -- start start 2024-05-21T15:18:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.965+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b5d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.965+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b5d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.965+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b5dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.965+0000 7f9b39c04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fcf0ac2c40 con 0x55fcf07be390 2024-05-21T15:18:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.965+0000 7f9b39c04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fcf0a6a580 con 0x55fcf079d6b0 2024-05-21T15:18:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.965+0000 7f9b39c04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fcf0abec00 con 0x55fcf07954d0 2024-05-21T15:18:55.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.972+0000 7f9b30a16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b5d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.973+0000 7f9b30a16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b5d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33182/0 (socket says 172.21.15.78:33182) 2024-05-21T15:18:55.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.973+0000 7f9b30a16640 1 -- 172.21.15.78:0/2182854997 learned_addr learned my addr 172.21.15.78:0/2182854997 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:55.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.975+0000 7f9b29ffb640 1 -- 172.21.15.78:0/1812470653 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9b24031480 con 0x7f9b2c07b810 2024-05-21T15:18:55.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.975+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/2182854997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b5dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.976+0000 7f9b29ffb640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9b1c0a2d30 0x7f9b1c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:55.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.976+0000 7f9b29ffb640 1 -- 172.21.15.78:0/1812470653 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9b240f07d0 con 0x7f9b2c07b810 2024-05-21T15:18:55.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.979+0000 7f9b30a16640 1 -- 172.21.15.78:0/2182854997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 msgr2=0x55fcf0b5dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.979+0000 7f9b30a16640 1 --2- 172.21.15.78:0/2182854997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b5dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.979+0000 7f9b30a16640 1 -- 172.21.15.78:0/2182854997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 msgr2=0x55fcf0b5d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.979+0000 7f9b30a16640 1 --2- 172.21.15.78:0/2182854997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b5d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.979+0000 7f9b30a16640 1 -- 172.21.15.78:0/2182854997 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf07d63f0 con 0x55fcf07954d0 2024-05-21T15:18:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.979+0000 7f9b31217640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9b1c0a2d30 0x7f9b1c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.982+0000 7f9b31217640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9b1c0a2d30 0x7f9b1c0a5240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9b240ee4c0 tx=0x7f9b2403d040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.987+0000 7f9b30a16640 1 --2- 172.21.15.78:0/2182854997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b5d740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800b7c0 tx=0x7f9b1800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe03a1a5c8c882ad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.988+0000 7f9b167fc640 1 -- 172.21.15.78:0/2182854997 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800d950 con 0x55fcf07954d0 2024-05-21T15:18:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.988+0000 7f9b167fc640 1 -- 172.21.15.78:0/2182854997 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b18018070 con 0x55fcf07954d0 2024-05-21T15:18:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:55.988+0000 7f9b167fc640 1 -- 172.21.15.78:0/2182854997 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b180123e0 con 0x55fcf07954d0 2024-05-21T15:18:56.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.002+0000 7f9b39c04740 1 -- 172.21.15.78:0/2182854997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 msgr2=0x55fcf0b5d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.002+0000 7f9b39c04740 1 --2- 172.21.15.78:0/2182854997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b5d740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800b7c0 tx=0x7f9b1800bc80 comp rx=0 tx=0).stop 2024-05-21T15:18:56.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.002+0000 7f9b39c04740 1 -- 172.21.15.78:0/2182854997 shutdown_connections 2024-05-21T15:18:56.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.002+0000 7f9b39c04740 1 --2- 172.21.15.78:0/2182854997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b5dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.002+0000 7f9b39c04740 1 --2- 172.21.15.78:0/2182854997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b5d740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.002+0000 7f9b39c04740 1 --2- 172.21.15.78:0/2182854997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b5d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.002+0000 7f9b39c04740 1 -- 172.21.15.78:0/2182854997 >> 172.21.15.78:0/2182854997 conn(0x55fcf07b24e0 msgr2=0x55fcf0b599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.002+0000 7f9b39c04740 1 -- 172.21.15.78:0/2182854997 shutdown_connections 2024-05-21T15:18:56.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.003+0000 7f9b39c04740 1 -- 172.21.15.78:0/2182854997 wait complete. 2024-05-21T15:18:56.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.003+0000 7f9b39c04740 1 -- start start 2024-05-21T15:18:56.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.003+0000 7f9b39c04740 1 Processor -- start 2024-05-21T15:18:56.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.004+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b738f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b39c04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fcf0b6a440 con 0x55fcf07be390 2024-05-21T15:18:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b39c04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fcf0a7dc40 con 0x55fcf07954d0 2024-05-21T15:18:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b39c04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fcf0b6a5c0 con 0x55fcf079d6b0 2024-05-21T15:18:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b39c04740 10 client.0 _ll_register_callbacks cb 0x55fcf07d5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b30a16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b30a16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b679c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33190/0 (socket says 172.21.15.78:33190) 2024-05-21T15:18:56.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.005+0000 7f9b30a16640 1 -- 172.21.15.78:0/926704417 learned_addr learned my addr 172.21.15.78:0/926704417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.007+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.007+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.007+0000 7f9b30a16640 1 -- 172.21.15.78:0/926704417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 msgr2=0x55fcf0b67480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.007+0000 7f9b30a16640 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b67480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.007+0000 7f9b30a16640 1 -- 172.21.15.78:0/926704417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 msgr2=0x55fcf0b738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.007+0000 7f9b30a16640 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.007+0000 7f9b30a16640 1 -- 172.21.15.78:0/926704417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf08a07d0 con 0x55fcf079d6b0 2024-05-21T15:18:56.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.008+0000 7f9b30a16640 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b679c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000c00 tx=0x7f9b18013480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e25beabe77d7f12 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.008+0000 7f9b0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:18:56.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.009+0000 7f9b0bfff640 1 -- 172.21.15.78:0/926704417 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800db50 con 0x55fcf079d6b0 2024-05-21T15:18:56.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.009+0000 7f9b39c04740 1 -- 172.21.15.78:0/926704417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b18018070 con 0x55fcf079d6b0 2024-05-21T15:18:56.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.009+0000 7f9b0bfff640 1 -- 172.21.15.78:0/926704417 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9b1800dcf0 con 0x55fcf079d6b0 2024-05-21T15:18:56.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.009+0000 7f9b0bfff640 1 -- 172.21.15.78:0/926704417 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b1802c470 con 0x55fcf079d6b0 2024-05-21T15:18:56.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.010+0000 7f9b39c04740 10 client.25915 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:56.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.010+0000 7f9b39c04740 1 -- 172.21.15.78:0/926704417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fcf0b768f0 con 0x55fcf079d6b0 2024-05-21T15:18:56.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.010+0000 7f9b0bfff640 1 -- 172.21.15.78:0/926704417 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1802caf0 con 0x55fcf079d6b0 2024-05-21T15:18:56.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.011+0000 7f9b0bfff640 1 -- 172.21.15.78:0/926704417 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9b1801c7c0 con 0x55fcf079d6b0 2024-05-21T15:18:56.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.011+0000 7f9b39c04740 10 client.25915 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:56.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.011+0000 7f9b39c04740 10 client.25915 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:18:56.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.011+0000 7f9b39c04740 20 client.25915 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:56.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.011+0000 7f9b39c04740 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55fcf077cf60 0x55fcf0b79d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.011+0000 7f9b39c04740 4 client.25915 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:56.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.011+0000 7f9b39c04740 1 -- 172.21.15.78:0/926704417 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9b18018070 con 0x55fcf077cf60 2024-05-21T15:18:56.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.012+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55fcf077cf60 0x55fcf0b79d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.014+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55fcf077cf60 0x55fcf0b79d10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9a07728455ef6ce server_cookie=536f70bca4e8682b in_seq=0 out_seq=0 2024-05-21T15:18:56.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.014+0000 7f9b0bfff640 10 client.25915 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:18:56.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.015+0000 7f9b0bfff640 1 -- 172.21.15.78:0/926704417 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9b18018070 con 0x55fcf077cf60 2024-05-21T15:18:56.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.015+0000 7f9b0bfff640 10 client.25915 handle_command_reply: tid=0 2024-05-21T15:18:56.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.016+0000 7f9b39c04740 1 client.25915 shutdown 2024-05-21T15:18:56.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.016+0000 7f9b39c04740 1 -- 172.21.15.78:0/926704417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 msgr2=0x55fcf0b679c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.016+0000 7f9b39c04740 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b679c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000c00 tx=0x7f9b18013480 comp rx=0 tx=0).stop 2024-05-21T15:18:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.018+0000 7f9b39c04740 1 -- 172.21.15.78:0/926704417 shutdown_connections 2024-05-21T15:18:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.018+0000 7f9b39c04740 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55fcf077cf60 0x55fcf0b79d10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.018+0000 7f9b39c04740 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.018+0000 7f9b0bfff640 0 client.25915 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:18:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.018+0000 7f9b39c04740 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b679c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.018+0000 7f9b39c04740 1 --2- 172.21.15.78:0/926704417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b67480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.018+0000 7f9b39c04740 1 -- 172.21.15.78:0/926704417 >> 172.21.15.78:0/926704417 conn(0x55fcf07b24e0 msgr2=0x55fcf0b5ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.018+0000 7f9b39c04740 1 -- 172.21.15.78:0/926704417 shutdown_connections 2024-05-21T15:18:56.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.019+0000 7f9b39c04740 1 -- 172.21.15.78:0/926704417 wait complete. 2024-05-21T15:18:56.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.019+0000 7f9b39c04740 20 client.25915 trim_cache size 0 max 16384 2024-05-21T15:18:56.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.051+0000 7f9b39c04740 1 Processor -- start 2024-05-21T15:18:56.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.051+0000 7f9b39c04740 1 -- start start 2024-05-21T15:18:56.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.051+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b54a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b58cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b2bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b54a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b30a16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b58cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b30a16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b58cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33192/0 (socket says 172.21.15.78:33192) 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b2bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b54a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45052/0 (socket says 172.21.15.78:45052) 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b30a16640 1 -- 172.21.15.78:0/687258047 learned_addr learned my addr 172.21.15.78:0/687258047 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b39c04740 1 -- 172.21.15.78:0/687258047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c073900 con 0x55fcf07be390 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.052+0000 7f9b39c04740 1 -- 172.21.15.78:0/687258047 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1c7fa0 con 0x55fcf07954d0 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.053+0000 7f9b39c04740 1 -- 172.21.15.78:0/687258047 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1c6a10 con 0x55fcf079d6b0 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.053+0000 7f9b30a16640 1 -- 172.21.15.78:0/687258047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 msgr2=0x55fcf0b54a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.053+0000 7f9b30a16640 1 --2- 172.21.15.78:0/687258047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b54a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.053+0000 7f9b30a16640 1 -- 172.21.15.78:0/687258047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 msgr2=0x55fcf0b734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.053+0000 7f9b30a16640 1 --2- 172.21.15.78:0/687258047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.053+0000 7f9b30a16640 1 -- 172.21.15.78:0/687258047 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf0b68060 con 0x55fcf079d6b0 2024-05-21T15:18:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.055+0000 7f9b30a16640 1 --2- 172.21.15.78:0/687258047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b58cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800dc90 tx=0x7f9b1800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b1b4a8a8d0a320e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.056+0000 7f9b15ffb640 1 -- 172.21.15.78:0/687258047 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18013d50 con 0x55fcf079d6b0 2024-05-21T15:18:56.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.056+0000 7f9b15ffb640 1 -- 172.21.15.78:0/687258047 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b18018040 con 0x55fcf079d6b0 2024-05-21T15:18:56.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.057+0000 7f9b15ffb640 1 -- 172.21.15.78:0/687258047 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18023630 con 0x55fcf079d6b0 2024-05-21T15:18:56.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.058+0000 7f9b39c04740 1 -- 172.21.15.78:0/687258047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 msgr2=0x55fcf0b58cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.058+0000 7f9b39c04740 1 --2- 172.21.15.78:0/687258047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b58cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800dc90 tx=0x7f9b1800dcc0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.058+0000 7f9b39c04740 1 -- 172.21.15.78:0/687258047 shutdown_connections 2024-05-21T15:18:56.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.058+0000 7f9b39c04740 1 --2- 172.21.15.78:0/687258047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0b734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.058+0000 7f9b39c04740 1 --2- 172.21.15.78:0/687258047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf079d6b0 0x55fcf0b58cb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.058+0000 7f9b39c04740 1 --2- 172.21.15.78:0/687258047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf07954d0 0x55fcf0b54a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.058+0000 7f9b39c04740 1 -- 172.21.15.78:0/687258047 >> 172.21.15.78:0/687258047 conn(0x55fcf07b24e0 msgr2=0x55fcf0b695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.062+0000 7f9b39c04740 1 -- 172.21.15.78:0/687258047 shutdown_connections 2024-05-21T15:18:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.062+0000 7f9b39c04740 1 -- 172.21.15.78:0/687258047 wait complete. 2024-05-21T15:18:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.062+0000 7f9b39c04740 1 -- start start 2024-05-21T15:18:56.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.063+0000 7f9b39c04740 1 Processor -- start 2024-05-21T15:18:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.063+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b6aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.063+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b6b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.063+0000 7f9b39c04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0bdff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.063+0000 7f9b39c04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1c7810 con 0x55fcf07be390 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.063+0000 7f9b39c04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c1c7110 con 0x55fcf079d6b0 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.063+0000 7f9b39c04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c10a1e0 con 0x55fcf07954d0 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.064+0000 7f9b39c04740 10 client.0 _ll_register_callbacks cb 0x7f9b1800b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b2bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b6aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b30a16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b6b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b2bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b6aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33208/0 (socket says 172.21.15.78:33208) 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b2bfff640 1 -- 172.21.15.78:0/1354485439 learned_addr learned my addr 172.21.15.78:0/1354485439 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0bdff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b30a16640 1 -- 172.21.15.78:0/1354485439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 msgr2=0x55fcf0b6aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b30a16640 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b6aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b30a16640 1 -- 172.21.15.78:0/1354485439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 msgr2=0x55fcf0bdff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b30a16640 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0bdff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.065+0000 7f9b30a16640 1 -- 172.21.15.78:0/1354485439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf0be04d0 con 0x55fcf079d6b0 2024-05-21T15:18:56.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.066+0000 7f9b30a16640 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b6b000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55fcf0b5ecd0 tx=0x7f9b18013480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3110e027d195f931 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.066+0000 7f9b0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:56.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.066+0000 7f9b0b7fe640 1 -- 172.21.15.78:0/1354485439 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18006ca0 con 0x55fcf079d6b0 2024-05-21T15:18:56.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.066+0000 7f9b39c04740 1 -- 172.21.15.78:0/1354485439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b18018040 con 0x55fcf079d6b0 2024-05-21T15:18:56.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.066+0000 7f9b0b7fe640 1 -- 172.21.15.78:0/1354485439 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9b1801c3f0 con 0x55fcf079d6b0 2024-05-21T15:18:56.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.067+0000 7f9b0b7fe640 1 -- 172.21.15.78:0/1354485439 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b1801c5f0 con 0x55fcf079d6b0 2024-05-21T15:18:56.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.069+0000 7f9b39c04740 10 client.25922 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:56.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.069+0000 7f9b39c04740 1 -- 172.21.15.78:0/1354485439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fcf0be1680 con 0x55fcf079d6b0 2024-05-21T15:18:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.069+0000 7f9b0b7fe640 1 -- 172.21.15.78:0/1354485439 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b18018040 con 0x55fcf079d6b0 2024-05-21T15:18:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.070+0000 7f9b0b7fe640 1 -- 172.21.15.78:0/1354485439 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9b18023630 con 0x55fcf079d6b0 2024-05-21T15:18:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.070+0000 7f9b39c04740 10 client.25922 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.070+0000 7f9b39c04740 10 client.25922 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:18:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.070+0000 7f9b39c04740 20 client.25922 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.070+0000 7f9b39c04740 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55fcf07523a0 0x55fcf0be3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.070+0000 7f9b39c04740 4 client.25922 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.070+0000 7f9b39c04740 1 -- 172.21.15.78:0/1354485439 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcf0b63250 con 0x55fcf07523a0 2024-05-21T15:18:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.070+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55fcf07523a0 0x55fcf0be3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.072+0000 7f9b2bfff640 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55fcf07523a0 0x55fcf0be3db0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9629222af4624700 server_cookie=e3198a62b736230e in_seq=0 out_seq=0 2024-05-21T15:18:56.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.072+0000 7f9b0b7fe640 10 client.25922 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:18:56.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.072+0000 7f9b0b7fe640 1 -- 172.21.15.78:0/1354485439 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcf0b63250 con 0x55fcf07523a0 2024-05-21T15:18:56.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.072+0000 7f9b0b7fe640 10 client.25922 handle_command_reply: tid=0 2024-05-21T15:18:56.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.072+0000 7f9b39c04740 1 client.25922 shutdown 2024-05-21T15:18:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.073+0000 7f9b39c04740 1 -- 172.21.15.78:0/1354485439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 msgr2=0x55fcf0b6b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.073+0000 7f9b39c04740 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b6b000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55fcf0b5ecd0 tx=0x7f9b18013480 comp rx=0 tx=0).stop 2024-05-21T15:18:56.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.079+0000 7f9b39c04740 1 -- 172.21.15.78:0/1354485439 shutdown_connections 2024-05-21T15:18:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.079+0000 7f9b39c04740 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55fcf07523a0 0x55fcf0be3db0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.079+0000 7f9b39c04740 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fcf079d6b0 0x55fcf0b6b000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.079+0000 7f9b39c04740 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fcf07be390 0x55fcf0bdff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.079+0000 7f9b39c04740 1 --2- 172.21.15.78:0/1354485439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fcf07954d0 0x55fcf0b6aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.079+0000 7f9b39c04740 1 -- 172.21.15.78:0/1354485439 >> 172.21.15.78:0/1354485439 conn(0x55fcf07b24e0 msgr2=0x55fcf0b68b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.079+0000 7f9b0b7fe640 0 client.25922 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:18:56.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.081+0000 7f9b39c04740 1 -- 172.21.15.78:0/1354485439 shutdown_connections 2024-05-21T15:18:56.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.081+0000 7f9b39c04740 1 -- 172.21.15.78:0/1354485439 wait complete. 2024-05-21T15:18:56.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.081+0000 7f9b39c04740 20 client.25922 trim_cache size 0 max 16384 2024-05-21T15:18:56.084 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.086+0000 7f9b177fe640 1 -- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9b1c0a2d30 msgr2=0x7f9b1c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.086+0000 7f9b177fe640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9b1c0a2d30 0x7f9b1c0a5240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9b240ee4c0 tx=0x7f9b2403d040 comp rx=0 tx=0).stop 2024-05-21T15:18:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.086+0000 7f9b177fe640 1 -- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b2c07b810 msgr2=0x7f9b2c07bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.086+0000 7f9b177fe640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b2c07b810 0x7f9b2c07bc20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9b240045e0 tx=0x7f9b24004610 comp rx=0 tx=0).stop 2024-05-21T15:18:56.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.088+0000 7f9b177fe640 1 -- 172.21.15.78:0/1812470653 shutdown_connections 2024-05-21T15:18:56.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.088+0000 7f9b177fe640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9b1c0a2d30 0x7f9b1c0a5240 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.088+0000 7f9b177fe640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b2c07b810 0x7f9b2c07bc20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.088+0000 7f9b177fe640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c10f3d0 0x7f9b2c07b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.088+0000 7f9b177fe640 1 --2- 172.21.15.78:0/1812470653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b2c10ef30 0x7f9b2c082220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.088+0000 7f9b177fe640 1 -- 172.21.15.78:0/1812470653 >> 172.21.15.78:0/1812470653 conn(0x7f9b2c074f00 msgr2=0x7f9b2c1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.089+0000 7f9b177fe640 1 -- 172.21.15.78:0/1812470653 shutdown_connections 2024-05-21T15:18:56.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.089+0000 7f9b177fe640 1 -- 172.21.15.78:0/1812470653 wait complete. 2024-05-21T15:18:56.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:55 smithi078 ceph-mon[31997]: pgmap v88: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 106 MiB/s wr, 15.76k op/s 2024-05-21T15:18:56.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:55 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1026964100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:18:56.115 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:18:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:55 smithi084 ceph-mon[36334]: pgmap v88: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 106 MiB/s wr, 15.76k op/s 2024-05-21T15:18:56.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:55 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1026964100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:18:56.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:55 smithi172 ceph-mon[36402]: pgmap v88: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 106 MiB/s wr, 15.76k op/s 2024-05-21T15:18:56.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:55 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1026964100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:18:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.331+0000 7fa71a157640 1 -- 172.21.15.78:0/2851673071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 msgr2=0x7fa71410e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.331+0000 7fa71a157640 1 --2- 172.21.15.78:0/2851673071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 0x7fa71410e110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa70800af00 tx=0x7fa70802e630 comp rx=0 tx=0).stop 2024-05-21T15:18:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.333+0000 7fa71a157640 1 -- 172.21.15.78:0/2851673071 shutdown_connections 2024-05-21T15:18:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.333+0000 7fa71a157640 1 --2- 172.21.15.78:0/2851673071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa71410e7a0 0x7fa714115f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.333+0000 7fa71a157640 1 --2- 172.21.15.78:0/2851673071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 0x7fa71410e110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.333+0000 7fa71a157640 1 --2- 172.21.15.78:0/2851673071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa714109750 0x7fa714109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.333+0000 7fa71a157640 1 -- 172.21.15.78:0/2851673071 >> 172.21.15.78:0/2851673071 conn(0x7fa71406dc00 msgr2=0x7fa71406e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.337+0000 7fa71a157640 1 -- 172.21.15.78:0/2851673071 shutdown_connections 2024-05-21T15:18:56.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.337+0000 7fa71a157640 1 -- 172.21.15.78:0/2851673071 wait complete. 2024-05-21T15:18:56.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.338+0000 7fa71a157640 1 Processor -- start 2024-05-21T15:18:56.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.338+0000 7fa71a157640 1 -- start start 2024-05-21T15:18:56.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa71a157640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa714109750 0x7fa7141a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa71a157640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 0x7fa7141a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa71a157640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa71410e7a0 0x7fa7141acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa71a157640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa714119450 con 0x7fa714109750 2024-05-21T15:18:56.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa71a157640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141192d0 con 0x7fa71410e7a0 2024-05-21T15:18:56.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa71a157640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141195d0 con 0x7fa71410dce0 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa712ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 0x7fa7141a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa712ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 0x7fa7141a8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33230/0 (socket says 172.21.15.78:33230) 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.339+0000 7fa712ffd640 1 -- 172.21.15.78:0/3094620537 learned_addr learned my addr 172.21.15.78:0/3094620537 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.340+0000 7fa713fff640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa71410e7a0 0x7fa7141acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.340+0000 7fa7137fe640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa714109750 0x7fa7141a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.340+0000 7fa712ffd640 1 -- 172.21.15.78:0/3094620537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa71410e7a0 msgr2=0x7fa7141acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.340+0000 7fa712ffd640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa71410e7a0 0x7fa7141acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.340+0000 7fa712ffd640 1 -- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa714109750 msgr2=0x7fa7141a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.340+0000 7fa712ffd640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa714109750 0x7fa7141a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.340+0000 7fa712ffd640 1 -- 172.21.15.78:0/3094620537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7141ad5a0 con 0x7fa71410dce0 2024-05-21T15:18:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.340+0000 7fa712ffd640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 0x7fa7141a8ca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa708000c00 tx=0x7fa70800a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.341+0000 7fa710ff9640 1 -- 172.21.15.78:0/3094620537 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080046c0 con 0x7fa71410dce0 2024-05-21T15:18:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.341+0000 7fa710ff9640 1 -- 172.21.15.78:0/3094620537 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa708042070 con 0x7fa71410dce0 2024-05-21T15:18:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.341+0000 7fa71a157640 1 -- 172.21.15.78:0/3094620537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7141ad830 con 0x7fa71410dce0 2024-05-21T15:18:56.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.341+0000 7fa71a157640 1 -- 172.21.15.78:0/3094620537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7141add70 con 0x7fa71410dce0 2024-05-21T15:18:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.344+0000 7fa710ff9640 1 -- 172.21.15.78:0/3094620537 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080035f0 con 0x7fa71410dce0 2024-05-21T15:18:56.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.345+0000 7fa720ffe740 1 Processor -- start 2024-05-21T15:18:56.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa720ffe740 1 -- start start 2024-05-21T15:18:56.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a41ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a4200260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42007a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa720ffe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5642a4165c40 con 0x5642a3e61390 2024-05-21T15:18:56.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa720ffe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5642a410d580 con 0x5642a3e384d0 2024-05-21T15:18:56.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa720ffe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5642a4161c00 con 0x5642a3e406b0 2024-05-21T15:18:56.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa710ff9640 1 -- 172.21.15.78:0/3094620537 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa70800f030 con 0x7fa71410dce0 2024-05-21T15:18:56.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.347+0000 7fa7137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a4200260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.348+0000 7fa713fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a41ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.348+0000 7fa713fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a41ffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52420/0 (socket says 172.21.15.78:52420) 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.348+0000 7fa713fff640 1 -- 172.21.15.78:0/2083832632 learned_addr learned my addr 172.21.15.78:0/2083832632 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.348+0000 7fa712ffd640 1 --2- 172.21.15.78:0/2083832632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.348+0000 7fa7137fe640 1 -- 172.21.15.78:0/2083832632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 msgr2=0x5642a42007a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.348+0000 7fa7137fe640 1 --2- 172.21.15.78:0/2083832632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42007a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.348+0000 7fa7137fe640 1 -- 172.21.15.78:0/2083832632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 msgr2=0x5642a41ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.348+0000 7fa7137fe640 1 --2- 172.21.15.78:0/2083832632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a41ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.349+0000 7fa7137fe640 1 -- 172.21.15.78:0/2083832632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a3e793f0 con 0x5642a3e384d0 2024-05-21T15:18:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.349+0000 7fa7137fe640 1 --2- 172.21.15.78:0/2083832632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a4200260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009f00 tx=0x7fa70c00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6ce7adc80c8446 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.349+0000 7fa6f9ffb640 1 -- 172.21.15.78:0/2083832632 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c01a070 con 0x5642a3e384d0 2024-05-21T15:18:56.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.349+0000 7fa710ff9640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa6fc0a2c60 0x7fa6fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.350+0000 7fa6f9ffb640 1 -- 172.21.15.78:0/2083832632 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa70c00aad0 con 0x5642a3e384d0 2024-05-21T15:18:56.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.350+0000 7fa6f9ffb640 1 -- 172.21.15.78:0/2083832632 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c015c00 con 0x5642a3e384d0 2024-05-21T15:18:56.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.351+0000 7fa720ffe740 1 -- 172.21.15.78:0/2083832632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 msgr2=0x5642a4200260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.351+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2083832632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a4200260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009f00 tx=0x7fa70c00c7d0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.351+0000 7fa713fff640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa6fc0a2c60 0x7fa6fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.351+0000 7fa720ffe740 1 -- 172.21.15.78:0/2083832632 shutdown_connections 2024-05-21T15:18:56.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.352+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2083832632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a4200260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.352+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2083832632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.352+0000 7fa713fff640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa6fc0a2c60 0x7fa6fc0a5170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5642a41fce90 tx=0x7fa704009060 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.352+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2083832632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a41ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.352+0000 7fa720ffe740 1 -- 172.21.15.78:0/2083832632 >> 172.21.15.78:0/2083832632 conn(0x5642a3e554e0 msgr2=0x5642a41fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.352+0000 7fa720ffe740 1 -- 172.21.15.78:0/2083832632 shutdown_connections 2024-05-21T15:18:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.352+0000 7fa710ff9640 1 -- 172.21.15.78:0/3094620537 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa7080efdd0 con 0x7fa71410dce0 2024-05-21T15:18:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.353+0000 7fa720ffe740 1 -- 172.21.15.78:0/2083832632 wait complete. 2024-05-21T15:18:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.353+0000 7fa720ffe740 1 -- start start 2024-05-21T15:18:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.354+0000 7fa720ffe740 1 Processor -- start 2024-05-21T15:18:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.354+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 0x5642a420be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e406b0 0x5642a420c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a420a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa720ffe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5642a420cfb0 con 0x5642a3e61390 2024-05-21T15:18:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa720ffe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5642a4120c40 con 0x5642a3e406b0 2024-05-21T15:18:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa720ffe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5642a420d130 con 0x5642a3e384d0 2024-05-21T15:18:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa720ffe740 10 client.0 _ll_register_callbacks cb 0x5642a3e78be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa713fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a420a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa713fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a420a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52436/0 (socket says 172.21.15.78:52436) 2024-05-21T15:18:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.355+0000 7fa713fff640 1 -- 172.21.15.78:0/2128640698 learned_addr learned my addr 172.21.15.78:0/2128640698 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.356+0000 7fa713fff640 1 -- 172.21.15.78:0/2128640698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 msgr2=0x5642a420be10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.356+0000 7fa713fff640 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 0x5642a420be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.356+0000 7fa713fff640 1 -- 172.21.15.78:0/2128640698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e406b0 msgr2=0x5642a420c350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.356+0000 7fa713fff640 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e406b0 0x5642a420c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.356+0000 7fa713fff640 1 -- 172.21.15.78:0/2128640698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a3f437d0 con 0x5642a3e61390 2024-05-21T15:18:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.357+0000 7fa713fff640 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a420a2b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa7040149c0 tx=0x7fa704004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb79db932c9912b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.357+0000 7fa6f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.357+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/2128640698 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704025070 con 0x5642a3e61390 2024-05-21T15:18:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.357+0000 7fa720ffe740 1 -- 172.21.15.78:0/2128640698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa70c00aad0 con 0x5642a3e61390 2024-05-21T15:18:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.357+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/2128640698 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa704021600 con 0x5642a3e61390 2024-05-21T15:18:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.357+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/2128640698 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa704031050 con 0x5642a3e61390 2024-05-21T15:18:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.358+0000 7fa720ffe740 10 client.16467 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.358+0000 7fa720ffe740 1 -- 172.21.15.78:0/2128640698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642a42198a0 con 0x5642a3e61390 2024-05-21T15:18:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.358+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/2128640698 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa70c00aad0 con 0x5642a3e61390 2024-05-21T15:18:56.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.359+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/2128640698 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa704020070 con 0x5642a3e61390 2024-05-21T15:18:56.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.360+0000 7fa720ffe740 10 client.16467 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:56.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.360+0000 7fa720ffe740 10 client.16467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:18:56.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.360+0000 7fa720ffe740 20 client.16467 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:56.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.360+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5642a3e1ff60 0x5642a421ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.360+0000 7fa720ffe740 4 client.16467 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:56.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.360+0000 7fa720ffe740 1 -- 172.21.15.78:0/2128640698 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642a4173370 con 0x5642a3e1ff60 2024-05-21T15:18:56.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.366+0000 7fa7137fe640 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5642a3e1ff60 0x5642a421ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.369+0000 7fa7137fe640 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5642a3e1ff60 0x5642a421ced0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2dd52c1dd532ccfc server_cookie=83d5883d6072eb67 in_seq=0 out_seq=0 2024-05-21T15:18:56.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.369+0000 7fa6f2ffd640 10 client.16467 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:18:56.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.371+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/2128640698 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642a4173370 con 0x5642a3e1ff60 2024-05-21T15:18:56.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.371+0000 7fa6f2ffd640 10 client.16467 handle_command_reply: tid=0 2024-05-21T15:18:56.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.371+0000 7fa720ffe740 1 client.16467 shutdown 2024-05-21T15:18:56.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.371+0000 7fa720ffe740 1 -- 172.21.15.78:0/2128640698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 msgr2=0x5642a420a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.372+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a420a2b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa7040149c0 tx=0x7fa704004700 comp rx=0 tx=0).stop 2024-05-21T15:18:56.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.372+0000 7fa720ffe740 1 -- 172.21.15.78:0/2128640698 shutdown_connections 2024-05-21T15:18:56.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.372+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5642a3e1ff60 0x5642a421ced0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.372+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e406b0 0x5642a420c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.372+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a420a2b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.372+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2128640698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 0x5642a420be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.372+0000 7fa720ffe740 1 -- 172.21.15.78:0/2128640698 >> 172.21.15.78:0/2128640698 conn(0x5642a3e554e0 msgr2=0x5642a4201a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.373+0000 7fa6f2ffd640 0 client.16467 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:18:56.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.376+0000 7fa720ffe740 1 -- 172.21.15.78:0/2128640698 shutdown_connections 2024-05-21T15:18:56.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.377+0000 7fa720ffe740 1 -- 172.21.15.78:0/2128640698 wait complete. 2024-05-21T15:18:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.378+0000 7fa720ffe740 20 client.16467 trim_cache size 0 max 16384 2024-05-21T15:18:56.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.410+0000 7fa720ffe740 1 Processor -- start 2024-05-21T15:18:56.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.410+0000 7fa720ffe740 1 -- start start 2024-05-21T15:18:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.411+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a42091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.411+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.411+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a4203a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.411+0000 7fa720ffe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141bd840 con 0x5642a3e61390 2024-05-21T15:18:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa720ffe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141bf4a0 con 0x5642a3e384d0 2024-05-21T15:18:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa7137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a42091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa713fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a4203a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa7137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a42091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45096/0 (socket says 172.21.15.78:45096) 2024-05-21T15:18:56.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa720ffe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141bdf10 con 0x5642a3e406b0 2024-05-21T15:18:56.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa7137fe640 1 -- 172.21.15.78:0/2444033708 learned_addr learned my addr 172.21.15.78:0/2444033708 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa712ffd640 1 --2- 172.21.15.78:0/2444033708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa712ffd640 1 -- 172.21.15.78:0/2444033708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 msgr2=0x5642a42091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa712ffd640 1 --2- 172.21.15.78:0/2444033708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a42091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa712ffd640 1 -- 172.21.15.78:0/2444033708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 msgr2=0x5642a4203a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa712ffd640 1 --2- 172.21.15.78:0/2444033708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a4203a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.412+0000 7fa712ffd640 1 -- 172.21.15.78:0/2444033708 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a3e20190 con 0x5642a3e406b0 2024-05-21T15:18:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.413+0000 7fa712ffd640 1 --2- 172.21.15.78:0/2444033708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42020e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7080ed4c0 tx=0x7fa70800ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc48fe8f0769702 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.415+0000 7fa6f97fa640 1 -- 172.21.15.78:0/2444033708 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080b8530 con 0x5642a3e406b0 2024-05-21T15:18:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.415+0000 7fa6f97fa640 1 -- 172.21.15.78:0/2444033708 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7080bc990 con 0x5642a3e406b0 2024-05-21T15:18:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.415+0000 7fa6f97fa640 1 -- 172.21.15.78:0/2444033708 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080bbbf0 con 0x5642a3e406b0 2024-05-21T15:18:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.416+0000 7fa720ffe740 1 -- 172.21.15.78:0/2444033708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 msgr2=0x5642a42020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.417+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2444033708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42020e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7080ed4c0 tx=0x7fa70800ac00 comp rx=0 tx=0).stop 2024-05-21T15:18:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.417+0000 7fa720ffe740 1 -- 172.21.15.78:0/2444033708 shutdown_connections 2024-05-21T15:18:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.417+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2444033708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e61390 0x5642a4203a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.417+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2444033708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e406b0 0x5642a42020e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.417+0000 7fa720ffe740 1 --2- 172.21.15.78:0/2444033708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e384d0 0x5642a42091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.417+0000 7fa720ffe740 1 -- 172.21.15.78:0/2444033708 >> 172.21.15.78:0/2444033708 conn(0x5642a3e554e0 msgr2=0x5642a42019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.418+0000 7fa720ffe740 1 -- 172.21.15.78:0/2444033708 shutdown_connections 2024-05-21T15:18:56.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.418+0000 7fa720ffe740 1 -- 172.21.15.78:0/2444033708 wait complete. 2024-05-21T15:18:56.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.419+0000 7fa720ffe740 1 -- start start 2024-05-21T15:18:56.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.420+0000 7fa720ffe740 1 Processor -- start 2024-05-21T15:18:56.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.420+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 0x5642a420f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.420+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e406b0 0x5642a420fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.421+0000 7fa720ffe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e61390 0x5642a4283280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.421+0000 7fa712ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e406b0 0x5642a420fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.421+0000 7fa712ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e406b0 0x5642a420fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52444/0 (socket says 172.21.15.78:52444) 2024-05-21T15:18:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.421+0000 7fa712ffd640 1 -- 172.21.15.78:0/1740293362 learned_addr learned my addr 172.21.15.78:0/1740293362 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.421+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141bed10 con 0x5642a3e406b0 2024-05-21T15:18:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.421+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141be610 con 0x5642a3e61390 2024-05-21T15:18:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.421+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141bb820 con 0x5642a3e384d0 2024-05-21T15:18:56.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.421+0000 7fa720ffe740 10 client.0 _ll_register_callbacks cb 0x7fa7040094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:56.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.424+0000 7fa712ffd640 1 -- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 msgr2=0x5642a420f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.422+0000 7fa7137fe640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 0x5642a420f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.425+0000 7fa712ffd640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 0x5642a420f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.425+0000 7fa712ffd640 1 -- 172.21.15.78:0/1740293362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e61390 msgr2=0x5642a4283280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.422+0000 7fa713fff640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e61390 0x5642a4283280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.425+0000 7fa712ffd640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e61390 0x5642a4283280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.425+0000 7fa712ffd640 1 -- 172.21.15.78:0/1740293362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a42837c0 con 0x5642a3e406b0 2024-05-21T15:18:56.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.425+0000 7fa7137fe640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 0x5642a420f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:56.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.425+0000 7fa713fff640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e61390 0x5642a4283280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.428+0000 7fa712ffd640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e406b0 0x5642a420fc00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa708005d20 tx=0x7fa708005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9802d514e3ddbe8a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.428+0000 7fa6f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:56.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.429+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/1740293362 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080bbdd0 con 0x5642a3e406b0 2024-05-21T15:18:56.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.430+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7080bc990 con 0x5642a3e406b0 2024-05-21T15:18:56.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.430+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/1740293362 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa708004e50 con 0x5642a3e406b0 2024-05-21T15:18:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.430+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/1740293362 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7080bc060 con 0x5642a3e406b0 2024-05-21T15:18:56.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.433+0000 7fa720ffe740 10 client.16473 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:56.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.433+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642a4284980 con 0x5642a3e406b0 2024-05-21T15:18:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.435+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/1740293362 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7080c0640 con 0x5642a3e406b0 2024-05-21T15:18:56.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.435+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/1740293362 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa7080c0170 con 0x5642a3e406b0 2024-05-21T15:18:56.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.436+0000 7fa720ffe740 10 client.16473 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:56.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.436+0000 7fa720ffe740 10 client.16473 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:18:56.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.436+0000 7fa720ffe740 20 client.16473 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:56.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.436+0000 7fa720ffe740 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5642a3df53a0 0x5642a4287060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.436+0000 7fa720ffe740 4 client.16473 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:56.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.436+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa7080bc990 con 0x5642a3df53a0 2024-05-21T15:18:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.437+0000 7fa7137fe640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5642a3df53a0 0x5642a4287060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.440+0000 7fa6f2ffd640 10 client.16473 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:18:56.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.440+0000 7fa7137fe640 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5642a3df53a0 0x5642a4287060 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2021e0069f6eb9a3 server_cookie=536a716094ca838e in_seq=0 out_seq=0 2024-05-21T15:18:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.441+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/1740293362 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa7080bc990 con 0x5642a3df53a0 2024-05-21T15:18:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.442+0000 7fa6f2ffd640 10 client.16473 handle_command_reply: tid=0 2024-05-21T15:18:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.442+0000 7fa720ffe740 1 client.16473 shutdown 2024-05-21T15:18:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e406b0 msgr2=0x5642a420fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa720ffe740 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e406b0 0x5642a420fc00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa708005d20 tx=0x7fa708005fd0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 shutdown_connections 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa720ffe740 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5642a3df53a0 0x5642a4287060 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa720ffe740 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642a3e61390 0x5642a4283280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa720ffe740 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642a3e406b0 0x5642a420fc00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa720ffe740 1 --2- 172.21.15.78:0/1740293362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642a3e384d0 0x5642a420f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 >> 172.21.15.78:0/1740293362 conn(0x5642a3e554e0 msgr2=0x5642a420b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.447+0000 7fa6f2ffd640 0 client.16473 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.448+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 shutdown_connections 2024-05-21T15:18:56.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.448+0000 7fa720ffe740 1 -- 172.21.15.78:0/1740293362 wait complete. 2024-05-21T15:18:56.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.454+0000 7fa720ffe740 20 client.16473 trim_cache size 0 max 16384 2024-05-21T15:18:56.461 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:56.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.458+0000 7fa71a157640 1 -- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa6fc0a2c60 msgr2=0x7fa6fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.458+0000 7fa71a157640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa6fc0a2c60 0x7fa6fc0a5170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5642a41fce90 tx=0x7fa704009060 comp rx=0 tx=0).stop 2024-05-21T15:18:56.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.458+0000 7fa71a157640 1 -- 172.21.15.78:0/3094620537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 msgr2=0x7fa7141a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.458+0000 7fa71a157640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 0x7fa7141a8ca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa708000c00 tx=0x7fa70800a950 comp rx=0 tx=0).stop 2024-05-21T15:18:56.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.459+0000 7fa71a157640 1 -- 172.21.15.78:0/3094620537 shutdown_connections 2024-05-21T15:18:56.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.459+0000 7fa71a157640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa71410e7a0 0x7fa7141acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.459+0000 7fa71a157640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa6fc0a2c60 0x7fa6fc0a5170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.459+0000 7fa71a157640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71410dce0 0x7fa7141a8ca0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa708000c00 tx=0x7fa70800a950 comp rx=0 tx=0).stop 2024-05-21T15:18:56.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.459+0000 7fa71a157640 1 --2- 172.21.15.78:0/3094620537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa714109750 0x7fa7141a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.459+0000 7fa71a157640 1 -- 172.21.15.78:0/3094620537 >> 172.21.15.78:0/3094620537 conn(0x7fa71406dc00 msgr2=0x7fa71410cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.459+0000 7fa71a157640 1 -- 172.21.15.78:0/3094620537 shutdown_connections 2024-05-21T15:18:56.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.459+0000 7fa71a157640 1 -- 172.21.15.78:0/3094620537 wait complete. 2024-05-21T15:18:56.486 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:18:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.695+0000 7f22257ff640 1 -- 172.21.15.78:0/2918242107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2220119770 msgr2=0x7f222011bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.695+0000 7f22257ff640 1 --2- 172.21.15.78:0/2918242107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2220119770 0x7f222011bb90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2210009900 tx=0x7f221002a4d0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.696+0000 7f22257ff640 1 -- 172.21.15.78:0/2918242107 shutdown_connections 2024-05-21T15:18:56.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.696+0000 7f22257ff640 1 --2- 172.21.15.78:0/2918242107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222011c0d0 0x7f222011e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.696+0000 7f22257ff640 1 --2- 172.21.15.78:0/2918242107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2220119770 0x7f222011bb90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.696+0000 7f22257ff640 1 --2- 172.21.15.78:0/2918242107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2220108f70 0x7f2220109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.696+0000 7f22257ff640 1 -- 172.21.15.78:0/2918242107 >> 172.21.15.78:0/2918242107 conn(0x7f222006d5b0 msgr2=0x7f222006da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.699+0000 7f22257ff640 1 -- 172.21.15.78:0/2918242107 shutdown_connections 2024-05-21T15:18:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.700+0000 7f22257ff640 1 -- 172.21.15.78:0/2918242107 wait complete. 2024-05-21T15:18:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.701+0000 7f22257ff640 1 Processor -- start 2024-05-21T15:18:56.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.704+0000 7f22257ff640 1 -- start start 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f22257ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2220108f70 0x7f22201a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f22257ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2220119770 0x7f22201a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f22257ff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222011c0d0 0x7f22201acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f22257ff640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2220121ae0 con 0x7f2220119770 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f22257ff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2220121960 con 0x7f2220108f70 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f22257ff640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2220121c60 con 0x7f222011c0d0 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f221effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2220108f70 0x7f22201a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f221effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2220108f70 0x7f22201a8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45118/0 (socket says 172.21.15.78:45118) 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.705+0000 7f221effd640 1 -- 172.21.15.78:0/1018599156 learned_addr learned my addr 172.21.15.78:0/1018599156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.706+0000 7f221f7fe640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222011c0d0 0x7f22201acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.707+0000 7f221e7fc640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2220119770 0x7f22201a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.709+0000 7f221f7fe640 1 -- 172.21.15.78:0/1018599156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2220108f70 msgr2=0x7f22201a8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.709+0000 7f221f7fe640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2220108f70 0x7f22201a8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.709+0000 7f221f7fe640 1 -- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2220119770 msgr2=0x7f22201a8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.709+0000 7f221f7fe640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2220119770 0x7f22201a8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.709+0000 7f221f7fe640 1 -- 172.21.15.78:0/1018599156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22201ad360 con 0x7f222011c0d0 2024-05-21T15:18:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.711+0000 7f221f7fe640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222011c0d0 0x7f22201acd20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f221400e970 tx=0x7f221400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.712+0000 7f2207fff640 1 -- 172.21.15.78:0/1018599156 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400ccc0 con 0x7f222011c0d0 2024-05-21T15:18:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.712+0000 7f2207fff640 1 -- 172.21.15.78:0/1018599156 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22140035f0 con 0x7f222011c0d0 2024-05-21T15:18:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.712+0000 7f2207fff640 1 -- 172.21.15.78:0/1018599156 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214010690 con 0x7f222011c0d0 2024-05-21T15:18:56.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.718+0000 7f22257ff640 1 -- 172.21.15.78:0/1018599156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22201ad650 con 0x7f222011c0d0 2024-05-21T15:18:56.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.718+0000 7f22257ff640 1 -- 172.21.15.78:0/1018599156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201adb20 con 0x7f222011c0d0 2024-05-21T15:18:56.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.722+0000 7f222c77c740 1 Processor -- start 2024-05-21T15:18:56.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.722+0000 7f222c77c740 1 -- start start 2024-05-21T15:18:56.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.722+0000 7f222c77c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d8e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.723+0000 7f222c77c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d51e4d0 0x55b88d8e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.723+0000 7f222c77c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.723+0000 7f222c77c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b88d84bc40 con 0x55b88d547390 2024-05-21T15:18:56.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.723+0000 7f222c77c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b88d7f3580 con 0x55b88d51e4d0 2024-05-21T15:18:56.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.723+0000 7f222c77c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b88d847c00 con 0x55b88d5266b0 2024-05-21T15:18:56.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.724+0000 7f221e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d8e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.724+0000 7f221e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d8e5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52484/0 (socket says 172.21.15.78:52484) 2024-05-21T15:18:56.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.724+0000 7f221e7fc640 1 -- 172.21.15.78:0/2159936125 learned_addr learned my addr 172.21.15.78:0/2159936125 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.724+0000 7f221e7fc640 1 --2- 172.21.15.78:0/2159936125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.727+0000 7f221e7fc640 1 -- 172.21.15.78:0/2159936125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 msgr2=0x55b88d8e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.729+0000 7f221e7fc640 1 --2- 172.21.15.78:0/2159936125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.729+0000 7f221e7fc640 1 -- 172.21.15.78:0/2159936125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d51e4d0 msgr2=0x55b88d8e64a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:18:56.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.729+0000 7f221e7fc640 1 --2- 172.21.15.78:0/2159936125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d51e4d0 0x55b88d8e64a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.729+0000 7f221e7fc640 1 -- 172.21.15.78:0/2159936125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88d55f3f0 con 0x55b88d547390 2024-05-21T15:18:56.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.730+0000 7f2207fff640 1 -- 172.21.15.78:0/1018599156 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f22140037a0 con 0x7f222011c0d0 2024-05-21T15:18:56.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.731+0000 7f2207fff640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f22080a2e40 0x7f22080a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.731+0000 7f2207fff640 1 -- 172.21.15.78:0/1018599156 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f22140cf8a0 con 0x7f222011c0d0 2024-05-21T15:18:56.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.739+0000 7f221effd640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f22080a2e40 0x7f22080a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.739+0000 7f221e7fc640 1 --2- 172.21.15.78:0/2159936125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d8e5f60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f221002aa40 tx=0x7f2210005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d505f0019da1df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.743+0000 7f22057fa640 1 -- 172.21.15.78:0/2159936125 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210044020 con 0x55b88d547390 2024-05-21T15:18:56.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.743+0000 7f22057fa640 1 -- 172.21.15.78:0/2159936125 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2210005000 con 0x55b88d547390 2024-05-21T15:18:56.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.743+0000 7f22057fa640 1 -- 172.21.15.78:0/2159936125 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210003a10 con 0x55b88d547390 2024-05-21T15:18:56.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.744+0000 7f222c77c740 1 -- 172.21.15.78:0/2159936125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 msgr2=0x55b88d8e5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.744+0000 7f222c77c740 1 --2- 172.21.15.78:0/2159936125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d8e5f60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f221002aa40 tx=0x7f2210005d50 comp rx=0 tx=0).stop 2024-05-21T15:18:56.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.744+0000 7f222c77c740 1 -- 172.21.15.78:0/2159936125 shutdown_connections 2024-05-21T15:18:56.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.744+0000 7f222c77c740 1 --2- 172.21.15.78:0/2159936125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d51e4d0 0x55b88d8e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.744+0000 7f222c77c740 1 --2- 172.21.15.78:0/2159936125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.744+0000 7f222c77c740 1 --2- 172.21.15.78:0/2159936125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d8e5f60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.744+0000 7f222c77c740 1 -- 172.21.15.78:0/2159936125 >> 172.21.15.78:0/2159936125 conn(0x55b88d53b4e0 msgr2=0x55b88d8e2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.752+0000 7f222c77c740 1 -- 172.21.15.78:0/2159936125 shutdown_connections 2024-05-21T15:18:56.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.752+0000 7f222c77c740 1 -- 172.21.15.78:0/2159936125 wait complete. 2024-05-21T15:18:56.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.753+0000 7f222c77c740 1 -- start start 2024-05-21T15:18:56.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.753+0000 7f222c77c740 1 Processor -- start 2024-05-21T15:18:56.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.754+0000 7f222c77c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.754+0000 7f222c77c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.754+0000 7f222c77c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 0x55b88d8f0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.754+0000 7f222c77c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b88d904260 con 0x55b88d51e4d0 2024-05-21T15:18:56.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.754+0000 7f222c77c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b88d806c40 con 0x55b88d547390 2024-05-21T15:18:56.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.754+0000 7f222c77c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b88d9043e0 con 0x55b88d5266b0 2024-05-21T15:18:56.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.754+0000 7f222c77c740 10 client.0 _ll_register_callbacks cb 0x55b88d55ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:56.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.755+0000 7f221effd640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f22080a2e40 0x7f22080a5350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55b88d8e7520 tx=0x7f2218009590 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52490/0 (socket says 172.21.15.78:52490) 2024-05-21T15:18:56.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221effd640 1 -- 172.21.15.78:0/2400209476 learned_addr learned my addr 172.21.15.78:0/2400209476 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221f7fe640 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 0x55b88d8f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221e7fc640 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221e7fc640 1 -- 172.21.15.78:0/2400209476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 msgr2=0x55b88d8f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221e7fc640 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 0x55b88d8f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221e7fc640 1 -- 172.21.15.78:0/2400209476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 msgr2=0x55b88d8f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221e7fc640 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.756+0000 7f221e7fc640 1 -- 172.21.15.78:0/2400209476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88d6297d0 con 0x55b88d5266b0 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.757+0000 7f221e7fc640 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8f24c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2210009900 tx=0x7f221002a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac2feefacd2801b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.757+0000 7f2202ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.758+0000 7f2202ff5640 1 -- 172.21.15.78:0/2400209476 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210040450 con 0x55b88d5266b0 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.758+0000 7f222c77c740 1 -- 172.21.15.78:0/2400209476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2210005000 con 0x55b88d5266b0 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.758+0000 7f2202ff5640 1 -- 172.21.15.78:0/2400209476 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f22100405f0 con 0x55b88d5266b0 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.758+0000 7f2202ff5640 1 -- 172.21.15.78:0/2400209476 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2210059050 con 0x55b88d5266b0 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.758+0000 7f222c77c740 10 client.25945 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.758+0000 7f222c77c740 1 -- 172.21.15.78:0/2400209476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b88d904260 con 0x55b88d5266b0 2024-05-21T15:18:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.759+0000 7f2202ff5640 1 -- 172.21.15.78:0/2400209476 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2210005000 con 0x55b88d5266b0 2024-05-21T15:18:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.760+0000 7f2202ff5640 1 -- 172.21.15.78:0/2400209476 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f221003e4c0 con 0x55b88d5266b0 2024-05-21T15:18:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.760+0000 7f222c77c740 10 client.25945 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.760+0000 7f222c77c740 10 client.25945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:18:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.760+0000 7f222c77c740 20 client.25945 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.760+0000 7f222c77c740 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b88d505f60 0x55b88d914010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.760+0000 7f222c77c740 4 client.25945 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:56.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.760+0000 7f222c77c740 1 -- 172.21.15.78:0/2400209476 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b88d859370 con 0x55b88d505f60 2024-05-21T15:18:56.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.765+0000 7f221effd640 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b88d505f60 0x55b88d914010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.766+0000 7f221effd640 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b88d505f60 0x55b88d914010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94c85b4d88a4edd server_cookie=767aab1785043d11 in_seq=0 out_seq=0 2024-05-21T15:18:56.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.766+0000 7f2202ff5640 10 client.25945 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:18:56.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.767+0000 7f2202ff5640 1 -- 172.21.15.78:0/2400209476 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b88d859370 con 0x55b88d505f60 2024-05-21T15:18:56.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.767+0000 7f2202ff5640 10 client.25945 handle_command_reply: tid=0 2024-05-21T15:18:56.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.767+0000 7f222c77c740 1 client.25945 shutdown 2024-05-21T15:18:56.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.767+0000 7f222c77c740 1 -- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 msgr2=0x55b88d8f24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.767+0000 7f222c77c740 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8f24c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2210009900 tx=0x7f221002a4d0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.769+0000 7f222c77c740 1 -- 172.21.15.78:0/2400209476 shutdown_connections 2024-05-21T15:18:56.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.769+0000 7f222c77c740 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b88d505f60 0x55b88d914010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.769+0000 7f222c77c740 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 0x55b88d8f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.769+0000 7f222c77c740 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d8f24c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.769+0000 7f222c77c740 1 --2- 172.21.15.78:0/2400209476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.769+0000 7f222c77c740 1 -- 172.21.15.78:0/2400209476 >> 172.21.15.78:0/2400209476 conn(0x55b88d53b4e0 msgr2=0x55b88d8e5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.769+0000 7f2202ff5640 0 client.25945 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:18:56.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.769+0000 7f222c77c740 1 -- 172.21.15.78:0/2400209476 shutdown_connections 2024-05-21T15:18:56.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.770+0000 7f222c77c740 1 -- 172.21.15.78:0/2400209476 wait complete. 2024-05-21T15:18:56.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.770+0000 7f222c77c740 20 client.25945 trim_cache size 0 max 16384 2024-05-21T15:18:56.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.802+0000 7f222c77c740 1 Processor -- start 2024-05-21T15:18:56.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.802+0000 7f222c77c740 1 -- start start 2024-05-21T15:18:56.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.802+0000 7f222c77c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.803+0000 7f222c77c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d909670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.803+0000 7f222c77c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 0x55b88d909bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.803+0000 7f222c77c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2220119580 con 0x55b88d51e4d0 2024-05-21T15:18:56.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.803+0000 7f222c77c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b6ae0 con 0x55b88d547390 2024-05-21T15:18:56.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.803+0000 7f222c77c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b5550 con 0x55b88d5266b0 2024-05-21T15:18:56.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.804+0000 7f221e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d909670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.804+0000 7f221e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d909670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33326/0 (socket says 172.21.15.78:33326) 2024-05-21T15:18:56.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.804+0000 7f221e7fc640 1 -- 172.21.15.78:0/293976184 learned_addr learned my addr 172.21.15.78:0/293976184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.804+0000 7f221f7fe640 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 0x55b88d909bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.804+0000 7f221effd640 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.805+0000 7f221effd640 1 -- 172.21.15.78:0/293976184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 msgr2=0x55b88d909670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.805+0000 7f221effd640 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d909670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.805+0000 7f221effd640 1 -- 172.21.15.78:0/293976184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 msgr2=0x55b88d909bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.805+0000 7f221effd640 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 0x55b88d909bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.805+0000 7f221effd640 1 -- 172.21.15.78:0/293976184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88d8e7550 con 0x55b88d51e4d0 2024-05-21T15:18:56.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.805+0000 7f221effd640 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f03c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55b88d914740 tx=0x7f2218014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9debd41dd3b91834 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.806+0000 7f2204ff9640 1 -- 172.21.15.78:0/293976184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218029070 con 0x55b88d51e4d0 2024-05-21T15:18:56.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.806+0000 7f2204ff9640 1 -- 172.21.15.78:0/293976184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2218016e40 con 0x55b88d51e4d0 2024-05-21T15:18:56.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 -- 172.21.15.78:0/293976184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 msgr2=0x55b88d8f03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f03c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55b88d914740 tx=0x7f2218014000 comp rx=0 tx=0).stop 2024-05-21T15:18:56.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 -- 172.21.15.78:0/293976184 shutdown_connections 2024-05-21T15:18:56.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d547390 0x55b88d909bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d5266b0 0x55b88d909670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 --2- 172.21.15.78:0/293976184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d51e4d0 0x55b88d8f03c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 -- 172.21.15.78:0/293976184 >> 172.21.15.78:0/293976184 conn(0x55b88d53b4e0 msgr2=0x55b88d8efcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 -- 172.21.15.78:0/293976184 shutdown_connections 2024-05-21T15:18:56.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.808+0000 7f222c77c740 1 -- 172.21.15.78:0/293976184 wait complete. 2024-05-21T15:18:56.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.809+0000 7f222c77c740 1 -- start start 2024-05-21T15:18:56.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.809+0000 7f222c77c740 1 Processor -- start 2024-05-21T15:18:56.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.810+0000 7f222c77c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d5266b0 0x55b88d908190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.810+0000 7f222c77c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d9086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.810+0000 7f221effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d5266b0 0x55b88d908190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.810+0000 7f221effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d5266b0 0x55b88d908190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45142/0 (socket says 172.21.15.78:45142) 2024-05-21T15:18:56.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.810+0000 7f221effd640 1 -- 172.21.15.78:0/3164061725 learned_addr learned my addr 172.21.15.78:0/3164061725 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.810+0000 7f222c77c740 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d4db3a0 0x55b88d908c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.810+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b6350 con 0x55b88d547390 2024-05-21T15:18:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.810+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b5c50 con 0x55b88d5266b0 2024-05-21T15:18:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.811+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f222010a1f0 con 0x55b88d4db3a0 2024-05-21T15:18:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.811+0000 7f222c77c740 10 client.0 _ll_register_callbacks cb 0x7f2210009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.811+0000 7f221f7fe640 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d4db3a0 0x55b88d908c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.812+0000 7f221e7fc640 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d9086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.812+0000 7f221effd640 1 -- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d4db3a0 msgr2=0x55b88d908c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.812+0000 7f221effd640 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d4db3a0 0x55b88d908c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.812+0000 7f221effd640 1 -- 172.21.15.78:0/3164061725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 msgr2=0x55b88d9086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.812+0000 7f221effd640 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d9086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.812+0000 7f221effd640 1 -- 172.21.15.78:0/3164061725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88d8dc880 con 0x55b88d5266b0 2024-05-21T15:18:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.813+0000 7f221effd640 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d5266b0 0x55b88d908190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2218016710 tx=0x7f2218015b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83f4c882678c324 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.813+0000 7f2202ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.814+0000 7f2202ff5640 1 -- 172.21.15.78:0/3164061725 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218032060 con 0x55b88d5266b0 2024-05-21T15:18:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.814+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b88d8ddb70 con 0x55b88d5266b0 2024-05-21T15:18:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.814+0000 7f2202ff5640 1 -- 172.21.15.78:0/3164061725 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2218015e30 con 0x55b88d5266b0 2024-05-21T15:18:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.815+0000 7f2202ff5640 1 -- 172.21.15.78:0/3164061725 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2218029040 con 0x55b88d5266b0 2024-05-21T15:18:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.815+0000 7f222c77c740 10 client.25943 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.815+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b88d97b340 con 0x55b88d5266b0 2024-05-21T15:18:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.815+0000 7f2202ff5640 1 -- 172.21.15.78:0/3164061725 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b88d8ddb70 con 0x55b88d5266b0 2024-05-21T15:18:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.817+0000 7f2202ff5640 1 -- 172.21.15.78:0/3164061725 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f221802c070 con 0x55b88d5266b0 2024-05-21T15:18:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.817+0000 7f222c77c740 10 client.25943 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.817+0000 7f222c77c740 10 client.25943 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:18:56.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.817+0000 7f222c77c740 20 client.25943 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.817+0000 7f222c77c740 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b88d97b910 0x55b88d97dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.817+0000 7f222c77c740 4 client.25943 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.817+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b88d8536c0 con 0x55b88d97b910 2024-05-21T15:18:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.818+0000 7f221e7fc640 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b88d97b910 0x55b88d97dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.819+0000 7f221e7fc640 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b88d97b910 0x55b88d97dd30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=482e2c947e2af61 server_cookie=20c84bbff8ad619 in_seq=0 out_seq=0 2024-05-21T15:18:56.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.819+0000 7f2202ff5640 10 client.25943 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:18:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.820+0000 7f2202ff5640 1 -- 172.21.15.78:0/3164061725 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b88d8536c0 con 0x55b88d97b910 2024-05-21T15:18:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.820+0000 7f2202ff5640 10 client.25943 handle_command_reply: tid=0 2024-05-21T15:18:56.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.820+0000 7f222c77c740 1 client.25943 shutdown 2024-05-21T15:18:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.824+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d5266b0 msgr2=0x55b88d908190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.824+0000 7f222c77c740 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d5266b0 0x55b88d908190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2218016710 tx=0x7f2218015b10 comp rx=0 tx=0).stop 2024-05-21T15:18:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.825+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 shutdown_connections 2024-05-21T15:18:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.825+0000 7f222c77c740 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b88d97b910 0x55b88d97dd30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.825+0000 7f222c77c740 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b88d4db3a0 0x55b88d908c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.825+0000 7f222c77c740 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b88d547390 0x55b88d9086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.825+0000 7f222c77c740 1 --2- 172.21.15.78:0/3164061725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b88d5266b0 0x55b88d908190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.825+0000 7f2202ff5640 0 client.25943 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:18:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.825+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 >> 172.21.15.78:0/3164061725 conn(0x55b88d53b4e0 msgr2=0x55b88d8e5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.826+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 shutdown_connections 2024-05-21T15:18:56.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.828+0000 7f222c77c740 1 -- 172.21.15.78:0/3164061725 wait complete. 2024-05-21T15:18:56.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.829+0000 7f222c77c740 20 client.25943 trim_cache size 0 max 16384 2024-05-21T15:18:56.835 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 -- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f22080a2e40 msgr2=0x7f22080a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f22080a2e40 0x7f22080a5350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55b88d8e7520 tx=0x7f2218009590 comp rx=0 tx=0).stop 2024-05-21T15:18:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 -- 172.21.15.78:0/1018599156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222011c0d0 msgr2=0x7f22201acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222011c0d0 0x7f22201acd20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f221400e970 tx=0x7f221400ee30 comp rx=0 tx=0).stop 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 -- 172.21.15.78:0/1018599156 shutdown_connections 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f22080a2e40 0x7f22080a5350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f222011c0d0 0x7f22201acd20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2220119770 0x7f22201a8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 --2- 172.21.15.78:0/1018599156 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2220108f70 0x7f22201a8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 -- 172.21.15.78:0/1018599156 >> 172.21.15.78:0/1018599156 conn(0x7f222006d5b0 msgr2=0x7f2220119b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 -- 172.21.15.78:0/1018599156 shutdown_connections 2024-05-21T15:18:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:56.830+0000 7f22257ff640 1 -- 172.21.15.78:0/1018599156 wait complete. 2024-05-21T15:18:56.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:18:57.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.066+0000 7f6735b79640 1 -- 172.21.15.78:0/1742755813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67301013b0 msgr2=0x7f67301017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.066+0000 7f6735b79640 1 --2- 172.21.15.78:0/1742755813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67301013b0 0x7f67301017c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6724009860 tx=0x7f672402a340 comp rx=0 tx=0).stop 2024-05-21T15:18:57.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.066+0000 7f6735b79640 1 -- 172.21.15.78:0/1742755813 shutdown_connections 2024-05-21T15:18:57.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.066+0000 7f6735b79640 1 --2- 172.21.15.78:0/1742755813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6730106400 0x7f673010dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.066+0000 7f6735b79640 1 --2- 172.21.15.78:0/1742755813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6730105940 0x7f6730105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.066+0000 7f6735b79640 1 --2- 172.21.15.78:0/1742755813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67301013b0 0x7f67301017c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.066+0000 7f6735b79640 1 -- 172.21.15.78:0/1742755813 >> 172.21.15.78:0/1742755813 conn(0x7f6730076830 msgr2=0x7f6730076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.068+0000 7f6735b79640 1 -- 172.21.15.78:0/1742755813 shutdown_connections 2024-05-21T15:18:57.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.068+0000 7f6735b79640 1 -- 172.21.15.78:0/1742755813 wait complete. 2024-05-21T15:18:57.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.070+0000 7f6735b79640 1 Processor -- start 2024-05-21T15:18:57.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.071+0000 7f6735b79640 1 -- start start 2024-05-21T15:18:57.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.071+0000 7f6735b79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67301013b0 0x7f67301974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.071+0000 7f6735b79640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6730105940 0x7f6730197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.071+0000 7f6735b79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6730106400 0x7f673019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.071+0000 7f6735b79640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6730110c90 con 0x7f67301013b0 2024-05-21T15:18:57.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.071+0000 7f6735b79640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6730110b10 con 0x7f6730105940 2024-05-21T15:18:57.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.071+0000 7f6735b79640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6730110e10 con 0x7f6730106400 2024-05-21T15:18:57.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.072+0000 7f672effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6730105940 0x7f6730197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.073+0000 7f672ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6730106400 0x7f673019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.073+0000 7f672effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6730105940 0x7f6730197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45156/0 (socket says 172.21.15.78:45156) 2024-05-21T15:18:57.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.073+0000 7f672effd640 1 -- 172.21.15.78:0/644019237 learned_addr learned my addr 172.21.15.78:0/644019237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.075+0000 7f672effd640 1 -- 172.21.15.78:0/644019237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6730106400 msgr2=0x7f673019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.075+0000 7f672effd640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6730106400 0x7f673019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.075+0000 7f672effd640 1 -- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67301013b0 msgr2=0x7f67301974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.075+0000 7f672effd640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67301013b0 0x7f67301974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.075+0000 7f672effd640 1 -- 172.21.15.78:0/644019237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673019c300 con 0x7f6730105940 2024-05-21T15:18:57.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.081+0000 7f672effd640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6730105940 0x7f6730197a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6720009ce0 tx=0x7f672000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.084+0000 7f672cff9640 1 -- 172.21.15.78:0/644019237 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672001f070 con 0x7f6730105940 2024-05-21T15:18:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.084+0000 7f672cff9640 1 -- 172.21.15.78:0/644019237 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672001a8f0 con 0x7f6730105940 2024-05-21T15:18:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.084+0000 7f672cff9640 1 -- 172.21.15.78:0/644019237 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720019800 con 0x7f6730105940 2024-05-21T15:18:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.086+0000 7f6735b79640 1 -- 172.21.15.78:0/644019237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673019c5f0 con 0x7f6730105940 2024-05-21T15:18:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.086+0000 7f6735b79640 1 -- 172.21.15.78:0/644019237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673019caf0 con 0x7f6730105940 2024-05-21T15:18:57.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.090+0000 7f673ca19740 1 Processor -- start 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.090+0000 7f673ca19740 1 -- start start 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f673ca19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f673ca19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f673ca19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 0x561d3a3a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f673ca19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561d3a2fdc40 con 0x561d39ff9390 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f673ca19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561d3a2a5580 con 0x561d39fd04d0 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f673ca19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d3a2f9c00 con 0x561d39fd86b0 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f672ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f672ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52524/0 (socket says 172.21.15.78:52524) 2024-05-21T15:18:57.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.091+0000 7f672ffff640 1 -- 172.21.15.78:0/2556445709 learned_addr learned my addr 172.21.15.78:0/2556445709 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.092+0000 7f672ffff640 1 -- 172.21.15.78:0/2556445709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 msgr2=0x561d3a3a0fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:18:57.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.092+0000 7f672ffff640 1 --2- 172.21.15.78:0/2556445709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 0x561d3a3a0fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.092+0000 7f672ffff640 1 -- 172.21.15.78:0/2556445709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 msgr2=0x561d3a3a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.092+0000 7f672ffff640 1 --2- 172.21.15.78:0/2556445709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.092+0000 7f672ffff640 1 -- 172.21.15.78:0/2556445709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d3a0113f0 con 0x561d39ff9390 2024-05-21T15:18:57.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.092+0000 7f672ffff640 1 --2- 172.21.15.78:0/2556445709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3a0530 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6728009f20 tx=0x7f672800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c70e8091f605e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.097+0000 7f6715ffb640 1 -- 172.21.15.78:0/2556445709 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672801a070 con 0x561d39ff9390 2024-05-21T15:18:57.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.097+0000 7f6715ffb640 1 -- 172.21.15.78:0/2556445709 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67280158f0 con 0x561d39ff9390 2024-05-21T15:18:57.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.097+0000 7f672cff9640 1 -- 172.21.15.78:0/644019237 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6720022070 con 0x7f6730105940 2024-05-21T15:18:57.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.098+0000 7f672cff9640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67180a2e40 0x7f67180a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.098+0000 7f673ca19740 1 -- 172.21.15.78:0/2556445709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 msgr2=0x561d3a3a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.098+0000 7f673ca19740 1 --2- 172.21.15.78:0/2556445709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3a0530 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6728009f20 tx=0x7f672800c8b0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.098+0000 7f672cff9640 1 -- 172.21.15.78:0/644019237 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f67200cf0e0 con 0x7f6730105940 2024-05-21T15:18:57.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.099+0000 7f673ca19740 1 -- 172.21.15.78:0/2556445709 shutdown_connections 2024-05-21T15:18:57.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.099+0000 7f673ca19740 1 --2- 172.21.15.78:0/2556445709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.099+0000 7f673ca19740 1 --2- 172.21.15.78:0/2556445709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 0x561d3a3a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.099+0000 7f673ca19740 1 --2- 172.21.15.78:0/2556445709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3a0530 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.099+0000 7f673ca19740 1 -- 172.21.15.78:0/2556445709 >> 172.21.15.78:0/2556445709 conn(0x561d39fed4e0 msgr2=0x561d3a39cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.099+0000 7f672f7fe640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67180a2e40 0x7f67180a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.100+0000 7f673ca19740 1 -- 172.21.15.78:0/2556445709 shutdown_connections 2024-05-21T15:18:57.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.100+0000 7f673ca19740 1 -- 172.21.15.78:0/2556445709 wait complete. 2024-05-21T15:18:57.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.100+0000 7f673ca19740 1 -- start start 2024-05-21T15:18:57.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.101+0000 7f673ca19740 1 Processor -- start 2024-05-21T15:18:57.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.101+0000 7f673ca19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.101+0000 7f673ca19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39fd86b0 0x561d3a3a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.101+0000 7f673ca19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39ff9390 0x561d3a3a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.101+0000 7f673ca19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561d3a394260 con 0x561d39fd86b0 2024-05-21T15:18:57.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.101+0000 7f673ca19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561d3a2b8c40 con 0x561d39fd04d0 2024-05-21T15:18:57.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.101+0000 7f673ca19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d3a3943e0 con 0x561d39ff9390 2024-05-21T15:18:57.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.101+0000 7f673ca19740 10 client.0 _ll_register_callbacks cb 0x561d3a010be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:57.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45170/0 (socket says 172.21.15.78:45170) 2024-05-21T15:18:57.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672f7fe640 1 -- 172.21.15.78:0/192034912 learned_addr learned my addr 172.21.15.78:0/192034912 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672effd640 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39fd86b0 0x561d3a3a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672ffff640 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39ff9390 0x561d3a3a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672f7fe640 1 -- 172.21.15.78:0/192034912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39ff9390 msgr2=0x561d3a3a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672f7fe640 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39ff9390 0x561d3a3a84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672f7fe640 1 -- 172.21.15.78:0/192034912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39fd86b0 msgr2=0x561d3a3a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672f7fe640 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39fd86b0 0x561d3a3a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.102+0000 7f672f7fe640 1 -- 172.21.15.78:0/192034912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d3a0db7d0 con 0x561d39fd04d0 2024-05-21T15:18:57.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.103+0000 7f672f7fe640 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a3520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x561d3a39e020 tx=0x7f6724030ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c36c675af65254c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.103+0000 7f670effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:57.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.104+0000 7f670effd640 1 -- 172.21.15.78:0/192034912 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724045070 con 0x561d39fd04d0 2024-05-21T15:18:57.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.104+0000 7f673ca19740 1 -- 172.21.15.78:0/192034912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67280158f0 con 0x561d39fd04d0 2024-05-21T15:18:57.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.104+0000 7f670effd640 1 -- 172.21.15.78:0/192034912 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f6724003ce0 con 0x561d39fd04d0 2024-05-21T15:18:57.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.105+0000 7f672f7fe640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67180a2e40 0x7f67180a5350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6724009830 tx=0x7f672403a000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.107+0000 7f670effd640 1 -- 172.21.15.78:0/192034912 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6724051030 con 0x561d39fd04d0 2024-05-21T15:18:57.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.107+0000 7f673ca19740 10 client.25949 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:57.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.107+0000 7f673ca19740 1 -- 172.21.15.78:0/192034912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561d3a3ac680 con 0x561d39fd04d0 2024-05-21T15:18:57.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.108+0000 7f670effd640 1 -- 172.21.15.78:0/192034912 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67280158f0 con 0x561d39fd04d0 2024-05-21T15:18:57.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.109+0000 7f670effd640 1 -- 172.21.15.78:0/192034912 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6724054070 con 0x561d39fd04d0 2024-05-21T15:18:57.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.109+0000 7f673ca19740 10 client.25949 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:57.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.109+0000 7f673ca19740 10 client.25949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:18:57.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.109+0000 7f673ca19740 20 client.25949 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:57.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.109+0000 7f673ca19740 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x561d39fb7f60 0x561d3a3acde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.109+0000 7f673ca19740 4 client.25949 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:57.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.109+0000 7f673ca19740 1 -- 172.21.15.78:0/192034912 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d3a30b370 con 0x561d39fb7f60 2024-05-21T15:18:57.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.110+0000 7f672effd640 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x561d39fb7f60 0x561d3a3acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.113+0000 7f672effd640 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x561d39fb7f60 0x561d3a3acde0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9022b8c96ec4713 server_cookie=6f578835fa02bc10 in_seq=0 out_seq=0 2024-05-21T15:18:57.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.113+0000 7f670effd640 10 client.25949 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:18:57.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.114+0000 7f670effd640 1 -- 172.21.15.78:0/192034912 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d3a30b370 con 0x561d39fb7f60 2024-05-21T15:18:57.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.114+0000 7f670effd640 10 client.25949 handle_command_reply: tid=0 2024-05-21T15:18:57.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.114+0000 7f673ca19740 1 client.25949 shutdown 2024-05-21T15:18:57.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 -- 172.21.15.78:0/192034912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 msgr2=0x561d3a3a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a3520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x561d3a39e020 tx=0x7f6724030ac0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 -- 172.21.15.78:0/192034912 shutdown_connections 2024-05-21T15:18:57.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x561d39fb7f60 0x561d3a3acde0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39ff9390 0x561d3a3a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39fd86b0 0x561d3a3a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 --2- 172.21.15.78:0/192034912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd04d0 0x561d3a3a3520 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 -- 172.21.15.78:0/192034912 >> 172.21.15.78:0/192034912 conn(0x561d39fed4e0 msgr2=0x561d3a3a0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f670effd640 0 client.25949 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:18:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 -- 172.21.15.78:0/192034912 shutdown_connections 2024-05-21T15:18:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 1 -- 172.21.15.78:0/192034912 wait complete. 2024-05-21T15:18:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.115+0000 7f673ca19740 20 client.25949 trim_cache size 0 max 16384 2024-05-21T15:18:57.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.145+0000 7f673ca19740 1 Processor -- start 2024-05-21T15:18:57.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.146+0000 7f673ca19740 1 -- start start 2024-05-21T15:18:57.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.146+0000 7f673ca19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd04d0 0x561d3a39a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.146+0000 7f673ca19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd86b0 0x561d3a3cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.147+0000 7f673ca19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.147+0000 7f672f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd04d0 0x561d3a39a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.147+0000 7f672f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd04d0 0x561d3a39a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33378/0 (socket says 172.21.15.78:33378) 2024-05-21T15:18:57.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.147+0000 7f672f7fe640 1 -- 172.21.15.78:0/44716884 learned_addr learned my addr 172.21.15.78:0/44716884 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.147+0000 7f673ca19740 1 -- 172.21.15.78:0/44716884 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ac7b0 con 0x561d39ff9390 2024-05-21T15:18:57.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.147+0000 7f673ca19740 1 -- 172.21.15.78:0/44716884 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ae410 con 0x561d39fd86b0 2024-05-21T15:18:57.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.147+0000 7f673ca19740 1 -- 172.21.15.78:0/44716884 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ace80 con 0x561d39fd04d0 2024-05-21T15:18:57.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.148+0000 7f672ffff640 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.149+0000 7f672effd640 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd86b0 0x561d3a3cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.149+0000 7f672f7fe640 1 -- 172.21.15.78:0/44716884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd86b0 msgr2=0x561d3a3cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.149+0000 7f672f7fe640 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd86b0 0x561d3a3cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.149+0000 7f672f7fe640 1 -- 172.21.15.78:0/44716884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 msgr2=0x561d3a3cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.149+0000 7f672f7fe640 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.149+0000 7f672f7fe640 1 -- 172.21.15.78:0/44716884 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d39fb8190 con 0x561d39fd04d0 2024-05-21T15:18:57.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.154+0000 7f672f7fe640 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd04d0 0x561d3a39a670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6724005eb0 tx=0x7f672402a840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace29589ec089d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.155+0000 7f67157fa640 1 -- 172.21.15.78:0/44716884 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724045070 con 0x561d39fd04d0 2024-05-21T15:18:57.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.155+0000 7f67157fa640 1 -- 172.21.15.78:0/44716884 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724005850 con 0x561d39fd04d0 2024-05-21T15:18:57.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.157+0000 7f67157fa640 1 -- 172.21.15.78:0/44716884 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724037040 con 0x561d39fd04d0 2024-05-21T15:18:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.158+0000 7f673ca19740 1 -- 172.21.15.78:0/44716884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd04d0 msgr2=0x561d3a39a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.158+0000 7f673ca19740 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd04d0 0x561d3a39a670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6724005eb0 tx=0x7f672402a840 comp rx=0 tx=0).stop 2024-05-21T15:18:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.158+0000 7f673ca19740 1 -- 172.21.15.78:0/44716884 shutdown_connections 2024-05-21T15:18:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.158+0000 7f673ca19740 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39fd86b0 0x561d3a3cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.158+0000 7f673ca19740 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.158+0000 7f673ca19740 1 --2- 172.21.15.78:0/44716884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd04d0 0x561d3a39a670 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6724005eb0 tx=0x7f672402a840 comp rx=0 tx=0).stop 2024-05-21T15:18:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.158+0000 7f673ca19740 1 -- 172.21.15.78:0/44716884 >> 172.21.15.78:0/44716884 conn(0x561d39fed4e0 msgr2=0x561d3a396f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.165+0000 7f673ca19740 1 -- 172.21.15.78:0/44716884 shutdown_connections 2024-05-21T15:18:57.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.165+0000 7f673ca19740 1 -- 172.21.15.78:0/44716884 wait complete. 2024-05-21T15:18:57.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.166+0000 7f673ca19740 1 -- start start 2024-05-21T15:18:57.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.166+0000 7f673ca19740 1 Processor -- start 2024-05-21T15:18:57.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f673ca19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 0x561d3a3d7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f673ca19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f673ca19740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39f8d3a0 0x561d3a3d4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f673ca19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67301adc80 con 0x561d39ff9390 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f673ca19740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ad580 con 0x561d39f8d3a0 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f673ca19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67301aa840 con 0x561d39fd86b0 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f673ca19740 10 client.0 _ll_register_callbacks cb 0x7f6724035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f672f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 0x561d3a3d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f672f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 0x561d3a3d7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33380/0 (socket says 172.21.15.78:33380) 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f672f7fe640 1 -- 172.21.15.78:0/1641431952 learned_addr learned my addr 172.21.15.78:0/1641431952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f672effd640 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.167+0000 7f672ffff640 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39f8d3a0 0x561d3a3d4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.168+0000 7f672effd640 1 -- 172.21.15.78:0/1641431952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 msgr2=0x561d3a3d7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.168+0000 7f672effd640 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 0x561d3a3d7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.168+0000 7f672effd640 1 -- 172.21.15.78:0/1641431952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39f8d3a0 msgr2=0x561d3a3d4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.168+0000 7f672effd640 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39f8d3a0 0x561d3a3d4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.168+0000 7f672effd640 1 -- 172.21.15.78:0/1641431952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d3a3d5610 con 0x561d39ff9390 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.168+0000 7f672effd640 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3d4a90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f67200cc770 tx=0x7f67200ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648a4f7556a58631 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.168+0000 7f670effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.169+0000 7f670effd640 1 -- 172.21.15.78:0/1641431952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200b3020 con 0x561d39ff9390 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.169+0000 7f673ca19740 1 -- 172.21.15.78:0/1641431952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6724005850 con 0x561d39ff9390 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.169+0000 7f670effd640 1 -- 172.21.15.78:0/1641431952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f672001aaa0 con 0x561d39ff9390 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.169+0000 7f670effd640 1 -- 172.21.15.78:0/1641431952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f67200b9050 con 0x561d39ff9390 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.170+0000 7f673ca19740 10 client.16506 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:57.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.170+0000 7f673ca19740 1 -- 172.21.15.78:0/1641431952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561d3a42e050 con 0x561d39ff9390 2024-05-21T15:18:57.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.170+0000 7f670effd640 1 -- 172.21.15.78:0/1641431952 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724005850 con 0x561d39ff9390 2024-05-21T15:18:57.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.172+0000 7f670effd640 1 -- 172.21.15.78:0/1641431952 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f67200b4330 con 0x561d39ff9390 2024-05-21T15:18:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.172+0000 7f673ca19740 10 client.16506 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.172+0000 7f673ca19740 10 client.16506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:18:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.172+0000 7f673ca19740 20 client.16506 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.172+0000 7f673ca19740 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x561d3a42e7a0 0x561d3a430bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.172+0000 7f673ca19740 4 client.16506 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.172+0000 7f673ca19740 1 -- 172.21.15.78:0/1641431952 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d3a3056c0 con 0x561d3a42e7a0 2024-05-21T15:18:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.172+0000 7f672f7fe640 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x561d3a42e7a0 0x561d3a430bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.173+0000 7f670effd640 10 client.16506 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:18:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.173+0000 7f672f7fe640 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x561d3a42e7a0 0x561d3a430bc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6b9c4ade15bb383 server_cookie=953d598a088fe484 in_seq=0 out_seq=0 2024-05-21T15:18:57.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.174+0000 7f670effd640 1 -- 172.21.15.78:0/1641431952 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d3a3056c0 con 0x561d3a42e7a0 2024-05-21T15:18:57.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.174+0000 7f670effd640 10 client.16506 handle_command_reply: tid=0 2024-05-21T15:18:57.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.174+0000 7f673ca19740 1 client.16506 shutdown 2024-05-21T15:18:57.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.174+0000 7f673ca19740 1 -- 172.21.15.78:0/1641431952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 msgr2=0x561d3a3d4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.174+0000 7f673ca19740 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3d4a90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f67200cc770 tx=0x7f67200ce040 comp rx=0 tx=0).stop 2024-05-21T15:18:57.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.175+0000 7f673ca19740 1 -- 172.21.15.78:0/1641431952 shutdown_connections 2024-05-21T15:18:57.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.175+0000 7f673ca19740 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x561d3a42e7a0 0x561d3a430bc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.175+0000 7f673ca19740 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561d39f8d3a0 0x561d3a3d4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.175+0000 7f673ca19740 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d39ff9390 0x561d3a3d4a90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.175+0000 7f673ca19740 1 --2- 172.21.15.78:0/1641431952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d39fd86b0 0x561d3a3d7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.175+0000 7f673ca19740 1 -- 172.21.15.78:0/1641431952 >> 172.21.15.78:0/1641431952 conn(0x561d39fed4e0 msgr2=0x561d3a39a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.175+0000 7f670effd640 0 client.16506 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:18:57.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.177+0000 7f673ca19740 1 -- 172.21.15.78:0/1641431952 shutdown_connections 2024-05-21T15:18:57.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.177+0000 7f673ca19740 1 -- 172.21.15.78:0/1641431952 wait complete. 2024-05-21T15:18:57.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.177+0000 7f673ca19740 20 client.16506 trim_cache size 0 max 16384 2024-05-21T15:18:57.182 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:57.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 -- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67180a2e40 msgr2=0x7f67180a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67180a2e40 0x7f67180a5350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6724009830 tx=0x7f672403a000 comp rx=0 tx=0).stop 2024-05-21T15:18:57.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 -- 172.21.15.78:0/644019237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6730105940 msgr2=0x7f6730197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6730105940 0x7f6730197a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6720009ce0 tx=0x7f672000c6a0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 -- 172.21.15.78:0/644019237 shutdown_connections 2024-05-21T15:18:57.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6730105940 0x7f6730197a00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67180a2e40 0x7f67180a5350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6730106400 0x7f673019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 --2- 172.21.15.78:0/644019237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67301013b0 0x7f67301974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.180+0000 7f6735b79640 1 -- 172.21.15.78:0/644019237 >> 172.21.15.78:0/644019237 conn(0x7f6730076830 msgr2=0x7f67300fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.181+0000 7f6735b79640 1 -- 172.21.15.78:0/644019237 shutdown_connections 2024-05-21T15:18:57.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.181+0000 7f6735b79640 1 -- 172.21.15.78:0/644019237 wait complete. 2024-05-21T15:18:57.208 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:18:57.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.408+0000 7f239ffb5640 1 -- 172.21.15.78:0/744292541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398113950 msgr2=0x7f2398115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.408+0000 7f239ffb5640 1 --2- 172.21.15.78:0/744292541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398113950 0x7f2398115d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2388009900 tx=0x7f238802a420 comp rx=0 tx=0).stop 2024-05-21T15:18:57.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.408+0000 7f239ffb5640 1 -- 172.21.15.78:0/744292541 shutdown_connections 2024-05-21T15:18:57.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.408+0000 7f239ffb5640 1 --2- 172.21.15.78:0/744292541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398113950 0x7f2398115d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.408+0000 7f239ffb5640 1 --2- 172.21.15.78:0/744292541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2398110ff0 0x7f2398113410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.408+0000 7f239ffb5640 1 --2- 172.21.15.78:0/744292541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23980764a0 0x7f23980768b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.408+0000 7f239ffb5640 1 -- 172.21.15.78:0/744292541 >> 172.21.15.78:0/744292541 conn(0x7f23981003a0 msgr2=0x7f23981027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.409+0000 7f239ffb5640 1 -- 172.21.15.78:0/744292541 shutdown_connections 2024-05-21T15:18:57.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.409+0000 7f239ffb5640 1 -- 172.21.15.78:0/744292541 wait complete. 2024-05-21T15:18:57.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.410+0000 7f239ffb5640 1 Processor -- start 2024-05-21T15:18:57.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.410+0000 7f239ffb5640 1 -- start start 2024-05-21T15:18:57.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239ffb5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980764a0 0x7f23980736b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239ffb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398110ff0 0x7f23980707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239ffb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398113950 0x7f2398070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239ffb5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2398119410 con 0x7f23980764a0 2024-05-21T15:18:57.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239ffb5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2398119290 con 0x7f2398110ff0 2024-05-21T15:18:57.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239ffb5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2398119590 con 0x7f2398113950 2024-05-21T15:18:57.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239d550640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398110ff0 0x7f23980707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239d550640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398110ff0 0x7f23980707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45190/0 (socket says 172.21.15.78:45190) 2024-05-21T15:18:57.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.411+0000 7f239d550640 1 -- 172.21.15.78:0/3666670714 learned_addr learned my addr 172.21.15.78:0/3666670714 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.412+0000 7f239d550640 1 -- 172.21.15.78:0/3666670714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398113950 msgr2=0x7f2398070d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:18:57.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.412+0000 7f239d550640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398113950 0x7f2398070d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.412+0000 7f239d550640 1 -- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980764a0 msgr2=0x7f23980736b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:18:57.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.412+0000 7f239d550640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980764a0 0x7f23980736b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.412+0000 7f239d550640 1 -- 172.21.15.78:0/3666670714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2398071500 con 0x7f2398110ff0 2024-05-21T15:18:57.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.412+0000 7f239d550640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398110ff0 0x7f23980707f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f239400e9c0 tx=0x7f239400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.412+0000 7f238effd640 1 -- 172.21.15.78:0/3666670714 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239400cd80 con 0x7f2398110ff0 2024-05-21T15:18:57.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.413+0000 7f238effd640 1 -- 172.21.15.78:0/3666670714 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2394005500 con 0x7f2398110ff0 2024-05-21T15:18:57.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.413+0000 7f238effd640 1 -- 172.21.15.78:0/3666670714 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394010690 con 0x7f2398110ff0 2024-05-21T15:18:57.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.413+0000 7f239ffb5640 1 -- 172.21.15.78:0/3666670714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239810d810 con 0x7f2398110ff0 2024-05-21T15:18:57.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.413+0000 7f239ffb5640 1 -- 172.21.15.78:0/3666670714 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239810dd10 con 0x7f2398110ff0 2024-05-21T15:18:57.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.417+0000 7f23a6e65740 1 Processor -- start 2024-05-21T15:18:57.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.418+0000 7f23a6e65740 1 -- start start 2024-05-21T15:18:57.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.418+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x5632684766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.418+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680ad4d0 0x563268474bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.418+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680b56b0 0x563268475290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.418+0000 7f23a6e65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5632683dac40 con 0x5632680ad4d0 2024-05-21T15:18:57.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.418+0000 7f23a6e65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563268382580 con 0x5632680d6390 2024-05-21T15:18:57.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.418+0000 7f23a6e65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5632683d6c00 con 0x5632680b56b0 2024-05-21T15:18:57.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.420+0000 7f239dd51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680ad4d0 0x563268474bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.420+0000 7f239dd51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680ad4d0 0x563268474bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52562/0 (socket says 172.21.15.78:52562) 2024-05-21T15:18:57.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.420+0000 7f239dd51640 1 -- 172.21.15.78:0/769622520 learned_addr learned my addr 172.21.15.78:0/769622520 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.420+0000 7f239e552640 1 --2- 172.21.15.78:0/769622520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x5632684766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.421+0000 7f239e552640 1 -- 172.21.15.78:0/769622520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680b56b0 msgr2=0x563268475290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:18:57.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.421+0000 7f239e552640 1 --2- 172.21.15.78:0/769622520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680b56b0 0x563268475290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.421+0000 7f239e552640 1 -- 172.21.15.78:0/769622520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680ad4d0 msgr2=0x563268474bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.421+0000 7f239e552640 1 --2- 172.21.15.78:0/769622520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680ad4d0 0x563268474bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.421+0000 7f239e552640 1 -- 172.21.15.78:0/769622520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632680ee3f0 con 0x5632680d6390 2024-05-21T15:18:57.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.422+0000 7f239e552640 1 --2- 172.21.15.78:0/769622520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x5632684766e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f238802a940 tx=0x7f2388009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d679c5084f0c095 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.422+0000 7f239ffb5640 1 -- 172.21.15.78:0/769622520 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388040070 con 0x5632680d6390 2024-05-21T15:18:57.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.422+0000 7f239ffb5640 1 -- 172.21.15.78:0/769622520 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2388004040 con 0x5632680d6390 2024-05-21T15:18:57.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.422+0000 7f239ffb5640 1 -- 172.21.15.78:0/769622520 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388040070 con 0x5632680d6390 2024-05-21T15:18:57.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.423+0000 7f23a6e65740 1 -- 172.21.15.78:0/769622520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 msgr2=0x5632684766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.423+0000 7f23a6e65740 1 --2- 172.21.15.78:0/769622520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x5632684766e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f238802a940 tx=0x7f2388009460 comp rx=0 tx=0).stop 2024-05-21T15:18:57.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.424+0000 7f23a6e65740 1 -- 172.21.15.78:0/769622520 shutdown_connections 2024-05-21T15:18:57.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.424+0000 7f23a6e65740 1 --2- 172.21.15.78:0/769622520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680b56b0 0x563268475290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.424+0000 7f23a6e65740 1 --2- 172.21.15.78:0/769622520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680ad4d0 0x563268474bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.424+0000 7f23a6e65740 1 --2- 172.21.15.78:0/769622520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x5632684766e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.424+0000 7f23a6e65740 1 -- 172.21.15.78:0/769622520 >> 172.21.15.78:0/769622520 conn(0x5632680ca4e0 msgr2=0x5632684717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.424+0000 7f238effd640 1 -- 172.21.15.78:0/3666670714 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f23940046c0 con 0x7f2398110ff0 2024-05-21T15:18:57.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.425+0000 7f238effd640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f23840a2d80 0x7f23840a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.426+0000 7f238effd640 1 -- 172.21.15.78:0/3666670714 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2394099e10 con 0x7f2398110ff0 2024-05-21T15:18:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.426+0000 7f23a6e65740 1 -- 172.21.15.78:0/769622520 shutdown_connections 2024-05-21T15:18:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.426+0000 7f239dd51640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f23840a2d80 0x7f23840a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.426+0000 7f23a6e65740 1 -- 172.21.15.78:0/769622520 wait complete. 2024-05-21T15:18:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.427+0000 7f23a6e65740 1 -- start start 2024-05-21T15:18:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.427+0000 7f23a6e65740 1 Processor -- start 2024-05-21T15:18:57.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x56326847f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x56326847fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 0x56326848b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f23a6e65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563268482080 con 0x563268094f60 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f23a6e65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563268395c40 con 0x5632680d6390 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f23a6e65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563268482200 con 0x5632680ad4d0 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f23a6e65740 10 client.0 _ll_register_callbacks cb 0x5632680edbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f239dd51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x56326847f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f239dd51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x56326847f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33400/0 (socket says 172.21.15.78:33400) 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.428+0000 7f239dd51640 1 -- 172.21.15.78:0/1320012877 learned_addr learned my addr 172.21.15.78:0/1320012877 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.429+0000 7f239dd51640 1 -- 172.21.15.78:0/1320012877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 msgr2=0x56326847fb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.429+0000 7f239dd51640 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x56326847fb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.429+0000 7f239dd51640 1 -- 172.21.15.78:0/1320012877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 msgr2=0x56326848b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.429+0000 7f239dd51640 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 0x56326848b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.429+0000 7f239dd51640 1 -- 172.21.15.78:0/1320012877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632681b87d0 con 0x5632680ad4d0 2024-05-21T15:18:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.430+0000 7f239dd51640 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x56326847f5e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2390013d80 tx=0x7f239000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efb8f0b2eea4f4c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.430+0000 7f239dd51640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f23840a2d80 0x7f23840a5290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2390005fd0 tx=0x7f239001e000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.430+0000 7f23817fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:18:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.430+0000 7f23817fa640 1 -- 172.21.15.78:0/1320012877 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390004cb0 con 0x5632680ad4d0 2024-05-21T15:18:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.430+0000 7f23a6e65740 1 -- 172.21.15.78:0/1320012877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2388004040 con 0x5632680ad4d0 2024-05-21T15:18:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.430+0000 7f23817fa640 1 -- 172.21.15.78:0/1320012877 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2390005050 con 0x5632680ad4d0 2024-05-21T15:18:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.430+0000 7f23817fa640 1 -- 172.21.15.78:0/1320012877 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f239002e050 con 0x5632680ad4d0 2024-05-21T15:18:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23a6e65740 10 client.25969 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23a6e65740 1 -- 172.21.15.78:0/1320012877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5632684b07b0 con 0x5632680ad4d0 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23817fa640 1 -- 172.21.15.78:0/1320012877 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2390023070 con 0x5632680ad4d0 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23817fa640 1 -- 172.21.15.78:0/1320012877 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f239001e7c0 con 0x5632680ad4d0 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23a6e65740 10 client.25969 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23a6e65740 10 client.25969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23a6e65740 20 client.25969 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23a6e65740 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56326806a3a0 0x5632684b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23a6e65740 4 client.25969 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.431+0000 7f23a6e65740 1 -- 172.21.15.78:0/1320012877 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2388004040 con 0x56326806a3a0 2024-05-21T15:18:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.433+0000 7f239d550640 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56326806a3a0 0x5632684b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.436+0000 7f239d550640 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56326806a3a0 0x5632684b4210 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d91d1d36a87aaaf server_cookie=c7b06a08ff45a59 in_seq=0 out_seq=0 2024-05-21T15:18:57.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.436+0000 7f23817fa640 10 client.25969 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:57.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.437+0000 7f23817fa640 1 -- 172.21.15.78:0/1320012877 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2388004040 con 0x56326806a3a0 2024-05-21T15:18:57.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.437+0000 7f23817fa640 10 client.25969 handle_command_reply: tid=0 2024-05-21T15:18:57.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.437+0000 7f23a6e65740 1 client.25969 shutdown 2024-05-21T15:18:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23a6e65740 1 -- 172.21.15.78:0/1320012877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 msgr2=0x56326847f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23a6e65740 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x56326847f5e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2390013d80 tx=0x7f239000f730 comp rx=0 tx=0).stop 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23a6e65740 1 -- 172.21.15.78:0/1320012877 shutdown_connections 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23a6e65740 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56326806a3a0 0x5632684b4210 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23a6e65740 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x56326847fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23a6e65740 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 0x56326848b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23a6e65740 1 --2- 172.21.15.78:0/1320012877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x56326847f5e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23a6e65740 1 -- 172.21.15.78:0/1320012877 >> 172.21.15.78:0/1320012877 conn(0x5632680ca4e0 msgr2=0x5632684707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.439+0000 7f23817fa640 0 client.25969 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.440+0000 7f23a6e65740 1 -- 172.21.15.78:0/1320012877 shutdown_connections 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.440+0000 7f23a6e65740 1 -- 172.21.15.78:0/1320012877 wait complete. 2024-05-21T15:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.440+0000 7f23a6e65740 20 client.25969 trim_cache size 0 max 16384 2024-05-21T15:18:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.468+0000 7f23a6e65740 1 Processor -- start 2024-05-21T15:18:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.468+0000 7f23a6e65740 1 -- start start 2024-05-21T15:18:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.468+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563268094f60 0x563268481520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.468+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268479090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.468+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680d6390 0x56326848b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.469+0000 7f23a6e65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23981b5290 con 0x5632680d6390 2024-05-21T15:18:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.469+0000 7f23a6e65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f23981b6ef0 con 0x563268094f60 2024-05-21T15:18:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.469+0000 7f23a6e65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23981b5960 con 0x5632680ad4d0 2024-05-21T15:18:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.469+0000 7f239d550640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268479090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.469+0000 7f239d550640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268479090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33408/0 (socket says 172.21.15.78:33408) 2024-05-21T15:18:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.469+0000 7f239d550640 1 -- 172.21.15.78:0/3636861991 learned_addr learned my addr 172.21.15.78:0/3636861991 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.469+0000 7f239e552640 1 --2- 172.21.15.78:0/3636861991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680d6390 0x56326848b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.470+0000 7f239e552640 1 -- 172.21.15.78:0/3636861991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 msgr2=0x563268479090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.470+0000 7f239e552640 1 --2- 172.21.15.78:0/3636861991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268479090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.470+0000 7f239e552640 1 -- 172.21.15.78:0/3636861991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563268094f60 msgr2=0x563268481520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:18:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.470+0000 7f239e552640 1 --2- 172.21.15.78:0/3636861991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563268094f60 0x563268481520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.470+0000 7f239e552640 1 -- 172.21.15.78:0/3636861991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632684b47e0 con 0x5632680d6390 2024-05-21T15:18:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.471+0000 7f239e552640 1 --2- 172.21.15.78:0/3636861991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680d6390 0x56326848b120 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2388005ab0 tx=0x7f238802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfb4968f7365feb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.471+0000 7f23837fe640 1 -- 172.21.15.78:0/3636861991 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388040070 con 0x5632680d6390 2024-05-21T15:18:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.471+0000 7f23837fe640 1 -- 172.21.15.78:0/3636861991 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2388004050 con 0x5632680d6390 2024-05-21T15:18:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.471+0000 7f23837fe640 1 -- 172.21.15.78:0/3636861991 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388035020 con 0x5632680d6390 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 -- 172.21.15.78:0/3636861991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680d6390 msgr2=0x56326848b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 --2- 172.21.15.78:0/3636861991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680d6390 0x56326848b120 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2388005ab0 tx=0x7f238802ab90 comp rx=0 tx=0).stop 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 -- 172.21.15.78:0/3636861991 shutdown_connections 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 --2- 172.21.15.78:0/3636861991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632680d6390 0x56326848b120 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 --2- 172.21.15.78:0/3636861991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268479090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 --2- 172.21.15.78:0/3636861991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563268094f60 0x563268481520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 -- 172.21.15.78:0/3636861991 >> 172.21.15.78:0/3636861991 conn(0x5632680ca4e0 msgr2=0x56326847e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 -- 172.21.15.78:0/3636861991 shutdown_connections 2024-05-21T15:18:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.473+0000 7f23a6e65740 1 -- 172.21.15.78:0/3636861991 wait complete. 2024-05-21T15:18:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.474+0000 7f23a6e65740 1 -- start start 2024-05-21T15:18:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.474+0000 7f23a6e65740 1 Processor -- start 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.475+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 0x5632684878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.475+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268482c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f23a6e65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x563268483170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f23a6e65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23981b6760 con 0x563268094f60 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f239d550640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268482c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f239d550640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268482c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33410/0 (socket says 172.21.15.78:33410) 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f239d550640 1 -- 172.21.15.78:0/2527086802 learned_addr learned my addr 172.21.15.78:0/2527086802 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f239e552640 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x563268483170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f23981b6060 con 0x5632680d6390 2024-05-21T15:18:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23981b3270 con 0x5632680ad4d0 2024-05-21T15:18:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.476+0000 7f23a6e65740 10 client.0 _ll_register_callbacks cb 0x7f2390009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f239e552640 1 -- 172.21.15.78:0/2527086802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 msgr2=0x563268482c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f239e552640 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268482c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f239dd51640 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 0x5632684878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f239e552640 1 -- 172.21.15.78:0/2527086802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 msgr2=0x5632684878e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f239e552640 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 0x5632684878e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f239e552640 1 -- 172.21.15.78:0/2527086802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563268483760 con 0x5632680d6390 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f239dd51640 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 0x5632684878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f239e552640 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x563268483170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f238802a940 tx=0x7f2388009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6287e54c059fd948 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f23817fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.477+0000 7f23817fa640 1 -- 172.21.15.78:0/2527086802 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388035070 con 0x5632680d6390 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.478+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2388004050 con 0x5632680d6390 2024-05-21T15:18:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.478+0000 7f23817fa640 1 -- 172.21.15.78:0/2527086802 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2388031c60 con 0x5632680d6390 2024-05-21T15:18:57.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.479+0000 7f23817fa640 1 -- 172.21.15.78:0/2527086802 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f238805f050 con 0x5632680d6390 2024-05-21T15:18:57.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.480+0000 7f23a6e65740 10 client.25967 fetch_fsmap learned FSMap version 30 2024-05-21T15:18:57.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.480+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56326851c340 con 0x5632680d6390 2024-05-21T15:18:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.480+0000 7f23817fa640 1 -- 172.21.15.78:0/2527086802 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2388004050 con 0x5632680d6390 2024-05-21T15:18:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.480+0000 7f23817fa640 1 -- 172.21.15.78:0/2527086802 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f238803a040 con 0x5632680d6390 2024-05-21T15:18:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.481+0000 7f23a6e65740 10 client.25967 fetch_fsmap finished waiting for FSMap version 30 2024-05-21T15:18:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.481+0000 7f23a6e65740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.481+0000 7f23a6e65740 20 client.25967 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.481+0000 7f23a6e65740 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56326851c610 0x56326851ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.481+0000 7f23a6e65740 4 client.25967 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:18:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.481+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632683e8370 con 0x56326851c610 2024-05-21T15:18:57.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.481+0000 7f239dd51640 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56326851c610 0x56326851ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:57.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.483+0000 7f239dd51640 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56326851c610 0x56326851ea30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c505fd87847afa41 server_cookie=17ada943b5d11399 in_seq=0 out_seq=0 2024-05-21T15:18:57.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.483+0000 7f23817fa640 10 client.25967 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:57.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.483+0000 7f23817fa640 1 -- 172.21.15.78:0/2527086802 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632683e8370 con 0x56326851c610 2024-05-21T15:18:57.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.483+0000 7f23817fa640 10 client.25967 handle_command_reply: tid=0 2024-05-21T15:18:57.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.483+0000 7f23a6e65740 1 client.25967 shutdown 2024-05-21T15:18:57.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.484+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 msgr2=0x563268483170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.484+0000 7f23a6e65740 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x563268483170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f238802a940 tx=0x7f2388009460 comp rx=0 tx=0).stop 2024-05-21T15:18:57.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.485+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 shutdown_connections 2024-05-21T15:18:57.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.485+0000 7f23a6e65740 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56326851c610 0x56326851ea30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.485+0000 7f23a6e65740 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632680d6390 0x563268483170 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.485+0000 7f23a6e65740 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632680ad4d0 0x563268482c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.485+0000 7f23a6e65740 1 --2- 172.21.15.78:0/2527086802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563268094f60 0x5632684878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.485+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 >> 172.21.15.78:0/2527086802 conn(0x5632680ca4e0 msgr2=0x56326846fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.486+0000 7f23817fa640 0 client.25967 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:57.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.492+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 shutdown_connections 2024-05-21T15:18:57.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.492+0000 7f23a6e65740 1 -- 172.21.15.78:0/2527086802 wait complete. 2024-05-21T15:18:57.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.496+0000 7f23a6e65740 20 client.25967 trim_cache size 0 max 16384 2024-05-21T15:18:57.499 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:18:57.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.497+0000 7f238cff9640 1 -- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f23840a2d80 msgr2=0x7f23840a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.497+0000 7f238cff9640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f23840a2d80 0x7f23840a5290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2390005fd0 tx=0x7f239001e000 comp rx=0 tx=0).stop 2024-05-21T15:18:57.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.497+0000 7f238cff9640 1 -- 172.21.15.78:0/3666670714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398110ff0 msgr2=0x7f23980707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:57.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.497+0000 7f238cff9640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398110ff0 0x7f23980707f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f239400e9c0 tx=0x7f239400ee80 comp rx=0 tx=0).stop 2024-05-21T15:18:57.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.499+0000 7f238cff9640 1 -- 172.21.15.78:0/3666670714 shutdown_connections 2024-05-21T15:18:57.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.499+0000 7f238cff9640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2398110ff0 0x7f23980707f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.499+0000 7f238cff9640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f23840a2d80 0x7f23840a5290 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.499+0000 7f238cff9640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2398113950 0x7f2398070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.499+0000 7f238cff9640 1 --2- 172.21.15.78:0/3666670714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23980764a0 0x7f23980736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:57.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.499+0000 7f238cff9640 1 -- 172.21.15.78:0/3666670714 >> 172.21.15.78:0/3666670714 conn(0x7f23981003a0 msgr2=0x7f2398111a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:57.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.499+0000 7f238cff9640 1 -- 172.21.15.78:0/3666670714 shutdown_connections 2024-05-21T15:18:57.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:57.499+0000 7f238cff9640 1 -- 172.21.15.78:0/3666670714 wait complete. 2024-05-21T15:18:58.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:57 smithi078 ceph-mon[31997]: pgmap v89: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 94 KiB/s rd, 113 MiB/s wr, 16.89k op/s 2024-05-21T15:18:58.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:57 smithi084 ceph-mon[36334]: pgmap v89: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 94 KiB/s rd, 113 MiB/s wr, 16.89k op/s 2024-05-21T15:18:58.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:57 smithi172 ceph-mon[36402]: pgmap v89: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 94 KiB/s rd, 113 MiB/s wr, 16.89k op/s 2024-05-21T15:18:58.522 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:18:58.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:18:58.523 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:18:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.754+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1798129152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98113af0 msgr2=0x7f9e98115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.754+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1798129152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98113af0 0x7f9e98115f10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009900 tx=0x7f9e8802a4d0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.755+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1798129152 shutdown_connections 2024-05-21T15:18:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.755+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1798129152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e98111190 0x7f9e981135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.755+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1798129152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98113af0 0x7f9e98115f10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.755+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1798129152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98074310 0x7f9e98074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.755+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1798129152 >> 172.21.15.78:0/1798129152 conn(0x7f9e98078230 msgr2=0x7f9e98078670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:58.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.760+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1798129152 shutdown_connections 2024-05-21T15:18:58.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.762+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1798129152 wait complete. 2024-05-21T15:18:58.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.764+0000 7f9e9e80d640 1 Processor -- start 2024-05-21T15:18:58.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.764+0000 7f9e9e80d640 1 -- start start 2024-05-21T15:18:58.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e9e80d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98074310 0x7f9e9810ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e9e80d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e98111190 0x7f9e98107d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e9e80d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98113af0 0x7f9e98108410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e9e80d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98119500 con 0x7f9e98074310 2024-05-21T15:18:58.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e9e80d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98119380 con 0x7f9e98111190 2024-05-21T15:18:58.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e9e80d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98119680 con 0x7f9e98113af0 2024-05-21T15:18:58.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98074310 0x7f9e9810ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e98111190 0x7f9e98107d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e98111190 0x7f9e98107d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44052/0 (socket says 172.21.15.78:44052) 2024-05-21T15:18:58.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.765+0000 7f9e977fe640 1 -- 172.21.15.78:0/1240497564 learned_addr learned my addr 172.21.15.78:0/1240497564 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.766+0000 7f9e97fff640 1 -- 172.21.15.78:0/1240497564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98113af0 msgr2=0x7f9e98108410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.766+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98113af0 0x7f9e98108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.766+0000 7f9e97fff640 1 -- 172.21.15.78:0/1240497564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e98111190 msgr2=0x7f9e98107d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.766+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e98111190 0x7f9e98107d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.766+0000 7f9e97fff640 1 -- 172.21.15.78:0/1240497564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9810c730 con 0x7f9e98074310 2024-05-21T15:18:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.767+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98074310 0x7f9e9810ec10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000be20 tx=0x7f9e90009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:58.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.767+0000 7f9e957fa640 1 -- 172.21.15.78:0/1240497564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90019070 con 0x7f9e98074310 2024-05-21T15:18:58.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.767+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1240497564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9810ca20 con 0x7f9e98074310 2024-05-21T15:18:58.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.768+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1240497564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9810cfc0 con 0x7f9e98074310 2024-05-21T15:18:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.768+0000 7f9e957fa640 1 -- 172.21.15.78:0/1240497564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90014920 con 0x7f9e98074310 2024-05-21T15:18:58.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.769+0000 7f9e957fa640 1 -- 172.21.15.78:0/1240497564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90023060 con 0x7f9e98074310 2024-05-21T15:18:58.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.773+0000 7f9e957fa640 1 -- 172.21.15.78:0/1240497564 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9e9001c070 con 0x7f9e98074310 2024-05-21T15:18:58.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.775+0000 7f9e957fa640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9e800a2c20 0x7f9e800a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.775+0000 7f9e977fe640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9e800a2c20 0x7f9e800a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.777+0000 7f9ea5792740 1 Processor -- start 2024-05-21T15:18:58.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.777+0000 7f9ea5792740 1 -- start start 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.777+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a8a410 0x558b22e29620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.778+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e29b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.778+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e2a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.778+0000 7f9e97fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a8a410 0x558b22e29620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.778+0000 7f9e97fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a8a410 0x558b22e29620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:45144/0 (socket says 172.21.15.78:45144) 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.778+0000 7f9e97fff640 1 -- 172.21.15.78:0/2478937579 learned_addr learned my addr 172.21.15.78:0/2478937579 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.778+0000 7f9ea5792740 1 -- 172.21.15.78:0/2478937579 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558b22d8e990 con 0x558b22a61550 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.778+0000 7f9ea5792740 1 -- 172.21.15.78:0/2478937579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b22d365b0 con 0x558b22a69730 2024-05-21T15:18:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.778+0000 7f9ea5792740 1 -- 172.21.15.78:0/2478937579 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558b22d8a950 con 0x558b22a8a410 2024-05-21T15:18:58.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.779+0000 7f9e9cdaa640 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e2a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.780+0000 7f9e977fe640 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e29b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.780+0000 7f9e957fa640 1 -- 172.21.15.78:0/1240497564 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9e90099350 con 0x7f9e98074310 2024-05-21T15:18:58.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.780+0000 7f9e97fff640 1 -- 172.21.15.78:0/2478937579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 msgr2=0x558b22e2a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:58.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.780+0000 7f9e97fff640 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e2a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.780+0000 7f9e97fff640 1 -- 172.21.15.78:0/2478937579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 msgr2=0x558b22e29b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:58.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.780+0000 7f9e97fff640 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e29b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.780+0000 7f9e97fff640 1 -- 172.21.15.78:0/2478937579 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b22b6a7e0 con 0x558b22a8a410 2024-05-21T15:18:58.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.780+0000 7f9e97fff640 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a8a410 0x558b22e29620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009940 tx=0x7f9e9009ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621c8b0a7767fbc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:58.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.781+0000 7f9e977fe640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9e800a2c20 0x7f9e800a5130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x558b22e2abe0 tx=0x7f9e8c0095b0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:58.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.788+0000 7f9e7e7fc640 1 -- 172.21.15.78:0/2478937579 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90099670 con 0x558b22a8a410 2024-05-21T15:18:58.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.788+0000 7f9e7e7fc640 1 -- 172.21.15.78:0/2478937579 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e900cc5a0 con 0x558b22a8a410 2024-05-21T15:18:58.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.788+0000 7f9e7e7fc640 1 -- 172.21.15.78:0/2478937579 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9009f360 con 0x558b22a8a410 2024-05-21T15:18:58.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.789+0000 7f9ea5792740 1 -- 172.21.15.78:0/2478937579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a8a410 msgr2=0x558b22e29620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.789+0000 7f9ea5792740 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a8a410 0x558b22e29620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009940 tx=0x7f9e9009ad20 comp rx=0 tx=0).stop 2024-05-21T15:18:58.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.790+0000 7f9ea5792740 1 -- 172.21.15.78:0/2478937579 shutdown_connections 2024-05-21T15:18:58.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.790+0000 7f9ea5792740 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e2a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.790+0000 7f9ea5792740 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.790+0000 7f9ea5792740 1 --2- 172.21.15.78:0/2478937579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a8a410 0x558b22e29620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.790+0000 7f9ea5792740 1 -- 172.21.15.78:0/2478937579 >> 172.21.15.78:0/2478937579 conn(0x558b22a7e560 msgr2=0x558b22e25dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.790+0000 7f9ea5792740 1 -- 172.21.15.78:0/2478937579 shutdown_connections 2024-05-21T15:18:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.791+0000 7f9ea5792740 1 -- 172.21.15.78:0/2478937579 wait complete. 2024-05-21T15:18:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.792+0000 7f9ea5792740 1 -- start start 2024-05-21T15:18:58.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.792+0000 7f9ea5792740 1 Processor -- start 2024-05-21T15:18:58.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.793+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e33730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.793+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e33c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a491f0 0x558b22e3fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9ea5792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558b22e36960 con 0x558b22a61550 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e33c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e977fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e33c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44080/0 (socket says 172.21.15.78:44080) 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e977fe640 1 -- 172.21.15.78:0/1749887076 learned_addr learned my addr 172.21.15.78:0/1749887076 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b22d49b40 con 0x558b22a69730 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558b22e36ae0 con 0x558b22a491f0 2024-05-21T15:18:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9ea5792740 10 client.0 _ll_register_callbacks cb 0x558b22d3e0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e9cdaa640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a491f0 0x558b22e3fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e33730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e977fe640 1 -- 172.21.15.78:0/1749887076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a491f0 msgr2=0x558b22e3fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e977fe640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a491f0 0x558b22e3fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e977fe640 1 -- 172.21.15.78:0/1749887076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 msgr2=0x558b22e33730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.794+0000 7f9e977fe640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e33730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.795+0000 7f9e977fe640 1 -- 172.21.15.78:0/1749887076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b22bcaf70 con 0x558b22a69730 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.795+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e33730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.795+0000 7f9e9cdaa640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a491f0 0x558b22e3fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.795+0000 7f9e977fe640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e33c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0176a0 tx=0x7f9e8c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8693fe0aa1c9241f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.795+0000 7f9e777fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.795+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e900cc5a0 con 0x558b22a69730 2024-05-21T15:18:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.795+0000 7f9e777fe640 1 -- 172.21.15.78:0/1749887076 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c02f050 con 0x558b22a69730 2024-05-21T15:18:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.796+0000 7f9e777fe640 1 -- 172.21.15.78:0/1749887076 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9e8c015970 con 0x558b22a69730 2024-05-21T15:18:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.796+0000 7f9e777fe640 1 -- 172.21.15.78:0/1749887076 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e8c025070 con 0x558b22a69730 2024-05-21T15:18:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.796+0000 7f9ea5792740 10 client.25976 fetch_fsmap learned FSMap version 31 2024-05-21T15:18:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.796+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({fsmap=31}) -- 0x558b22a62170 con 0x558b22a69730 2024-05-21T15:18:58.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.797+0000 7f9e777fe640 1 -- 172.21.15.78:0/1749887076 <== mon.2 v2:172.21.15.84:3300/0 4 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f9e8c020070 con 0x558b22a69730 2024-05-21T15:18:58.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.797+0000 7f9ea5792740 10 client.25976 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:18:58.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.797+0000 7f9ea5792740 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.797+0000 7f9ea5792740 20 client.25976 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.798+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558b22a1e630 0x558b22e45c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.798+0000 7f9ea5792740 4 client.25976 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.798+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e900cc5a0 con 0x558b22a1e630 2024-05-21T15:18:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.798+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558b22a1e630 0x558b22e45c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.800+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558b22a1e630 0x558b22e45c10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7aff528235360d1 server_cookie=c1ca3a3f00466a7b in_seq=0 out_seq=0 2024-05-21T15:18:58.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.800+0000 7f9e777fe640 10 client.25976 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.802+0000 7f9e777fe640 1 -- 172.21.15.78:0/1749887076 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9e900cc5a0 con 0x558b22a1e630 2024-05-21T15:18:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.802+0000 7f9e777fe640 10 client.25976 handle_command_reply: tid=0 2024-05-21T15:18:58.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.802+0000 7f9ea5792740 1 client.25976 shutdown 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 msgr2=0x558b22e33c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e33c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0176a0 tx=0x7f9e8c004770 comp rx=0 tx=0).stop 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 shutdown_connections 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558b22a1e630 0x558b22e45c10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a69730 0x558b22e33c70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9e777fe640 0 client.25976 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a491f0 0x558b22e3fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1749887076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a61550 0x558b22e33730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.803+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 >> 172.21.15.78:0/1749887076 conn(0x558b22a7e560 msgr2=0x558b22e295c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.804+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 shutdown_connections 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.804+0000 7f9ea5792740 1 -- 172.21.15.78:0/1749887076 wait complete. 2024-05-21T15:18:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.804+0000 7f9ea5792740 20 client.25976 trim_cache size 0 max 16384 2024-05-21T15:18:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.850+0000 7f9ea5792740 1 Processor -- start 2024-05-21T15:18:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.851+0000 7f9ea5792740 1 -- start start 2024-05-21T15:18:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.851+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a1e630 0x558b22e35e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.851+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e3be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.852+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a61550 0x558b22e3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.852+0000 7f9ea5792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8400e910 con 0x558b22a491f0 2024-05-21T15:18:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.852+0000 7f9ea5792740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84010600 con 0x558b22a1e630 2024-05-21T15:18:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.852+0000 7f9ea5792740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8400efe0 con 0x558b22a61550 2024-05-21T15:18:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.853+0000 7f9e97fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a1e630 0x558b22e35e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e9cdaa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a61550 0x558b22e3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e3be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e3be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47450/0 (socket says 172.21.15.78:47450) 2024-05-21T15:18:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e977fe640 1 -- 172.21.15.78:0/3434016696 learned_addr learned my addr 172.21.15.78:0/3434016696 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e977fe640 1 -- 172.21.15.78:0/3434016696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a61550 msgr2=0x558b22e3c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e977fe640 1 --2- 172.21.15.78:0/3434016696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a61550 0x558b22e3c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e977fe640 1 -- 172.21.15.78:0/3434016696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a1e630 msgr2=0x558b22e35e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e977fe640 1 --2- 172.21.15.78:0/3434016696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a1e630 0x558b22e35e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.854+0000 7f9e977fe640 1 -- 172.21.15.78:0/3434016696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b22e3c890 con 0x558b22a491f0 2024-05-21T15:18:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.855+0000 7f9e977fe640 1 --2- 172.21.15.78:0/3434016696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e3be10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c010e30 tx=0x7f9e8c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1fb7959c1e17b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.858+0000 7f9e7dffb640 1 -- 172.21.15.78:0/3434016696 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c02f040 con 0x558b22a491f0 2024-05-21T15:18:58.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.858+0000 7f9e7dffb640 1 -- 172.21.15.78:0/3434016696 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c01b070 con 0x558b22a491f0 2024-05-21T15:18:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.859+0000 7f9ea5792740 1 -- 172.21.15.78:0/3434016696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 msgr2=0x558b22e3be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.859+0000 7f9ea5792740 1 --2- 172.21.15.78:0/3434016696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e3be10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c010e30 tx=0x7f9e8c020000 comp rx=0 tx=0).stop 2024-05-21T15:18:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.863+0000 7f9ea5792740 1 -- 172.21.15.78:0/3434016696 shutdown_connections 2024-05-21T15:18:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.863+0000 7f9ea5792740 1 --2- 172.21.15.78:0/3434016696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a61550 0x558b22e3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.863+0000 7f9ea5792740 1 --2- 172.21.15.78:0/3434016696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e3be10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.863+0000 7f9ea5792740 1 --2- 172.21.15.78:0/3434016696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a1e630 0x558b22e35e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.863+0000 7f9ea5792740 1 -- 172.21.15.78:0/3434016696 >> 172.21.15.78:0/3434016696 conn(0x558b22a7e560 msgr2=0x558b22e2d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.863+0000 7f9ea5792740 1 -- 172.21.15.78:0/3434016696 shutdown_connections 2024-05-21T15:18:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.864+0000 7f9ea5792740 1 -- 172.21.15.78:0/3434016696 wait complete. 2024-05-21T15:18:58.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.865+0000 7f9ea5792740 1 -- start start 2024-05-21T15:18:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.866+0000 7f9ea5792740 1 Processor -- start 2024-05-21T15:18:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.867+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a1e630 0x558b22e37bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.867+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e38130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.867+0000 7f9ea5792740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a61550 0x558b22eacf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.867+0000 7f9ea5792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8400fe70 con 0x558b22a491f0 2024-05-21T15:18:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.867+0000 7f9ea5792740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8400f770 con 0x558b22a61550 2024-05-21T15:18:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.867+0000 7f9ea5792740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8400c8f0 con 0x558b22a1e630 2024-05-21T15:18:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.867+0000 7f9ea5792740 10 client.0 _ll_register_callbacks cb 0x7f9e8c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.867+0000 7f9e97fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a1e630 0x558b22e37bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e97fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a1e630 0x558b22e37bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:45176/0 (socket says 172.21.15.78:45176) 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e97fff640 1 -- 172.21.15.78:0/1439079563 learned_addr learned my addr 172.21.15.78:0/1439079563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e9cdaa640 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a61550 0x558b22eacf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e97fff640 1 -- 172.21.15.78:0/1439079563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a61550 msgr2=0x558b22eacf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a61550 0x558b22eacf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e97fff640 1 -- 172.21.15.78:0/1439079563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 msgr2=0x558b22e38130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e38130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e97fff640 1 -- 172.21.15.78:0/1439079563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b22e38a10 con 0x558b22a1e630 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.868+0000 7f9e97fff640 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a1e630 0x558b22e37bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e900a4030 tx=0x7f9e90097680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd82c39571e57d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.869+0000 7f9e777fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.869+0000 7f9e777fe640 1 -- 172.21.15.78:0/1439079563 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900cc7a0 con 0x558b22a1e630 2024-05-21T15:18:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.869+0000 7f9ea5792740 1 -- 172.21.15.78:0/1439079563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e8c01b070 con 0x558b22a1e630 2024-05-21T15:18:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.869+0000 7f9e777fe640 1 -- 172.21.15.78:0/1439079563 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9e900bb080 con 0x558b22a1e630 2024-05-21T15:18:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.869+0000 7f9e777fe640 1 -- 172.21.15.78:0/1439079563 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e900bb280 con 0x558b22a1e630 2024-05-21T15:18:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.870+0000 7f9ea5792740 10 client.25981 fetch_fsmap learned FSMap version 31 2024-05-21T15:18:58.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.870+0000 7f9ea5792740 1 -- 172.21.15.78:0/1439079563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b22eae590 con 0x558b22a1e630 2024-05-21T15:18:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.874+0000 7f9e777fe640 1 -- 172.21.15.78:0/1439079563 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c01b070 con 0x558b22a1e630 2024-05-21T15:18:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.874+0000 7f9e777fe640 1 -- 172.21.15.78:0/1439079563 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f9e9009a700 con 0x558b22a1e630 2024-05-21T15:18:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.874+0000 7f9ea5792740 10 client.25981 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:18:58.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.874+0000 7f9ea5792740 10 client.25981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.875+0000 7f9ea5792740 20 client.25981 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.875+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558b22a69730 0x558b22eb09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.875+0000 7f9ea5792740 4 client.25981 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:18:58.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.875+0000 7f9ea5792740 1 -- 172.21.15.78:0/1439079563 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b22d9c0c0 con 0x558b22a69730 2024-05-21T15:18:58.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.880+0000 7f9e977fe640 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558b22a69730 0x558b22eb09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:58.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.883+0000 7f9e977fe640 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558b22a69730 0x558b22eb09d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b152d002a26a114 server_cookie=5c9ad424ff51a748 in_seq=0 out_seq=0 2024-05-21T15:18:58.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.884+0000 7f9e777fe640 10 client.25981 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.885+0000 7f9e777fe640 1 -- 172.21.15.78:0/1439079563 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b22d9c0c0 con 0x558b22a69730 2024-05-21T15:18:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.885+0000 7f9e777fe640 10 client.25981 handle_command_reply: tid=0 2024-05-21T15:18:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.885+0000 7f9ea5792740 1 client.25981 shutdown 2024-05-21T15:18:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 -- 172.21.15.78:0/1439079563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a1e630 msgr2=0x558b22e37bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a1e630 0x558b22e37bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e900a4030 tx=0x7f9e90097680 comp rx=0 tx=0).stop 2024-05-21T15:18:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 -- 172.21.15.78:0/1439079563 shutdown_connections 2024-05-21T15:18:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558b22a69730 0x558b22eb09d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b22a61550 0x558b22eacf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558b22a491f0 0x558b22e38130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 --2- 172.21.15.78:0/1439079563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b22a1e630 0x558b22e37bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 -- 172.21.15.78:0/1439079563 >> 172.21.15.78:0/1439079563 conn(0x558b22a7e560 msgr2=0x558b22e2db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9e777fe640 0 client.25981 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 -- 172.21.15.78:0/1439079563 shutdown_connections 2024-05-21T15:18:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.886+0000 7f9ea5792740 1 -- 172.21.15.78:0/1439079563 wait complete. 2024-05-21T15:18:58.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.887+0000 7f9ea5792740 20 client.25981 trim_cache size 0 max 16384 2024-05-21T15:18:58.890 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:18:58.890 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:18:58.890 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "d9787171-9806-42b7-87e6-aedbe0144e55", 2024-05-21T15:18:58.891 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:18:58.891 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:18:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9e800a2c20 msgr2=0x7f9e800a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9e800a2c20 0x7f9e800a5130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x558b22e2abe0 tx=0x7f9e8c0095b0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98074310 msgr2=0x7f9e9810ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98074310 0x7f9e9810ec10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000be20 tx=0x7f9e90009f70 comp rx=0 tx=0).stop 2024-05-21T15:18:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1240497564 shutdown_connections 2024-05-21T15:18:58.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e98111190 0x7f9e98107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9e800a2c20 0x7f9e800a5130 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e98113af0 0x7f9e98108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 --2- 172.21.15.78:0/1240497564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e98074310 0x7f9e9810ec10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.894+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1240497564 >> 172.21.15.78:0/1240497564 conn(0x7f9e98078230 msgr2=0x7f9e981115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:58.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.896+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1240497564 shutdown_connections 2024-05-21T15:18:58.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:58.896+0000 7f9e9e80d640 1 -- 172.21.15.78:0/1240497564 wait complete. 2024-05-21T15:18:58.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:18:58.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:18:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.128+0000 7f7a96ebc640 1 -- 172.21.15.78:0/2235653480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90109060 msgr2=0x7f7a90109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.128+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/2235653480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90109060 0x7f7a90109470 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009860 tx=0x7f7a8002a410 comp rx=0 tx=0).stop 2024-05-21T15:18:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.133+0000 7f7a96ebc640 1 -- 172.21.15.78:0/2235653480 shutdown_connections 2024-05-21T15:18:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.133+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/2235653480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a90119770 0x7f7a9011bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.133+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/2235653480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9011c0d0 0x7f7a9011e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.133+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/2235653480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a90109060 0x7f7a90109470 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.133+0000 7f7a96ebc640 1 -- 172.21.15.78:0/2235653480 >> 172.21.15.78:0/2235653480 conn(0x7f7a9006d750 msgr2=0x7f7a9006dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.135+0000 7f7a96ebc640 1 -- 172.21.15.78:0/2235653480 shutdown_connections 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.135+0000 7f7a96ebc640 1 -- 172.21.15.78:0/2235653480 wait complete. 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.136+0000 7f7a96ebc640 1 Processor -- start 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.136+0000 7f7a96ebc640 1 -- start start 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.137+0000 7f7a96ebc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a90109060 0x7f7a90119110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.137+0000 7f7a96ebc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a90119770 0x7f7a90112230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.137+0000 7f7a96ebc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a9011c0d0 0x7f7a90112820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.137+0000 7f7a96ebc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90121c20 con 0x7f7a9011c0d0 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.137+0000 7f7a96ebc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90121aa0 con 0x7f7a90119770 2024-05-21T15:18:59.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.137+0000 7f7a96ebc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90121da0 con 0x7f7a90109060 2024-05-21T15:18:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.138+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a90119770 0x7f7a90112230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.138+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a90119770 0x7f7a90112230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44110/0 (socket says 172.21.15.78:44110) 2024-05-21T15:18:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.138+0000 7f7a8ffff640 1 -- 172.21.15.78:0/3206993588 learned_addr learned my addr 172.21.15.78:0/3206993588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.140+0000 7f7a8ffff640 1 -- 172.21.15.78:0/3206993588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a90109060 msgr2=0x7f7a90119110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.140+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a90109060 0x7f7a90119110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.140+0000 7f7a8ffff640 1 -- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a9011c0d0 msgr2=0x7f7a90112820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.140+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a9011c0d0 0x7f7a90112820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.140+0000 7f7a8ffff640 1 -- 172.21.15.78:0/3206993588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a90113040 con 0x7f7a90119770 2024-05-21T15:18:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.140+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a90119770 0x7f7a90112230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400d950 tx=0x7f7a8400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.141+0000 7f7a8dffb640 1 -- 172.21.15.78:0/3206993588 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84017070 con 0x7f7a90119770 2024-05-21T15:18:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.141+0000 7f7a96ebc640 1 -- 172.21.15.78:0/3206993588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a90116d50 con 0x7f7a90119770 2024-05-21T15:18:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.141+0000 7f7a8dffb640 1 -- 172.21.15.78:0/3206993588 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a84009c30 con 0x7f7a90119770 2024-05-21T15:18:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.141+0000 7f7a96ebc640 1 -- 172.21.15.78:0/3206993588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a901172c0 con 0x7f7a90119770 2024-05-21T15:18:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.141+0000 7f7a8dffb640 1 -- 172.21.15.78:0/3206993588 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84020060 con 0x7f7a90119770 2024-05-21T15:18:59.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:58 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:59.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.146+0000 7f7a8dffb640 1 -- 172.21.15.78:0/3206993588 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7a840109d0 con 0x7f7a90119770 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.147+0000 7f7a8dffb640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a780a2e40 0x7f7a780a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.147+0000 7f7a8dffb640 1 -- 172.21.15.78:0/3206993588 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7a840cf4b0 con 0x7f7a90119770 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.148+0000 7f7a9de37740 1 Processor -- start 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.148+0000 7f7a9de37740 1 -- start start 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.148+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 0x564d32828d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.148+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32829270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.148+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d328297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.148+0000 7f7a9de37740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d3278ed00 con 0x564d3248a390 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.148+0000 7f7a9de37740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d327364c0 con 0x564d324614d0 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.148+0000 7f7a9de37740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d3278acc0 con 0x564d324696b0 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.149+0000 7f7a95459640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d328297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.149+0000 7f7a95459640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d328297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:45198/0 (socket says 172.21.15.78:45198) 2024-05-21T15:18:59.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.149+0000 7f7a95459640 1 -- 172.21.15.78:0/4175779024 learned_addr learned my addr 172.21.15.78:0/4175779024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:59.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.149+0000 7f7a94c58640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a780a2e40 0x7f7a780a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.149+0000 7f7a94c58640 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 0x564d32828d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.149+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32829270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.150+0000 7f7a95459640 1 -- 172.21.15.78:0/4175779024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 msgr2=0x564d32829270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.150+0000 7f7a95459640 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32829270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.150+0000 7f7a95459640 1 -- 172.21.15.78:0/4175779024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 msgr2=0x564d32828d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.150+0000 7f7a95459640 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 0x564d32828d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.150+0000 7f7a95459640 1 -- 172.21.15.78:0/4175779024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d3256c7d0 con 0x564d324696b0 2024-05-21T15:18:59.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.152+0000 7f7a95459640 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d328297b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009e20 tx=0x7f7a88030730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c18d78d257bbb9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.152+0000 7f7a76ffd640 1 -- 172.21.15.78:0/4175779024 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88004d20 con 0x564d324696b0 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.152+0000 7f7a76ffd640 1 -- 172.21.15.78:0/4175779024 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a88030e80 con 0x564d324696b0 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.153+0000 7f7a76ffd640 1 -- 172.21.15.78:0/4175779024 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8803a9d0 con 0x564d324696b0 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 -- 172.21.15.78:0/4175779024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 msgr2=0x564d328297b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d328297b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009e20 tx=0x7f7a88030730 comp rx=0 tx=0).stop 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a94c58640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a780a2e40 0x7f7a780a5350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009780 tx=0x7f7a8002a910 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 -- 172.21.15.78:0/4175779024 shutdown_connections 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32829270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d328297b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 --2- 172.21.15.78:0/4175779024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 0x564d32828d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 -- 172.21.15.78:0/4175779024 >> 172.21.15.78:0/4175779024 conn(0x564d3247e4e0 msgr2=0x564d328254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:59.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 -- 172.21.15.78:0/4175779024 shutdown_connections 2024-05-21T15:18:59.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.154+0000 7f7a9de37740 1 -- 172.21.15.78:0/4175779024 wait complete. 2024-05-21T15:18:59.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.155+0000 7f7a9de37740 1 -- start start 2024-05-21T15:18:59.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.155+0000 7f7a9de37740 1 Processor -- start 2024-05-21T15:18:59.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.156+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32834d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.156+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 0x564d328352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.156+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d328331f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.156+0000 7f7a9de37740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d328360d0 con 0x564d324696b0 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.156+0000 7f7a9de37740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d32749b90 con 0x564d324614d0 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.156+0000 7f7a9de37740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d32836250 con 0x564d3248a390 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.156+0000 7f7a9de37740 10 client.0 _ll_register_callbacks cb 0x564d32785da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.156+0000 7f7a95459640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d328331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.157+0000 7f7a95459640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d328331f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:45214/0 (socket says 172.21.15.78:45214) 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.157+0000 7f7a95459640 1 -- 172.21.15.78:0/573286719 learned_addr learned my addr 172.21.15.78:0/573286719 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:59.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.157+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 0x564d328352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.157+0000 7f7a94c58640 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32834d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.159+0000 7f7a95459640 1 -- 172.21.15.78:0/573286719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 msgr2=0x564d32834d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.159+0000 7f7a95459640 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32834d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.159+0000 7f7a95459640 1 -- 172.21.15.78:0/573286719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 msgr2=0x564d328352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.159+0000 7f7a95459640 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 0x564d328352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.159+0000 7f7a95459640 1 -- 172.21.15.78:0/573286719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d324a23f0 con 0x564d3248a390 2024-05-21T15:18:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.160+0000 7f7a95459640 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d328331f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009540 tx=0x7f7a88004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d8cdd9240349c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.160+0000 7f7a6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:18:59.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.160+0000 7f7a6ffff640 1 -- 172.21.15.78:0/573286719 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88003b80 con 0x564d3248a390 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.160+0000 7f7a9de37740 1 -- 172.21.15.78:0/573286719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564d3279c430 con 0x564d3248a390 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.160+0000 7f7a6ffff640 1 -- 172.21.15.78:0/573286719 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7a88003d20 con 0x564d3248a390 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.161+0000 7f7a6ffff640 1 -- 172.21.15.78:0/573286719 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7a8803a650 con 0x564d3248a390 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.161+0000 7f7a9de37740 10 client.25990 fetch_fsmap learned FSMap version 31 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.161+0000 7f7a9de37740 1 -- 172.21.15.78:0/573286719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564d324620f0 con 0x564d3248a390 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a6ffff640 1 -- 172.21.15.78:0/573286719 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564d3279c430 con 0x564d3248a390 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a6ffff640 1 -- 172.21.15.78:0/573286719 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f7a880044b0 con 0x564d3248a390 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a9de37740 10 client.25990 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a9de37740 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a9de37740 20 client.25990 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a9de37740 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d32448f60 0x564d32845f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a9de37740 4 client.25990 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:18:59.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a9de37740 1 -- 172.21.15.78:0/573286719 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d32796780 con 0x564d32448f60 2024-05-21T15:18:59.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.163+0000 7f7a94c58640 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d32448f60 0x564d32845f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.166+0000 7f7a94c58640 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d32448f60 0x564d32845f60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6e9b60477d1240 server_cookie=53a20a929841623b in_seq=0 out_seq=0 2024-05-21T15:18:59.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.167+0000 7f7a6ffff640 10 client.25990 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:59.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.168+0000 7f7a6ffff640 1 -- 172.21.15.78:0/573286719 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d32796780 con 0x564d32448f60 2024-05-21T15:18:59.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.168+0000 7f7a6ffff640 10 client.25990 handle_command_reply: tid=0 2024-05-21T15:18:59.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.168+0000 7f7a9de37740 1 client.25990 shutdown 2024-05-21T15:18:59.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.169+0000 7f7a9de37740 1 -- 172.21.15.78:0/573286719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 msgr2=0x564d328331f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.169+0000 7f7a9de37740 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d328331f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009540 tx=0x7f7a88004d50 comp rx=0 tx=0).stop 2024-05-21T15:18:59.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.172+0000 7f7a9de37740 1 -- 172.21.15.78:0/573286719 shutdown_connections 2024-05-21T15:18:59.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.172+0000 7f7a9de37740 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d32448f60 0x564d32845f60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.172+0000 7f7a9de37740 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d328331f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.172+0000 7f7a9de37740 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 0x564d328352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.172+0000 7f7a9de37740 1 --2- 172.21.15.78:0/573286719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32834d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.172+0000 7f7a9de37740 1 -- 172.21.15.78:0/573286719 >> 172.21.15.78:0/573286719 conn(0x564d3247e4e0 msgr2=0x564d3282aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:59.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.172+0000 7f7a6ffff640 0 client.25990 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:59.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.175+0000 7f7a9de37740 1 -- 172.21.15.78:0/573286719 shutdown_connections 2024-05-21T15:18:59.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.176+0000 7f7a9de37740 1 -- 172.21.15.78:0/573286719 wait complete. 2024-05-21T15:18:59.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.176+0000 7f7a9de37740 20 client.25990 trim_cache size 0 max 16384 2024-05-21T15:18:59.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:58 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:59.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.215+0000 7f7a9de37740 1 Processor -- start 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.216+0000 7f7a9de37740 1 -- start start 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.216+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32831ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.216+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 0x564d3284b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.216+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d3284bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.216+0000 7f7a94c58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32831ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.216+0000 7f7a94c58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32831ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44138/0 (socket says 172.21.15.78:44138) 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a94c58640 1 -- 172.21.15.78:0/1473278245 learned_addr learned my addr 172.21.15.78:0/1473278245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a9de37740 1 -- 172.21.15.78:0/1473278245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90130820 con 0x564d324696b0 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a9de37740 1 -- 172.21.15.78:0/1473278245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90132510 con 0x564d324614d0 2024-05-21T15:18:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a9de37740 1 -- 172.21.15.78:0/1473278245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90130ef0 con 0x564d3248a390 2024-05-21T15:18:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 0x564d3284b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a95459640 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d3284bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a94c58640 1 -- 172.21.15.78:0/1473278245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 msgr2=0x564d3284bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a94c58640 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d3284bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a94c58640 1 -- 172.21.15.78:0/1473278245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 msgr2=0x564d3284b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a94c58640 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 0x564d3284b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.217+0000 7f7a94c58640 1 -- 172.21.15.78:0/1473278245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d328350c0 con 0x564d324614d0 2024-05-21T15:18:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.218+0000 7f7a94c58640 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32831ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a90113a80 tx=0x7f7a80043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3214817cf8311bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.218+0000 7f7a767fc640 1 -- 172.21.15.78:0/1473278245 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80058070 con 0x564d324614d0 2024-05-21T15:18:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.219+0000 7f7a767fc640 1 -- 172.21.15.78:0/1473278245 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a800046c0 con 0x564d324614d0 2024-05-21T15:18:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a767fc640 1 -- 172.21.15.78:0/1473278245 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80053550 con 0x564d324614d0 2024-05-21T15:18:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 -- 172.21.15.78:0/1473278245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 msgr2=0x564d32831ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32831ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a90113a80 tx=0x7f7a80043000 comp rx=0 tx=0).stop 2024-05-21T15:18:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 -- 172.21.15.78:0/1473278245 shutdown_connections 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d3248a390 0x564d3284bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d324696b0 0x564d3284b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 --2- 172.21.15.78:0/1473278245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d32831ac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 -- 172.21.15.78:0/1473278245 >> 172.21.15.78:0/1473278245 conn(0x564d3247e4e0 msgr2=0x564d32845ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 -- 172.21.15.78:0/1473278245 shutdown_connections 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.220+0000 7f7a9de37740 1 -- 172.21.15.78:0/1473278245 wait complete. 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.221+0000 7f7a9de37740 1 -- start start 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.221+0000 7f7a9de37740 1 Processor -- start 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.221+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d3283b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d32837b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a9de37740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 0x564d32838040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a9de37740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90131d80 con 0x564d3248a390 2024-05-21T15:18:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a9de37740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90131680 con 0x564d324614d0 2024-05-21T15:18:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a9de37740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9010a3c0 con 0x564d324696b0 2024-05-21T15:18:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a9de37740 10 client.0 _ll_register_callbacks cb 0x7f7a880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:18:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a94c58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d3283b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d32837b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.222+0000 7f7a94c58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d3283b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44154/0 (socket says 172.21.15.78:44154) 2024-05-21T15:18:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.223+0000 7f7a94c58640 1 -- 172.21.15.78:0/2664066811 learned_addr learned my addr 172.21.15.78:0/2664066811 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:18:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.223+0000 7f7a95459640 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 0x564d32838040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.224+0000 7f7a94c58640 1 -- 172.21.15.78:0/2664066811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 msgr2=0x564d32837b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.224+0000 7f7a94c58640 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d32837b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.224+0000 7f7a94c58640 1 -- 172.21.15.78:0/2664066811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 msgr2=0x564d32838040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:18:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.224+0000 7f7a94c58640 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 0x564d32838040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.224+0000 7f7a94c58640 1 -- 172.21.15.78:0/2664066811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d32838580 con 0x564d324614d0 2024-05-21T15:18:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.224+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d32837b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:18:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.224+0000 7f7a94c58640 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d3283b7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a800052b0 tx=0x7f7a800052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f235a14b750e0ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:18:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.224+0000 7f7a6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:18:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.225+0000 7f7a6ffff640 1 -- 172.21.15.78:0/2664066811 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80058040 con 0x564d324614d0 2024-05-21T15:18:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.225+0000 7f7a9de37740 1 -- 172.21.15.78:0/2664066811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a800046c0 con 0x564d324614d0 2024-05-21T15:18:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.228+0000 7f7a6ffff640 1 -- 172.21.15.78:0/2664066811 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7a80004870 con 0x564d324614d0 2024-05-21T15:18:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.228+0000 7f7a6ffff640 1 -- 172.21.15.78:0/2664066811 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7a8006c050 con 0x564d324614d0 2024-05-21T15:18:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.229+0000 7f7a9de37740 10 client.26000 fetch_fsmap learned FSMap version 31 2024-05-21T15:18:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.229+0000 7f7a9de37740 1 -- 172.21.15.78:0/2664066811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564d328acf80 con 0x564d324614d0 2024-05-21T15:18:59.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.232+0000 7f7a6ffff640 1 -- 172.21.15.78:0/2664066811 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a800046c0 con 0x564d324614d0 2024-05-21T15:18:59.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.232+0000 7f7a6ffff640 1 -- 172.21.15.78:0/2664066811 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f7a8005b070 con 0x564d324614d0 2024-05-21T15:18:59.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.232+0000 7f7a9de37740 10 client.26000 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:18:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.232+0000 7f7a9de37740 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:18:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.232+0000 7f7a9de37740 20 client.26000 populate_metadata read hostname 'smithi078' 2024-05-21T15:18:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.232+0000 7f7a9de37740 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d3241e3a0 0x564d328af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:18:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.232+0000 7f7a9de37740 4 client.26000 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:18:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.232+0000 7f7a9de37740 1 -- 172.21.15.78:0/2664066811 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d327136f0 con 0x564d3241e3a0 2024-05-21T15:18:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.234+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d3241e3a0 0x564d328af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:18:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.237+0000 7f7a8ffff640 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d3241e3a0 0x564d328af740 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2630c098fca6a1b3 server_cookie=82cfc866a39eb2da in_seq=0 out_seq=0 2024-05-21T15:18:59.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.238+0000 7f7a6ffff640 10 client.26000 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.239+0000 7f7a6ffff640 1 -- 172.21.15.78:0/2664066811 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x564d327136f0 con 0x564d3241e3a0 2024-05-21T15:18:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.239+0000 7f7a6ffff640 10 client.26000 handle_command_reply: tid=0 2024-05-21T15:18:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.239+0000 7f7a9de37740 1 client.26000 shutdown 2024-05-21T15:18:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.239+0000 7f7a9de37740 1 -- 172.21.15.78:0/2664066811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 msgr2=0x564d3283b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.239+0000 7f7a9de37740 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d3283b7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a800052b0 tx=0x7f7a800052e0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.241+0000 7f7a9de37740 1 -- 172.21.15.78:0/2664066811 shutdown_connections 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.241+0000 7f7a9de37740 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d3241e3a0 0x564d328af740 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.242+0000 7f7a6ffff640 0 client.26000 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.242+0000 7f7a9de37740 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d3248a390 0x564d32838040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.242+0000 7f7a9de37740 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d324696b0 0x564d32837b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.242+0000 7f7a9de37740 1 --2- 172.21.15.78:0/2664066811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d324614d0 0x564d3283b7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.242+0000 7f7a9de37740 1 -- 172.21.15.78:0/2664066811 >> 172.21.15.78:0/2664066811 conn(0x564d3247e4e0 msgr2=0x564d32831a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.242+0000 7f7a9de37740 1 -- 172.21.15.78:0/2664066811 shutdown_connections 2024-05-21T15:18:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.242+0000 7f7a9de37740 1 -- 172.21.15.78:0/2664066811 wait complete. 2024-05-21T15:18:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.242+0000 7f7a9de37740 20 client.26000 trim_cache size 0 max 16384 2024-05-21T15:18:59.246 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (0 inodes in the stack)", 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: "d9787171-9806-42b7-87e6-aedbe0144e55": { 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "d9787171-9806-42b7-87e6-aedbe0144e55", 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:18:59.247 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:18:59.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.246+0000 7f7a96ebc640 1 -- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a780a2e40 msgr2=0x7f7a780a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.246+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a780a2e40 0x7f7a780a5350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009780 tx=0x7f7a8002a910 comp rx=0 tx=0).stop 2024-05-21T15:18:59.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.246+0000 7f7a96ebc640 1 -- 172.21.15.78:0/3206993588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a90119770 msgr2=0x7f7a90112230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.246+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a90119770 0x7f7a90112230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400d950 tx=0x7f7a8400de10 comp rx=0 tx=0).stop 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.247+0000 7f7a96ebc640 1 -- 172.21.15.78:0/3206993588 shutdown_connections 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.247+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a90119770 0x7f7a90112230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.247+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a780a2e40 0x7f7a780a5350 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.247+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a9011c0d0 0x7f7a90112820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.247+0000 7f7a96ebc640 1 --2- 172.21.15.78:0/3206993588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a90109060 0x7f7a90119110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.247+0000 7f7a96ebc640 1 -- 172.21.15.78:0/3206993588 >> 172.21.15.78:0/3206993588 conn(0x7f7a9006d750 msgr2=0x7f7a90119b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.247+0000 7f7a96ebc640 1 -- 172.21.15.78:0/3206993588 shutdown_connections 2024-05-21T15:18:59.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:18:59.247+0000 7f7a96ebc640 1 -- 172.21.15.78:0/3206993588 wait complete. 2024-05-21T15:18:59.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:58 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:18:59.272 INFO:tasks.cephfs.filesystem:scrub status for tag:d9787171-9806-42b7-87e6-aedbe0144e55 - {'path': '/', 'tag': 'd9787171-9806-42b7-87e6-aedbe0144e55', 'options': 'recursive,force'} 2024-05-21T15:19:00.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: pgmap v90: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 88 KiB/s rd, 69 MiB/s wr, 10.74k op/s 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:00.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:18:59 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:00.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: pgmap v90: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 88 KiB/s rd, 69 MiB/s wr, 10.74k op/s 2024-05-21T15:19:00.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:19:00.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:00.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:00.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:00.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:00.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:00.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:00.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:00.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:18:59 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:00.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: pgmap v90: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 88 KiB/s rd, 69 MiB/s wr, 10.74k op/s 2024-05-21T15:19:00.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:19:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:18:59 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:00 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:01.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:00 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:00 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:00 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:00 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:00 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:01.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:00 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:00 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:00 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:00 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:00 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:01.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:00 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:00 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:00 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:01.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:00 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:02.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:01 smithi084 ceph-mon[36334]: pgmap v91: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 198 KiB/s rd, 152 MiB/s wr, 20.58k op/s 2024-05-21T15:19:02.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:01 smithi172 ceph-mon[36402]: pgmap v91: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 198 KiB/s rd, 152 MiB/s wr, 20.58k op/s 2024-05-21T15:19:02.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:01 smithi078 ceph-mon[31997]: pgmap v91: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 198 KiB/s rd, 152 MiB/s wr, 20.58k op/s 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 -- 172.21.15.78:0/879218598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f310810a670 msgr2=0x7f310810ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 --2- 172.21.15.78:0/879218598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f310810a670 0x7f310810ca90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009900 tx=0x7f30f802a410 comp rx=0 tx=0).stop 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 -- 172.21.15.78:0/879218598 shutdown_connections 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 --2- 172.21.15.78:0/879218598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f310810d060 0x7f310810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 --2- 172.21.15.78:0/879218598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f310810a670 0x7f310810ca90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 --2- 172.21.15.78:0/879218598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3108103c10 0x7f3108104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 -- 172.21.15.78:0/879218598 >> 172.21.15.78:0/879218598 conn(0x7f31080fd210 msgr2=0x7f31080ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 -- 172.21.15.78:0/879218598 shutdown_connections 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.427+0000 7f31106fe640 1 -- 172.21.15.78:0/879218598 wait complete. 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.428+0000 7f31106fe640 1 Processor -- start 2024-05-21T15:19:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.428+0000 7f31106fe640 1 -- start start 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f31106fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3108103c10 0x7f31081997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f31106fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f310810a670 0x7f3108199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f31106fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f310810d060 0x7f310819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f31106fe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3108112e10 con 0x7f3108103c10 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f31106fe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3108112c90 con 0x7f310810a670 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f31106fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3108112f90 con 0x7f310810d060 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310dc99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f310810a670 0x7f3108199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310dc99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f310810a670 0x7f3108199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44166/0 (socket says 172.21.15.78:44166) 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310e49a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3108103c10 0x7f31081997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310dc99640 1 -- 172.21.15.78:0/3099207367 learned_addr learned my addr 172.21.15.78:0/3099207367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310ec9b640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f310810d060 0x7f310819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310dc99640 1 -- 172.21.15.78:0/3099207367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f310810d060 msgr2=0x7f310819dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310dc99640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f310810d060 0x7f310819dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310dc99640 1 -- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3108103c10 msgr2=0x7f31081997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310dc99640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3108103c10 0x7f31081997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.429+0000 7f310dc99640 1 -- 172.21.15.78:0/3099207367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310819e630 con 0x7f310810a670 2024-05-21T15:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.430+0000 7f310dc99640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f310810a670 0x7f3108199d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f30f8006010 tx=0x7f30f80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.430+0000 7f30ff7fe640 1 -- 172.21.15.78:0/3099207367 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f803f030 con 0x7f310810a670 2024-05-21T15:19:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.430+0000 7f30ff7fe640 1 -- 172.21.15.78:0/3099207367 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8003ce0 con 0x7f310810a670 2024-05-21T15:19:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.430+0000 7f31106fe640 1 -- 172.21.15.78:0/3099207367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310819e8c0 con 0x7f310810a670 2024-05-21T15:19:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.430+0000 7f31106fe640 1 -- 172.21.15.78:0/3099207367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310819ee00 con 0x7f310810a670 2024-05-21T15:19:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.431+0000 7f30ff7fe640 1 -- 172.21.15.78:0/3099207367 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8035070 con 0x7f310810a670 2024-05-21T15:19:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.433+0000 7f30ff7fe640 1 -- 172.21.15.78:0/3099207367 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f30f8042030 con 0x7f310810a670 2024-05-21T15:19:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.434+0000 7f31175a2740 1 Processor -- start 2024-05-21T15:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.434+0000 7f31175a2740 1 -- start start 2024-05-21T15:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.434+0000 7f31175a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f31175a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4e6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f30ff7fe640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30f40a2d70 0x7f30f40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4e5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:45252/0 (socket says 172.21.15.78:45252) 2024-05-21T15:19:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310e49a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310e49a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4e6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47522/0 (socket says 172.21.15.78:47522) 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 -- 172.21.15.78:0/957966924 learned_addr learned my addr 172.21.15.78:0/957966924 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f31175a2740 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f31175a2740 1 -- 172.21.15.78:0/957966924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bc9b44b340 con 0x55bc9b11e2f0 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f31175a2740 1 -- 172.21.15.78:0/957966924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bc9b3f3290 con 0x55bc9b1264d0 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f31175a2740 1 -- 172.21.15.78:0/957966924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bc9b447300 con 0x55bc9b1471b0 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f30ff7fe640 1 -- 172.21.15.78:0/3099207367 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f30f80ef8a0 con 0x7f310810a670 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 -- 172.21.15.78:0/957966924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 msgr2=0x55bc9b4e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4e69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 -- 172.21.15.78:0/957966924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 msgr2=0x55bc9b4e6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4e6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.435+0000 7f310ec9b640 1 -- 172.21.15.78:0/957966924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc9b4178e0 con 0x55bc9b1471b0 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.436+0000 7f310ec9b640 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4e5f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31000118b0 tx=0x7f3100009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba9bbc63a29faa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.436+0000 7f310dc99640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30f40a2d70 0x7f30f40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.436+0000 7f30fcff9640 1 -- 172.21.15.78:0/957966924 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310001a030 con 0x55bc9b1471b0 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.436+0000 7f30fcff9640 1 -- 172.21.15.78:0/957966924 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3100011110 con 0x55bc9b1471b0 2024-05-21T15:19:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.436+0000 7f30fcff9640 1 -- 172.21.15.78:0/957966924 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100005280 con 0x55bc9b1471b0 2024-05-21T15:19:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.437+0000 7f310dc99640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30f40a2d70 0x7f30f40a5280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005ab0 tx=0x7f30f8031a60 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.437+0000 7f31175a2740 1 -- 172.21.15.78:0/957966924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 msgr2=0x55bc9b4e5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.437+0000 7f31175a2740 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4e5f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31000118b0 tx=0x7f3100009c80 comp rx=0 tx=0).stop 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.437+0000 7f31175a2740 1 -- 172.21.15.78:0/957966924 shutdown_connections 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.437+0000 7f31175a2740 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.437+0000 7f31175a2740 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4e6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.437+0000 7f31175a2740 1 --2- 172.21.15.78:0/957966924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4e5f50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.437+0000 7f31175a2740 1 -- 172.21.15.78:0/957966924 >> 172.21.15.78:0/957966924 conn(0x55bc9b13b300 msgr2=0x55bc9b4e2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.438+0000 7f31175a2740 1 -- 172.21.15.78:0/957966924 shutdown_connections 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.438+0000 7f31175a2740 1 -- 172.21.15.78:0/957966924 wait complete. 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.438+0000 7f31175a2740 1 -- start start 2024-05-21T15:19:03.445 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.438+0000 7f31175a2740 1 Processor -- start 2024-05-21T15:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f31175a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f31175a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f31175a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f31175a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bc9b4f7670 con 0x55bc9b11e2f0 2024-05-21T15:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f31175a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bc9b406790 con 0x55bc9b1264d0 2024-05-21T15:19:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f31175a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bc9b4f77f0 con 0x55bc9b1471b0 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f31175a2740 10 client.0 _ll_register_callbacks cb 0x55bc9b441a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310e49a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310e49a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44182/0 (socket says 172.21.15.78:44182) 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310e49a640 1 -- 172.21.15.78:0/941318804 learned_addr learned my addr 172.21.15.78:0/941318804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310ec9b640 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310ec9b640 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310e49a640 1 -- 172.21.15.78:0/941318804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 msgr2=0x55bc9b4f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310e49a640 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310e49a640 1 -- 172.21.15.78:0/941318804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 msgr2=0x55bc9b4f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310e49a640 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310e49a640 1 -- 172.21.15.78:0/941318804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc9b15f210 con 0x55bc9b1264d0 2024-05-21T15:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.439+0000 7f310ec9b640 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f310e49a640 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f6b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f310400bf40 tx=0x7f31040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36cfd0042fb7af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f30f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f30f1ffb640 1 -- 172.21.15.78:0/941318804 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104004020 con 0x55bc9b1264d0 2024-05-21T15:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f30f1ffb640 1 -- 172.21.15.78:0/941318804 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f31040041c0 con 0x55bc9b1264d0 2024-05-21T15:19:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f31175a2740 1 -- 172.21.15.78:0/941318804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3100011110 con 0x55bc9b1264d0 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f30f1ffb640 1 -- 172.21.15.78:0/941318804 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3104004400 con 0x55bc9b1264d0 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f31175a2740 10 client.26012 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f31175a2740 1 -- 172.21.15.78:0/941318804 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bc9b22b620 con 0x55bc9b1264d0 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.440+0000 7f30f1ffb640 1 -- 172.21.15.78:0/941318804 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3100011110 con 0x55bc9b1264d0 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f30f1ffb640 1 -- 172.21.15.78:0/941318804 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f3104003af0 con 0x55bc9b1264d0 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f31175a2740 10 client.26012 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f31175a2740 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f31175a2740 20 client.26012 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f31175a2740 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bc9b105d80 0x55bc9b506dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f31175a2740 4 client.26012 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f31175a2740 1 -- 172.21.15.78:0/941318804 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc9b458a70 con 0x55bc9b105d80 2024-05-21T15:19:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f310ec9b640 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bc9b105d80 0x55bc9b506dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.441+0000 7f30f1ffb640 10 client.26012 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.442+0000 7f310ec9b640 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bc9b105d80 0x55bc9b506dc0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbc0cfac90383b server_cookie=e365a5d8bf437578 in_seq=0 out_seq=0 2024-05-21T15:19:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.442+0000 7f30f1ffb640 1 -- 172.21.15.78:0/941318804 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc9b458a70 con 0x55bc9b105d80 2024-05-21T15:19:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.442+0000 7f30f1ffb640 10 client.26012 handle_command_reply: tid=0 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.442+0000 7f31175a2740 1 client.26012 shutdown 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 -- 172.21.15.78:0/941318804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 msgr2=0x55bc9b4f6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f6b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f310400bf40 tx=0x7f31040118d0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 -- 172.21.15.78:0/941318804 shutdown_connections 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bc9b105d80 0x55bc9b506dc0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f30f1ffb640 0 client.26012 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f6b10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1471b0 0x55bc9b4f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 --2- 172.21.15.78:0/941318804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 -- 172.21.15.78:0/941318804 >> 172.21.15.78:0/941318804 conn(0x55bc9b13b300 msgr2=0x55bc9b4e5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 -- 172.21.15.78:0/941318804 shutdown_connections 2024-05-21T15:19:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 1 -- 172.21.15.78:0/941318804 wait complete. 2024-05-21T15:19:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.443+0000 7f31175a2740 20 client.26012 trim_cache size 0 max 16384 2024-05-21T15:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f31175a2740 1 Processor -- start 2024-05-21T15:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f31175a2740 1 -- start start 2024-05-21T15:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f31175a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b105d80 0x55bc9b4e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f31175a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f31175a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b1264d0 0x55bc9b4ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f31175a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31081aca50 con 0x55bc9b1264d0 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f31175a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f31081ae740 con 0x55bc9b11e2f0 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f31175a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f31081ad120 con 0x55bc9b105d80 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.463+0000 7f310ec9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b1264d0 0x55bc9b4ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4ee2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44198/0 (socket says 172.21.15.78:44198) 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 -- 172.21.15.78:0/776947022 learned_addr learned my addr 172.21.15.78:0/776947022 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310ec9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b105d80 0x55bc9b4e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 -- 172.21.15.78:0/776947022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b105d80 msgr2=0x55bc9b4e2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 --2- 172.21.15.78:0/776947022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b105d80 0x55bc9b4e2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 -- 172.21.15.78:0/776947022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b1264d0 msgr2=0x55bc9b4ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 --2- 172.21.15.78:0/776947022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b1264d0 0x55bc9b4ee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 -- 172.21.15.78:0/776947022 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc9b4e7410 con 0x55bc9b11e2f0 2024-05-21T15:19:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.464+0000 7f310e49a640 1 --2- 172.21.15.78:0/776947022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4ee2c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f310400c010 tx=0x7f310401a7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61843c615b20c34f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:03.471 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.465+0000 7f30f3fff640 1 -- 172.21.15.78:0/776947022 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310402e020 con 0x55bc9b11e2f0 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.465+0000 7f30f3fff640 1 -- 172.21.15.78:0/776947022 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3104003fe0 con 0x55bc9b11e2f0 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.465+0000 7f30f3fff640 1 -- 172.21.15.78:0/776947022 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040055b0 con 0x55bc9b11e2f0 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 -- 172.21.15.78:0/776947022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b11e2f0 msgr2=0x55bc9b4ee2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 --2- 172.21.15.78:0/776947022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4ee2c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f310400c010 tx=0x7f310401a7f0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 -- 172.21.15.78:0/776947022 shutdown_connections 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 --2- 172.21.15.78:0/776947022 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4ee2c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 --2- 172.21.15.78:0/776947022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b1264d0 0x55bc9b4ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 --2- 172.21.15.78:0/776947022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b105d80 0x55bc9b4e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 -- 172.21.15.78:0/776947022 >> 172.21.15.78:0/776947022 conn(0x55bc9b13b300 msgr2=0x55bc9b4e21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 -- 172.21.15.78:0/776947022 shutdown_connections 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.466+0000 7f31175a2740 1 -- 172.21.15.78:0/776947022 wait complete. 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 1 -- start start 2024-05-21T15:19:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 1 Processor -- start 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b105d80 0x55bc9b4fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31081adfb0 con 0x55bc9b11e2f0 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f31081ad8b0 con 0x55bc9b105d80 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f31081aaa30 con 0x55bc9b1264d0 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f31175a2740 10 client.0 _ll_register_callbacks cb 0x7f310400bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f310ec9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b105d80 0x55bc9b4fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f310e49a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f310ec9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b105d80 0x55bc9b4fce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:44202/0 (socket says 172.21.15.78:44202) 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f310ec9b640 1 -- 172.21.15.78:0/1186917943 learned_addr learned my addr 172.21.15.78:0/1186917943 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.467+0000 7f310ec9b640 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f310ec9b640 1 -- 172.21.15.78:0/1186917943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1264d0 msgr2=0x55bc9b4f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f310ec9b640 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f310ec9b640 1 -- 172.21.15.78:0/1186917943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 msgr2=0x55bc9b4f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f310ec9b640 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f310ec9b640 1 -- 172.21.15.78:0/1186917943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc9b4f8c00 con 0x55bc9b105d80 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f310ec9b640 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b105d80 0x55bc9b4fce00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3100011880 tx=0x7f3100022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6fa7a89111f5613 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f30f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f30f1ffb640 1 -- 172.21.15.78:0/1186917943 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100003b70 con 0x55bc9b105d80 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f31175a2740 1 -- 172.21.15.78:0/1186917943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3104003fe0 con 0x55bc9b105d80 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.468+0000 7f30f1ffb640 1 -- 172.21.15.78:0/1186917943 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3100003d10 con 0x55bc9b105d80 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f30f1ffb640 1 -- 172.21.15.78:0/1186917943 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f310002d280 con 0x55bc9b105d80 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f31175a2740 10 client.26018 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f31175a2740 1 -- 172.21.15.78:0/1186917943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bc9b4e8fe0 con 0x55bc9b105d80 2024-05-21T15:19:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f30f1ffb640 1 -- 172.21.15.78:0/1186917943 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3104003fe0 con 0x55bc9b105d80 2024-05-21T15:19:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f30f1ffb640 1 -- 172.21.15.78:0/1186917943 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f31000054b0 con 0x55bc9b105d80 2024-05-21T15:19:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f31175a2740 10 client.26018 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f31175a2740 10 client.26018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f31175a2740 20 client.26018 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f31175a2740 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bc9b1471b0 0x55bc9b4eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f31175a2740 4 client.26018 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:19:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.469+0000 7f31175a2740 1 -- 172.21.15.78:0/1186917943 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc9b452dc0 con 0x55bc9b1471b0 2024-05-21T15:19:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.470+0000 7f310e49a640 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bc9b1471b0 0x55bc9b4eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.470+0000 7f30f1ffb640 10 client.26018 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.470+0000 7f310e49a640 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bc9b1471b0 0x55bc9b4eb590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db8840f62ea45c6e server_cookie=13d43fc5d7fd173a in_seq=0 out_seq=0 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.491+0000 7f30f1ffb640 1 -- 172.21.15.78:0/1186917943 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc9b452dc0 con 0x55bc9b1471b0 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.491+0000 7f30f1ffb640 10 client.26018 handle_command_reply: tid=0 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.491+0000 7f31175a2740 1 client.26018 shutdown 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.491+0000 7f31175a2740 1 -- 172.21.15.78:0/1186917943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b105d80 msgr2=0x55bc9b4fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.491+0000 7f31175a2740 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b105d80 0x55bc9b4fce00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3100011880 tx=0x7f3100022040 comp rx=0 tx=0).stop 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 1 -- 172.21.15.78:0/1186917943 shutdown_connections 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bc9b1471b0 0x55bc9b4eb590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bc9b1264d0 0x55bc9b4f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bc9b11e2f0 0x55bc9b4f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f30f1ffb640 0 client.26018 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 1 --2- 172.21.15.78:0/1186917943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bc9b105d80 0x55bc9b4fce00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 1 -- 172.21.15.78:0/1186917943 >> 172.21.15.78:0/1186917943 conn(0x55bc9b13b300 msgr2=0x55bc9b4e86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 1 -- 172.21.15.78:0/1186917943 shutdown_connections 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 1 -- 172.21.15.78:0/1186917943 wait complete. 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31175a2740 20 client.26018 trim_cache size 0 max 16384 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:19:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:19:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31106fe640 1 -- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30f40a2d70 msgr2=0x7f30f40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31106fe640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30f40a2d70 0x7f30f40a5280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f30f8005ab0 tx=0x7f30f8031a60 comp rx=0 tx=0).stop 2024-05-21T15:19:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31106fe640 1 -- 172.21.15.78:0/3099207367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f310810a670 msgr2=0x7f3108199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.492+0000 7f31106fe640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f310810a670 0x7f3108199d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f30f8006010 tx=0x7f30f80042f0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.493+0000 7f31106fe640 1 -- 172.21.15.78:0/3099207367 shutdown_connections 2024-05-21T15:19:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.493+0000 7f31106fe640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f310810a670 0x7f3108199d30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.493+0000 7f31106fe640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30f40a2d70 0x7f30f40a5280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.493+0000 7f31106fe640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f310810d060 0x7f310819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.493+0000 7f31106fe640 1 --2- 172.21.15.78:0/3099207367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3108103c10 0x7f31081997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.493+0000 7f31106fe640 1 -- 172.21.15.78:0/3099207367 >> 172.21.15.78:0/3099207367 conn(0x7f31080fd210 msgr2=0x7f310810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.493+0000 7f31106fe640 1 -- 172.21.15.78:0/3099207367 shutdown_connections 2024-05-21T15:19:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:03.493+0000 7f31106fe640 1 -- 172.21.15.78:0/3099207367 wait complete. 2024-05-21T15:19:04.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:03 smithi084 ceph-mon[36334]: pgmap v92: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 187 KiB/s rd, 134 MiB/s wr, 17.45k op/s 2024-05-21T15:19:04.149 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:03 smithi078 ceph-mon[31997]: pgmap v92: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 187 KiB/s rd, 134 MiB/s wr, 17.45k op/s 2024-05-21T15:19:04.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:03 smithi172 ceph-mon[36402]: pgmap v92: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 187 KiB/s rd, 134 MiB/s wr, 17.45k op/s 2024-05-21T15:19:06.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:05 smithi084 ceph-mon[36334]: pgmap v93: 97 pgs: 97 active+clean; 14 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 217 KiB/s rd, 139 MiB/s wr, 18.21k op/s 2024-05-21T15:19:06.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:05 smithi172 ceph-mon[36402]: pgmap v93: 97 pgs: 97 active+clean; 14 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 217 KiB/s rd, 139 MiB/s wr, 18.21k op/s 2024-05-21T15:19:06.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:05 smithi078 ceph-mon[31997]: pgmap v93: 97 pgs: 97 active+clean; 14 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 217 KiB/s rd, 139 MiB/s wr, 18.21k op/s 2024-05-21T15:19:08.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:07 smithi084 ceph-mon[36334]: pgmap v94: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 168 KiB/s rd, 134 MiB/s wr, 17.38k op/s 2024-05-21T15:19:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:07 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:07 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:07 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:07 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:08.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:07 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:07 smithi172 ceph-mon[36402]: pgmap v94: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 168 KiB/s rd, 134 MiB/s wr, 17.38k op/s 2024-05-21T15:19:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:07 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:07 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:07 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:07 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:08.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:07 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:08.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:07 smithi078 ceph-mon[31997]: pgmap v94: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 168 KiB/s rd, 134 MiB/s wr, 17.38k op/s 2024-05-21T15:19:08.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:07 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:08.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:07 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:08.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:07 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:08.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:07 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:08.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:07 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:10.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:09 smithi172 ceph-mon[36402]: pgmap v95: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 142 KiB/s rd, 101 MiB/s wr, 12.41k op/s 2024-05-21T15:19:10.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:09 smithi078 ceph-mon[31997]: pgmap v95: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 142 KiB/s rd, 101 MiB/s wr, 12.41k op/s 2024-05-21T15:19:10.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:09 smithi084 ceph-mon[36334]: pgmap v95: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 142 KiB/s rd, 101 MiB/s wr, 12.41k op/s 2024-05-21T15:19:12.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:11 smithi078 ceph-mon[31997]: pgmap v96: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 153 KiB/s rd, 118 MiB/s wr, 15.15k op/s 2024-05-21T15:19:12.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:11 smithi084 ceph-mon[36334]: pgmap v96: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 153 KiB/s rd, 118 MiB/s wr, 15.15k op/s 2024-05-21T15:19:12.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:11 smithi172 ceph-mon[36402]: pgmap v96: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 153 KiB/s rd, 118 MiB/s wr, 15.15k op/s 2024-05-21T15:19:14.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:13 smithi172 ceph-mon[36402]: pgmap v97: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 5.30k op/s 2024-05-21T15:19:14.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:13 smithi078 ceph-mon[31997]: pgmap v97: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 5.30k op/s 2024-05-21T15:19:14.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:13 smithi084 ceph-mon[36334]: pgmap v97: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 5.30k op/s 2024-05-21T15:19:16.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:15 smithi078 ceph-mon[31997]: pgmap v98: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 51 KiB/s rd, 38 MiB/s wr, 5.79k op/s 2024-05-21T15:19:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:15 smithi084 ceph-mon[36334]: pgmap v98: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 51 KiB/s rd, 38 MiB/s wr, 5.79k op/s 2024-05-21T15:19:16.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:15 smithi172 ceph-mon[36402]: pgmap v98: 97 pgs: 97 active+clean; 15 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 51 KiB/s rd, 38 MiB/s wr, 5.79k op/s 2024-05-21T15:19:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.670+0000 7f7ebe651640 1 -- 172.21.15.78:0/3964744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 msgr2=0x7f7eb8106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.670+0000 7f7ebe651640 1 --2- 172.21.15.78:0/3964744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 0x7f7eb8106880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009ca0 tx=0x7f7eb002a8d0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.671+0000 7f7ebe651640 1 -- 172.21.15.78:0/3964744 shutdown_connections 2024-05-21T15:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.671+0000 7f7ebe651640 1 --2- 172.21.15.78:0/3964744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7eb8106ea0 0x7f7eb8107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.671+0000 7f7ebe651640 1 --2- 172.21.15.78:0/3964744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 0x7f7eb8106880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.671+0000 7f7ebe651640 1 --2- 172.21.15.78:0/3964744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eb8101a20 0x7f7eb8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.671+0000 7f7ebe651640 1 -- 172.21.15.78:0/3964744 >> 172.21.15.78:0/3964744 conn(0x7f7eb8077100 msgr2=0x7f7eb8077540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.672+0000 7f7ebe651640 1 -- 172.21.15.78:0/3964744 shutdown_connections 2024-05-21T15:19:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.673+0000 7f7ebe651640 1 -- 172.21.15.78:0/3964744 wait complete. 2024-05-21T15:19:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.674+0000 7f7ebe651640 1 Processor -- start 2024-05-21T15:19:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.678+0000 7f7ebe651640 1 -- start start 2024-05-21T15:19:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.678+0000 7f7ebe651640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7eb8101a20 0x7f7eb8073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.678+0000 7f7ebe651640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 0x7f7eb806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.678+0000 7f7ebe651640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eb8106ea0 0x7f7eb806db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.678+0000 7f7ebe651640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb806bb80 con 0x7f7eb8106ea0 2024-05-21T15:19:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.678+0000 7f7ebe651640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb806ba00 con 0x7f7eb8101a20 2024-05-21T15:19:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.678+0000 7f7ebe651640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb806bd00 con 0x7f7eb8106470 2024-05-21T15:19:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.679+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 0x7f7eb806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.679+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 0x7f7eb806d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:33776/0 (socket says 172.21.15.78:33776) 2024-05-21T15:19:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.679+0000 7f7eb77fe640 1 -- 172.21.15.78:0/2338752106 learned_addr learned my addr 172.21.15.78:0/2338752106 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.679+0000 7f7eb77fe640 1 -- 172.21.15.78:0/2338752106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7eb8101a20 msgr2=0x7f7eb8073540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:19:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.679+0000 7f7eb77fe640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7eb8101a20 0x7f7eb8073540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.679+0000 7f7eb77fe640 1 -- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eb8106ea0 msgr2=0x7f7eb806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.679+0000 7f7eb77fe640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eb8106ea0 0x7f7eb806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.679+0000 7f7eb77fe640 1 -- 172.21.15.78:0/2338752106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb806e2c0 con 0x7f7eb8106470 2024-05-21T15:19:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.680+0000 7f7eb77fe640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 0x7f7eb806d650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7eb002aec0 tx=0x7f7eb0003e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.680+0000 7f7eb57fa640 1 -- 172.21.15.78:0/2338752106 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0031b10 con 0x7f7eb8106470 2024-05-21T15:19:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.680+0000 7f7ebe651640 1 -- 172.21.15.78:0/2338752106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb81b4b90 con 0x7f7eb8106470 2024-05-21T15:19:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.680+0000 7f7ebe651640 1 -- 172.21.15.78:0/2338752106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb81b4ff0 con 0x7f7eb8106470 2024-05-21T15:19:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.682+0000 7f7eb57fa640 1 -- 172.21.15.78:0/2338752106 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb0040070 con 0x7f7eb8106470 2024-05-21T15:19:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.682+0000 7f7eb57fa640 1 -- 172.21.15.78:0/2338752106 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb002fbb0 con 0x7f7eb8106470 2024-05-21T15:19:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.685+0000 7f7ec55d0740 1 Processor -- start 2024-05-21T15:19:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.685+0000 7f7ec55d0740 1 -- start start 2024-05-21T15:19:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.686+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc981b0 0x556bbd037080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.686+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.686+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc774d0 0x556bbd037b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.686+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556bbcf9c340 con 0x556bbcc981b0 2024-05-21T15:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.686+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556bbcf44290 con 0x556bbcc6f2f0 2024-05-21T15:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.686+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556bbcf98300 con 0x556bbcc774d0 2024-05-21T15:19:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.687+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.687+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0375c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:42446/0 (socket says 172.21.15.78:42446) 2024-05-21T15:19:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.687+0000 7f7eb7fff640 1 -- 172.21.15.78:0/2355616098 learned_addr learned my addr 172.21.15.78:0/2355616098 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.689+0000 7f7ebcbee640 1 --2- 172.21.15.78:0/2355616098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc981b0 0x556bbd037080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.690+0000 7f7eb7fff640 1 -- 172.21.15.78:0/2355616098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc774d0 msgr2=0x556bbd037b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:19:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.690+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/2355616098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc774d0 0x556bbd037b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.690+0000 7f7eb7fff640 1 -- 172.21.15.78:0/2355616098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc981b0 msgr2=0x556bbd037080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.691+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/2355616098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc981b0 0x556bbd037080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.691+0000 7f7eb7fff640 1 -- 172.21.15.78:0/2355616098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bbcf688e0 con 0x556bbcc6f2f0 2024-05-21T15:19:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.691+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/2355616098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0375c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009bd0 tx=0x7f7eac00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b209222226c47f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.692+0000 7f7eb57fa640 1 -- 172.21.15.78:0/2338752106 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7eb002fd50 con 0x7f7eb8106470 2024-05-21T15:19:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.692+0000 7f7e9e7fc640 1 -- 172.21.15.78:0/2355616098 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac005020 con 0x556bbcc6f2f0 2024-05-21T15:19:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.693+0000 7f7e9e7fc640 1 -- 172.21.15.78:0/2355616098 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac0047a0 con 0x556bbcc6f2f0 2024-05-21T15:19:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.693+0000 7f7e9e7fc640 1 -- 172.21.15.78:0/2355616098 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac010690 con 0x556bbcc6f2f0 2024-05-21T15:19:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.693+0000 7f7eb57fa640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ea00a2e30 0x7f7ea00a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.694+0000 7f7eb57fa640 1 -- 172.21.15.78:0/2338752106 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7eb00f0e60 con 0x7f7eb8106470 2024-05-21T15:19:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.694+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2355616098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 msgr2=0x556bbd0375c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.694+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/2355616098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0375c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009bd0 tx=0x7f7eac00e720 comp rx=0 tx=0).stop 2024-05-21T15:19:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.696+0000 7f7ebcbee640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ea00a2e30 0x7f7ea00a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.696+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2355616098 shutdown_connections 2024-05-21T15:19:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.696+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/2355616098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0375c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.696+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/2355616098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc774d0 0x556bbd037b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.696+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/2355616098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc981b0 0x556bbd037080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.696+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2355616098 >> 172.21.15.78:0/2355616098 conn(0x556bbcc8c300 msgr2=0x556bbd033830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.701+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2355616098 shutdown_connections 2024-05-21T15:19:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.701+0000 7f7ebcbee640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ea00a2e30 0x7f7ea00a5340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80058c0 tx=0x7f7ea800b260 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.701+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2355616098 wait complete. 2024-05-21T15:19:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.701+0000 7f7ec55d0740 1 -- start start 2024-05-21T15:19:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.702+0000 7f7ec55d0740 1 Processor -- start 2024-05-21T15:19:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.706+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc6f2f0 0x556bbd043120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.706+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc774d0 0x556bbd043680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.706+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 0x556bbd0415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.706+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556bbd0441b0 con 0x556bbcc6f2f0 2024-05-21T15:19:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.706+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556bbcf57790 con 0x556bbcc774d0 2024-05-21T15:19:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.706+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556bbd044330 con 0x556bbcc981b0 2024-05-21T15:19:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.706+0000 7f7ec55d0740 10 client.0 _ll_register_callbacks cb 0x556bbcf92a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.707+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc774d0 0x556bbd043680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.707+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc774d0 0x556bbd043680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:42454/0 (socket says 172.21.15.78:42454) 2024-05-21T15:19:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.707+0000 7f7eb77fe640 1 -- 172.21.15.78:0/254739796 learned_addr learned my addr 172.21.15.78:0/254739796 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.707+0000 7f7eb77fe640 1 -- 172.21.15.78:0/254739796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 msgr2=0x556bbd0415c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.707+0000 7f7eb77fe640 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 0x556bbd0415c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.707+0000 7f7eb77fe640 1 -- 172.21.15.78:0/254739796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc6f2f0 msgr2=0x556bbd043120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.707+0000 7f7eb77fe640 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc6f2f0 0x556bbd043120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.707+0000 7f7eb77fe640 1 -- 172.21.15.78:0/254739796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bbccb0210 con 0x556bbcc774d0 2024-05-21T15:19:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.710+0000 7f7eb77fe640 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc774d0 0x556bbd043680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00bd0d0 tx=0x7f7eb00bb8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37cbe75ef8573afe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.710+0000 7f7e977fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.712+0000 7f7e977fe640 1 -- 172.21.15.78:0/254739796 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00d5020 con 0x556bbcc774d0 2024-05-21T15:19:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.712+0000 7f7ec55d0740 1 -- 172.21.15.78:0/254739796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eac0047a0 con 0x556bbcc774d0 2024-05-21T15:19:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.713+0000 7f7e977fe640 1 -- 172.21.15.78:0/254739796 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7eb00c5bb0 con 0x556bbcc774d0 2024-05-21T15:19:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.714+0000 7f7e977fe640 1 -- 172.21.15.78:0/254739796 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7eb00ee980 con 0x556bbcc774d0 2024-05-21T15:19:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.716+0000 7f7ec55d0740 10 client.26027 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.716+0000 7f7ec55d0740 1 -- 172.21.15.78:0/254739796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556bbcd7c620 con 0x556bbcc774d0 2024-05-21T15:19:16.722 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7e977fe640 1 -- 172.21.15.78:0/254739796 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac0047a0 con 0x556bbcc774d0 2024-05-21T15:19:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7e977fe640 1 -- 172.21.15.78:0/254739796 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f7eb004d070 con 0x556bbcc774d0 2024-05-21T15:19:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7ec55d0740 10 client.26027 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7ec55d0740 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7ec55d0740 20 client.26027 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bbcc56d80 0x556bbd053d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7ec55d0740 4 client.26027 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7ec55d0740 1 -- 172.21.15.78:0/254739796 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bbcfa9a70 con 0x556bbcc56d80 2024-05-21T15:19:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.720+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bbcc56d80 0x556bbd053d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.721+0000 7f7e977fe640 10 client.26027 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.722+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bbcc56d80 0x556bbd053d90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4a049079bdad13 server_cookie=44ee3dda5685bc51 in_seq=0 out_seq=0 2024-05-21T15:19:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.727+0000 7f7e977fe640 1 -- 172.21.15.78:0/254739796 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556bbcfa9a70 con 0x556bbcc56d80 2024-05-21T15:19:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.727+0000 7f7e977fe640 10 client.26027 handle_command_reply: tid=0 2024-05-21T15:19:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.727+0000 7f7ec55d0740 1 client.26027 shutdown 2024-05-21T15:19:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.728+0000 7f7ec55d0740 1 -- 172.21.15.78:0/254739796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc774d0 msgr2=0x556bbd043680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.728+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc774d0 0x556bbd043680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00bd0d0 tx=0x7f7eb00bb8e0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.730+0000 7f7ec55d0740 1 -- 172.21.15.78:0/254739796 shutdown_connections 2024-05-21T15:19:16.732 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.730+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bbcc56d80 0x556bbd053d90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.730+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc774d0 0x556bbd043680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.730+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 0x556bbd0415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.730+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/254739796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc6f2f0 0x556bbd043120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.730+0000 7f7ec55d0740 1 -- 172.21.15.78:0/254739796 >> 172.21.15.78:0/254739796 conn(0x556bbcc8c300 msgr2=0x556bbd038da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:16.733 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.730+0000 7f7e977fe640 0 client.26027 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.733+0000 7f7ec55d0740 1 -- 172.21.15.78:0/254739796 shutdown_connections 2024-05-21T15:19:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.733+0000 7f7ec55d0740 1 -- 172.21.15.78:0/254739796 wait complete. 2024-05-21T15:19:16.737 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.734+0000 7f7ec55d0740 20 client.26027 trim_cache size 0 max 16384 2024-05-21T15:19:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.775+0000 7f7ec55d0740 1 Processor -- start 2024-05-21T15:19:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.775+0000 7f7ec55d0740 1 -- start start 2024-05-21T15:19:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc6f2f0 0x556bbd03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 0x556bbd03b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc981b0 0x556bbd038c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea400e950 con 0x556bbcc774d0 2024-05-21T15:19:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea4010640 con 0x556bbcc981b0 2024-05-21T15:19:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7ec55d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea400f020 con 0x556bbcc6f2f0 2024-05-21T15:19:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7ebcbee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc981b0 0x556bbd038c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 0x556bbd03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 0x556bbd03b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42610/0 (socket says 172.21.15.78:42610) 2024-05-21T15:19:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.776+0000 7f7eb77fe640 1 -- 172.21.15.78:0/2746098713 learned_addr learned my addr 172.21.15.78:0/2746098713 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.777+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/2746098713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc6f2f0 0x556bbd03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.778+0000 7f7eb7fff640 1 -- 172.21.15.78:0/2746098713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc981b0 msgr2=0x556bbd038c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.778+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/2746098713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc981b0 0x556bbd038c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.779+0000 7f7eb7fff640 1 -- 172.21.15.78:0/2746098713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 msgr2=0x556bbd03b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.779+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/2746098713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 0x556bbd03b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.779+0000 7f7eb7fff640 1 -- 172.21.15.78:0/2746098713 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bbd02eb90 con 0x556bbcc6f2f0 2024-05-21T15:19:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.783+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/2746098713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc6f2f0 0x556bbd03ad30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x556bbd038430 tx=0x7f7eac005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3efa0fce2a600117 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.784+0000 7f7e9dffb640 1 -- 172.21.15.78:0/2746098713 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac01d040 con 0x556bbcc6f2f0 2024-05-21T15:19:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.784+0000 7f7e9dffb640 1 -- 172.21.15.78:0/2746098713 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac005520 con 0x556bbcc6f2f0 2024-05-21T15:19:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.784+0000 7f7e9dffb640 1 -- 172.21.15.78:0/2746098713 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac026110 con 0x556bbcc6f2f0 2024-05-21T15:19:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.787+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2746098713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc6f2f0 msgr2=0x556bbd03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.788+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/2746098713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc6f2f0 0x556bbd03ad30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x556bbd038430 tx=0x7f7eac005db0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.792+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2746098713 shutdown_connections 2024-05-21T15:19:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.792+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/2746098713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc981b0 0x556bbd038c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.792+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/2746098713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 0x556bbd03b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.792+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/2746098713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc6f2f0 0x556bbd03ad30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.792+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2746098713 >> 172.21.15.78:0/2746098713 conn(0x556bbcc8c300 msgr2=0x556bbd03fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.798+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2746098713 shutdown_connections 2024-05-21T15:19:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.798+0000 7f7ec55d0740 1 -- 172.21.15.78:0/2746098713 wait complete. 2024-05-21T15:19:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.798+0000 7f7ec55d0740 1 -- start start 2024-05-21T15:19:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.799+0000 7f7ec55d0740 1 Processor -- start 2024-05-21T15:19:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 0x556bbd046d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7ec55d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 0x556bbd0472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:42466/0 (socket says 172.21.15.78:42466) 2024-05-21T15:19:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7eb7fff640 1 -- 172.21.15.78:0/3326269888 learned_addr learned my addr 172.21.15.78:0/3326269888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea400feb0 con 0x556bbcc774d0 2024-05-21T15:19:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea400f7b0 con 0x556bbcc6f2f0 2024-05-21T15:19:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.800+0000 7f7ebcbee640 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 0x556bbd0472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.801+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea400c930 con 0x556bbcc981b0 2024-05-21T15:19:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.801+0000 7f7ec55d0740 10 client.0 _ll_register_callbacks cb 0x7f7eb00093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.801+0000 7f7eb7fff640 1 -- 172.21.15.78:0/3326269888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 msgr2=0x556bbd0472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.801+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 0x556bbd0472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.801+0000 7f7eb7fff640 1 -- 172.21.15.78:0/3326269888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 msgr2=0x556bbd046d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.801+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 0x556bbd046d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.801+0000 7f7eb7fff640 1 -- 172.21.15.78:0/3326269888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bbd047810 con 0x556bbcc6f2f0 2024-05-21T15:19:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.801+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eac005520 con 0x556bbcc6f2f0 2024-05-21T15:19:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.802+0000 7f7eb7fff640 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0499f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00ec20 tx=0x7f7eac009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a1fedffa44a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.802+0000 7f7e977fe640 10 client.26030 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.802+0000 7f7e977fe640 1 -- 172.21.15.78:0/3326269888 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac034060 con 0x556bbcc6f2f0 2024-05-21T15:19:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.803+0000 7f7e977fe640 1 -- 172.21.15.78:0/3326269888 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7eac0047d0 con 0x556bbcc6f2f0 2024-05-21T15:19:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.803+0000 7f7e977fe640 1 -- 172.21.15.78:0/3326269888 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7eac009cc0 con 0x556bbcc6f2f0 2024-05-21T15:19:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.805+0000 7f7ec55d0740 10 client.26030 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.806+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({fsmap=31}) -- 0x556bbd0bbca0 con 0x556bbcc6f2f0 2024-05-21T15:19:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.806+0000 7f7e977fe640 1 -- 172.21.15.78:0/3326269888 <== mon.2 v2:172.21.15.84:3300/0 4 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f7eac004af0 con 0x556bbcc6f2f0 2024-05-21T15:19:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.807+0000 7f7ec55d0740 10 client.26030 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.807+0000 7f7ec55d0740 10 client.26030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.807+0000 7f7ec55d0740 20 client.26030 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.807+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bbcc2c1c0 0x556bbd0be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.807+0000 7f7ec55d0740 4 client.26030 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:19:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.807+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7eac005520 con 0x556bbcc2c1c0 2024-05-21T15:19:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.807+0000 7f7eb77fe640 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bbcc2c1c0 0x556bbd0be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.809+0000 7f7eb77fe640 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bbcc2c1c0 0x556bbd0be3c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459928197dcbf1dc server_cookie=abdbd78dacdc74e0 in_seq=0 out_seq=0 2024-05-21T15:19:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:16.809+0000 7f7e977fe640 10 client.26030 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.326+0000 7f7e977fe640 1 -- 172.21.15.78:0/3326269888 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7eac005520 con 0x556bbcc2c1c0 2024-05-21T15:19:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.326+0000 7f7e977fe640 10 client.26030 handle_command_reply: tid=0 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.326+0000 7f7ec55d0740 1 client.26030 shutdown 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 msgr2=0x556bbd0499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0499f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00ec20 tx=0x7f7eac009370 comp rx=0 tx=0).stop 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 shutdown_connections 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bbcc2c1c0 0x556bbd0be3c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bbcc981b0 0x556bbd0472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bbcc774d0 0x556bbd046d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7ec55d0740 1 --2- 172.21.15.78:0/3326269888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bbcc6f2f0 0x556bbd0499f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 >> 172.21.15.78:0/3326269888 conn(0x556bbcc8c300 msgr2=0x556bbd038c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.327+0000 7f7e977fe640 0 client.26030 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.328+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 shutdown_connections 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.328+0000 7f7ec55d0740 1 -- 172.21.15.78:0/3326269888 wait complete. 2024-05-21T15:19:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.328+0000 7f7ec55d0740 20 client.26030 trim_cache size 0 max 16384 2024-05-21T15:19:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:19:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:19:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:19:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 -- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ea00a2e30 msgr2=0x7f7ea00a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ea00a2e30 0x7f7ea00a5340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80058c0 tx=0x7f7ea800b260 comp rx=0 tx=0).stop 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 -- 172.21.15.78:0/2338752106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 msgr2=0x7f7eb806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 0x7f7eb806d650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7eb002aec0 tx=0x7f7eb0003e60 comp rx=0 tx=0).stop 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 -- 172.21.15.78:0/2338752106 shutdown_connections 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ea00a2e30 0x7f7ea00a5340 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7eb8106ea0 0x7f7eb806db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7eb8106470 0x7f7eb806d650 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 --2- 172.21.15.78:0/2338752106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7eb8101a20 0x7f7eb8073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 -- 172.21.15.78:0/2338752106 >> 172.21.15.78:0/2338752106 conn(0x7f7eb8077100 msgr2=0x7f7eb81043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 -- 172.21.15.78:0/2338752106 shutdown_connections 2024-05-21T15:19:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:17.329+0000 7f7ebe651640 1 -- 172.21.15.78:0/2338752106 wait complete. 2024-05-21T15:19:18.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:17 smithi078 ceph-mon[31997]: pgmap v99: 97 pgs: 97 active+clean; 15 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 23 KiB/s rd, 63 MiB/s wr, 10.61k op/s 2024-05-21T15:19:18.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:17 smithi084 ceph-mon[36334]: pgmap v99: 97 pgs: 97 active+clean; 15 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 23 KiB/s rd, 63 MiB/s wr, 10.61k op/s 2024-05-21T15:19:18.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:17 smithi172 ceph-mon[36402]: pgmap v99: 97 pgs: 97 active+clean; 15 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 23 KiB/s rd, 63 MiB/s wr, 10.61k op/s 2024-05-21T15:19:19.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:18 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:19:19.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:18 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:19:19.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:18 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:19:20.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:19 smithi078 ceph-mon[31997]: pgmap v100: 97 pgs: 97 active+clean; 15 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 20 KiB/s rd, 50 MiB/s wr, 8.81k op/s 2024-05-21T15:19:20.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:19 smithi084 ceph-mon[36334]: pgmap v100: 97 pgs: 97 active+clean; 15 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 20 KiB/s rd, 50 MiB/s wr, 8.81k op/s 2024-05-21T15:19:20.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:19 smithi172 ceph-mon[36402]: pgmap v100: 97 pgs: 97 active+clean; 15 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 20 KiB/s rd, 50 MiB/s wr, 8.81k op/s 2024-05-21T15:19:21.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:20 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:19:21.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:20 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:19:21.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:20 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:19:21.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:20 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:19:21.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:20 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:19:21.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:20 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:19:21.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:20 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:19:21.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:20 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:19:21.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:20 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:19:22.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:22 smithi078 ceph-mon[31997]: pgmap v101: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 106 MiB/s wr, 18.60k op/s 2024-05-21T15:19:22.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:22 smithi084 ceph-mon[36334]: pgmap v101: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 106 MiB/s wr, 18.60k op/s 2024-05-21T15:19:22.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:22 smithi172 ceph-mon[36402]: pgmap v101: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 106 MiB/s wr, 18.60k op/s 2024-05-21T15:19:24.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:24 smithi078 ceph-mon[31997]: pgmap v102: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 25 KiB/s rd, 89 MiB/s wr, 15.86k op/s 2024-05-21T15:19:24.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:24 smithi084 ceph-mon[36334]: pgmap v102: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 25 KiB/s rd, 89 MiB/s wr, 15.86k op/s 2024-05-21T15:19:24.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:24 smithi172 ceph-mon[36402]: pgmap v102: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 25 KiB/s rd, 89 MiB/s wr, 15.86k op/s 2024-05-21T15:19:26.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:26 smithi078 ceph-mon[31997]: pgmap v103: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 89 MiB/s wr, 16.01k op/s 2024-05-21T15:19:26.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:26 smithi084 ceph-mon[36334]: pgmap v103: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 89 MiB/s wr, 16.01k op/s 2024-05-21T15:19:26.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:26 smithi172 ceph-mon[36402]: pgmap v103: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 89 MiB/s wr, 16.01k op/s 2024-05-21T15:19:28.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:28 smithi078 ceph-mon[31997]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 99 MiB/s wr, 17.93k op/s 2024-05-21T15:19:28.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:28 smithi084 ceph-mon[36334]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 99 MiB/s wr, 17.93k op/s 2024-05-21T15:19:28.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:28 smithi172 ceph-mon[36402]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 99 MiB/s wr, 17.93k op/s 2024-05-21T15:19:29.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:19:29.273 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:19:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.463+0000 7fd41c58f640 1 -- 172.21.15.78:0/1868944410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd414074310 msgr2=0x7fd414074720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.463+0000 7fd41c58f640 1 --2- 172.21.15.78:0/1868944410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd414074310 0x7fd414074720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd410009900 tx=0x7fd41002a4b0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.464+0000 7fd41c58f640 1 -- 172.21.15.78:0/1868944410 shutdown_connections 2024-05-21T15:19:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.464+0000 7fd41c58f640 1 --2- 172.21.15.78:0/1868944410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414113af0 0x7fd414115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.464+0000 7fd41c58f640 1 --2- 172.21.15.78:0/1868944410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd414111190 0x7fd4141135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.464+0000 7fd41c58f640 1 --2- 172.21.15.78:0/1868944410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd414074310 0x7fd414074720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.464+0000 7fd41c58f640 1 -- 172.21.15.78:0/1868944410 >> 172.21.15.78:0/1868944410 conn(0x7fd4140780b0 msgr2=0x7fd4140784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:29.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.465+0000 7fd41c58f640 1 -- 172.21.15.78:0/1868944410 shutdown_connections 2024-05-21T15:19:29.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.465+0000 7fd41c58f640 1 -- 172.21.15.78:0/1868944410 wait complete. 2024-05-21T15:19:29.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.467+0000 7fd41c58f640 1 Processor -- start 2024-05-21T15:19:29.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.469+0000 7fd41c58f640 1 -- start start 2024-05-21T15:19:29.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.470+0000 7fd41c58f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd414074310 0x7fd41419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.470+0000 7fd41c58f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd414111190 0x7fd4141a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.470+0000 7fd41c58f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414113af0 0x7fd4141a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.470+0000 7fd41c58f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141194b0 con 0x7fd414111190 2024-05-21T15:19:29.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.470+0000 7fd41c58f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd414119330 con 0x7fd414074310 2024-05-21T15:19:29.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.470+0000 7fd41c58f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd414119630 con 0x7fd414113af0 2024-05-21T15:19:29.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.471+0000 7fd41ab2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414113af0 0x7fd4141a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.471+0000 7fd41ab2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414113af0 0x7fd4141a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:43876/0 (socket says 172.21.15.78:43876) 2024-05-21T15:19:29.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.471+0000 7fd41ab2c640 1 -- 172.21.15.78:0/3326504463 learned_addr learned my addr 172.21.15.78:0/3326504463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:29.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.472+0000 7fd419b2a640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd414111190 0x7fd4141a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.472+0000 7fd41ab2c640 1 -- 172.21.15.78:0/3326504463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd414074310 msgr2=0x7fd41419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.472+0000 7fd41ab2c640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd414074310 0x7fd41419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.472+0000 7fd41ab2c640 1 -- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd414111190 msgr2=0x7fd4141a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.472+0000 7fd41ab2c640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd414111190 0x7fd4141a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.472+0000 7fd41ab2c640 1 -- 172.21.15.78:0/3326504463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4141a4b90 con 0x7fd414113af0 2024-05-21T15:19:29.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.473+0000 7fd41ab2c640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414113af0 0x7fd4141a4550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00eb10 tx=0x7fd40c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:29.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.475+0000 7fd40b7fe640 1 -- 172.21.15.78:0/3326504463 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c004b20 con 0x7fd414113af0 2024-05-21T15:19:29.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.475+0000 7fd40b7fe640 1 -- 172.21.15.78:0/3326504463 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40c004cc0 con 0x7fd414113af0 2024-05-21T15:19:29.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.475+0000 7fd40b7fe640 1 -- 172.21.15.78:0/3326504463 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c010760 con 0x7fd414113af0 2024-05-21T15:19:29.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.475+0000 7fd41c58f640 1 -- 172.21.15.78:0/3326504463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4141a4e80 con 0x7fd414113af0 2024-05-21T15:19:29.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.475+0000 7fd41c58f640 1 -- 172.21.15.78:0/3326504463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41406b570 con 0x7fd414113af0 2024-05-21T15:19:29.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.483+0000 7fd423436740 1 Processor -- start 2024-05-21T15:19:29.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.483+0000 7fd423436740 1 -- start start 2024-05-21T15:19:29.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.484+0000 7fd423436740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683b6390 0x562968754d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.484+0000 7fd40b7fe640 1 -- 172.21.15.78:0/3326504463 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd40c010900 con 0x7fd414113af0 2024-05-21T15:19:29.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.484+0000 7fd423436740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56296838d4d0 0x562968755270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.484+0000 7fd423436740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x5629687557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.484+0000 7fd423436740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5629686bad00 con 0x5629683b6390 2024-05-21T15:19:29.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.484+0000 7fd423436740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5629686624c0 con 0x5629683956b0 2024-05-21T15:19:29.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.484+0000 7fd423436740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629686b6cc0 con 0x56296838d4d0 2024-05-21T15:19:29.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.485+0000 7fd40b7fe640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd4000a2ed0 0x7fd4000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.485+0000 7fd40b7fe640 1 -- 172.21.15.78:0/3326504463 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd40c0d0b90 con 0x7fd414113af0 2024-05-21T15:19:29.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.486+0000 7fd41a32b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56296838d4d0 0x562968755270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.486+0000 7fd41a32b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56296838d4d0 0x562968755270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:43886/0 (socket says 172.21.15.78:43886) 2024-05-21T15:19:29.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.486+0000 7fd41a32b640 1 -- 172.21.15.78:0/1612065954 learned_addr learned my addr 172.21.15.78:0/1612065954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:29.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.486+0000 7fd41ab2c640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd4000a2ed0 0x7fd4000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.487+0000 7fd41ab2c640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd4000a2ed0 0x7fd4000a53e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd40c005ba0 tx=0x7fd40c00efd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:29.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.488+0000 7fd419b2a640 1 --2- 172.21.15.78:0/1612065954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x5629687557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.488+0000 7fd41a32b640 1 -- 172.21.15.78:0/1612065954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 msgr2=0x5629687557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:29.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.488+0000 7fd41a32b640 1 --2- 172.21.15.78:0/1612065954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x5629687557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.488+0000 7fd41a32b640 1 -- 172.21.15.78:0/1612065954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683b6390 msgr2=0x562968754d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:29.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.488+0000 7fd41a32b640 1 --2- 172.21.15.78:0/1612065954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683b6390 0x562968754d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.488+0000 7fd41a32b640 1 -- 172.21.15.78:0/1612065954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629684987d0 con 0x56296838d4d0 2024-05-21T15:19:29.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.489+0000 7fd41a32b640 1 --2- 172.21.15.78:0/1612065954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56296838d4d0 0x562968755270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd41002aaa0 tx=0x7fd410031510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=732645b25ebc3b75 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:29.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.490+0000 7fd408bfe640 1 -- 172.21.15.78:0/1612065954 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41003e070 con 0x56296838d4d0 2024-05-21T15:19:29.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.490+0000 7fd408bfe640 1 -- 172.21.15.78:0/1612065954 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd410004460 con 0x56296838d4d0 2024-05-21T15:19:29.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.490+0000 7fd408bfe640 1 -- 172.21.15.78:0/1612065954 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410035040 con 0x56296838d4d0 2024-05-21T15:19:29.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.492+0000 7fd423436740 1 -- 172.21.15.78:0/1612065954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56296838d4d0 msgr2=0x562968755270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.492+0000 7fd423436740 1 --2- 172.21.15.78:0/1612065954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56296838d4d0 0x562968755270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd41002aaa0 tx=0x7fd410031510 comp rx=0 tx=0).stop 2024-05-21T15:19:29.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.498+0000 7fd423436740 1 -- 172.21.15.78:0/1612065954 shutdown_connections 2024-05-21T15:19:29.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.498+0000 7fd423436740 1 --2- 172.21.15.78:0/1612065954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x5629687557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.498+0000 7fd423436740 1 --2- 172.21.15.78:0/1612065954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56296838d4d0 0x562968755270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.498+0000 7fd423436740 1 --2- 172.21.15.78:0/1612065954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683b6390 0x562968754d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.498+0000 7fd423436740 1 -- 172.21.15.78:0/1612065954 >> 172.21.15.78:0/1612065954 conn(0x5629683aa4e0 msgr2=0x5629687514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:29.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.498+0000 7fd423436740 1 -- 172.21.15.78:0/1612065954 shutdown_connections 2024-05-21T15:19:29.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.498+0000 7fd423436740 1 -- 172.21.15.78:0/1612065954 wait complete. 2024-05-21T15:19:29.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 1 -- start start 2024-05-21T15:19:29.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 1 Processor -- start 2024-05-21T15:19:29.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56296838d4d0 0x56296875f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683956b0 0x56296875f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 0x562968765540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562968761fc0 con 0x5629683956b0 2024-05-21T15:19:29.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562968675b90 con 0x56296838d4d0 2024-05-21T15:19:29.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562968762140 con 0x5629683b6390 2024-05-21T15:19:29.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.499+0000 7fd423436740 10 client.0 _ll_register_callbacks cb 0x5629686b1da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:29.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.502+0000 7fd419b2a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56296838d4d0 0x56296875f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.502+0000 7fd419b2a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 0x562968765540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.502+0000 7fd419b2a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56296838d4d0 0x56296875f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45732/0 (socket says 172.21.15.78:45732) 2024-05-21T15:19:29.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.502+0000 7fd419b2a640 1 -- 172.21.15.78:0/3852128265 learned_addr learned my addr 172.21.15.78:0/3852128265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:29.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.504+0000 7fd41a32b640 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683956b0 0x56296875f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.507+0000 7fd41a32b640 1 -- 172.21.15.78:0/3852128265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 msgr2=0x562968765540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:29.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.507+0000 7fd41a32b640 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 0x562968765540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.507+0000 7fd41a32b640 1 -- 172.21.15.78:0/3852128265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56296838d4d0 msgr2=0x56296875f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:29.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.507+0000 7fd41a32b640 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56296838d4d0 0x56296875f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.507+0000 7fd41a32b640 1 -- 172.21.15.78:0/3852128265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629683ce3f0 con 0x5629683956b0 2024-05-21T15:19:29.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.509+0000 7fd419b2a640 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56296838d4d0 0x56296875f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.514+0000 7fd41a32b640 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683956b0 0x56296875f8f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x5629687562f0 tx=0x7fd410003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8148e1a3aac73d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.516+0000 7fd3fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:19:29.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.522+0000 7fd3fdffb640 1 -- 172.21.15.78:0/3852128265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41004b070 con 0x5629683956b0 2024-05-21T15:19:29.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.523+0000 7fd423436740 1 -- 172.21.15.78:0/3852128265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd410004460 con 0x5629683956b0 2024-05-21T15:19:29.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.523+0000 7fd3fdffb640 1 -- 172.21.15.78:0/3852128265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd4100035f0 con 0x5629683956b0 2024-05-21T15:19:29.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.527+0000 7fd3fdffb640 1 -- 172.21.15.78:0/3852128265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd41003e070 con 0x5629683956b0 2024-05-21T15:19:29.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.527+0000 7fd423436740 10 client.16575 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:29.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.527+0000 7fd423436740 1 -- 172.21.15.78:0/3852128265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56296838e0f0 con 0x5629683956b0 2024-05-21T15:19:29.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.529+0000 7fd3fdffb640 1 -- 172.21.15.78:0/3852128265 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd410004460 con 0x5629683956b0 2024-05-21T15:19:29.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.529+0000 7fd3fdffb640 1 -- 172.21.15.78:0/3852128265 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fd41004e020 con 0x5629683956b0 2024-05-21T15:19:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.529+0000 7fd423436740 10 client.16575 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.529+0000 7fd423436740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.529+0000 7fd423436740 20 client.16575 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.529+0000 7fd423436740 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562968374f60 0x562968771cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.529+0000 7fd423436740 4 client.16575 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.529+0000 7fd423436740 1 -- 172.21.15.78:0/3852128265 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629686c8430 con 0x562968374f60 2024-05-21T15:19:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.530+0000 7fd419b2a640 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562968374f60 0x562968771cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.533+0000 7fd419b2a640 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562968374f60 0x562968771cf0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f48bc19e34bf87 server_cookie=f125856ce8232200 in_seq=0 out_seq=0 2024-05-21T15:19:29.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.533+0000 7fd3fdffb640 10 client.16575 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.535+0000 7fd3fdffb640 1 -- 172.21.15.78:0/3852128265 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629686c8430 con 0x562968374f60 2024-05-21T15:19:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.535+0000 7fd3fdffb640 10 client.16575 handle_command_reply: tid=0 2024-05-21T15:19:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.535+0000 7fd423436740 1 client.16575 shutdown 2024-05-21T15:19:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.536+0000 7fd423436740 1 -- 172.21.15.78:0/3852128265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683956b0 msgr2=0x56296875f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.536+0000 7fd423436740 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683956b0 0x56296875f8f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x5629687562f0 tx=0x7fd410003ee0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.536+0000 7fd423436740 1 -- 172.21.15.78:0/3852128265 shutdown_connections 2024-05-21T15:19:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.536+0000 7fd423436740 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562968374f60 0x562968771cf0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.536+0000 7fd423436740 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 0x562968765540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.536+0000 7fd423436740 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5629683956b0 0x56296875f8f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.536+0000 7fd423436740 1 --2- 172.21.15.78:0/3852128265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56296838d4d0 0x56296875f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.536+0000 7fd423436740 1 -- 172.21.15.78:0/3852128265 >> 172.21.15.78:0/3852128265 conn(0x5629683aa4e0 msgr2=0x562968756a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.537+0000 7fd3fdffb640 0 client.16575 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.537+0000 7fd423436740 1 -- 172.21.15.78:0/3852128265 shutdown_connections 2024-05-21T15:19:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.537+0000 7fd423436740 1 -- 172.21.15.78:0/3852128265 wait complete. 2024-05-21T15:19:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.538+0000 7fd423436740 20 client.16575 trim_cache size 0 max 16384 2024-05-21T15:19:29.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.579+0000 7fd423436740 1 Processor -- start 2024-05-21T15:19:29.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.579+0000 7fd423436740 1 -- start start 2024-05-21T15:19:29.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.580+0000 7fd423436740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968757200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.580+0000 7fd423436740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x562968765440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.580+0000 7fd423436740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 0x56296875a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.580+0000 7fd423436740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f800ea60 con 0x56296838d4d0 2024-05-21T15:19:29.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.580+0000 7fd423436740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f8010750 con 0x5629683956b0 2024-05-21T15:19:29.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.580+0000 7fd423436740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f800f130 con 0x5629683b6390 2024-05-21T15:19:29.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.581+0000 7fd41a32b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x562968765440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.581+0000 7fd41a32b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x562968765440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45748/0 (socket says 172.21.15.78:45748) 2024-05-21T15:19:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.581+0000 7fd41a32b640 1 -- 172.21.15.78:0/87924213 learned_addr learned my addr 172.21.15.78:0/87924213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.581+0000 7fd419b2a640 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 0x56296875a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.581+0000 7fd419b2a640 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968757200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.583+0000 7fd41a32b640 1 -- 172.21.15.78:0/87924213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 msgr2=0x56296875a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.583+0000 7fd41a32b640 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 0x56296875a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.583+0000 7fd41a32b640 1 -- 172.21.15.78:0/87924213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 msgr2=0x562968757200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.583+0000 7fd41a32b640 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968757200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.583+0000 7fd41a32b640 1 -- 172.21.15.78:0/87924213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562968754ac0 con 0x5629683956b0 2024-05-21T15:19:29.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.583+0000 7fd41a32b640 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x562968765440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd410003f50 tx=0x7fd410050440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c2810df24528ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.585+0000 7fd3fffff640 1 -- 172.21.15.78:0/87924213 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410058020 con 0x5629683956b0 2024-05-21T15:19:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.585+0000 7fd3fffff640 1 -- 172.21.15.78:0/87924213 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd410004460 con 0x5629683956b0 2024-05-21T15:19:29.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.585+0000 7fd3fffff640 1 -- 172.21.15.78:0/87924213 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41004b070 con 0x5629683956b0 2024-05-21T15:19:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.587+0000 7fd423436740 1 -- 172.21.15.78:0/87924213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 msgr2=0x562968765440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.587+0000 7fd423436740 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x562968765440 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd410003f50 tx=0x7fd410050440 comp rx=0 tx=0).stop 2024-05-21T15:19:29.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.589+0000 7fd423436740 1 -- 172.21.15.78:0/87924213 shutdown_connections 2024-05-21T15:19:29.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.589+0000 7fd423436740 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683956b0 0x562968765440 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.589+0000 7fd423436740 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683b6390 0x56296875a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.589+0000 7fd423436740 1 --2- 172.21.15.78:0/87924213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968757200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.589+0000 7fd423436740 1 -- 172.21.15.78:0/87924213 >> 172.21.15.78:0/87924213 conn(0x5629683aa4e0 msgr2=0x56296875da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:29.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.593+0000 7fd423436740 1 -- 172.21.15.78:0/87924213 shutdown_connections 2024-05-21T15:19:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.593+0000 7fd423436740 1 -- 172.21.15.78:0/87924213 wait complete. 2024-05-21T15:19:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.594+0000 7fd423436740 1 -- start start 2024-05-21T15:19:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.594+0000 7fd423436740 1 Processor -- start 2024-05-21T15:19:29.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.595+0000 7fd423436740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968766d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.595+0000 7fd423436740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683956b0 0x562968767290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.595+0000 7fd423436740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683b6390 0x5629687d8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.595+0000 7fd423436740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f800ffc0 con 0x56296838d4d0 2024-05-21T15:19:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.595+0000 7fd423436740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f800f8c0 con 0x5629683b6390 2024-05-21T15:19:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.595+0000 7fd423436740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f800ca40 con 0x5629683956b0 2024-05-21T15:19:29.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.595+0000 7fd423436740 10 client.0 _ll_register_callbacks cb 0x7fd410009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.598+0000 7fd419b2a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683b6390 0x5629687d8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.598+0000 7fd419b2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968766d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.598+0000 7fd419b2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968766d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36674/0 (socket says 172.21.15.78:36674) 2024-05-21T15:19:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.598+0000 7fd419b2a640 1 -- 172.21.15.78:0/1056909490 learned_addr learned my addr 172.21.15.78:0/1056909490 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:29.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.598+0000 7fd41a32b640 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683956b0 0x562968767290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.599+0000 7fd419b2a640 1 -- 172.21.15.78:0/1056909490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683956b0 msgr2=0x562968767290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.599+0000 7fd419b2a640 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683956b0 0x562968767290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.599+0000 7fd419b2a640 1 -- 172.21.15.78:0/1056909490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 msgr2=0x562968766d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.599+0000 7fd419b2a640 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968766d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.599+0000 7fd419b2a640 1 -- 172.21.15.78:0/1056909490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629687e95d0 con 0x5629683b6390 2024-05-21T15:19:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.600+0000 7fd419b2a640 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683b6390 0x5629687d8050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd404006dc0 tx=0x7fd40401fc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63dd5c67247bab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:29.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.600+0000 7fd3fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.603+0000 7fd3fdffb640 1 -- 172.21.15.78:0/1056909490 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404030060 con 0x5629683b6390 2024-05-21T15:19:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.603+0000 7fd423436740 1 -- 172.21.15.78:0/1056909490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd410004460 con 0x5629683b6390 2024-05-21T15:19:29.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.603+0000 7fd3fdffb640 1 -- 172.21.15.78:0/1056909490 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd40401f1b0 con 0x5629683b6390 2024-05-21T15:19:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.604+0000 7fd3fdffb640 1 -- 172.21.15.78:0/1056909490 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd40401f3b0 con 0x5629683b6390 2024-05-21T15:19:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.604+0000 7fd423436740 10 client.26042 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.604+0000 7fd423436740 1 -- 172.21.15.78:0/1056909490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5629687ea8f0 con 0x5629683b6390 2024-05-21T15:19:29.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.604+0000 7fd3fdffb640 1 -- 172.21.15.78:0/1056909490 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd410004460 con 0x5629683b6390 2024-05-21T15:19:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.606+0000 7fd3fdffb640 1 -- 172.21.15.78:0/1056909490 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fd40401fce0 con 0x5629683b6390 2024-05-21T15:19:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.606+0000 7fd423436740 10 client.26042 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.606+0000 7fd423436740 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.606+0000 7fd423436740 20 client.26042 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.606+0000 7fd423436740 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56296834a3a0 0x5629687ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.606+0000 7fd423436740 4 client.26042 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:19:29.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.606+0000 7fd423436740 1 -- 172.21.15.78:0/1056909490 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629686c2780 con 0x56296834a3a0 2024-05-21T15:19:29.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.609+0000 7fd41a32b640 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56296834a3a0 0x5629687ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.616+0000 7fd3fdffb640 10 client.26042 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.616+0000 7fd41a32b640 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56296834a3a0 0x5629687ed0a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87e12ba780a794 server_cookie=a986b96871d4ccca in_seq=0 out_seq=0 2024-05-21T15:19:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.618+0000 7fd3fdffb640 1 -- 172.21.15.78:0/1056909490 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5629686c2780 con 0x56296834a3a0 2024-05-21T15:19:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.618+0000 7fd3fdffb640 10 client.26042 handle_command_reply: tid=0 2024-05-21T15:19:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.618+0000 7fd423436740 1 client.26042 shutdown 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd423436740 1 -- 172.21.15.78:0/1056909490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683b6390 msgr2=0x5629687d8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd423436740 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683b6390 0x5629687d8050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd404006dc0 tx=0x7fd40401fc70 comp rx=0 tx=0).stop 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd423436740 1 -- 172.21.15.78:0/1056909490 shutdown_connections 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd423436740 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56296834a3a0 0x5629687ed0a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd423436740 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5629683b6390 0x5629687d8050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd423436740 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629683956b0 0x562968767290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd423436740 1 --2- 172.21.15.78:0/1056909490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56296838d4d0 0x562968766d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd3fdffb640 0 client.26042 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.621+0000 7fd423436740 1 -- 172.21.15.78:0/1056909490 >> 172.21.15.78:0/1056909490 conn(0x5629683aa4e0 msgr2=0x56296875a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.622+0000 7fd423436740 1 -- 172.21.15.78:0/1056909490 shutdown_connections 2024-05-21T15:19:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.622+0000 7fd423436740 1 -- 172.21.15.78:0/1056909490 wait complete. 2024-05-21T15:19:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.624+0000 7fd423436740 20 client.26042 trim_cache size 0 max 16384 2024-05-21T15:19:29.628 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:19:29.628 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:19:29.628 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:19:29.628 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:19:29.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.625+0000 7fd41c58f640 1 -- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd4000a2ed0 msgr2=0x7fd4000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.625+0000 7fd41c58f640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd4000a2ed0 0x7fd4000a53e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd40c005ba0 tx=0x7fd40c00efd0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.625+0000 7fd41c58f640 1 -- 172.21.15.78:0/3326504463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414113af0 msgr2=0x7fd4141a4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.625+0000 7fd41c58f640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414113af0 0x7fd4141a4550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00eb10 tx=0x7fd40c00c2d0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.626+0000 7fd41c58f640 1 -- 172.21.15.78:0/3326504463 shutdown_connections 2024-05-21T15:19:29.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.626+0000 7fd41c58f640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd4000a2ed0 0x7fd4000a53e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.626+0000 7fd41c58f640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414113af0 0x7fd4141a4550 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.626+0000 7fd41c58f640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd414111190 0x7fd4141a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.626+0000 7fd41c58f640 1 --2- 172.21.15.78:0/3326504463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd414074310 0x7fd41419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.626+0000 7fd41c58f640 1 -- 172.21.15.78:0/3326504463 >> 172.21.15.78:0/3326504463 conn(0x7fd4140780b0 msgr2=0x7fd4141115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:29.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.626+0000 7fd41c58f640 1 -- 172.21.15.78:0/3326504463 shutdown_connections 2024-05-21T15:19:29.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.626+0000 7fd41c58f640 1 -- 172.21.15.78:0/3326504463 wait complete. 2024-05-21T15:19:29.649 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:19:29.649 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:19:29.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.856+0000 7f55dc102640 1 -- 172.21.15.78:0/2681998031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d4109d70 msgr2=0x7f55d410a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.856+0000 7f55dc102640 1 --2- 172.21.15.78:0/2681998031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d4109d70 0x7f55d410a180 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f55d000da00 tx=0x7f55d002c550 comp rx=0 tx=0).stop 2024-05-21T15:19:29.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.857+0000 7f55dc102640 1 -- 172.21.15.78:0/2681998031 shutdown_connections 2024-05-21T15:19:29.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.857+0000 7f55dc102640 1 --2- 172.21.15.78:0/2681998031 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d4114290 0x7f55d41166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.857+0000 7f55dc102640 1 --2- 172.21.15.78:0/2681998031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d4109d70 0x7f55d410a180 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.857+0000 7f55dc102640 1 --2- 172.21.15.78:0/2681998031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d410f9e0 0x7f55d4109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.857+0000 7f55dc102640 1 -- 172.21.15.78:0/2681998031 >> 172.21.15.78:0/2681998031 conn(0x7f55d406d760 msgr2=0x7f55d406dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:29.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.862+0000 7f55dc102640 1 -- 172.21.15.78:0/2681998031 shutdown_connections 2024-05-21T15:19:29.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.865+0000 7f55dc102640 1 -- 172.21.15.78:0/2681998031 wait complete. 2024-05-21T15:19:29.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.866+0000 7f55dc102640 1 Processor -- start 2024-05-21T15:19:29.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.867+0000 7f55dc102640 1 -- start start 2024-05-21T15:19:29.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.867+0000 7f55dc102640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d4109d70 0x7f55d41ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.867+0000 7f55dc102640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d410f9e0 0x7f55d41ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.867+0000 7f55dc102640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d4114290 0x7f55d410d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.867+0000 7f55dc102640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4118ab0 con 0x7f55d410f9e0 2024-05-21T15:19:29.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.867+0000 7f55dc102640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4118930 con 0x7f55d4109d70 2024-05-21T15:19:29.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.867+0000 7f55dc102640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4118c30 con 0x7f55d4114290 2024-05-21T15:19:29.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.868+0000 7f55da69f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d4114290 0x7f55d410d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.868+0000 7f55d969d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d410f9e0 0x7f55d41ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.868+0000 7f55d969d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d410f9e0 0x7f55d41ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36694/0 (socket says 172.21.15.78:36694) 2024-05-21T15:19:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.868+0000 7f55d969d640 1 -- 172.21.15.78:0/13142348 learned_addr learned my addr 172.21.15.78:0/13142348 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:29.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.870+0000 7f55d969d640 1 -- 172.21.15.78:0/13142348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d4114290 msgr2=0x7f55d410d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:29.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.870+0000 7f55d969d640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d4114290 0x7f55d410d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.870+0000 7f55d969d640 1 -- 172.21.15.78:0/13142348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d4109d70 msgr2=0x7f55d41ad310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:19:29.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.870+0000 7f55d969d640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d4109d70 0x7f55d41ad310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:29.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.870+0000 7f55d969d640 1 -- 172.21.15.78:0/13142348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d410dda0 con 0x7f55d410f9e0 2024-05-21T15:19:29.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.870+0000 7f55d969d640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d410f9e0 0x7f55d41ad850 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f55d0033630 tx=0x7f55d0033660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.871+0000 7f55caffd640 1 -- 172.21.15.78:0/13142348 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0040020 con 0x7f55d410f9e0 2024-05-21T15:19:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.871+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d410e030 con 0x7f55d410f9e0 2024-05-21T15:19:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.871+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d4079db0 con 0x7f55d410f9e0 2024-05-21T15:19:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.872+0000 7f55caffd640 1 -- 172.21.15.78:0/13142348 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d0005d10 con 0x7f55d410f9e0 2024-05-21T15:19:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.872+0000 7f55caffd640 1 -- 172.21.15.78:0/13142348 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0037070 con 0x7f55d410f9e0 2024-05-21T15:19:29.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.872+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d410abe0 con 0x7f55d410f9e0 2024-05-21T15:19:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.878+0000 7f55caffd640 1 -- 172.21.15.78:0/13142348 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f55d0043020 con 0x7f55d410f9e0 2024-05-21T15:19:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.879+0000 7f55caffd640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55c00a2e00 0x7f55c00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.879+0000 7f55caffd640 1 -- 172.21.15.78:0/13142348 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f55d00f0880 con 0x7f55d410f9e0 2024-05-21T15:19:29.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.882+0000 7f55d9e9e640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55c00a2e00 0x7f55c00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:29.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.884+0000 7f55caffd640 1 -- 172.21.15.78:0/13142348 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55d00bd3e0 con 0x7f55d410f9e0 2024-05-21T15:19:29.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:29.892+0000 7f55d9e9e640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55c00a2e00 0x7f55c00a5310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f55d4068c80 tx=0x7f55c4006bd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.316+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55d4068cf0 con 0x7f55d410f9e0 2024-05-21T15:19:30.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.318+0000 7f55caffd640 1 -- 172.21.15.78:0/13142348 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11940 (secure 0 0 0) 0x7f55d00bcce0 con 0x7f55d410f9e0 2024-05-21T15:19:30.321 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:19:30.321 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:18:57.755829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527,24526,24502,24419,24448]},"id":1}]} 2024-05-21T15:19:30.321 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-05-21T15:19:30.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.322+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55c00a2e00 msgr2=0x7f55c00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.322+0000 7f55dc102640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55c00a2e00 0x7f55c00a5310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f55d4068c80 tx=0x7f55c4006bd0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.322+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d410f9e0 msgr2=0x7f55d41ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.322+0000 7f55dc102640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d410f9e0 0x7f55d41ad850 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f55d0033630 tx=0x7f55d0033660 comp rx=0 tx=0).stop 2024-05-21T15:19:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.324+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 shutdown_connections 2024-05-21T15:19:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.325+0000 7f55dc102640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55c00a2e00 0x7f55c00a5310 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.325+0000 7f55dc102640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55d4114290 0x7f55d410d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.325+0000 7f55dc102640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55d410f9e0 0x7f55d41ad850 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.325+0000 7f55dc102640 1 --2- 172.21.15.78:0/13142348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55d4109d70 0x7f55d41ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.325+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 >> 172.21.15.78:0/13142348 conn(0x7f55d406d760 msgr2=0x7f55d4072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.325+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 shutdown_connections 2024-05-21T15:19:30.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.325+0000 7f55dc102640 1 -- 172.21.15.78:0/13142348 wait complete. 2024-05-21T15:19:30.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:30 smithi078 ceph-mon[31997]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 70 MiB/s wr, 12.36k op/s 2024-05-21T15:19:30.346 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:19:30.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:30 smithi084 ceph-mon[36334]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 70 MiB/s wr, 12.36k op/s 2024-05-21T15:19:30.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:30 smithi172 ceph-mon[36402]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 70 MiB/s wr, 12.36k op/s 2024-05-21T15:19:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.536+0000 7f83f2468640 1 -- 172.21.15.78:0/972863301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec1140b0 msgr2=0x7f83ec1164d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.536+0000 7f83f2468640 1 --2- 172.21.15.78:0/972863301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec1140b0 0x7f83ec1164d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009ca0 tx=0x7f83e402a8d0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.538+0000 7f83f2468640 1 -- 172.21.15.78:0/972863301 shutdown_connections 2024-05-21T15:19:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.538+0000 7f83f2468640 1 --2- 172.21.15.78:0/972863301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec1140b0 0x7f83ec1164d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.538+0000 7f83f2468640 1 --2- 172.21.15.78:0/972863301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec109d70 0x7f83ec10a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.538+0000 7f83f2468640 1 --2- 172.21.15.78:0/972863301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83ec10f890 0x7f83ec109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.538+0000 7f83f2468640 1 -- 172.21.15.78:0/972863301 >> 172.21.15.78:0/972863301 conn(0x7f83ec06e690 msgr2=0x7f83ec06eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.542+0000 7f83f2468640 1 -- 172.21.15.78:0/972863301 shutdown_connections 2024-05-21T15:19:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.542+0000 7f83f2468640 1 -- 172.21.15.78:0/972863301 wait complete. 2024-05-21T15:19:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.543+0000 7f83f2468640 1 Processor -- start 2024-05-21T15:19:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.543+0000 7f83f2468640 1 -- start start 2024-05-21T15:19:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f2468640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec109d70 0x7f83ec1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f2468640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec10f890 0x7f83ec1a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f2468640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83ec1140b0 0x7f83ec1aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f2468640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec1193b0 con 0x7f83ec1140b0 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f2468640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec119230 con 0x7f83ec10f890 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f2468640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec119530 con 0x7f83ec109d70 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f0c65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec10f890 0x7f83ec1a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f0c65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec10f890 0x7f83ec1a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45768/0 (socket says 172.21.15.78:45768) 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f0c65640 1 -- 172.21.15.78:0/2680539062 learned_addr learned my addr 172.21.15.78:0/2680539062 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f1466640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec109d70 0x7f83ec1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f0c65640 1 -- 172.21.15.78:0/2680539062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec109d70 msgr2=0x7f83ec1a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f0c65640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec109d70 0x7f83ec1a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f0c65640 1 -- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83ec1140b0 msgr2=0x7f83ec1aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.544+0000 7f83f0c65640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83ec1140b0 0x7f83ec1aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.545+0000 7f83f0c65640 1 -- 172.21.15.78:0/2680539062 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec1aad10 con 0x7f83ec10f890 2024-05-21T15:19:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.545+0000 7f83f0c65640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec10f890 0x7f83ec1a4d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00af30 tx=0x7f83dc00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.545+0000 7f83e27fc640 1 -- 172.21.15.78:0/2680539062 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc03d070 con 0x7f83ec10f890 2024-05-21T15:19:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.545+0000 7f83e27fc640 1 -- 172.21.15.78:0/2680539062 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc00cc70 con 0x7f83ec10f890 2024-05-21T15:19:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.545+0000 7f83e27fc640 1 -- 172.21.15.78:0/2680539062 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc00b5c0 con 0x7f83ec10f890 2024-05-21T15:19:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.546+0000 7f83f2468640 1 -- 172.21.15.78:0/2680539062 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec1ab000 con 0x7f83ec10f890 2024-05-21T15:19:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.546+0000 7f83f2468640 1 -- 172.21.15.78:0/2680539062 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec19fb90 con 0x7f83ec10f890 2024-05-21T15:19:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.552+0000 7f83fa571740 1 Processor -- start 2024-05-21T15:19:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.553+0000 7f83fa571740 1 -- start start 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.553+0000 7f83fa571740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789ffa1b0 0x55c78a399290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.553+0000 7f83fa571740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 0x55c78a3997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.553+0000 7f83fa571740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd94d0 0x55c78a399d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.553+0000 7f83fa571740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c78a2fe340 con 0x55c789fd12f0 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.553+0000 7f83fa571740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c78a2a6290 con 0x55c789ffa1b0 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.553+0000 7f83fa571740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c78a2fa300 con 0x55c789fd94d0 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1466640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 0x55c78a3997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1466640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 0x55c78a3997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36732/0 (socket says 172.21.15.78:36732) 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1466640 1 -- 172.21.15.78:0/1378201905 learned_addr learned my addr 172.21.15.78:0/1378201905 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1378201905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789ffa1b0 0x55c78a399290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83e27fc640 1 -- 172.21.15.78:0/2680539062 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83dc040070 con 0x7f83ec10f890 2024-05-21T15:19:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1c67640 1 -- 172.21.15.78:0/1378201905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd94d0 msgr2=0x55c78a399d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1378201905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd94d0 0x55c78a399d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1c67640 1 -- 172.21.15.78:0/1378201905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 msgr2=0x55c78a3997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1378201905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 0x55c78a3997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.554+0000 7f83f1c67640 1 -- 172.21.15.78:0/1378201905 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78a2ca8e0 con 0x55c789ffa1b0 2024-05-21T15:19:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.555+0000 7f83e27fc640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83d80a2dc0 0x7f83d80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.556+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1378201905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789ffa1b0 0x55c78a399290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009bf0 tx=0x7f83e402f320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de79f401b7210346 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.556+0000 7f83f1466640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83d80a2dc0 0x7f83d80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.556+0000 7f83d77fe640 1 -- 172.21.15.78:0/1378201905 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4031d80 con 0x55c789ffa1b0 2024-05-21T15:19:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.556+0000 7f83d77fe640 1 -- 172.21.15.78:0/1378201905 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e403f070 con 0x55c789ffa1b0 2024-05-21T15:19:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.556+0000 7f83d77fe640 1 -- 172.21.15.78:0/1378201905 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e40042a0 con 0x55c789ffa1b0 2024-05-21T15:19:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.556+0000 7f83e27fc640 1 -- 172.21.15.78:0/2680539062 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f83dc0bb4c0 con 0x7f83ec10f890 2024-05-21T15:19:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.558+0000 7f83fa571740 1 -- 172.21.15.78:0/1378201905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789ffa1b0 msgr2=0x55c78a399290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.558+0000 7f83fa571740 1 --2- 172.21.15.78:0/1378201905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789ffa1b0 0x55c78a399290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009bf0 tx=0x7f83e402f320 comp rx=0 tx=0).stop 2024-05-21T15:19:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.558+0000 7f83fa571740 1 -- 172.21.15.78:0/1378201905 shutdown_connections 2024-05-21T15:19:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.558+0000 7f83fa571740 1 --2- 172.21.15.78:0/1378201905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd94d0 0x55c78a399d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.558+0000 7f83fa571740 1 --2- 172.21.15.78:0/1378201905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 0x55c78a3997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.558+0000 7f83fa571740 1 --2- 172.21.15.78:0/1378201905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789ffa1b0 0x55c78a399290 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.558+0000 7f83fa571740 1 -- 172.21.15.78:0/1378201905 >> 172.21.15.78:0/1378201905 conn(0x55c789fee300 msgr2=0x55c78a395a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.560+0000 7f83fa571740 1 -- 172.21.15.78:0/1378201905 shutdown_connections 2024-05-21T15:19:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.562+0000 7f83fa571740 1 -- 172.21.15.78:0/1378201905 wait complete. 2024-05-21T15:19:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.563+0000 7f83fa571740 1 -- start start 2024-05-21T15:19:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.563+0000 7f83fa571740 1 Processor -- start 2024-05-21T15:19:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.563+0000 7f83f1466640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83d80a2dc0 0x7f83d80a52d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55c78a39a640 tx=0x7f83e8009110 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.564+0000 7f83fa571740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd12f0 0x55c78a3a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.564+0000 7f83fa571740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 0x55c78a3a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.564+0000 7f83fa571740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789ffa1b0 0x55c78a3a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.564+0000 7f83f1466640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd12f0 0x55c78a3a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.564+0000 7f83f1466640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd12f0 0x55c78a3a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:43956/0 (socket says 172.21.15.78:43956) 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.564+0000 7f83f0c65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 0x55c78a3a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.564+0000 7f83f0c65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 0x55c78a3a58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45786/0 (socket says 172.21.15.78:45786) 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83f1466640 1 -- 172.21.15.78:0/4125918148 learned_addr learned my addr 172.21.15.78:0/4125918148 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c78a3a6360 con 0x55c789ffa1b0 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c78a2b9790 con 0x55c789fd94d0 2024-05-21T15:19:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c78a3a64e0 con 0x55c789fd12f0 2024-05-21T15:19:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83fa571740 10 client.0 _ll_register_callbacks cb 0x55c78a2f4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83f1466640 1 -- 172.21.15.78:0/4125918148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 msgr2=0x55c78a3a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83f1466640 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 0x55c78a3a58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83f1466640 1 -- 172.21.15.78:0/4125918148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789ffa1b0 msgr2=0x55c78a3a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.565+0000 7f83f1466640 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789ffa1b0 0x55c78a3a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.566+0000 7f83f1466640 1 -- 172.21.15.78:0/4125918148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78a012210 con 0x55c789fd12f0 2024-05-21T15:19:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.567+0000 7f83f1466640 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd12f0 0x55c78a3a5340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009680 tx=0x7f83e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9f786ceac2d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.570+0000 7f83d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:19:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.570+0000 7f83d4ff9640 1 -- 172.21.15.78:0/4125918148 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8025070 con 0x55c789fd12f0 2024-05-21T15:19:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.570+0000 7f83d4ff9640 1 -- 172.21.15.78:0/4125918148 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f83e80216c0 con 0x55c789fd12f0 2024-05-21T15:19:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.570+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e403f070 con 0x55c789fd12f0 2024-05-21T15:19:30.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.570+0000 7fc971b31640 1 -- 172.21.15.78:0/4017210253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 msgr2=0x7fc96c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.570+0000 7fc96a7fc640 1 -- 172.21.15.78:0/4017210253 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c035020 con 0x7fc96c10d8f0 2024-05-21T15:19:30.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.570+0000 7fc971b31640 1 --2- 172.21.15.78:0/4017210253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 0x7fc96c10dd20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009900 tx=0x7fc95c02a3f0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.571+0000 7f83d4ff9640 1 -- 172.21.15.78:0/4125918148 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f83e802f050 con 0x55c789fd12f0 2024-05-21T15:19:30.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.571+0000 7fc971b31640 1 -- 172.21.15.78:0/4017210253 shutdown_connections 2024-05-21T15:19:30.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.571+0000 7fc971b31640 1 --2- 172.21.15.78:0/4017210253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 0x7fc96c10dd20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.571+0000 7fc971b31640 1 --2- 172.21.15.78:0/4017210253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc96c10e3b0 0x7fc96c115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.571+0000 7fc971b31640 1 --2- 172.21.15.78:0/4017210253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c109360 0x7fc96c109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.571+0000 7fc971b31640 1 -- 172.21.15.78:0/4017210253 >> 172.21.15.78:0/4017210253 conn(0x7fc96c075fa0 msgr2=0x7fc96c076400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.571+0000 7fc971b31640 1 -- 172.21.15.78:0/4017210253 shutdown_connections 2024-05-21T15:19:30.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.571+0000 7fc971b31640 1 -- 172.21.15.78:0/4017210253 wait complete. 2024-05-21T15:19:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.572+0000 7f83fa571740 10 client.26050 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.572+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c78a0de620 con 0x55c789fd12f0 2024-05-21T15:19:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.572+0000 7f83d4ff9640 1 -- 172.21.15.78:0/4125918148 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e403f070 con 0x55c789fd12f0 2024-05-21T15:19:30.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.572+0000 7fc971b31640 1 Processor -- start 2024-05-21T15:19:30.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.572+0000 7fc971b31640 1 -- start start 2024-05-21T15:19:30.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.573+0000 7fc971b31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c109360 0x7fc96c083540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.573+0000 7fc971b31640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 0x7fc96c07b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.574+0000 7f83d4ff9640 1 -- 172.21.15.78:0/4125918148 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f83e8020070 con 0x55c789fd12f0 2024-05-21T15:19:30.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.573+0000 7fc971b31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc96c10e3b0 0x7fc96c07bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.573+0000 7fc971b31640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c119430 con 0x7fc96c10e3b0 2024-05-21T15:19:30.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.573+0000 7fc971b31640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1192b0 con 0x7fc96c10d8f0 2024-05-21T15:19:30.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.573+0000 7fc971b31640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1195b0 con 0x7fc96c109360 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.574+0000 7f83fa571740 10 client.26050 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.574+0000 7f83fa571740 10 client.26050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.574+0000 7f83fa571740 20 client.26050 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.574+0000 7f83fa571740 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c789fb8d80 0x55c78a3b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.574+0000 7f83fa571740 4 client.26050 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.574+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c78a30ba70 con 0x55c789fb8d80 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.574+0000 7f83f0c65640 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c789fb8d80 0x55c78a3b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.575+0000 7f83f0c65640 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c789fb8d80 0x55c78a3b6150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15bff8a185b8db25 server_cookie=8f0ca3b4516f0df4 in_seq=0 out_seq=0 2024-05-21T15:19:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.575+0000 7f83d4ff9640 10 client.26050 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.576+0000 7f83d4ff9640 1 -- 172.21.15.78:0/4125918148 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c78a30ba70 con 0x55c789fb8d80 2024-05-21T15:19:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.576+0000 7f83d4ff9640 10 client.26050 handle_command_reply: tid=0 2024-05-21T15:19:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 client.26050 shutdown 2024-05-21T15:19:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd12f0 msgr2=0x55c78a3a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd12f0 0x55c78a3a5340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009680 tx=0x7f83e80046c0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 shutdown_connections 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c789fb8d80 0x55c78a3b6150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 0x55c78a3a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789ffa1b0 0x55c78a3a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 --2- 172.21.15.78:0/4125918148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789fd12f0 0x55c78a3a5340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 >> 172.21.15.78:0/4125918148 conn(0x55c789fee300 msgr2=0x55c78a399060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83d4ff9640 0 client.26050 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 shutdown_connections 2024-05-21T15:19:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.577+0000 7f83fa571740 1 -- 172.21.15.78:0/4125918148 wait complete. 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.574+0000 7fc96b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c109360 0x7fc96c083540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.574+0000 7fc96b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c109360 0x7fc96c083540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:43988/0 (socket says 172.21.15.78:43988) 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.574+0000 7fc96b7fe640 1 -- 172.21.15.78:0/497351643 learned_addr learned my addr 172.21.15.78:0/497351643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.574+0000 7fc96affd640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 0x7fc96c07b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.575+0000 7fc96b7fe640 1 -- 172.21.15.78:0/497351643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 msgr2=0x7fc96c07b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.575+0000 7fc96b7fe640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 0x7fc96c07b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.575+0000 7fc96b7fe640 1 -- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc96c10e3b0 msgr2=0x7fc96c07bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.575+0000 7fc96b7fe640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc96c10e3b0 0x7fc96c07bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.575+0000 7fc96b7fe640 1 -- 172.21.15.78:0/497351643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c07c210 con 0x7fc96c109360 2024-05-21T15:19:30.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.575+0000 7fc96affd640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 0x7fc96c07b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:19:30.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.575+0000 7fc96b7fe640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c109360 0x7fc96c083540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc96000e9f0 tx=0x7fc96000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.578+0000 7fc968ff9640 1 -- 172.21.15.78:0/497351643 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96000cd80 con 0x7fc96c109360 2024-05-21T15:19:30.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.578+0000 7fc968ff9640 1 -- 172.21.15.78:0/497351643 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc960010910 con 0x7fc96c109360 2024-05-21T15:19:30.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.578+0000 7fc968ff9640 1 -- 172.21.15.78:0/497351643 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96000cd80 con 0x7fc96c109360 2024-05-21T15:19:30.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.581+0000 7fc971b31640 1 -- 172.21.15.78:0/497351643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c1b4b50 con 0x7fc96c109360 2024-05-21T15:19:30.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.581+0000 7fc971b31640 1 -- 172.21.15.78:0/497351643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c1b4fc0 con 0x7fc96c109360 2024-05-21T15:19:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.584+0000 7f83fa571740 20 client.26050 trim_cache size 0 max 16384 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.586+0000 7fc9789dd740 1 Processor -- start 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.586+0000 7fc9789dd740 1 -- start start 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c53093d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c53094260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 0x561c530947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc9789dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c52ff9c40 con 0x561c52cd46b0 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc9789dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561c52fa1580 con 0x561c52cf5390 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc9789dd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561c52ff5c00 con 0x561c52ccc4d0 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc96bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c53093d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc96bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c53093d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45828/0 (socket says 172.21.15.78:45828) 2024-05-21T15:19:30.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.587+0000 7fc96bfff640 1 -- 172.21.15.78:0/2453578958 learned_addr learned my addr 172.21.15.78:0/2453578958 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.588+0000 7fc96affd640 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 0x561c530947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.588+0000 7fc96bfff640 1 -- 172.21.15.78:0/2453578958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 msgr2=0x561c53094260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:30.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.588+0000 7fc96bfff640 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c53094260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.588+0000 7fc96bfff640 1 -- 172.21.15.78:0/2453578958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 msgr2=0x561c530947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.588+0000 7fc96bfff640 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 0x561c530947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.588+0000 7fc96bfff640 1 -- 172.21.15.78:0/2453578958 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c52d0d3f0 con 0x561c52cf5390 2024-05-21T15:19:30.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.589+0000 7fc96affd640 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 0x561c530947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:19:30.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.589+0000 7fc96bfff640 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c53093d20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc964007bb0 tx=0x7fc96400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6871854fc809eadb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.589+0000 7fc968ff9640 1 -- 172.21.15.78:0/497351643 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc96001f020 con 0x7fc96c109360 2024-05-21T15:19:30.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.591+0000 7fc968ff9640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc9540a3130 0x7fc9540a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.591+0000 7fc968ff9640 1 -- 172.21.15.78:0/497351643 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc96000cb80 con 0x7fc96c109360 2024-05-21T15:19:30.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.591+0000 7fc96b7fe640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc9540a3130 0x7fc9540a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.592+0000 7fc96b7fe640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc9540a3130 0x7fc9540a5640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc960095fc0 tx=0x7fc96009fa90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.595+0000 7fc951ffb640 1 -- 172.21.15.78:0/2453578958 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964014070 con 0x561c52cf5390 2024-05-21T15:19:30.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.595+0000 7fc951ffb640 1 -- 172.21.15.78:0/2453578958 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96400fb30 con 0x561c52cf5390 2024-05-21T15:19:30.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.595+0000 7fc951ffb640 1 -- 172.21.15.78:0/2453578958 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96400eaf0 con 0x561c52cf5390 2024-05-21T15:19:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.597+0000 7fc9789dd740 1 -- 172.21.15.78:0/2453578958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 msgr2=0x561c53093d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.597+0000 7fc9789dd740 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c53093d20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc964007bb0 tx=0x7fc96400c6a0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.597+0000 7fc9789dd740 1 -- 172.21.15.78:0/2453578958 shutdown_connections 2024-05-21T15:19:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.597+0000 7fc9789dd740 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 0x561c530947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.597+0000 7fc9789dd740 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c53094260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.597+0000 7fc9789dd740 1 --2- 172.21.15.78:0/2453578958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c53093d20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.597+0000 7fc9789dd740 1 -- 172.21.15.78:0/2453578958 >> 172.21.15.78:0/2453578958 conn(0x561c52ce94e0 msgr2=0x561c530904d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.599+0000 7fc9789dd740 1 -- 172.21.15.78:0/2453578958 shutdown_connections 2024-05-21T15:19:30.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.599+0000 7fc9789dd740 1 -- 172.21.15.78:0/2453578958 wait complete. 2024-05-21T15:19:30.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.599+0000 7fc9789dd740 1 -- start start 2024-05-21T15:19:30.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.600+0000 7fc9789dd740 1 Processor -- start 2024-05-21T15:19:30.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c5309e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 0x561c5309e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc96bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c5309e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc96bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c5309e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:43994/0 (socket says 172.21.15.78:43994) 2024-05-21T15:19:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc96bfff640 1 -- 172.21.15.78:0/49993502 learned_addr learned my addr 172.21.15.78:0/49993502 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc9789dd740 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c530aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c530a1010 con 0x561c52cd46b0 2024-05-21T15:19:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561c52fb4c40 con 0x561c52cf5390 2024-05-21T15:19:30.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.601+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561c530a1190 con 0x561c52ccc4d0 2024-05-21T15:19:30.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc9789dd740 10 client.0 _ll_register_callbacks cb 0x561c52d0cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc96bfff640 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c530aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc96bfff640 1 -- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 msgr2=0x561c530aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc96bfff640 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c530aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc96bfff640 1 -- 172.21.15.78:0/49993502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 msgr2=0x561c5309e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc96bfff640 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 0x561c5309e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc96bfff640 1 -- 172.21.15.78:0/49993502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c52dd77d0 con 0x561c52ccc4d0 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc96400fb30 con 0x561c52ccc4d0 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.602+0000 7fc96bfff640 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c5309e340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc96400e590 tx=0x7fc96400e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3935229cba40388 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.603+0000 7fc94affd640 10 client.26059 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.603+0000 7fc94affd640 1 -- 172.21.15.78:0/49993502 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96402d050 con 0x561c52ccc4d0 2024-05-21T15:19:30.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.603+0000 7fc94affd640 1 -- 172.21.15.78:0/49993502 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc96400f650 con 0x561c52ccc4d0 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.603+0000 7fc94affd640 1 -- 172.21.15.78:0/49993502 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc964014070 con 0x561c52ccc4d0 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.603+0000 7fc9789dd740 10 client.26059 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.603+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=31}) -- 0x561c530ad590 con 0x561c52ccc4d0 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.604+0000 7fc94affd640 1 -- 172.21.15.78:0/49993502 <== mon.1 v2:172.21.15.172:3300/0 4 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fc964030020 con 0x561c52ccc4d0 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.604+0000 7fc9789dd740 10 client.26059 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.604+0000 7fc9789dd740 10 client.26059 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.604+0000 7fc9789dd740 20 client.26059 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.604+0000 7fc9789dd740 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561c52cb3f60 0x561c530b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.604+0000 7fc9789dd740 4 client.26059 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.604+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c53007370 con 0x561c52cb3f60 2024-05-21T15:19:30.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.605+0000 7fc96affd640 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561c52cb3f60 0x561c530b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.606+0000 7fc96affd640 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561c52cb3f60 0x561c530b0f10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65b23632974132a8 server_cookie=85953b21fa67d265 in_seq=0 out_seq=0 2024-05-21T15:19:30.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.606+0000 7fc94affd640 10 client.26059 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:19:30.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.607+0000 7fc94affd640 1 -- 172.21.15.78:0/49993502 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c53007370 con 0x561c52cb3f60 2024-05-21T15:19:30.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.607+0000 7fc94affd640 10 client.26059 handle_command_reply: tid=0 2024-05-21T15:19:30.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.607+0000 7fc9789dd740 1 client.26059 shutdown 2024-05-21T15:19:30.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.608+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 msgr2=0x561c5309e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.608+0000 7fc9789dd740 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c5309e340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc96400e590 tx=0x7fc96400e7c0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.610+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 shutdown_connections 2024-05-21T15:19:30.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.610+0000 7fc9789dd740 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561c52cb3f60 0x561c530b0f10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.610+0000 7fc9789dd740 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cf5390 0x561c530aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.610+0000 7fc9789dd740 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cd46b0 0x561c5309e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.610+0000 7fc9789dd740 1 --2- 172.21.15.78:0/49993502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52ccc4d0 0x561c5309e340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.610+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 >> 172.21.15.78:0/49993502 conn(0x561c52ce94e0 msgr2=0x561c53095a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.611+0000 7fc94affd640 0 client.26059 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:19:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.611+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 shutdown_connections 2024-05-21T15:19:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.611+0000 7fc9789dd740 1 -- 172.21.15.78:0/49993502 wait complete. 2024-05-21T15:19:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.612+0000 7fc9789dd740 20 client.26059 trim_cache size 0 max 16384 2024-05-21T15:19:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.621+0000 7f83fa571740 1 Processor -- start 2024-05-21T15:19:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.622+0000 7f83fa571740 1 -- start start 2024-05-21T15:19:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.622+0000 7f83fa571740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd12f0 0x55c78a39d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.622+0000 7f83fa571740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd94d0 0x55c78a3bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.622+0000 7f83f1466640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd12f0 0x55c78a39d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.623+0000 7f83f1466640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd12f0 0x55c78a39d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45856/0 (socket says 172.21.15.78:45856) 2024-05-21T15:19:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.623+0000 7f83f1466640 1 -- 172.21.15.78:0/1127498087 learned_addr learned my addr 172.21.15.78:0/1127498087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.623+0000 7f83fa571740 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a3bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.623+0000 7f83fa571740 1 -- 172.21.15.78:0/1127498087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec07b750 con 0x55c789fd94d0 2024-05-21T15:19:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.623+0000 7f83fa571740 1 -- 172.21.15.78:0/1127498087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec07d440 con 0x55c789fd12f0 2024-05-21T15:19:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.623+0000 7f83fa571740 1 -- 172.21.15.78:0/1127498087 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec07be20 con 0x55c789ffa1b0 2024-05-21T15:19:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.624+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a3bbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.627 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.624+0000 7f83f0c65640 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd94d0 0x55c78a3bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.625+0000 7f83f1c67640 1 -- 172.21.15.78:0/1127498087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd12f0 msgr2=0x55c78a39d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.625+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd12f0 0x55c78a39d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.625+0000 7f83f1c67640 1 -- 172.21.15.78:0/1127498087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd94d0 msgr2=0x55c78a3bb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.625+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd94d0 0x55c78a3bb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.625+0000 7f83f1c67640 1 -- 172.21.15.78:0/1127498087 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c78a39a670 con 0x55c789ffa1b0 2024-05-21T15:19:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.626+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a3bbe50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f83e402afd0 tx=0x7f83e4031d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f532cdd1e38581 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.626+0000 7f83d6ffd640 1 -- 172.21.15.78:0/1127498087 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4003b70 con 0x55c789ffa1b0 2024-05-21T15:19:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.626+0000 7f83d6ffd640 1 -- 172.21.15.78:0/1127498087 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4049070 con 0x55c789ffa1b0 2024-05-21T15:19:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.626+0000 7f83d6ffd640 1 -- 172.21.15.78:0/1127498087 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4004530 con 0x55c789ffa1b0 2024-05-21T15:19:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.627+0000 7f83fa571740 1 -- 172.21.15.78:0/1127498087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 msgr2=0x55c78a3bbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.627+0000 7f83fa571740 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a3bbe50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f83e402afd0 tx=0x7f83e4031d80 comp rx=0 tx=0).stop 2024-05-21T15:19:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.628+0000 7f83fa571740 1 -- 172.21.15.78:0/1127498087 shutdown_connections 2024-05-21T15:19:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.628+0000 7f83fa571740 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a3bbe50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.628+0000 7f83fa571740 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd94d0 0x55c78a3bb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.628+0000 7f83fa571740 1 --2- 172.21.15.78:0/1127498087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd12f0 0x55c78a39d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.628+0000 7f83fa571740 1 -- 172.21.15.78:0/1127498087 >> 172.21.15.78:0/1127498087 conn(0x55c789fee300 msgr2=0x55c78a39cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.628+0000 7f83fa571740 1 -- 172.21.15.78:0/1127498087 shutdown_connections 2024-05-21T15:19:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.631+0000 7f83fa571740 1 -- 172.21.15.78:0/1127498087 wait complete. 2024-05-21T15:19:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.632+0000 7f83fa571740 1 -- start start 2024-05-21T15:19:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.632+0000 7f83fa571740 1 Processor -- start 2024-05-21T15:19:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83fa571740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 0x55c78a3aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83fa571740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 0x55c78a3ab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83fa571740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a416510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83fa571740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec07ccb0 con 0x55c789fd12f0 2024-05-21T15:19:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83fa571740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec07c5b0 con 0x55c789fd94d0 2024-05-21T15:19:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83fa571740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec0797e0 con 0x55c789ffa1b0 2024-05-21T15:19:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83fa571740 10 client.0 _ll_register_callbacks cb 0x7f83e8009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83f1c67640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a416510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83f1c67640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a416510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44004/0 (socket says 172.21.15.78:44004) 2024-05-21T15:19:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.633+0000 7f83f1c67640 1 -- 172.21.15.78:0/1484755292 learned_addr learned my addr 172.21.15.78:0/1484755292 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.634+0000 7f83f1c67640 1 -- 172.21.15.78:0/1484755292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 msgr2=0x55c78a3ab200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.634+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 0x55c78a3ab200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.634+0000 7f83f1c67640 1 -- 172.21.15.78:0/1484755292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 msgr2=0x55c78a3aacc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.634+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 0x55c78a3aacc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.634+0000 7f83f1c67640 1 -- 172.21.15.78:0/1484755292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78a416aa0 con 0x55c789ffa1b0 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.634+0000 7f83f1c67640 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a416510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f83e402ae10 tx=0x7f83e40046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a27b2363fd7cee server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.634+0000 7f83d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.635+0000 7f83d4ff9640 1 -- 172.21.15.78:0/1484755292 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4056070 con 0x55c789ffa1b0 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.635+0000 7f83d4ff9640 1 -- 172.21.15.78:0/1484755292 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f83e4003d80 con 0x55c789ffa1b0 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.635+0000 7f83fa571740 1 -- 172.21.15.78:0/1484755292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e4049070 con 0x55c789ffa1b0 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.635+0000 7f83d4ff9640 1 -- 172.21.15.78:0/1484755292 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f83e4062050 con 0x55c789ffa1b0 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.635+0000 7f83fa571740 10 client.26065 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.635+0000 7f83fa571740 1 -- 172.21.15.78:0/1484755292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c78a41ddf0 con 0x55c789ffa1b0 2024-05-21T15:19:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.635+0000 7f83d4ff9640 1 -- 172.21.15.78:0/1484755292 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4049070 con 0x55c789ffa1b0 2024-05-21T15:19:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.635+0000 7f83d4ff9640 1 -- 172.21.15.78:0/1484755292 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f83e402f920 con 0x55c789ffa1b0 2024-05-21T15:19:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.636+0000 7f83fa571740 10 client.26065 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.636+0000 7f83fa571740 10 client.26065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.636+0000 7f83fa571740 20 client.26065 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.636+0000 7f83fa571740 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c789f8e1c0 0x55c78a4200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.636+0000 7f83fa571740 4 client.26065 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:19:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.636+0000 7f83fa571740 1 -- 172.21.15.78:0/1484755292 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c78a305dc0 con 0x55c789f8e1c0 2024-05-21T15:19:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.636+0000 7f83f1466640 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c789f8e1c0 0x55c78a4200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.638+0000 7f83f1466640 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c789f8e1c0 0x55c78a4200a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46dfb4bdf628c838 server_cookie=2fe1f254f6523ffc in_seq=0 out_seq=0 2024-05-21T15:19:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.638+0000 7f83d4ff9640 10 client.26065 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.641+0000 7fc9789dd740 1 Processor -- start 2024-05-21T15:19:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.641+0000 7fc9789dd740 1 -- start start 2024-05-21T15:19:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.641+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52ccc4d0 0x561c530961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.641+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cd46b0 0x561c530aa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.641+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cf5390 0x561c530b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc9789dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1bdc00 con 0x561c52ccc4d0 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc9789dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1bf8f0 con 0x561c52cd46b0 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc9789dd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1be2d0 con 0x561c52cf5390 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc96affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cd46b0 0x561c530aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc96affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cd46b0 0x561c530aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45858/0 (socket says 172.21.15.78:45858) 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc96affd640 1 -- 172.21.15.78:0/1305441410 learned_addr learned my addr 172.21.15.78:0/1305441410 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc96affd640 1 -- 172.21.15.78:0/1305441410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cf5390 msgr2=0x561c530b6710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc96affd640 1 --2- 172.21.15.78:0/1305441410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cf5390 0x561c530b6710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc96affd640 1 -- 172.21.15.78:0/1305441410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52ccc4d0 msgr2=0x561c530961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc96affd640 1 --2- 172.21.15.78:0/1305441410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52ccc4d0 0x561c530961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.642+0000 7fc96affd640 1 -- 172.21.15.78:0/1305441410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c530b14e0 con 0x561c52cd46b0 2024-05-21T15:19:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.643+0000 7fc96affd640 1 --2- 172.21.15.78:0/1305441410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cd46b0 0x561c530aa060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009850 tx=0x7fc95c004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b7520125f4727da server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.644+0000 7fc9517fa640 1 -- 172.21.15.78:0/1305441410 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c035070 con 0x561c52cd46b0 2024-05-21T15:19:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.644+0000 7fc9517fa640 1 -- 172.21.15.78:0/1305441410 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c005510 con 0x561c52cd46b0 2024-05-21T15:19:30.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.644+0000 7fc9517fa640 1 -- 172.21.15.78:0/1305441410 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c004d70 con 0x561c52cd46b0 2024-05-21T15:19:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 -- 172.21.15.78:0/1305441410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cd46b0 msgr2=0x561c530aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 --2- 172.21.15.78:0/1305441410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cd46b0 0x561c530aa060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009850 tx=0x7fc95c004520 comp rx=0 tx=0).stop 2024-05-21T15:19:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 -- 172.21.15.78:0/1305441410 shutdown_connections 2024-05-21T15:19:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 --2- 172.21.15.78:0/1305441410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52cd46b0 0x561c530aa060 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 --2- 172.21.15.78:0/1305441410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cf5390 0x561c530b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 --2- 172.21.15.78:0/1305441410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52ccc4d0 0x561c530961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 -- 172.21.15.78:0/1305441410 >> 172.21.15.78:0/1305441410 conn(0x561c52ce94e0 msgr2=0x561c5309f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 -- 172.21.15.78:0/1305441410 shutdown_connections 2024-05-21T15:19:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.645+0000 7fc9789dd740 1 -- 172.21.15.78:0/1305441410 wait complete. 2024-05-21T15:19:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.646+0000 7fc9789dd740 1 -- start start 2024-05-21T15:19:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.646+0000 7fc9789dd740 1 Processor -- start 2024-05-21T15:19:30.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.647+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52ccc4d0 0x561c530a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.647+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cd46b0 0x561c530a6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.647+0000 7fc9789dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cf5390 0x561c530a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.647+0000 7fc96affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cd46b0 0x561c530a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.648+0000 7fc96affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cd46b0 0x561c530a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44026/0 (socket says 172.21.15.78:44026) 2024-05-21T15:19:30.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.648+0000 7fc96bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cf5390 0x561c530a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.649+0000 7fc9789dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1bf160 con 0x561c52cf5390 2024-05-21T15:19:30.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.649+0000 7fc9789dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1bea60 con 0x561c52ccc4d0 2024-05-21T15:19:30.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.649+0000 7fc9789dd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c1bbbe0 con 0x561c52cd46b0 2024-05-21T15:19:30.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.649+0000 7fc96bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cf5390 0x561c530a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36800/0 (socket says 172.21.15.78:36800) 2024-05-21T15:19:30.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.649+0000 7fc96bfff640 1 -- 172.21.15.78:0/975350887 learned_addr learned my addr 172.21.15.78:0/975350887 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.649+0000 7fc96bfff640 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52ccc4d0 0x561c530a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.649+0000 7fc9789dd740 10 client.0 _ll_register_callbacks cb 0x7fc964007a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:30.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.651+0000 7fc96affd640 1 -- 172.21.15.78:0/975350887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52ccc4d0 msgr2=0x561c530a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.651+0000 7fc96affd640 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52ccc4d0 0x561c530a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.651+0000 7fc96affd640 1 -- 172.21.15.78:0/975350887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cf5390 msgr2=0x561c530a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.651+0000 7fc96affd640 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cf5390 0x561c530a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.651+0000 7fc96affd640 1 -- 172.21.15.78:0/975350887 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c530a1ae0 con 0x561c52cd46b0 2024-05-21T15:19:30.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.653+0000 7fc96affd640 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cd46b0 0x561c530a6930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x561c53096750 tx=0x7fc95c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8d790eefd911e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.653+0000 7fc94affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:19:30.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.654+0000 7fc94affd640 1 -- 172.21.15.78:0/975350887 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c053060 con 0x561c52cd46b0 2024-05-21T15:19:30.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.654+0000 7fc94affd640 1 -- 172.21.15.78:0/975350887 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc95c003800 con 0x561c52cd46b0 2024-05-21T15:19:30.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.654+0000 7fc9789dd740 1 -- 172.21.15.78:0/975350887 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95c005510 con 0x561c52cd46b0 2024-05-21T15:19:30.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.654+0000 7fc94affd640 1 -- 172.21.15.78:0/975350887 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc95c035040 con 0x561c52cd46b0 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.654+0000 7fc9789dd740 10 client.26068 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.654+0000 7fc9789dd740 1 -- 172.21.15.78:0/975350887 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561c53117bf0 con 0x561c52cd46b0 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.655+0000 7fc94affd640 1 -- 172.21.15.78:0/975350887 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c0046c0 con 0x561c52cd46b0 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.655+0000 7fc94affd640 1 -- 172.21.15.78:0/975350887 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fc95c03a040 con 0x561c52cd46b0 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.655+0000 7fc9789dd740 10 client.26068 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.655+0000 7fc9789dd740 10 client.26068 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.655+0000 7fc9789dd740 20 client.26068 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.655+0000 7fc9789dd740 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561c52c893a0 0x561c5311a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.655+0000 7fc9789dd740 4 client.26068 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.656+0000 7fc9789dd740 1 -- 172.21.15.78:0/975350887 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc95c005510 con 0x561c52c893a0 2024-05-21T15:19:30.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.656+0000 7fc96bfff640 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561c52c893a0 0x561c5311a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.657+0000 7fc96bfff640 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561c52c893a0 0x561c5311a250 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b7937bef8923ea3 server_cookie=fbfa70dc7e2949d7 in_seq=0 out_seq=0 2024-05-21T15:19:30.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.657+0000 7fc94affd640 10 client.26068 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:19:30.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.657+0000 7fc94affd640 1 -- 172.21.15.78:0/975350887 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc95c005510 con 0x561c52c893a0 2024-05-21T15:19:30.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.657+0000 7fc94affd640 10 client.26068 handle_command_reply: tid=0 2024-05-21T15:19:30.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.657+0000 7fc9789dd740 1 client.26068 shutdown 2024-05-21T15:19:30.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 -- 172.21.15.78:0/975350887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cd46b0 msgr2=0x561c530a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cd46b0 0x561c530a6930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x561c53096750 tx=0x7fc95c004610 comp rx=0 tx=0).stop 2024-05-21T15:19:30.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 -- 172.21.15.78:0/975350887 shutdown_connections 2024-05-21T15:19:30.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561c52c893a0 0x561c5311a250 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c52cf5390 0x561c530a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc94affd640 0 client.26068 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:19:30.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c52cd46b0 0x561c530a6930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 --2- 172.21.15.78:0/975350887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561c52ccc4d0 0x561c530a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 -- 172.21.15.78:0/975350887 >> 172.21.15.78:0/975350887 conn(0x561c52ce94e0 msgr2=0x561c53095f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.658+0000 7fc9789dd740 1 -- 172.21.15.78:0/975350887 shutdown_connections 2024-05-21T15:19:30.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.659+0000 7fc9789dd740 1 -- 172.21.15.78:0/975350887 wait complete. 2024-05-21T15:19:30.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.659+0000 7fc9789dd740 20 client.26068 trim_cache size 0 max 16384 2024-05-21T15:19:30.663 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:19:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.661+0000 7fc971b31640 1 -- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc9540a3130 msgr2=0x7fc9540a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc9540a3130 0x7fc9540a5640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc960095fc0 tx=0x7fc96009fa90 comp rx=0 tx=0).stop 2024-05-21T15:19:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 -- 172.21.15.78:0/497351643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c109360 msgr2=0x7fc96c083540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c109360 0x7fc96c083540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc96000e9f0 tx=0x7fc96000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 -- 172.21.15.78:0/497351643 shutdown_connections 2024-05-21T15:19:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96c10d8f0 0x7fc96c07b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc9540a3130 0x7fc9540a5640 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc96c10e3b0 0x7fc96c07bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 --2- 172.21.15.78:0/497351643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96c109360 0x7fc96c083540 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 -- 172.21.15.78:0/497351643 >> 172.21.15.78:0/497351643 conn(0x7fc96c075fa0 msgr2=0x7fc96c0792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.662+0000 7fc971b31640 1 -- 172.21.15.78:0/497351643 shutdown_connections 2024-05-21T15:19:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.663+0000 7fc971b31640 1 -- 172.21.15.78:0/497351643 wait complete. 2024-05-21T15:19:30.687 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:19:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.732+0000 7f83d4ff9640 1 -- 172.21.15.78:0/1484755292 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c78a305dc0 con 0x55c789f8e1c0 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.732+0000 7f83d4ff9640 10 client.26065 handle_command_reply: tid=0 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.732+0000 7f83fa571740 1 client.26065 shutdown 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 -- 172.21.15.78:0/1484755292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 msgr2=0x55c78a416510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a416510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f83e402ae10 tx=0x7f83e40046e0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 -- 172.21.15.78:0/1484755292 shutdown_connections 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c789f8e1c0 0x55c78a4200a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c789fd94d0 0x55c78a3ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c789ffa1b0 0x55c78a416510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 --2- 172.21.15.78:0/1484755292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c789fd12f0 0x55c78a3aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 -- 172.21.15.78:0/1484755292 >> 172.21.15.78:0/1484755292 conn(0x55c789fee300 msgr2=0x55c78a399020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83d4ff9640 0 client.26065 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 -- 172.21.15.78:0/1484755292 shutdown_connections 2024-05-21T15:19:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 1 -- 172.21.15.78:0/1484755292 wait complete. 2024-05-21T15:19:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.733+0000 7f83fa571740 20 client.26065 trim_cache size 0 max 16384 2024-05-21T15:19:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:19:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:19:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:19:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:19:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.736+0000 7f83f2468640 1 -- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83d80a2dc0 msgr2=0x7f83d80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.736+0000 7f83f2468640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83d80a2dc0 0x7f83d80a52d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55c78a39a640 tx=0x7f83e8009110 comp rx=0 tx=0).stop 2024-05-21T15:19:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.736+0000 7f83f2468640 1 -- 172.21.15.78:0/2680539062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec10f890 msgr2=0x7f83ec1a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.736+0000 7f83f2468640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec10f890 0x7f83ec1a4d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00af30 tx=0x7f83dc00ee60 comp rx=0 tx=0).stop 2024-05-21T15:19:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.736+0000 7f83f2468640 1 -- 172.21.15.78:0/2680539062 shutdown_connections 2024-05-21T15:19:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.736+0000 7f83f2468640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83ec10f890 0x7f83ec1a4d30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.736+0000 7f83f2468640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83d80a2dc0 0x7f83d80a52d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.737+0000 7f83f2468640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83ec1140b0 0x7f83ec1aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.737+0000 7f83f2468640 1 --2- 172.21.15.78:0/2680539062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec109d70 0x7f83ec1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.737+0000 7f83f2468640 1 -- 172.21.15.78:0/2680539062 >> 172.21.15.78:0/2680539062 conn(0x7f83ec06e690 msgr2=0x7f83ec110070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.738+0000 7f83f2468640 1 -- 172.21.15.78:0/2680539062 shutdown_connections 2024-05-21T15:19:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:30.738+0000 7f83f2468640 1 -- 172.21.15.78:0/2680539062 wait complete. 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.860+0000 7f3896291640 1 -- 172.21.15.78:0/325668600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f389010c330 msgr2=0x7f389010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.860+0000 7f3896291640 1 --2- 172.21.15.78:0/325668600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f389010c330 0x7f389010e750 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3880009860 tx=0x7f388002a360 comp rx=0 tx=0).stop 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.861+0000 7f3896291640 1 -- 172.21.15.78:0/325668600 shutdown_connections 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.861+0000 7f3896291640 1 --2- 172.21.15.78:0/325668600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3890101aa0 0x7f3890101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.861+0000 7f3896291640 1 --2- 172.21.15.78:0/325668600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f389010c330 0x7f389010e750 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.861+0000 7f3896291640 1 --2- 172.21.15.78:0/325668600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38901079d0 0x7f3890101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.861+0000 7f3896291640 1 -- 172.21.15.78:0/325668600 >> 172.21.15.78:0/325668600 conn(0x7f38900770b0 msgr2=0x7f38900774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.861+0000 7f3896291640 1 -- 172.21.15.78:0/325668600 shutdown_connections 2024-05-21T15:19:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.861+0000 7f3896291640 1 -- 172.21.15.78:0/325668600 wait complete. 2024-05-21T15:19:30.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.862+0000 7f3896291640 1 Processor -- start 2024-05-21T15:19:30.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.862+0000 7f3896291640 1 -- start start 2024-05-21T15:19:30.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.862+0000 7f3896291640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3890101aa0 0x7f389019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f3896291640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38901079d0 0x7f389019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f3896291640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389010c330 0x7f389019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f3896291640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3890111ce0 con 0x7f38901079d0 2024-05-21T15:19:30.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f3896291640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3890111b60 con 0x7f389010c330 2024-05-21T15:19:30.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f3896291640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3890111e60 con 0x7f3890101aa0 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3890101aa0 0x7f389019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3890101aa0 0x7f389019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44046/0 (socket says 172.21.15.78:44046) 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 -- 172.21.15.78:0/3828143482 learned_addr learned my addr 172.21.15.78:0/3828143482 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38901079d0 0x7f389019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f389482e640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389010c330 0x7f389019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 -- 172.21.15.78:0/3828143482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389010c330 msgr2=0x7f389019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389010c330 0x7f389019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 -- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38901079d0 msgr2=0x7f389019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38901079d0 0x7f389019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 -- 172.21.15.78:0/3828143482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38901a5850 con 0x7f3890101aa0 2024-05-21T15:19:30.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.863+0000 7f388ffff640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3890101aa0 0x7f389019a540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f388800c3a0 tx=0x7f388800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.864+0000 7f388d7fa640 1 -- 172.21.15.78:0/3828143482 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388801b070 con 0x7f3890101aa0 2024-05-21T15:19:30.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.864+0000 7f388d7fa640 1 -- 172.21.15.78:0/3828143482 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f388800bb10 con 0x7f3890101aa0 2024-05-21T15:19:30.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.864+0000 7f388d7fa640 1 -- 172.21.15.78:0/3828143482 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888016c20 con 0x7f3890101aa0 2024-05-21T15:19:30.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.864+0000 7f3896291640 1 -- 172.21.15.78:0/3828143482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38901a5b40 con 0x7f3890101aa0 2024-05-21T15:19:30.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.864+0000 7f3896291640 1 -- 172.21.15.78:0/3828143482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38901a6040 con 0x7f3890101aa0 2024-05-21T15:19:30.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.868+0000 7f389d20e740 1 Processor -- start 2024-05-21T15:19:30.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.868+0000 7f389d20e740 1 -- start start 2024-05-21T15:19:30.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.868+0000 7f389d20e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559effa2390 0x5559f0340d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.868+0000 7f389d20e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 0x5559f0341260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.868+0000 7f389d20e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 0x5559f03417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.868+0000 7f389d20e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5559f02a6c40 con 0x5559eff816b0 2024-05-21T15:19:30.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.868+0000 7f389d20e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5559f024e580 con 0x5559eff794d0 2024-05-21T15:19:30.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.868+0000 7f389d20e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559f02a2c00 con 0x5559effa2390 2024-05-21T15:19:30.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f389482e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559effa2390 0x5559f0340d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f389482e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559effa2390 0x5559f0340d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44054/0 (socket says 172.21.15.78:44054) 2024-05-21T15:19:30.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f389482e640 1 -- 172.21.15.78:0/902158917 learned_addr learned my addr 172.21.15.78:0/902158917 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f388f7fe640 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 0x5559f03417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f388d7fa640 1 -- 172.21.15.78:0/3828143482 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f388801e070 con 0x7f3890101aa0 2024-05-21T15:19:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f389482e640 1 -- 172.21.15.78:0/902158917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 msgr2=0x5559f0341260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:19:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f389482e640 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 0x5559f0341260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f389482e640 1 -- 172.21.15.78:0/902158917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 msgr2=0x5559f03417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f389482e640 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 0x5559f03417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f389482e640 1 -- 172.21.15.78:0/902158917 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559effba3f0 con 0x5559effa2390 2024-05-21T15:19:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.869+0000 7f388f7fe640 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 0x5559f03417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.870+0000 7f389482e640 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559effa2390 0x5559f0340d20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3880034000 tx=0x7f3880005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb09ee6725e1b40f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.870+0000 7f388d7fa640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f38780a2d80 0x7f38780a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.871+0000 7f388ffff640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f38780a2d80 0x7f38780a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.871+0000 7f38767fc640 1 -- 172.21.15.78:0/902158917 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880037070 con 0x5559effa2390 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.871+0000 7f38767fc640 1 -- 172.21.15.78:0/902158917 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3880042070 con 0x5559effa2390 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.871+0000 7f38767fc640 1 -- 172.21.15.78:0/902158917 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38800052a0 con 0x5559effa2390 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.871+0000 7f388d7fa640 1 -- 172.21.15.78:0/3828143482 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f38880cf3d0 con 0x7f3890101aa0 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 -- 172.21.15.78:0/902158917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559effa2390 msgr2=0x5559f0340d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f388ffff640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f38780a2d80 0x7f38780a5290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5559f033e810 tx=0x7f38880166c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559effa2390 0x5559f0340d20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3880034000 tx=0x7f3880005e40 comp rx=0 tx=0).stop 2024-05-21T15:19:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 -- 172.21.15.78:0/902158917 shutdown_connections 2024-05-21T15:19:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 0x5559f0341260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 0x5559f03417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 --2- 172.21.15.78:0/902158917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559effa2390 0x5559f0340d20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 -- 172.21.15.78:0/902158917 >> 172.21.15.78:0/902158917 conn(0x5559eff964e0 msgr2=0x5559f033d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 -- 172.21.15.78:0/902158917 shutdown_connections 2024-05-21T15:19:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 -- 172.21.15.78:0/902158917 wait complete. 2024-05-21T15:19:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.872+0000 7f389d20e740 1 -- start start 2024-05-21T15:19:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.873+0000 7f389d20e740 1 Processor -- start 2024-05-21T15:19:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.873+0000 7f389d20e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f034b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.873+0000 7f389d20e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff816b0 0x5559f034b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.873+0000 7f389d20e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559effa2390 0x5559f03514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.873+0000 7f389d20e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5559f034dfb0 con 0x5559effa2390 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.873+0000 7f389d20e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5559f0261c40 con 0x5559eff816b0 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f389d20e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559f034e130 con 0x5559eff794d0 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f389482e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f034b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff816b0 0x5559f034b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f389d20e740 10 client.0 _ll_register_callbacks cb 0x5559effb9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f389482e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f034b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44070/0 (socket says 172.21.15.78:44070) 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f389482e640 1 -- 172.21.15.78:0/848904417 learned_addr learned my addr 172.21.15.78:0/848904417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f389482e640 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559effa2390 0x5559f03514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f388f7fe640 1 -- 172.21.15.78:0/848904417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 msgr2=0x5559f034b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f388f7fe640 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f034b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f388f7fe640 1 -- 172.21.15.78:0/848904417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559effa2390 msgr2=0x5559f03514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f388f7fe640 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559effa2390 0x5559f03514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f388f7fe640 1 -- 172.21.15.78:0/848904417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f00847d0 con 0x5559eff816b0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f388f7fe640 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff816b0 0x5559f034b920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3890074f80 tx=0x7f3884011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff8a0abf016ab077 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.874+0000 7f386f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.875+0000 7f386f7fe640 1 -- 172.21.15.78:0/848904417 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884004030 con 0x5559eff816b0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.875+0000 7f389d20e740 1 -- 172.21.15.78:0/848904417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3880042070 con 0x5559eff816b0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.875+0000 7f386f7fe640 1 -- 172.21.15.78:0/848904417 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f38840041d0 con 0x5559eff816b0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.875+0000 7f386f7fe640 1 -- 172.21.15.78:0/848904417 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f38840043d0 con 0x5559eff816b0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.875+0000 7f389d20e740 10 client.26081 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.875+0000 7f389d20e740 1 -- 172.21.15.78:0/848904417 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5559f035a830 con 0x5559eff816b0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.875+0000 7f386f7fe640 1 -- 172.21.15.78:0/848904417 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3880042070 con 0x5559eff816b0 2024-05-21T15:19:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f386f7fe640 1 -- 172.21.15.78:0/848904417 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f3884003970 con 0x5559eff816b0 2024-05-21T15:19:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f389d20e740 10 client.26081 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f389d20e740 10 client.26081 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:19:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f389d20e740 20 client.26081 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f389d20e740 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5559eff60f60 0x5559f035e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f389d20e740 4 client.26081 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f389d20e740 1 -- 172.21.15.78:0/848904417 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559f02b4370 con 0x5559eff60f60 2024-05-21T15:19:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f389482e640 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5559eff60f60 0x5559f035e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f386f7fe640 10 client.26081 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.876+0000 7f389482e640 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5559eff60f60 0x5559f035e1c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce50b4d5efe12bfa server_cookie=b5d95255c5f69695 in_seq=0 out_seq=0 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.877+0000 7f386f7fe640 1 -- 172.21.15.78:0/848904417 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5559f02b4370 con 0x5559eff60f60 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f386f7fe640 10 client.26081 handle_command_reply: tid=0 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 client.26081 shutdown 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 -- 172.21.15.78:0/848904417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff816b0 msgr2=0x5559f034b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff816b0 0x5559f034b920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3890074f80 tx=0x7f3884011810 comp rx=0 tx=0).stop 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 -- 172.21.15.78:0/848904417 shutdown_connections 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5559eff60f60 0x5559f035e1c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff816b0 0x5559f034b920 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559effa2390 0x5559f03514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 --2- 172.21.15.78:0/848904417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f034b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f386f7fe640 0 client.26081 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:19:30.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 -- 172.21.15.78:0/848904417 >> 172.21.15.78:0/848904417 conn(0x5559eff964e0 msgr2=0x5559f0342a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 -- 172.21.15.78:0/848904417 shutdown_connections 2024-05-21T15:19:30.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 1 -- 172.21.15.78:0/848904417 wait complete. 2024-05-21T15:19:30.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.878+0000 7f389d20e740 20 client.26081 trim_cache size 0 max 16384 2024-05-21T15:19:30.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.899+0000 7f389d20e740 1 Processor -- start 2024-05-21T15:19:30.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.899+0000 7f389d20e740 1 -- start start 2024-05-21T15:19:30.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389d20e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff60f60 0x5559f03513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389d20e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 0x5559f0344a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389d20e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff816b0 0x5559f0344fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389d20e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38901aea70 con 0x5559eff60f60 2024-05-21T15:19:30.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389d20e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b0760 con 0x5559eff794d0 2024-05-21T15:19:30.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389d20e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38901af140 con 0x5559eff816b0 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389482e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff60f60 0x5559f03513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 0x5559f0344a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389482e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff60f60 0x5559f03513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36860/0 (socket says 172.21.15.78:36860) 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389482e640 1 -- 172.21.15.78:0/2546898455 learned_addr learned my addr 172.21.15.78:0/2546898455 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f389482e640 1 --2- 172.21.15.78:0/2546898455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff816b0 0x5559f0344fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f388f7fe640 1 -- 172.21.15.78:0/2546898455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff816b0 msgr2=0x5559f0344fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f388f7fe640 1 --2- 172.21.15.78:0/2546898455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff816b0 0x5559f0344fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f388f7fe640 1 -- 172.21.15.78:0/2546898455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff60f60 msgr2=0x5559f03513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f388f7fe640 1 --2- 172.21.15.78:0/2546898455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff60f60 0x5559f03513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.900+0000 7f388f7fe640 1 -- 172.21.15.78:0/2546898455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f033e0c0 con 0x5559eff794d0 2024-05-21T15:19:30.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.901+0000 7f388f7fe640 1 --2- 172.21.15.78:0/2546898455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 0x5559f0344a80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5559f034c7a0 tx=0x7f388400bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3798179ce60ce6b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.901+0000 7f3875ffb640 1 -- 172.21.15.78:0/2546898455 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884003fd0 con 0x5559eff794d0 2024-05-21T15:19:30.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.901+0000 7f3875ffb640 1 -- 172.21.15.78:0/2546898455 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3884004170 con 0x5559eff794d0 2024-05-21T15:19:30.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.901+0000 7f3875ffb640 1 -- 172.21.15.78:0/2546898455 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884010760 con 0x5559eff794d0 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.902+0000 7f389d20e740 1 -- 172.21.15.78:0/2546898455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 msgr2=0x5559f0344a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.902+0000 7f389d20e740 1 --2- 172.21.15.78:0/2546898455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 0x5559f0344a80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5559f034c7a0 tx=0x7f388400bb80 comp rx=0 tx=0).stop 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.902+0000 7f389d20e740 1 -- 172.21.15.78:0/2546898455 shutdown_connections 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.902+0000 7f389d20e740 1 --2- 172.21.15.78:0/2546898455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff794d0 0x5559f0344a80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.902+0000 7f389d20e740 1 --2- 172.21.15.78:0/2546898455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff816b0 0x5559f0344fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.902+0000 7f389d20e740 1 --2- 172.21.15.78:0/2546898455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff60f60 0x5559f03513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.902+0000 7f389d20e740 1 -- 172.21.15.78:0/2546898455 >> 172.21.15.78:0/2546898455 conn(0x5559eff964e0 msgr2=0x5559f0349aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.903+0000 7f389d20e740 1 -- 172.21.15.78:0/2546898455 shutdown_connections 2024-05-21T15:19:30.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.903+0000 7f389d20e740 1 -- 172.21.15.78:0/2546898455 wait complete. 2024-05-21T15:19:30.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.903+0000 7f389d20e740 1 -- start start 2024-05-21T15:19:30.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.903+0000 7f389d20e740 1 Processor -- start 2024-05-21T15:19:30.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389d20e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff60f60 0x5559f0344f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389d20e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f0342810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389d20e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 0x5559f0342d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389482e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff60f60 0x5559f0344f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389482e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff60f60 0x5559f0344f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45942/0 (socket says 172.21.15.78:45942) 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f0342810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f0342810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44088/0 (socket says 172.21.15.78:44088) 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389482e640 1 -- 172.21.15.78:0/2431497203 learned_addr learned my addr 172.21.15.78:0/2431497203 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38901affd0 con 0x5559eff816b0 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38901af8d0 con 0x5559eff60f60 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38901aca50 con 0x5559eff794d0 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.904+0000 7f389d20e740 10 client.0 _ll_register_callbacks cb 0x7f388400b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.905+0000 7f388f7fe640 1 -- 172.21.15.78:0/2431497203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff60f60 msgr2=0x5559f0344f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.905+0000 7f388f7fe640 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff60f60 0x5559f0344f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.905+0000 7f388f7fe640 1 -- 172.21.15.78:0/2431497203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 msgr2=0x5559f0342d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.905+0000 7f388f7fe640 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 0x5559f0342d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.905+0000 7f388f7fe640 1 -- 172.21.15.78:0/2431497203 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f0343310 con 0x5559eff794d0 2024-05-21T15:19:30.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.905+0000 7f389482e640 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff60f60 0x5559f0344f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.906+0000 7f388f7fe640 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f0342810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5559f0345c50 tx=0x7f3884005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cf3ebf11053dcd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.906+0000 7f386f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.906+0000 7f386f7fe640 1 -- 172.21.15.78:0/2431497203 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884010700 con 0x5559eff794d0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.906+0000 7f386f7fe640 1 -- 172.21.15.78:0/2431497203 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f38840108a0 con 0x5559eff794d0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.906+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3884004170 con 0x5559eff794d0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.906+0000 7f386f7fe640 1 -- 172.21.15.78:0/2431497203 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3884010b00 con 0x5559eff794d0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.906+0000 7f389d20e740 10 client.26086 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.906+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5559f03580b0 con 0x5559eff794d0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f386f7fe640 1 -- 172.21.15.78:0/2431497203 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3884004170 con 0x5559eff794d0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f386f7fe640 1 -- 172.21.15.78:0/2431497203 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f388403b030 con 0x5559eff794d0 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f389d20e740 10 client.26086 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f389d20e740 10 client.26086 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:19:30.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f389d20e740 20 client.26086 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:30.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f389d20e740 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5559effa2390 0x5559f03dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:30.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f389d20e740 4 client.26086 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:19:30.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559f02ae6c0 con 0x5559effa2390 2024-05-21T15:19:30.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.907+0000 7f389482e640 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5559effa2390 0x5559f03dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:30.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.908+0000 7f386f7fe640 10 client.26086 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:19:30.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.908+0000 7f389482e640 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5559effa2390 0x5559f03dda70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdda280ca3915cc1 server_cookie=7b5b7d35f9869a63 in_seq=0 out_seq=0 2024-05-21T15:19:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.908+0000 7f386f7fe640 1 -- 172.21.15.78:0/2431497203 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559f02ae6c0 con 0x5559effa2390 2024-05-21T15:19:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.908+0000 7f386f7fe640 10 client.26086 handle_command_reply: tid=0 2024-05-21T15:19:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.908+0000 7f389d20e740 1 client.26086 shutdown 2024-05-21T15:19:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 msgr2=0x5559f0342810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f0342810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5559f0345c50 tx=0x7f3884005270 comp rx=0 tx=0).stop 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 shutdown_connections 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5559effa2390 0x5559f03dda70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559eff816b0 0x5559f0342d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559eff794d0 0x5559f0342810 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 --2- 172.21.15.78:0/2431497203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559eff60f60 0x5559f0344f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 >> 172.21.15.78:0/2431497203 conn(0x5559eff964e0 msgr2=0x5559f0340ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f386f7fe640 0 client.26086 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 shutdown_connections 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 1 -- 172.21.15.78:0/2431497203 wait complete. 2024-05-21T15:19:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.909+0000 7f389d20e740 20 client.26086 trim_cache size 0 max 16384 2024-05-21T15:19:30.915 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:19:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 -- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f38780a2d80 msgr2=0x7f38780a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f38780a2d80 0x7f38780a5290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5559f033e810 tx=0x7f38880166c0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 -- 172.21.15.78:0/3828143482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3890101aa0 msgr2=0x7f389019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3890101aa0 0x7f389019a540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f388800c3a0 tx=0x7f388800c860 comp rx=0 tx=0).stop 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 -- 172.21.15.78:0/3828143482 shutdown_connections 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f389010c330 0x7f389019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f38780a2d80 0x7f38780a5290 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38901079d0 0x7f389019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 --2- 172.21.15.78:0/3828143482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3890101aa0 0x7f389019a540 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.910+0000 7f3896291640 1 -- 172.21.15.78:0/3828143482 >> 172.21.15.78:0/3828143482 conn(0x7f38900770b0 msgr2=0x7f3890103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.911+0000 7f3896291640 1 -- 172.21.15.78:0/3828143482 shutdown_connections 2024-05-21T15:19:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:30.911+0000 7f3896291640 1 -- 172.21.15.78:0/3828143482 wait complete. 2024-05-21T15:19:30.929 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:19:31.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.115+0000 7fc26219d640 1 -- 172.21.15.78:0/2498054707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10b000 msgr2=0x7fc25c10b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.115+0000 7fc26219d640 1 --2- 172.21.15.78:0/2498054707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10b000 0x7fc25c10b410 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009900 tx=0x7fc24c02a420 comp rx=0 tx=0).stop 2024-05-21T15:19:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.116+0000 7fc26219d640 1 -- 172.21.15.78:0/2498054707 shutdown_connections 2024-05-21T15:19:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.116+0000 7fc26219d640 1 --2- 172.21.15.78:0/2498054707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25c10fa20 0x7fc25c10fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.116+0000 7fc26219d640 1 --2- 172.21.15.78:0/2498054707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc25c1104e0 0x7fc25c116a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.116+0000 7fc26219d640 1 --2- 172.21.15.78:0/2498054707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10b000 0x7fc25c10b410 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.116+0000 7fc26219d640 1 -- 172.21.15.78:0/2498054707 >> 172.21.15.78:0/2498054707 conn(0x7fc25c06d580 msgr2=0x7fc25c06d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.118+0000 7fc26219d640 1 -- 172.21.15.78:0/2498054707 shutdown_connections 2024-05-21T15:19:31.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.118+0000 7fc26219d640 1 -- 172.21.15.78:0/2498054707 wait complete. 2024-05-21T15:19:31.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.119+0000 7fc26219d640 1 Processor -- start 2024-05-21T15:19:31.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.119+0000 7fc26219d640 1 -- start start 2024-05-21T15:19:31.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.119+0000 7fc26219d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc25c10b000 0x7fc25c1a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.119+0000 7fc26219d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10fa20 0x7fc25c1a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26219d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25c1104e0 0x7fc25c1a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26119b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc25c10b000 0x7fc25c1a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26119b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc25c10b000 0x7fc25c1a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44102/0 (socket says 172.21.15.78:44102) 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26099a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10fa20 0x7fc25c1a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26119b640 1 -- 172.21.15.78:0/402655558 learned_addr learned my addr 172.21.15.78:0/402655558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c11d620 con 0x7fc25c10fa20 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c11d4a0 con 0x7fc25c1104e0 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c11d7a0 con 0x7fc25c10b000 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26199c640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25c1104e0 0x7fc25c1a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26099a640 1 -- 172.21.15.78:0/402655558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc25c10b000 msgr2=0x7fc25c1a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26099a640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc25c10b000 0x7fc25c1a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26099a640 1 -- 172.21.15.78:0/402655558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25c1104e0 msgr2=0x7fc25c1a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26099a640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25c1104e0 0x7fc25c1a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26099a640 1 -- 172.21.15.78:0/402655558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25c1a8de0 con 0x7fc25c10fa20 2024-05-21T15:19:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.120+0000 7fc26199c640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25c1104e0 0x7fc25c1a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:19:31.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.121+0000 7fc26099a640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10fa20 0x7fc25c1a44e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9c0 tx=0x7fc25800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.121+0000 7fc2527fc640 1 -- 172.21.15.78:0/402655558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25800ccc0 con 0x7fc25c10fa20 2024-05-21T15:19:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.121+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25c1a90d0 con 0x7fc25c10fa20 2024-05-21T15:19:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.121+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25c1b0a50 con 0x7fc25c10fa20 2024-05-21T15:19:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.121+0000 7fc2527fc640 1 -- 172.21.15.78:0/402655558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc258010910 con 0x7fc25c10fa20 2024-05-21T15:19:31.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.121+0000 7fc2527fc640 1 -- 172.21.15.78:0/402655558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258018880 con 0x7fc25c10fa20 2024-05-21T15:19:31.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26a37b740 1 Processor -- start 2024-05-21T15:19:31.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26a37b740 1 -- start start 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb6cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 0x5642cbb6d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7ad6b0 0x5642cbb6d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26a37b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5642cbad2c40 con 0x5642cb7ce390 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26a37b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5642cba7a580 con 0x5642cb7a54d0 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26a37b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5642cbacec00 con 0x5642cb7ad6b0 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26199c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb6cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26199c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb6cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36888/0 (socket says 172.21.15.78:36888) 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.125+0000 7fc26199c640 1 -- 172.21.15.78:0/2609602428 learned_addr learned my addr 172.21.15.78:0/2609602428 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc26119b640 1 --2- 172.21.15.78:0/2609602428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 0x5642cbb6d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc26199c640 1 -- 172.21.15.78:0/2609602428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7ad6b0 msgr2=0x5642cbb6d9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:31.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc26199c640 1 --2- 172.21.15.78:0/2609602428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7ad6b0 0x5642cbb6d9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc26199c640 1 -- 172.21.15.78:0/2609602428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 msgr2=0x5642cbb6d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc26199c640 1 --2- 172.21.15.78:0/2609602428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 0x5642cbb6d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc26199c640 1 -- 172.21.15.78:0/2609602428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cb7e63f0 con 0x5642cb7ce390 2024-05-21T15:19:31.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc26199c640 1 --2- 172.21.15.78:0/2609602428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb6cf60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc254009fd0 tx=0x7fc2540108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f655d3539ff69f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc2527fc640 1 -- 172.21.15.78:0/402655558 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc258010ac0 con 0x7fc25c10fa20 2024-05-21T15:19:31.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.126+0000 7fc2477fe640 1 -- 172.21.15.78:0/2609602428 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254011760 con 0x5642cb7ce390 2024-05-21T15:19:31.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.127+0000 7fc2477fe640 1 -- 172.21.15.78:0/2609602428 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc254011de0 con 0x5642cb7ce390 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.127+0000 7fc2477fe640 1 -- 172.21.15.78:0/2609602428 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25400e3c0 con 0x5642cb7ce390 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.127+0000 7fc2527fc640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2480a2d70 0x7fc2480a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26119b640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2480a2d70 0x7fc2480a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc2527fc640 1 -- 172.21.15.78:0/402655558 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc2580cf730 con 0x7fc25c10fa20 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 -- 172.21.15.78:0/2609602428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 msgr2=0x5642cbb6cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2609602428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb6cf60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc254009fd0 tx=0x7fc2540108e0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 -- 172.21.15.78:0/2609602428 shutdown_connections 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2609602428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 0x5642cbb6d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2609602428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7ad6b0 0x5642cbb6d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2609602428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb6cf60 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 -- 172.21.15.78:0/2609602428 >> 172.21.15.78:0/2609602428 conn(0x5642cb7c24e0 msgr2=0x5642cbb69710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 -- 172.21.15.78:0/2609602428 shutdown_connections 2024-05-21T15:19:31.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.128+0000 7fc26a37b740 1 -- 172.21.15.78:0/2609602428 wait complete. 2024-05-21T15:19:31.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.129+0000 7fc26a37b740 1 -- start start 2024-05-21T15:19:31.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.129+0000 7fc26119b640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2480a2d70 0x7fc2480a5280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5642cbb6a6a0 tx=0x7fc24c005a00 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.129+0000 7fc26a37b740 1 Processor -- start 2024-05-21T15:19:31.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.130+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7a54d0 0x5642cbb77620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.130+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb77b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.130+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb78cf60 0x5642cbb7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26a37b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5642cbb7a2b0 con 0x5642cb7ce390 2024-05-21T15:19:31.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26a37b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5642cba8dc40 con 0x5642cb78cf60 2024-05-21T15:19:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26a37b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5642cbb7a430 con 0x5642cb7a54d0 2024-05-21T15:19:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26199c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb78cf60 0x5642cbb7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26199c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb78cf60 0x5642cbb7d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:45966/0 (socket says 172.21.15.78:45966) 2024-05-21T15:19:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26199c640 1 -- 172.21.15.78:0/2671716116 learned_addr learned my addr 172.21.15.78:0/2671716116 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26a37b740 10 client.0 _ll_register_callbacks cb 0x5642cb7e5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26099a640 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.131+0000 7fc26119b640 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7a54d0 0x5642cbb77620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.132+0000 7fc26199c640 1 -- 172.21.15.78:0/2671716116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7a54d0 msgr2=0x5642cbb77620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.132+0000 7fc26199c640 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7a54d0 0x5642cbb77620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.132+0000 7fc26199c640 1 -- 172.21.15.78:0/2671716116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 msgr2=0x5642cbb77b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.132+0000 7fc26199c640 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb77b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.132+0000 7fc26199c640 1 -- 172.21.15.78:0/2671716116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cb8b07d0 con 0x5642cb78cf60 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.133+0000 7fc26199c640 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb78cf60 0x5642cbb7d800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc254011fd0 tx=0x7fc25401b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9961667c2d9bbe5d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.133+0000 7fc244ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.133+0000 7fc244ff9640 1 -- 172.21.15.78:0/2671716116 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254022070 con 0x5642cb78cf60 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.133+0000 7fc26a37b740 1 -- 172.21.15.78:0/2671716116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc254011de0 con 0x5642cb78cf60 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.133+0000 7fc244ff9640 1 -- 172.21.15.78:0/2671716116 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc254011560 con 0x5642cb78cf60 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.134+0000 7fc244ff9640 1 -- 172.21.15.78:0/2671716116 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc254004e50 con 0x5642cb78cf60 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.134+0000 7fc26a37b740 10 client.26099 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.134+0000 7fc26a37b740 1 -- 172.21.15.78:0/2671716116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5642cbb86b30 con 0x5642cb78cf60 2024-05-21T15:19:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.134+0000 7fc244ff9640 1 -- 172.21.15.78:0/2671716116 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc254011de0 con 0x5642cb78cf60 2024-05-21T15:19:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.135+0000 7fc244ff9640 1 -- 172.21.15.78:0/2671716116 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fc2540056c0 con 0x5642cb78cf60 2024-05-21T15:19:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.135+0000 7fc26a37b740 10 client.26099 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.135+0000 7fc26a37b740 10 client.26099 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:19:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.135+0000 7fc26a37b740 20 client.26099 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.135+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5642cb7623a0 0x5642cbb89ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.135+0000 7fc26a37b740 4 client.26099 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.135+0000 7fc26a37b740 1 -- 172.21.15.78:0/2671716116 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cbae0370 con 0x5642cb7623a0 2024-05-21T15:19:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.136+0000 7fc26119b640 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5642cb7623a0 0x5642cbb89ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.136+0000 7fc244ff9640 10 client.26099 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:19:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.136+0000 7fc26119b640 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5642cb7623a0 0x5642cbb89ef0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b5b19430642f414 server_cookie=3a16fdfe8e9f65f3 in_seq=0 out_seq=0 2024-05-21T15:19:31.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.138+0000 7fc244ff9640 1 -- 172.21.15.78:0/2671716116 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642cbae0370 con 0x5642cb7623a0 2024-05-21T15:19:31.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.138+0000 7fc244ff9640 10 client.26099 handle_command_reply: tid=0 2024-05-21T15:19:31.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.138+0000 7fc26a37b740 1 client.26099 shutdown 2024-05-21T15:19:31.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 -- 172.21.15.78:0/2671716116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb78cf60 msgr2=0x5642cbb7d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb78cf60 0x5642cbb7d800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc254011fd0 tx=0x7fc25401b8f0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 -- 172.21.15.78:0/2671716116 shutdown_connections 2024-05-21T15:19:31.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5642cb7623a0 0x5642cbb89ef0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb78cf60 0x5642cbb7d800 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7ce390 0x5642cbb77b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 --2- 172.21.15.78:0/2671716116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7a54d0 0x5642cbb77620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc244ff9640 0 client.26099 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:19:31.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 -- 172.21.15.78:0/2671716116 >> 172.21.15.78:0/2671716116 conn(0x5642cb7c24e0 msgr2=0x5642cbb6ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 -- 172.21.15.78:0/2671716116 shutdown_connections 2024-05-21T15:19:31.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 1 -- 172.21.15.78:0/2671716116 wait complete. 2024-05-21T15:19:31.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.139+0000 7fc26a37b740 20 client.26099 trim_cache size 0 max 16384 2024-05-21T15:19:31.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.165+0000 7fc26a37b740 1 Processor -- start 2024-05-21T15:19:31.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.167+0000 7fc26a37b740 1 -- start start 2024-05-21T15:19:31.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.167+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb78cf60 0x5642cbb7d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.167+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 0x5642cbb8f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.167+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7ce390 0x5642cbb8fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.167+0000 7fc26a37b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc24000e9a0 con 0x5642cb78cf60 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.167+0000 7fc26a37b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc240010690 con 0x5642cb7a54d0 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.167+0000 7fc26a37b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc24000f070 con 0x5642cb7ce390 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.168+0000 7fc26119b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb78cf60 0x5642cbb7d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.168+0000 7fc26119b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb78cf60 0x5642cbb7d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36894/0 (socket says 172.21.15.78:36894) 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.168+0000 7fc26119b640 1 -- 172.21.15.78:0/140377269 learned_addr learned my addr 172.21.15.78:0/140377269 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.169+0000 7fc26119b640 1 -- 172.21.15.78:0/140377269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7ce390 msgr2=0x5642cbb8fa30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.169+0000 7fc26119b640 1 --2- 172.21.15.78:0/140377269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7ce390 0x5642cbb8fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.169+0000 7fc26119b640 1 -- 172.21.15.78:0/140377269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 msgr2=0x5642cbb8f4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.169+0000 7fc26119b640 1 --2- 172.21.15.78:0/140377269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 0x5642cbb8f4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.169+0000 7fc26119b640 1 -- 172.21.15.78:0/140377269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cbb72fd0 con 0x5642cb78cf60 2024-05-21T15:19:31.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.169+0000 7fc26119b640 1 --2- 172.21.15.78:0/140377269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb78cf60 0x5642cbb7d700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009850 tx=0x7fc24c038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901b6669ad55f65b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.170+0000 7fc246ffd640 1 -- 172.21.15.78:0/140377269 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c055070 con 0x5642cb78cf60 2024-05-21T15:19:31.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.171+0000 7fc246ffd640 1 -- 172.21.15.78:0/140377269 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc24c005520 con 0x5642cb78cf60 2024-05-21T15:19:31.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.171+0000 7fc246ffd640 1 -- 172.21.15.78:0/140377269 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c035040 con 0x5642cb78cf60 2024-05-21T15:19:31.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.172+0000 7fc26a37b740 1 -- 172.21.15.78:0/140377269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb78cf60 msgr2=0x5642cbb7d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.172+0000 7fc26a37b740 1 --2- 172.21.15.78:0/140377269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb78cf60 0x5642cbb7d700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009850 tx=0x7fc24c038000 comp rx=0 tx=0).stop 2024-05-21T15:19:31.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.176+0000 7fc26a37b740 1 -- 172.21.15.78:0/140377269 shutdown_connections 2024-05-21T15:19:31.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.176+0000 7fc26a37b740 1 --2- 172.21.15.78:0/140377269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7a54d0 0x5642cbb8f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.176+0000 7fc26a37b740 1 --2- 172.21.15.78:0/140377269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb7ce390 0x5642cbb8fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.176+0000 7fc26a37b740 1 --2- 172.21.15.78:0/140377269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb78cf60 0x5642cbb7d700 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.176+0000 7fc26a37b740 1 -- 172.21.15.78:0/140377269 >> 172.21.15.78:0/140377269 conn(0x5642cb7c24e0 msgr2=0x5642cbb75d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.179+0000 7fc26a37b740 1 -- 172.21.15.78:0/140377269 shutdown_connections 2024-05-21T15:19:31.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.179+0000 7fc26a37b740 1 -- 172.21.15.78:0/140377269 wait complete. 2024-05-21T15:19:31.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.179+0000 7fc26a37b740 1 -- start start 2024-05-21T15:19:31.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.180+0000 7fc26a37b740 1 Processor -- start 2024-05-21T15:19:31.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.180+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb78cf60 0x5642cbb7ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.180+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7a54d0 0x5642cbb7f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.180+0000 7fc26a37b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7ce390 0x5642cbbf88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.180+0000 7fc26a37b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc24000ff00 con 0x5642cb7a54d0 2024-05-21T15:19:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.180+0000 7fc26a37b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc24000f800 con 0x5642cb7ce390 2024-05-21T15:19:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.180+0000 7fc26a37b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc24000c980 con 0x5642cb78cf60 2024-05-21T15:19:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.180+0000 7fc26a37b740 10 client.0 _ll_register_callbacks cb 0x7fc2540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:31.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26119b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb78cf60 0x5642cbb7ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26099a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7a54d0 0x5642cbb7f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26099a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7a54d0 0x5642cbb7f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36900/0 (socket says 172.21.15.78:36900) 2024-05-21T15:19:31.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26099a640 1 -- 172.21.15.78:0/3851677793 learned_addr learned my addr 172.21.15.78:0/3851677793 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26199c640 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7ce390 0x5642cbbf88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26119b640 1 -- 172.21.15.78:0/3851677793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7ce390 msgr2=0x5642cbbf88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26119b640 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7ce390 0x5642cbbf88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26119b640 1 -- 172.21.15.78:0/3851677793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7a54d0 msgr2=0x5642cbb7f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26119b640 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7a54d0 0x5642cbb7f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.184+0000 7fc26119b640 1 -- 172.21.15.78:0/3851677793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cbbf8de0 con 0x5642cb78cf60 2024-05-21T15:19:31.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.185+0000 7fc26119b640 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb78cf60 0x5642cbb7ef40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006010 tx=0x7fc24c02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cbc8d23cabca4b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.185+0000 7fc244ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:19:31.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.190+0000 7fc244ff9640 1 -- 172.21.15.78:0/3851677793 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c055040 con 0x5642cb78cf60 2024-05-21T15:19:31.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.190+0000 7fc26a37b740 1 -- 172.21.15.78:0/3851677793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc24c005520 con 0x5642cb78cf60 2024-05-21T15:19:31.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.190+0000 7fc244ff9640 1 -- 172.21.15.78:0/3851677793 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fc24c005040 con 0x5642cb78cf60 2024-05-21T15:19:31.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.191+0000 7fc244ff9640 1 -- 172.21.15.78:0/3851677793 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc24c005240 con 0x5642cb78cf60 2024-05-21T15:19:31.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.191+0000 7fc26a37b740 10 client.26098 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:31.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.191+0000 7fc26a37b740 1 -- 172.21.15.78:0/3851677793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5642cbbf9f70 con 0x5642cb78cf60 2024-05-21T15:19:31.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.193+0000 7fc244ff9640 1 -- 172.21.15.78:0/3851677793 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc24c005520 con 0x5642cb78cf60 2024-05-21T15:19:31.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.193+0000 7fc244ff9640 1 -- 172.21.15.78:0/3851677793 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fc24c050ce0 con 0x5642cb78cf60 2024-05-21T15:19:31.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.193+0000 7fc26a37b740 10 client.26098 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:31.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.193+0000 7fc26a37b740 10 client.26098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:19:31.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.193+0000 7fc26a37b740 20 client.26098 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:31.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.193+0000 7fc26a37b740 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5642cbbfa6e0 0x5642cbbfcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.193+0000 7fc26a37b740 4 client.26098 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:19:31.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.193+0000 7fc26a37b740 1 -- 172.21.15.78:0/3851677793 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642cbada6c0 con 0x5642cbbfa6e0 2024-05-21T15:19:31.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.194+0000 7fc26099a640 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5642cbbfa6e0 0x5642cbbfcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.200+0000 7fc26099a640 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5642cbbfa6e0 0x5642cbbfcb00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fca9ce63edb5e55 server_cookie=ca18e73da98ab011 in_seq=0 out_seq=0 2024-05-21T15:19:31.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.200+0000 7fc244ff9640 10 client.26098 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:19:31.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.201+0000 7fc244ff9640 1 -- 172.21.15.78:0/3851677793 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642cbada6c0 con 0x5642cbbfa6e0 2024-05-21T15:19:31.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.201+0000 7fc244ff9640 10 client.26098 handle_command_reply: tid=0 2024-05-21T15:19:31.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.201+0000 7fc26a37b740 1 client.26098 shutdown 2024-05-21T15:19:31.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.201+0000 7fc26a37b740 1 -- 172.21.15.78:0/3851677793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb78cf60 msgr2=0x5642cbb7ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.201+0000 7fc26a37b740 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb78cf60 0x5642cbb7ef40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006010 tx=0x7fc24c02a420 comp rx=0 tx=0).stop 2024-05-21T15:19:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.202+0000 7fc26a37b740 1 -- 172.21.15.78:0/3851677793 shutdown_connections 2024-05-21T15:19:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.202+0000 7fc26a37b740 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5642cbbfa6e0 0x5642cbbfcb00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.202+0000 7fc26a37b740 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642cb7ce390 0x5642cbbf88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.202+0000 7fc26a37b740 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642cb7a54d0 0x5642cbb7f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.202+0000 7fc26a37b740 1 --2- 172.21.15.78:0/3851677793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642cb78cf60 0x5642cbb7ef40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.202+0000 7fc26a37b740 1 -- 172.21.15.78:0/3851677793 >> 172.21.15.78:0/3851677793 conn(0x5642cb7c24e0 msgr2=0x5642cbb6eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.202+0000 7fc244ff9640 0 client.26098 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:19:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.203+0000 7fc26a37b740 1 -- 172.21.15.78:0/3851677793 shutdown_connections 2024-05-21T15:19:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.203+0000 7fc26a37b740 1 -- 172.21.15.78:0/3851677793 wait complete. 2024-05-21T15:19:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.204+0000 7fc26a37b740 20 client.26098 trim_cache size 0 max 16384 2024-05-21T15:19:31.209 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:19:31.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.206+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2480a2d70 msgr2=0x7fc2480a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2480a2d70 0x7fc2480a5280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5642cbb6a6a0 tx=0x7fc24c005a00 comp rx=0 tx=0).stop 2024-05-21T15:19:31.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10fa20 msgr2=0x7fc25c1a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10fa20 0x7fc25c1a44e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9c0 tx=0x7fc25800ee80 comp rx=0 tx=0).stop 2024-05-21T15:19:31.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 shutdown_connections 2024-05-21T15:19:31.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc25c1104e0 0x7fc25c1a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2480a2d70 0x7fc2480a5280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc25c10fa20 0x7fc25c1a44e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 --2- 172.21.15.78:0/402655558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc25c10b000 0x7fc25c1a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.207+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 >> 172.21.15.78:0/402655558 conn(0x7fc25c06d580 msgr2=0x7fc25c1109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.209+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 shutdown_connections 2024-05-21T15:19:31.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.209+0000 7fc26219d640 1 -- 172.21.15.78:0/402655558 wait complete. 2024-05-21T15:19:31.234 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:19:31.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:31 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/13142348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:19:31.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:31 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/13142348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:19:31.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.418+0000 7f760f832640 1 -- 172.21.15.78:0/929096504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 msgr2=0x7f7608102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.418+0000 7f760f832640 1 --2- 172.21.15.78:0/929096504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 0x7f7608102670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7604009860 tx=0x7f760402a340 comp rx=0 tx=0).stop 2024-05-21T15:19:31.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.419+0000 7f760f832640 1 -- 172.21.15.78:0/929096504 shutdown_connections 2024-05-21T15:19:31.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.419+0000 7f760f832640 1 --2- 172.21.15.78:0/929096504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7608103220 0x7f760810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.419+0000 7f760f832640 1 --2- 172.21.15.78:0/929096504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7608108f70 0x7f7608102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.419+0000 7f760f832640 1 --2- 172.21.15.78:0/929096504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 0x7f7608102670 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.419+0000 7f760f832640 1 -- 172.21.15.78:0/929096504 >> 172.21.15.78:0/929096504 conn(0x7f76080fe1b0 msgr2=0x7f7608100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.419+0000 7f760f832640 1 -- 172.21.15.78:0/929096504 shutdown_connections 2024-05-21T15:19:31.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.419+0000 7f760f832640 1 -- 172.21.15.78:0/929096504 wait complete. 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760f832640 1 Processor -- start 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760f832640 1 -- start start 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760f832640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7608103220 0x7f760819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760f832640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 0x7f760819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760f832640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7608108f70 0x7f7608199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760f832640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7608112e60 con 0x7f7608108f70 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760f832640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7608112ce0 con 0x7f7608108ad0 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760f832640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7608112fe0 con 0x7f7608103220 2024-05-21T15:19:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760cdcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 0x7f760819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760ddcf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7608108f70 0x7f7608199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760cdcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 0x7f760819fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:46006/0 (socket says 172.21.15.78:46006) 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.420+0000 7f760cdcd640 1 -- 172.21.15.78:0/4173721305 learned_addr learned my addr 172.21.15.78:0/4173721305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.421+0000 7f760ddcf640 1 -- 172.21.15.78:0/4173721305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7608103220 msgr2=0x7f760819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.421+0000 7f760ddcf640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7608103220 0x7f760819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.421+0000 7f760ddcf640 1 -- 172.21.15.78:0/4173721305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 msgr2=0x7f760819fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.421+0000 7f760ddcf640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 0x7f760819fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.421+0000 7f760ddcf640 1 -- 172.21.15.78:0/4173721305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7608199d90 con 0x7f7608108f70 2024-05-21T15:19:31.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.421+0000 7f760cdcd640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 0x7f760819fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.422+0000 7f760ddcf640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7608108f70 0x7f7608199660 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f760000ed70 tx=0x7f760000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.422+0000 7f75fe7fc640 1 -- 172.21.15.78:0/4173721305 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760000ee60 con 0x7f7608108f70 2024-05-21T15:19:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.422+0000 7f75fe7fc640 1 -- 172.21.15.78:0/4173721305 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7600003b70 con 0x7f7608108f70 2024-05-21T15:19:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.422+0000 7f75fe7fc640 1 -- 172.21.15.78:0/4173721305 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000178e0 con 0x7f7608108f70 2024-05-21T15:19:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.423+0000 7f760f832640 1 -- 172.21.15.78:0/4173721305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760819a020 con 0x7f7608108f70 2024-05-21T15:19:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.423+0000 7f760f832640 1 -- 172.21.15.78:0/4173721305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76081a6280 con 0x7f7608108f70 2024-05-21T15:19:31.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.428+0000 7f76167bb740 1 Processor -- start 2024-05-21T15:19:31.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.428+0000 7f75fe7fc640 1 -- 172.21.15.78:0/4173721305 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7600004020 con 0x7f7608108f70 2024-05-21T15:19:31.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.428+0000 7f76167bb740 1 -- start start 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.428+0000 7f76167bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b62d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.428+0000 7f76167bb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f079b4d0 0x5628f0b63260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.428+0000 7f76167bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 0x5628f0b637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.428+0000 7f76167bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5628f0ac8c40 con 0x5628f07a36b0 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f76167bb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5628f0a70580 con 0x5628f079b4d0 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f76167bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5628f0ac4c00 con 0x5628f07c4390 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760cdcd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b62d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f079b4d0 0x5628f0b63260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f079b4d0 0x5628f0b63260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:46016/0 (socket says 172.21.15.78:46016) 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760d5ce640 1 -- 172.21.15.78:0/448385171 learned_addr learned my addr 172.21.15.78:0/448385171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760cdcd640 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 0x5628f0b637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760d5ce640 1 -- 172.21.15.78:0/448385171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 msgr2=0x5628f0b62d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760d5ce640 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b62d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f75fe7fc640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f75f40a2e80 0x7f75f40a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760d5ce640 1 -- 172.21.15.78:0/448385171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 msgr2=0x5628f0b637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760d5ce640 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 0x5628f0b637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760d5ce640 1 -- 172.21.15.78:0/448385171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628f07dc3f0 con 0x5628f079b4d0 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f760cdcd640 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b62d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.429+0000 7f75fe7fc640 1 -- 172.21.15.78:0/4173721305 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f76000cf4e0 con 0x7f7608108f70 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.430+0000 7f760d5ce640 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f079b4d0 0x5628f0b63260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7604005d00 tx=0x7f7604009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b3c6978498af45 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.430+0000 7f760ddcf640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f75f40a2e80 0x7f75f40a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.430+0000 7f75f37fe640 1 -- 172.21.15.78:0/448385171 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604037040 con 0x5628f079b4d0 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.430+0000 7f75f37fe640 1 -- 172.21.15.78:0/448385171 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7604040070 con 0x5628f079b4d0 2024-05-21T15:19:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.430+0000 7f75f37fe640 1 -- 172.21.15.78:0/448385171 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604030a10 con 0x5628f079b4d0 2024-05-21T15:19:31.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:31 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/13142348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.431+0000 7f76167bb740 1 -- 172.21.15.78:0/448385171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f079b4d0 msgr2=0x5628f0b63260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.431+0000 7f760ddcf640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f75f40a2e80 0x7f75f40a5390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7600098480 tx=0x7f7600011040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.431+0000 7f76167bb740 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f079b4d0 0x5628f0b63260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7604005d00 tx=0x7f7604009520 comp rx=0 tx=0).stop 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.432+0000 7f76167bb740 1 -- 172.21.15.78:0/448385171 shutdown_connections 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.432+0000 7f76167bb740 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f079b4d0 0x5628f0b63260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.432+0000 7f76167bb740 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 0x5628f0b637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.432+0000 7f76167bb740 1 --2- 172.21.15.78:0/448385171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b62d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.432+0000 7f76167bb740 1 -- 172.21.15.78:0/448385171 >> 172.21.15.78:0/448385171 conn(0x5628f07b84e0 msgr2=0x5628f0b5f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.432+0000 7f76167bb740 1 -- 172.21.15.78:0/448385171 shutdown_connections 2024-05-21T15:19:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.432+0000 7f76167bb740 1 -- 172.21.15.78:0/448385171 wait complete. 2024-05-21T15:19:31.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.432+0000 7f76167bb740 1 -- start start 2024-05-21T15:19:31.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.433+0000 7f76167bb740 1 Processor -- start 2024-05-21T15:19:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f76167bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b6d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f76167bb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f07a36b0 0x5628f0b6d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f76167bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b73490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f76167bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5628f0b6ffb0 con 0x5628f079b4d0 2024-05-21T15:19:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f760d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f07a36b0 0x5628f0b6d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f760cdcd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b6d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f760cdcd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b6d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36916/0 (socket says 172.21.15.78:36916) 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f760d5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f07a36b0 0x5628f0b6d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:46024/0 (socket says 172.21.15.78:46024) 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f760cdcd640 1 -- 172.21.15.78:0/2195706598 learned_addr learned my addr 172.21.15.78:0/2195706598 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5628f0a83c40 con 0x5628f07a36b0 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5628f0b70130 con 0x5628f07c4390 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.434+0000 7f76167bb740 10 client.0 _ll_register_callbacks cb 0x5628f07dbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f760cdcd640 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b73490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f760d5ce640 1 -- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 msgr2=0x5628f0b73490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f760d5ce640 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b73490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f760d5ce640 1 -- 172.21.15.78:0/2195706598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 msgr2=0x5628f0b6d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f760d5ce640 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b6d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f760d5ce640 1 -- 172.21.15.78:0/2195706598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628f08a67d0 con 0x5628f07a36b0 2024-05-21T15:19:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f760cdcd640 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b73490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:19:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f760d5ce640 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f07a36b0 0x5628f0b6d8b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5628f0b642e0 tx=0x7f760402aac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03dd0d073bdcc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.435+0000 7f75f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.436+0000 7f75f0ff9640 1 -- 172.21.15.78:0/2195706598 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604048070 con 0x5628f07a36b0 2024-05-21T15:19:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.436+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7604040070 con 0x5628f07a36b0 2024-05-21T15:19:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.436+0000 7f75f0ff9640 1 -- 172.21.15.78:0/2195706598 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7604004040 con 0x5628f07a36b0 2024-05-21T15:19:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.437+0000 7f75f0ff9640 1 -- 172.21.15.78:0/2195706598 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7604054050 con 0x5628f07a36b0 2024-05-21T15:19:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.437+0000 7f76167bb740 10 client.26114 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.437+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628f0b7c760 con 0x5628f07a36b0 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.438+0000 7f75f0ff9640 1 -- 172.21.15.78:0/2195706598 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7604040070 con 0x5628f07a36b0 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.438+0000 7f75f0ff9640 1 -- 172.21.15.78:0/2195706598 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f760403a070 con 0x5628f07a36b0 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.438+0000 7f76167bb740 10 client.26114 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.438+0000 7f76167bb740 10 client.26114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.438+0000 7f76167bb740 20 client.26114 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.438+0000 7f76167bb740 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5628f0782f60 0x5628f0b7fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.438+0000 7f76167bb740 4 client.26114 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.438+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628f0ad6370 con 0x5628f0782f60 2024-05-21T15:19:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.439+0000 7f760cdcd640 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5628f0782f60 0x5628f0b7fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.441+0000 7f760cdcd640 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5628f0782f60 0x5628f0b7fac0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e4a872ce4b77abb server_cookie=fe16a5520928d81e in_seq=0 out_seq=0 2024-05-21T15:19:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.441+0000 7f75f0ff9640 10 client.26114 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:19:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.443+0000 7f75f0ff9640 1 -- 172.21.15.78:0/2195706598 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628f0ad6370 con 0x5628f0782f60 2024-05-21T15:19:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.443+0000 7f75f0ff9640 10 client.26114 handle_command_reply: tid=0 2024-05-21T15:19:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.443+0000 7f76167bb740 1 client.26114 shutdown 2024-05-21T15:19:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.443+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f07a36b0 msgr2=0x5628f0b6d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.443+0000 7f76167bb740 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f07a36b0 0x5628f0b6d8b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5628f0b642e0 tx=0x7f760402aac0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.443+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 shutdown_connections 2024-05-21T15:19:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.443+0000 7f76167bb740 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5628f0782f60 0x5628f0b7fac0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.443+0000 7f75f0ff9640 0 client.26114 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:19:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.444+0000 7f76167bb740 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f07a36b0 0x5628f0b6d8b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.444+0000 7f76167bb740 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07c4390 0x5628f0b73490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.444+0000 7f76167bb740 1 --2- 172.21.15.78:0/2195706598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b6d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.444+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 >> 172.21.15.78:0/2195706598 conn(0x5628f07b84e0 msgr2=0x5628f0b64a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.444+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 shutdown_connections 2024-05-21T15:19:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.444+0000 7f76167bb740 1 -- 172.21.15.78:0/2195706598 wait complete. 2024-05-21T15:19:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.444+0000 7f76167bb740 20 client.26114 trim_cache size 0 max 16384 2024-05-21T15:19:31.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.470+0000 7f76167bb740 1 Processor -- start 2024-05-21T15:19:31.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.470+0000 7f76167bb740 1 -- start start 2024-05-21T15:19:31.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.470+0000 7f76167bb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b6d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f76167bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f079b4d0 0x5628f0b6efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f76167bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 0x5628f0b66a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f76167bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76081b0de0 con 0x5628f07a36b0 2024-05-21T15:19:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f76167bb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f76081b2ad0 con 0x5628f0782f60 2024-05-21T15:19:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f760cdcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b6d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f760cdcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b6d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:46028/0 (socket says 172.21.15.78:46028) 2024-05-21T15:19:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f76167bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f76081b14b0 con 0x5628f079b4d0 2024-05-21T15:19:31.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f760cdcd640 1 -- 172.21.15.78:0/2444508658 learned_addr learned my addr 172.21.15.78:0/2444508658 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.471+0000 7f760cdcd640 1 --2- 172.21.15.78:0/2444508658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 0x5628f0b66a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.472+0000 7f760cdcd640 1 -- 172.21.15.78:0/2444508658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f079b4d0 msgr2=0x5628f0b6efb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:19:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.472+0000 7f760cdcd640 1 --2- 172.21.15.78:0/2444508658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f079b4d0 0x5628f0b6efb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.472+0000 7f760cdcd640 1 -- 172.21.15.78:0/2444508658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 msgr2=0x5628f0b66a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.472+0000 7f760cdcd640 1 --2- 172.21.15.78:0/2444508658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 0x5628f0b66a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.472+0000 7f760cdcd640 1 -- 172.21.15.78:0/2444508658 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628f0b64540 con 0x5628f0782f60 2024-05-21T15:19:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.472+0000 7f760cdcd640 1 --2- 172.21.15.78:0/2444508658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b6d6d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f75f8001eb0 tx=0x7f75f8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd09475c04361766 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.472+0000 7f75f2ffd640 1 -- 172.21.15.78:0/2444508658 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8003a70 con 0x5628f0782f60 2024-05-21T15:19:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.473+0000 7f75f2ffd640 1 -- 172.21.15.78:0/2444508658 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f8005500 con 0x5628f0782f60 2024-05-21T15:19:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.473+0000 7f75f2ffd640 1 -- 172.21.15.78:0/2444508658 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8004a40 con 0x5628f0782f60 2024-05-21T15:19:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 -- 172.21.15.78:0/2444508658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 msgr2=0x5628f0b6d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 --2- 172.21.15.78:0/2444508658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b6d6d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f75f8001eb0 tx=0x7f75f8013040 comp rx=0 tx=0).stop 2024-05-21T15:19:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 -- 172.21.15.78:0/2444508658 shutdown_connections 2024-05-21T15:19:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 --2- 172.21.15.78:0/2444508658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f07a36b0 0x5628f0b66a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 --2- 172.21.15.78:0/2444508658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f079b4d0 0x5628f0b6efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 --2- 172.21.15.78:0/2444508658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b6d6d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 -- 172.21.15.78:0/2444508658 >> 172.21.15.78:0/2444508658 conn(0x5628f07b84e0 msgr2=0x5628f0b6e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 -- 172.21.15.78:0/2444508658 shutdown_connections 2024-05-21T15:19:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.474+0000 7f76167bb740 1 -- 172.21.15.78:0/2444508658 wait complete. 2024-05-21T15:19:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.475+0000 7f76167bb740 1 -- start start 2024-05-21T15:19:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.475+0000 7f76167bb740 1 Processor -- start 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.475+0000 7f76167bb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b76710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f76167bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b76c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f76167bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07a36b0 0x5628f0b77190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f76167bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76081b2340 con 0x5628f079b4d0 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f76167bb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f76081b1c40 con 0x5628f0782f60 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f76167bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f76081aedc0 con 0x5628f07a36b0 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f76167bb740 10 client.0 _ll_register_callbacks cb 0x7f75f80115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f760d5ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b76c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f760d5ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b76c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36924/0 (socket says 172.21.15.78:36924) 2024-05-21T15:19:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.476+0000 7f760d5ce640 1 -- 172.21.15.78:0/3394280587 learned_addr learned my addr 172.21.15.78:0/3394280587 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f760d5ce640 1 -- 172.21.15.78:0/3394280587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07a36b0 msgr2=0x5628f0b77190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f760d5ce640 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07a36b0 0x5628f0b77190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f760d5ce640 1 -- 172.21.15.78:0/3394280587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 msgr2=0x5628f0b76710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f760d5ce640 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b76710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f760d5ce640 1 -- 172.21.15.78:0/3394280587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628f0bee4f0 con 0x5628f079b4d0 2024-05-21T15:19:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f760d5ce640 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b76c50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7604003e20 tx=0x7f7604030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a31f966a514367c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f75f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:19:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f75f0ff9640 1 -- 172.21.15.78:0/3394280587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604048030 con 0x5628f079b4d0 2024-05-21T15:19:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.477+0000 7f75f0ff9640 1 -- 172.21.15.78:0/3394280587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f76040043a0 con 0x5628f079b4d0 2024-05-21T15:19:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.478+0000 7f76167bb740 1 -- 172.21.15.78:0/3394280587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f8005500 con 0x5628f079b4d0 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.478+0000 7f75f0ff9640 1 -- 172.21.15.78:0/3394280587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f760405e050 con 0x5628f079b4d0 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.478+0000 7f76167bb740 10 client.16647 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.478+0000 7f76167bb740 1 -- 172.21.15.78:0/3394280587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628f0bef750 con 0x5628f079b4d0 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.479+0000 7f75f0ff9640 1 -- 172.21.15.78:0/3394280587 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f8005500 con 0x5628f079b4d0 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.479+0000 7f75f0ff9640 1 -- 172.21.15.78:0/3394280587 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f760403a070 con 0x5628f079b4d0 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.479+0000 7f76167bb740 10 client.16647 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.479+0000 7f76167bb740 10 client.16647 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.479+0000 7f76167bb740 20 client.16647 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.479+0000 7f76167bb740 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5628f07c4390 0x5628f0bf1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.479+0000 7f76167bb740 4 client.16647 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:19:31.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.479+0000 7f76167bb740 1 -- 172.21.15.78:0/3394280587 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628f0ad06c0 con 0x5628f07c4390 2024-05-21T15:19:31.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.480+0000 7f760cdcd640 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5628f07c4390 0x5628f0bf1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.482+0000 7f760cdcd640 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5628f07c4390 0x5628f0bf1e20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b8a27aa283e21e0 server_cookie=ca2edf126db652ba in_seq=0 out_seq=0 2024-05-21T15:19:31.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.482+0000 7f75f0ff9640 10 client.16647 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.482+0000 7f75f0ff9640 1 -- 172.21.15.78:0/3394280587 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628f0ad06c0 con 0x5628f07c4390 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.482+0000 7f75f0ff9640 10 client.16647 handle_command_reply: tid=0 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 client.16647 shutdown 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 -- 172.21.15.78:0/3394280587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 msgr2=0x5628f0b76c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b76c50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7604003e20 tx=0x7f7604030ad0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 -- 172.21.15.78:0/3394280587 shutdown_connections 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5628f07c4390 0x5628f0bf1e20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628f07a36b0 0x5628f0b77190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628f079b4d0 0x5628f0b76c50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 --2- 172.21.15.78:0/3394280587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5628f0782f60 0x5628f0b76710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f75f0ff9640 0 client.16647 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:19:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 -- 172.21.15.78:0/3394280587 >> 172.21.15.78:0/3394280587 conn(0x5628f07b84e0 msgr2=0x5628f0b62ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 -- 172.21.15.78:0/3394280587 shutdown_connections 2024-05-21T15:19:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 1 -- 172.21.15.78:0/3394280587 wait complete. 2024-05-21T15:19:31.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.483+0000 7f76167bb740 20 client.16647 trim_cache size 0 max 16384 2024-05-21T15:19:31.488 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:19:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 -- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f75f40a2e80 msgr2=0x7f75f40a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f75f40a2e80 0x7f75f40a5390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7600098480 tx=0x7f7600011040 comp rx=0 tx=0).stop 2024-05-21T15:19:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 -- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7608108f70 msgr2=0x7f7608199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7608108f70 0x7f7608199660 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f760000ed70 tx=0x7f760000c6e0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 -- 172.21.15.78:0/4173721305 shutdown_connections 2024-05-21T15:19:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7608108ad0 0x7f760819fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f75f40a2e80 0x7f75f40a5390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7608108f70 0x7f7608199660 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 --2- 172.21.15.78:0/4173721305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7608103220 0x7f760819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 -- 172.21.15.78:0/4173721305 >> 172.21.15.78:0/4173721305 conn(0x7f76080fe1b0 msgr2=0x7f76080fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.485+0000 7f760f832640 1 -- 172.21.15.78:0/4173721305 shutdown_connections 2024-05-21T15:19:31.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.486+0000 7f760f832640 1 -- 172.21.15.78:0/4173721305 wait complete. 2024-05-21T15:19:31.507 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:19:31.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.686+0000 7fde5e93e640 1 -- 172.21.15.78:0/3605840659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde58106400 msgr2=0x7fde5810dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.686+0000 7fde5e93e640 1 --2- 172.21.15.78:0/3605840659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde58106400 0x7fde5810dba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fde50009ca0 tx=0x7fde5002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.688+0000 7fde5e93e640 1 -- 172.21.15.78:0/3605840659 shutdown_connections 2024-05-21T15:19:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.688+0000 7fde5e93e640 1 --2- 172.21.15.78:0/3605840659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde58106400 0x7fde5810dba0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.688+0000 7fde5e93e640 1 --2- 172.21.15.78:0/3605840659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde58105940 0x7fde58105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.688+0000 7fde5e93e640 1 --2- 172.21.15.78:0/3605840659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde581013b0 0x7fde581017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.688+0000 7fde5e93e640 1 -- 172.21.15.78:0/3605840659 >> 172.21.15.78:0/3605840659 conn(0x7fde58076830 msgr2=0x7fde58076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.688+0000 7fde5e93e640 1 -- 172.21.15.78:0/3605840659 shutdown_connections 2024-05-21T15:19:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.688+0000 7fde5e93e640 1 -- 172.21.15.78:0/3605840659 wait complete. 2024-05-21T15:19:31.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.689+0000 7fde5e93e640 1 Processor -- start 2024-05-21T15:19:31.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.690+0000 7fde5e93e640 1 -- start start 2024-05-21T15:19:31.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.690+0000 7fde5e93e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde581013b0 0x7fde581975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde5e93e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde58105940 0x7fde58197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde5e93e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde58106400 0x7fde5819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde581013b0 0x7fde581975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde5e93e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde58110c30 con 0x7fde58106400 2024-05-21T15:19:31.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde5e93e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fde58110ab0 con 0x7fde58105940 2024-05-21T15:19:31.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde581013b0 0x7fde581975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44176/0 (socket says 172.21.15.78:44176) 2024-05-21T15:19:31.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde5e93e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde58110db0 con 0x7fde581013b0 2024-05-21T15:19:31.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde57fff640 1 -- 172.21.15.78:0/1943955142 learned_addr learned my addr 172.21.15.78:0/1943955142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde577fe640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde58105940 0x7fde58197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde5cedb640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde58106400 0x7fde5819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde57fff640 1 -- 172.21.15.78:0/1943955142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde58105940 msgr2=0x7fde58197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde57fff640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde58105940 0x7fde58197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde57fff640 1 -- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde58106400 msgr2=0x7fde5819be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde57fff640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde58106400 0x7fde5819be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.691+0000 7fde57fff640 1 -- 172.21.15.78:0/1943955142 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5819c450 con 0x7fde581013b0 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.692+0000 7fde57fff640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde581013b0 0x7fde581975f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde4800eb20 tx=0x7fde4800c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.692+0000 7fde557fa640 1 -- 172.21.15.78:0/1943955142 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48004220 con 0x7fde581013b0 2024-05-21T15:19:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.692+0000 7fde5e93e640 1 -- 172.21.15.78:0/1943955142 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5819c740 con 0x7fde581013b0 2024-05-21T15:19:31.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.692+0000 7fde557fa640 1 -- 172.21.15.78:0/1943955142 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4800ce10 con 0x7fde581013b0 2024-05-21T15:19:31.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.693+0000 7fde557fa640 1 -- 172.21.15.78:0/1943955142 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48013910 con 0x7fde581013b0 2024-05-21T15:19:31.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.693+0000 7fde5e93e640 1 -- 172.21.15.78:0/1943955142 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde581a40c0 con 0x7fde581013b0 2024-05-21T15:19:31.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.696+0000 7fde557fa640 1 -- 172.21.15.78:0/1943955142 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fde480035f0 con 0x7fde581013b0 2024-05-21T15:19:31.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.701+0000 7fde657ee740 1 Processor -- start 2024-05-21T15:19:31.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.701+0000 7fde657ee740 1 -- start start 2024-05-21T15:19:31.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde557fa640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fde400a2df0 0x7fde400a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde657ee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da2840ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde657ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da2840b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde657ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da2840b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde657ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da28370c40 con 0x55da280434d0 2024-05-21T15:19:31.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde657ee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55da28318580 con 0x55da2806c390 2024-05-21T15:19:31.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde657ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da2836cc00 con 0x55da2804b6b0 2024-05-21T15:19:31.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde5cedb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da2840b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde5cedb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da2840b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44178/0 (socket says 172.21.15.78:44178) 2024-05-21T15:19:31.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde5cedb640 1 -- 172.21.15.78:0/3468652964 learned_addr learned my addr 172.21.15.78:0/3468652964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde5cedb640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fde400a2df0 0x7fde400a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.702+0000 7fde577fe640 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da2840b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.703+0000 7fde557fa640 1 -- 172.21.15.78:0/1943955142 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fde480d37c0 con 0x7fde581013b0 2024-05-21T15:19:31.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.703+0000 7fde57fff640 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da2840ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.703+0000 7fde577fe640 1 -- 172.21.15.78:0/3468652964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 msgr2=0x55da2840b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.703+0000 7fde577fe640 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da2840b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.703+0000 7fde577fe640 1 -- 172.21.15.78:0/3468652964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 msgr2=0x55da2840ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.703+0000 7fde577fe640 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da2840ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.703+0000 7fde577fe640 1 -- 172.21.15.78:0/3468652964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da280843f0 con 0x55da280434d0 2024-05-21T15:19:31.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.703+0000 7fde57fff640 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da2840ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:19:31.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.704+0000 7fde577fe640 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da2840b260 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fde58074460 tx=0x7fde4c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0f6881dcef1d17 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.705+0000 7fde3e7fc640 1 -- 172.21.15.78:0/3468652964 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c010e90 con 0x55da280434d0 2024-05-21T15:19:31.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.705+0000 7fde3e7fc640 1 -- 172.21.15.78:0/3468652964 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c0035f0 con 0x55da280434d0 2024-05-21T15:19:31.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.705+0000 7fde3e7fc640 1 -- 172.21.15.78:0/3468652964 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0116a0 con 0x55da280434d0 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.706+0000 7fde657ee740 1 -- 172.21.15.78:0/3468652964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 msgr2=0x55da2840b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.706+0000 7fde657ee740 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da2840b260 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fde58074460 tx=0x7fde4c0103d0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.706+0000 7fde5cedb640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fde400a2df0 0x7fde400a5300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde50009a60 tx=0x7fde500095c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.707+0000 7fde657ee740 1 -- 172.21.15.78:0/3468652964 shutdown_connections 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.707+0000 7fde657ee740 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da2840b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.707+0000 7fde657ee740 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da2840b260 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.707+0000 7fde657ee740 1 --2- 172.21.15.78:0/3468652964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da2840ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.707+0000 7fde657ee740 1 -- 172.21.15.78:0/3468652964 >> 172.21.15.78:0/3468652964 conn(0x55da280604e0 msgr2=0x55da284074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.707+0000 7fde657ee740 1 -- 172.21.15.78:0/3468652964 shutdown_connections 2024-05-21T15:19:31.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.707+0000 7fde657ee740 1 -- 172.21.15.78:0/3468652964 wait complete. 2024-05-21T15:19:31.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.707+0000 7fde657ee740 1 -- start start 2024-05-21T15:19:31.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.708+0000 7fde657ee740 1 Processor -- start 2024-05-21T15:19:31.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde657ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da28416d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde657ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da284172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde657ee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da28415240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde657ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da28417fb0 con 0x55da280434d0 2024-05-21T15:19:31.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde657ee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55da2832bc40 con 0x55da2806c390 2024-05-21T15:19:31.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde657ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da28418130 con 0x55da2804b6b0 2024-05-21T15:19:31.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da28416d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da28416d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36958/0 (socket says 172.21.15.78:36958) 2024-05-21T15:19:31.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde57fff640 1 -- 172.21.15.78:0/336596017 learned_addr learned my addr 172.21.15.78:0/336596017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde5cedb640 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da28415240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde657ee740 10 client.0 _ll_register_callbacks cb 0x55da28083be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:31.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.709+0000 7fde577fe640 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da284172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde57fff640 1 -- 172.21.15.78:0/336596017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 msgr2=0x55da284172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde57fff640 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da284172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde57fff640 1 -- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 msgr2=0x55da28415240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde57fff640 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da28415240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde57fff640 1 -- 172.21.15.78:0/336596017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2814e7d0 con 0x55da280434d0 2024-05-21T15:19:31.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde5cedb640 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da28415240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:31.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde57fff640 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da28416d80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c8d0 tx=0x7fde480a0570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e53c53a02260e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde377fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:19:31.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde377fe640 1 -- 172.21.15.78:0/336596017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde480a89a0 con 0x55da280434d0 2024-05-21T15:19:31.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.710+0000 7fde377fe640 1 -- 172.21.15.78:0/336596017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fde4809c3a0 con 0x55da280434d0 2024-05-21T15:19:31.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.711+0000 7fde657ee740 1 -- 172.21.15.78:0/336596017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4c0035f0 con 0x55da280434d0 2024-05-21T15:19:31.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.711+0000 7fde377fe640 1 -- 172.21.15.78:0/336596017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fde48003b60 con 0x55da280434d0 2024-05-21T15:19:31.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.711+0000 7fde657ee740 10 client.16659 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:31.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.711+0000 7fde657ee740 1 -- 172.21.15.78:0/336596017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55da284247a0 con 0x55da280434d0 2024-05-21T15:19:31.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.712+0000 7fde377fe640 1 -- 172.21.15.78:0/336596017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde48003d00 con 0x55da280434d0 2024-05-21T15:19:31.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.712+0000 7fde377fe640 1 -- 172.21.15.78:0/336596017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fde480b6030 con 0x55da280434d0 2024-05-21T15:19:31.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.712+0000 7fde657ee740 10 client.16659 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:31.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.712+0000 7fde657ee740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:31.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.712+0000 7fde657ee740 20 client.16659 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:31.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.712+0000 7fde657ee740 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da2802af60 0x55da28427d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.712+0000 7fde657ee740 4 client.16659 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:31.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.712+0000 7fde657ee740 1 -- 172.21.15.78:0/336596017 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fde4c0035f0 con 0x55da2802af60 2024-05-21T15:19:31.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.715+0000 7fde577fe640 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da2802af60 0x55da28427d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.716+0000 7fde377fe640 10 client.16659 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:31.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.716+0000 7fde577fe640 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da2802af60 0x55da28427d50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b08e4f7fdbdc721 server_cookie=a8f12b893f006b15 in_seq=0 out_seq=0 2024-05-21T15:19:31.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.718+0000 7fde377fe640 1 -- 172.21.15.78:0/336596017 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fde4c0035f0 con 0x55da2802af60 2024-05-21T15:19:31.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.718+0000 7fde377fe640 10 client.16659 handle_command_reply: tid=0 2024-05-21T15:19:31.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.718+0000 7fde657ee740 1 client.16659 shutdown 2024-05-21T15:19:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 -- 172.21.15.78:0/336596017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 msgr2=0x55da28416d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da28416d80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c8d0 tx=0x7fde480a0570 comp rx=0 tx=0).stop 2024-05-21T15:19:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 -- 172.21.15.78:0/336596017 shutdown_connections 2024-05-21T15:19:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da2802af60 0x55da28427d50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da2806c390 0x55da28415240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2804b6b0 0x55da284172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde377fe640 0 client.16659 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:31.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 --2- 172.21.15.78:0/336596017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da280434d0 0x55da28416d80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 -- 172.21.15.78:0/336596017 >> 172.21.15.78:0/336596017 conn(0x55da280604e0 msgr2=0x55da2840ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 -- 172.21.15.78:0/336596017 shutdown_connections 2024-05-21T15:19:31.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.719+0000 7fde657ee740 1 -- 172.21.15.78:0/336596017 wait complete. 2024-05-21T15:19:31.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.720+0000 7fde657ee740 20 client.16659 trim_cache size 0 max 16384 2024-05-21T15:19:31.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.745+0000 7fde657ee740 1 Processor -- start 2024-05-21T15:19:31.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.745+0000 7fde657ee740 1 -- start start 2024-05-21T15:19:31.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.745+0000 7fde657ee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2840f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.745+0000 7fde657ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2840bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde657ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 0x55da28406b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde657ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde581acbd0 con 0x55da2804b6b0 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde657ee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ae8c0 con 0x55da280434d0 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde657ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ad2a0 con 0x55da2806c390 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde5cedb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 0x55da28406b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2840bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2840bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36974/0 (socket says 172.21.15.78:36974) 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde577fe640 1 -- 172.21.15.78:0/2403559391 learned_addr learned my addr 172.21.15.78:0/2403559391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde57fff640 1 --2- 172.21.15.78:0/2403559391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2840f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde577fe640 1 -- 172.21.15.78:0/2403559391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 msgr2=0x55da28406b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde577fe640 1 --2- 172.21.15.78:0/2403559391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 0x55da28406b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde577fe640 1 -- 172.21.15.78:0/2403559391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 msgr2=0x55da2840f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde577fe640 1 --2- 172.21.15.78:0/2403559391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2840f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.746+0000 7fde577fe640 1 -- 172.21.15.78:0/2403559391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da284080b0 con 0x55da2804b6b0 2024-05-21T15:19:31.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.747+0000 7fde577fe640 1 --2- 172.21.15.78:0/2403559391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2840bb80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x55da28408810 tx=0x7fde4c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15312c96ca08800a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.747+0000 7fde3dffb640 1 -- 172.21.15.78:0/2403559391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0050c0 con 0x55da2804b6b0 2024-05-21T15:19:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.747+0000 7fde3dffb640 1 -- 172.21.15.78:0/2403559391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c0035f0 con 0x55da2804b6b0 2024-05-21T15:19:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.748+0000 7fde3dffb640 1 -- 172.21.15.78:0/2403559391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0295a0 con 0x55da2804b6b0 2024-05-21T15:19:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.749+0000 7fde657ee740 1 -- 172.21.15.78:0/2403559391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 msgr2=0x55da2840bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.749+0000 7fde657ee740 1 --2- 172.21.15.78:0/2403559391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2840bb80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x55da28408810 tx=0x7fde4c00b5b0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.749+0000 7fde657ee740 1 -- 172.21.15.78:0/2403559391 shutdown_connections 2024-05-21T15:19:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.749+0000 7fde657ee740 1 --2- 172.21.15.78:0/2403559391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 0x55da28406b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.750+0000 7fde657ee740 1 --2- 172.21.15.78:0/2403559391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2840bb80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.750+0000 7fde657ee740 1 --2- 172.21.15.78:0/2403559391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2840f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.750+0000 7fde657ee740 1 -- 172.21.15.78:0/2403559391 >> 172.21.15.78:0/2403559391 conn(0x55da280604e0 msgr2=0x55da2840e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.750+0000 7fde657ee740 1 -- 172.21.15.78:0/2403559391 shutdown_connections 2024-05-21T15:19:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.750+0000 7fde657ee740 1 -- 172.21.15.78:0/2403559391 wait complete. 2024-05-21T15:19:31.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.751+0000 7fde657ee740 1 -- start start 2024-05-21T15:19:31.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.751+0000 7fde657ee740 1 Processor -- start 2024-05-21T15:19:31.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.752+0000 7fde657ee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2841a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.752+0000 7fde657ee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2841acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.752+0000 7fde657ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 0x55da284968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.752+0000 7fde657ee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ae130 con 0x55da2804b6b0 2024-05-21T15:19:31.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.752+0000 7fde657ee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ada30 con 0x55da280434d0 2024-05-21T15:19:31.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.752+0000 7fde657ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde581aabb0 con 0x55da2806c390 2024-05-21T15:19:31.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.752+0000 7fde657ee740 10 client.0 _ll_register_callbacks cb 0x7fde4809f690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2841a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2841a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:46108/0 (socket says 172.21.15.78:46108) 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 -- 172.21.15.78:0/1684994872 learned_addr learned my addr 172.21.15.78:0/1684994872 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde5cedb640 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 0x55da284968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde577fe640 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2841acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 -- 172.21.15.78:0/1684994872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 msgr2=0x55da284968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 0x55da284968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 -- 172.21.15.78:0/1684994872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 msgr2=0x55da2841acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2841acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 -- 172.21.15.78:0/1684994872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da28496e10 con 0x55da280434d0 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde577fe640 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2841acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:19:31.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde57fff640 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2841a780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde4807ee20 tx=0x7fde4809bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a43219af19a2af33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.753+0000 7fde377fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.754+0000 7fde377fe640 1 -- 172.21.15.78:0/1684994872 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4809f3d0 con 0x55da280434d0 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.754+0000 7fde377fe640 1 -- 172.21.15.78:0/1684994872 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fde480a8510 con 0x55da280434d0 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.754+0000 7fde657ee740 1 -- 172.21.15.78:0/1684994872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4c0035f0 con 0x55da280434d0 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.754+0000 7fde377fe640 1 -- 172.21.15.78:0/1684994872 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fde48003d00 con 0x55da280434d0 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.754+0000 7fde657ee740 10 client.26132 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.754+0000 7fde657ee740 1 -- 172.21.15.78:0/1684994872 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55da28498020 con 0x55da280434d0 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.755+0000 7fde377fe640 1 -- 172.21.15.78:0/1684994872 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c0035f0 con 0x55da280434d0 2024-05-21T15:19:31.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.755+0000 7fde377fe640 1 -- 172.21.15.78:0/1684994872 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fde48086050 con 0x55da280434d0 2024-05-21T15:19:31.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.755+0000 7fde657ee740 10 client.26132 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:31.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.755+0000 7fde657ee740 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:31.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.755+0000 7fde657ee740 20 client.26132 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:31.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.755+0000 7fde657ee740 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da280003a0 0x55da2849a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:31.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.755+0000 7fde657ee740 4 client.26132 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:19:31.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.755+0000 7fde657ee740 1 -- 172.21.15.78:0/1684994872 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da2837e370 con 0x55da280003a0 2024-05-21T15:19:31.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.756+0000 7fde577fe640 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da280003a0 0x55da2849a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:31.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.757+0000 7fde577fe640 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da280003a0 0x55da2849a6a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f460c6d42ad61a server_cookie=5498a64211353273 in_seq=0 out_seq=0 2024-05-21T15:19:31.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.757+0000 7fde377fe640 10 client.26132 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:31.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde377fe640 1 -- 172.21.15.78:0/1684994872 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da2837e370 con 0x55da280003a0 2024-05-21T15:19:31.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde377fe640 10 client.26132 handle_command_reply: tid=0 2024-05-21T15:19:31.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 client.26132 shutdown 2024-05-21T15:19:31.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 -- 172.21.15.78:0/1684994872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 msgr2=0x55da2841a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2841a780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde4807ee20 tx=0x7fde4809bfd0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 -- 172.21.15.78:0/1684994872 shutdown_connections 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da280003a0 0x55da2849a6a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da2806c390 0x55da284968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da2804b6b0 0x55da2841acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 --2- 172.21.15.78:0/1684994872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da280434d0 0x55da2841a780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 -- 172.21.15.78:0/1684994872 >> 172.21.15.78:0/1684994872 conn(0x55da280604e0 msgr2=0x55da28413b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde377fe640 0 client.26132 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.758+0000 7fde657ee740 1 -- 172.21.15.78:0/1684994872 shutdown_connections 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.759+0000 7fde657ee740 1 -- 172.21.15.78:0/1684994872 wait complete. 2024-05-21T15:19:31.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.759+0000 7fde657ee740 20 client.26132 trim_cache size 0 max 16384 2024-05-21T15:19:31.763 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:19:31.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 -- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fde400a2df0 msgr2=0x7fde400a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fde400a2df0 0x7fde400a5300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde50009a60 tx=0x7fde500095c0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 -- 172.21.15.78:0/1943955142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde581013b0 msgr2=0x7fde581975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:31.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde581013b0 0x7fde581975f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde4800eb20 tx=0x7fde4800c3d0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 -- 172.21.15.78:0/1943955142 shutdown_connections 2024-05-21T15:19:31.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde58105940 0x7fde58197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fde400a2df0 0x7fde400a5300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde58106400 0x7fde5819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 --2- 172.21.15.78:0/1943955142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde581013b0 0x7fde581975f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.761+0000 7fde5e93e640 1 -- 172.21.15.78:0/1943955142 >> 172.21.15.78:0/1943955142 conn(0x7fde58076830 msgr2=0x7fde580fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.762+0000 7fde5e93e640 1 -- 172.21.15.78:0/1943955142 shutdown_connections 2024-05-21T15:19:31.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:31.762+0000 7fde5e93e640 1 -- 172.21.15.78:0/1943955142 wait complete. 2024-05-21T15:19:32.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:32 smithi078 ceph-mon[31997]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 40 KiB/s rd, 90 MiB/s wr, 16.13k op/s 2024-05-21T15:19:32.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:32 smithi084 ceph-mon[36334]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 40 KiB/s rd, 90 MiB/s wr, 16.13k op/s 2024-05-21T15:19:32.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:32 smithi172 ceph-mon[36402]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 40 KiB/s rd, 90 MiB/s wr, 16.13k op/s 2024-05-21T15:19:32.782 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:19:32.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:19:32.782 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:19:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.948+0000 7fa30befe640 1 -- 172.21.15.78:0/1064947044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa30410e3a0 msgr2=0x7fa304115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.948+0000 7fa30befe640 1 --2- 172.21.15.78:0/1064947044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa30410e3a0 0x7fa304115b40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009ca0 tx=0x7fa2fc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.948+0000 7fa30befe640 1 -- 172.21.15.78:0/1064947044 shutdown_connections 2024-05-21T15:19:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.948+0000 7fa30befe640 1 --2- 172.21.15.78:0/1064947044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa30410d8e0 0x7fa30410dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.948+0000 7fa30befe640 1 --2- 172.21.15.78:0/1064947044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa30410e3a0 0x7fa304115b40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.948+0000 7fa30befe640 1 --2- 172.21.15.78:0/1064947044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa304109350 0x7fa304109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.948+0000 7fa30befe640 1 -- 172.21.15.78:0/1064947044 >> 172.21.15.78:0/1064947044 conn(0x7fa304075050 msgr2=0x7fa304075460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.948+0000 7fa30befe640 1 -- 172.21.15.78:0/1064947044 shutdown_connections 2024-05-21T15:19:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.949+0000 7fa30befe640 1 -- 172.21.15.78:0/1064947044 wait complete. 2024-05-21T15:19:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.950+0000 7fa30befe640 1 Processor -- start 2024-05-21T15:19:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.950+0000 7fa30befe640 1 -- start start 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.950+0000 7fa30befe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa304109350 0x7fa3041a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa30befe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa30410d8e0 0x7fa3041a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa30befe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa30410e3a0 0x7fa30419f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa309c9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa304109350 0x7fa3041a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa30befe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa304119450 con 0x7fa30410d8e0 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa30befe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041192d0 con 0x7fa30410e3a0 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa30befe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041195d0 con 0x7fa304109350 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa309c9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa304109350 0x7fa3041a57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44224/0 (socket says 172.21.15.78:44224) 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa309c9a640 1 -- 172.21.15.78:0/3176985257 learned_addr learned my addr 172.21.15.78:0/3176985257 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.951+0000 7fa30a49b640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa30410e3a0 0x7fa30419f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.952+0000 7fa309c9a640 1 -- 172.21.15.78:0/3176985257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa30410e3a0 msgr2=0x7fa30419f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.952+0000 7fa309c9a640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa30410e3a0 0x7fa30419f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.952+0000 7fa309c9a640 1 -- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa30410d8e0 msgr2=0x7fa3041a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.952+0000 7fa309c9a640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa30410d8e0 0x7fa3041a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.952+0000 7fa309c9a640 1 -- 172.21.15.78:0/3176985257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3041a0050 con 0x7fa304109350 2024-05-21T15:19:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.953+0000 7fa309c9a640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa304109350 0x7fa3041a57a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa30000e9f0 tx=0x7fa30000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.953+0000 7fa2faffd640 1 -- 172.21.15.78:0/3176985257 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30000cd80 con 0x7fa304109350 2024-05-21T15:19:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.953+0000 7fa2faffd640 1 -- 172.21.15.78:0/3176985257 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa300005500 con 0x7fa304109350 2024-05-21T15:19:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.953+0000 7fa2faffd640 1 -- 172.21.15.78:0/3176985257 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300010690 con 0x7fa304109350 2024-05-21T15:19:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.953+0000 7fa30befe640 1 -- 172.21.15.78:0/3176985257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3041a0340 con 0x7fa304109350 2024-05-21T15:19:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.953+0000 7fa30befe640 1 -- 172.21.15.78:0/3176985257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3041a07e0 con 0x7fa304109350 2024-05-21T15:19:32.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.958+0000 7fa312da6740 1 Processor -- start 2024-05-21T15:19:32.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.958+0000 7fa312da6740 1 -- start start 2024-05-21T15:19:32.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa312da6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed771620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa312da6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed771b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa312da6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3b1730 0x556bed7720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa312da6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556bed6d6990 con 0x556bed3d2410 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa312da6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556bed67e5b0 con 0x556bed3b1730 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa312da6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556bed6d2950 con 0x556bed3a9550 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa30a49b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed771620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa30a49b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed771620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37010/0 (socket says 172.21.15.78:37010) 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa30a49b640 1 -- 172.21.15.78:0/4153057476 learned_addr learned my addr 172.21.15.78:0/4153057476 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.959+0000 7fa309499640 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3b1730 0x556bed7720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa30a49b640 1 -- 172.21.15.78:0/4153057476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 msgr2=0x556bed771b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:19:32.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa30a49b640 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed771b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa30a49b640 1 -- 172.21.15.78:0/4153057476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3b1730 msgr2=0x556bed7720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa30a49b640 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3b1730 0x556bed7720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa30a49b640 1 -- 172.21.15.78:0/4153057476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bed4b27e0 con 0x556bed3d2410 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa309499640 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3b1730 0x556bed7720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa2faffd640 1 -- 172.21.15.78:0/3176985257 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa3000046c0 con 0x7fa304109350 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa30a49b640 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed771620 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc035000 tx=0x7fa2fc0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a1446e88a2fa43 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa2effff640 1 -- 172.21.15.78:0/4153057476 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc003970 con 0x556bed3d2410 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.960+0000 7fa2effff640 1 -- 172.21.15.78:0/4153057476 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2fc036030 con 0x556bed3d2410 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.961+0000 7fa2effff640 1 -- 172.21.15.78:0/4153057476 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc004410 con 0x556bed3d2410 2024-05-21T15:19:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.961+0000 7fa2faffd640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa2f00a2e00 0x7fa2f00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.961+0000 7fa2faffd640 1 -- 172.21.15.78:0/3176985257 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa3000cf940 con 0x7fa304109350 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.961+0000 7fa309c9a640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa2f00a2e00 0x7fa2f00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.962+0000 7fa312da6740 1 -- 172.21.15.78:0/4153057476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 msgr2=0x556bed771620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.962+0000 7fa312da6740 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed771620 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc035000 tx=0x7fa2fc0094a0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.962+0000 7fa309c9a640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa2f00a2e00 0x7fa2f00a5310 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x556bed76f110 tx=0x7fa300009310 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.962+0000 7fa312da6740 1 -- 172.21.15.78:0/4153057476 shutdown_connections 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.962+0000 7fa312da6740 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3b1730 0x556bed7720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.962+0000 7fa312da6740 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed771b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.962+0000 7fa312da6740 1 --2- 172.21.15.78:0/4153057476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed771620 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.962+0000 7fa312da6740 1 -- 172.21.15.78:0/4153057476 >> 172.21.15.78:0/4153057476 conn(0x556bed3c6560 msgr2=0x556bed76ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.963+0000 7fa312da6740 1 -- 172.21.15.78:0/4153057476 shutdown_connections 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.963+0000 7fa312da6740 1 -- 172.21.15.78:0/4153057476 wait complete. 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.963+0000 7fa312da6740 1 -- start start 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.963+0000 7fa312da6740 1 Processor -- start 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa312da6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3a9550 0x556bed77d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa312da6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3b1730 0x556bed77dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa312da6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed77bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa312da6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556bed77e790 con 0x556bed3d2410 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa312da6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556bed691b40 con 0x556bed3a9550 2024-05-21T15:19:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa312da6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556bed77e910 con 0x556bed3b1730 2024-05-21T15:19:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa312da6740 10 client.0 _ll_register_callbacks cb 0x556bed6860f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa30a49b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed77bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa30a49b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed77bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37016/0 (socket says 172.21.15.78:37016) 2024-05-21T15:19:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa30a49b640 1 -- 172.21.15.78:0/2582625895 learned_addr learned my addr 172.21.15.78:0/2582625895 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa30a49b640 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3a9550 0x556bed77d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.964+0000 7fa309499640 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3b1730 0x556bed77dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.965+0000 7fa30a49b640 1 -- 172.21.15.78:0/2582625895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3b1730 msgr2=0x556bed77dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.965+0000 7fa30a49b640 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3b1730 0x556bed77dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.965+0000 7fa30a49b640 1 -- 172.21.15.78:0/2582625895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 msgr2=0x556bed77bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.965+0000 7fa30a49b640 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed77bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.965+0000 7fa30a49b640 1 -- 172.21.15.78:0/2582625895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bed512f70 con 0x556bed3a9550 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.965+0000 7fa30a49b640 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3a9550 0x556bed77d6d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009bf0 tx=0x7fa2fc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e220699bf37277d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.965+0000 7fa2ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.966+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2582625895 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc059060 con 0x556bed3a9550 2024-05-21T15:19:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.966+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2582625895 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa2fc004190 con 0x556bed3a9550 2024-05-21T15:19:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.968+0000 7fa312da6740 1 -- 172.21.15.78:0/2582625895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2fc036030 con 0x556bed3a9550 2024-05-21T15:19:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.968+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2582625895 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa2fc004390 con 0x556bed3a9550 2024-05-21T15:19:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.968+0000 7fa312da6740 10 client.26144 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.968+0000 7fa312da6740 1 -- 172.21.15.78:0/2582625895 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556bed3aa170 con 0x556bed3a9550 2024-05-21T15:19:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2582625895 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2fc036030 con 0x556bed3a9550 2024-05-21T15:19:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2582625895 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fa2fc043070 con 0x556bed3a9550 2024-05-21T15:19:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa312da6740 10 client.26144 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa312da6740 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:32.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa312da6740 20 client.26144 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa312da6740 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bed3911f0 0x556bed78de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa312da6740 4 client.26144 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa312da6740 1 -- 172.21.15.78:0/2582625895 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bed6e40c0 con 0x556bed3911f0 2024-05-21T15:19:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.969+0000 7fa309499640 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bed3911f0 0x556bed78de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.970+0000 7fa309499640 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bed3911f0 0x556bed78de20 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=461d79bbd9d17137 server_cookie=1fa280d0cce49184 in_seq=0 out_seq=0 2024-05-21T15:19:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.970+0000 7fa2ed7fa640 10 client.26144 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.971+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2582625895 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556bed6e40c0 con 0x556bed3911f0 2024-05-21T15:19:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.971+0000 7fa2ed7fa640 10 client.26144 handle_command_reply: tid=0 2024-05-21T15:19:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 client.26144 shutdown 2024-05-21T15:19:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 -- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3a9550 msgr2=0x556bed77d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3a9550 0x556bed77d6d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009bf0 tx=0x7fa2fc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 -- 172.21.15.78:0/2582625895 shutdown_connections 2024-05-21T15:19:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bed3911f0 0x556bed78de20 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3d2410 0x556bed77bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3b1730 0x556bed77dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 --2- 172.21.15.78:0/2582625895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3a9550 0x556bed77d6d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa312da6740 1 -- 172.21.15.78:0/2582625895 >> 172.21.15.78:0/2582625895 conn(0x556bed3c6560 msgr2=0x556bed771450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.972+0000 7fa2ed7fa640 0 client.26144 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.973+0000 7fa312da6740 1 -- 172.21.15.78:0/2582625895 shutdown_connections 2024-05-21T15:19:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.973+0000 7fa312da6740 1 -- 172.21.15.78:0/2582625895 wait complete. 2024-05-21T15:19:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:32.974+0000 7fa312da6740 20 client.26144 trim_cache size 0 max 16384 2024-05-21T15:19:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.000+0000 7fa312da6740 1 Processor -- start 2024-05-21T15:19:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.000+0000 7fa312da6740 1 -- start start 2024-05-21T15:19:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa312da6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed771ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa312da6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed783ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa312da6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 0x556bed7840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa30a49b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed771ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa312da6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041b5000 con 0x556bed3b1730 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa312da6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041b6cf0 con 0x556bed3d2410 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa312da6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041b56d0 con 0x556bed3a9550 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa309499640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed783ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa30a49b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed771ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44236/0 (socket says 172.21.15.78:44236) 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa30a49b640 1 -- 172.21.15.78:0/2076786604 learned_addr learned my addr 172.21.15.78:0/2076786604 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa30a49b640 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 0x556bed7840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa309499640 1 -- 172.21.15.78:0/2076786604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 msgr2=0x556bed771ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa309499640 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed771ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa309499640 1 -- 172.21.15.78:0/2076786604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 msgr2=0x556bed7840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa309499640 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 0x556bed7840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.001+0000 7fa309499640 1 -- 172.21.15.78:0/2076786604 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bed768e90 con 0x556bed3b1730 2024-05-21T15:19:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.002+0000 7fa309499640 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed783ba0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40058c0 tx=0x7fa2f400d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a33782ee977769 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.002+0000 7fa30a49b640 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 0x556bed7840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.002+0000 7fa2ef7fe640 1 -- 172.21.15.78:0/2076786604 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f400de60 con 0x556bed3b1730 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.003+0000 7fa2ef7fe640 1 -- 172.21.15.78:0/2076786604 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f401a810 con 0x556bed3b1730 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.003+0000 7fa2ef7fe640 1 -- 172.21.15.78:0/2076786604 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4011b20 con 0x556bed3b1730 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 -- 172.21.15.78:0/2076786604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 msgr2=0x556bed783ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed783ba0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40058c0 tx=0x7fa2f400d440 comp rx=0 tx=0).stop 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 -- 172.21.15.78:0/2076786604 shutdown_connections 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 0x556bed7840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed783ba0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 --2- 172.21.15.78:0/2076786604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed771ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 -- 172.21.15.78:0/2076786604 >> 172.21.15.78:0/2076786604 conn(0x556bed3c6560 msgr2=0x556bed7713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 -- 172.21.15.78:0/2076786604 shutdown_connections 2024-05-21T15:19:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 -- 172.21.15.78:0/2076786604 wait complete. 2024-05-21T15:19:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 -- start start 2024-05-21T15:19:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.004+0000 7fa312da6740 1 Processor -- start 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.005+0000 7fa312da6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed782ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.005+0000 7fa312da6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed7830e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.005+0000 7fa312da6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 0x556bed7f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.005+0000 7fa312da6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041b6560 con 0x556bed3b1730 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.005+0000 7fa312da6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041b5e60 con 0x556bed3d2410 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.005+0000 7fa312da6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041b2fe0 con 0x556bed3a9550 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.005+0000 7fa312da6740 10 client.0 _ll_register_callbacks cb 0x7fa2fc0098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed7830e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed7830e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37044/0 (socket says 172.21.15.78:37044) 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 -- 172.21.15.78:0/2674263509 learned_addr learned my addr 172.21.15.78:0/2674263509 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:33.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 -- 172.21.15.78:0/2674263509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 msgr2=0x556bed782ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed782ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 -- 172.21.15.78:0/2674263509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 msgr2=0x556bed7f3fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 0x556bed7f3fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 -- 172.21.15.78:0/2674263509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bed7839a0 con 0x556bed3b1730 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.006+0000 7fa309499640 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed7830e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4004460 tx=0x7fa2f400d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718922e67e81c6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.007+0000 7fa2ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.007+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2674263509 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f401aa80 con 0x556bed3b1730 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.007+0000 7fa312da6740 1 -- 172.21.15.78:0/2674263509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f401a810 con 0x556bed3b1730 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.007+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2674263509 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa2f400dc60 con 0x556bed3b1730 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.008+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2674263509 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa2f401a330 con 0x556bed3b1730 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.008+0000 7fa312da6740 10 client.16677 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:33.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.008+0000 7fa312da6740 1 -- 172.21.15.78:0/2674263509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556bed7f55e0 con 0x556bed3b1730 2024-05-21T15:19:33.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.008+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2674263509 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f401a810 con 0x556bed3b1730 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.008+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2674263509 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fa2f402a020 con 0x556bed3b1730 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.008+0000 7fa312da6740 10 client.16677 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.008+0000 7fa312da6740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.008+0000 7fa312da6740 20 client.16677 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.009+0000 7fa312da6740 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bed366630 0x556bed7f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.009+0000 7fa312da6740 4 client.16677 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.009+0000 7fa312da6740 1 -- 172.21.15.78:0/2674263509 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556bed6de410 con 0x556bed366630 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.009+0000 7fa30a49b640 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bed366630 0x556bed7f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.010+0000 7fa30a49b640 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bed366630 0x556bed7f7d10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9beda3932d682ea0 server_cookie=bcf5b9395cd2c623 in_seq=0 out_seq=0 2024-05-21T15:19:33.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.010+0000 7fa2ed7fa640 10 client.16677 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.012+0000 7fa2ed7fa640 1 -- 172.21.15.78:0/2674263509 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556bed6de410 con 0x556bed366630 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.012+0000 7fa2ed7fa640 10 client.16677 handle_command_reply: tid=0 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.012+0000 7fa312da6740 1 client.16677 shutdown 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.012+0000 7fa312da6740 1 -- 172.21.15.78:0/2674263509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 msgr2=0x556bed7830e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.012+0000 7fa312da6740 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed7830e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4004460 tx=0x7fa2f400d820 comp rx=0 tx=0).stop 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa312da6740 1 -- 172.21.15.78:0/2674263509 shutdown_connections 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa312da6740 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556bed366630 0x556bed7f7d10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa2ed7fa640 0 client.16677 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa312da6740 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556bed3d2410 0x556bed7f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa312da6740 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556bed3b1730 0x556bed7830e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa312da6740 1 --2- 172.21.15.78:0/2674263509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556bed3a9550 0x556bed782ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa312da6740 1 -- 172.21.15.78:0/2674263509 >> 172.21.15.78:0/2674263509 conn(0x556bed3c6560 msgr2=0x556bed77d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa312da6740 1 -- 172.21.15.78:0/2674263509 shutdown_connections 2024-05-21T15:19:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.013+0000 7fa312da6740 1 -- 172.21.15.78:0/2674263509 wait complete. 2024-05-21T15:19:33.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.014+0000 7fa312da6740 20 client.16677 trim_cache size 0 max 16384 2024-05-21T15:19:33.018 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:19:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:19:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "565139b8-07e0-480a-9d81-c69e3ad35f70", 2024-05-21T15:19:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:19:33.019 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:19:33.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.014+0000 7fa30befe640 1 -- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa2f00a2e00 msgr2=0x7fa2f00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.014+0000 7fa30befe640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa2f00a2e00 0x7fa2f00a5310 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x556bed76f110 tx=0x7fa300009310 comp rx=0 tx=0).stop 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.014+0000 7fa30befe640 1 -- 172.21.15.78:0/3176985257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa304109350 msgr2=0x7fa3041a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.014+0000 7fa30befe640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa304109350 0x7fa3041a57a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa30000e9f0 tx=0x7fa30000edf0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.015+0000 7fa30befe640 1 -- 172.21.15.78:0/3176985257 shutdown_connections 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.015+0000 7fa30befe640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa30410e3a0 0x7fa30419f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.015+0000 7fa30befe640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa2f00a2e00 0x7fa2f00a5310 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.015+0000 7fa30befe640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa30410d8e0 0x7fa3041a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.015+0000 7fa30befe640 1 --2- 172.21.15.78:0/3176985257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa304109350 0x7fa3041a57a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.015+0000 7fa30befe640 1 -- 172.21.15.78:0/3176985257 >> 172.21.15.78:0/3176985257 conn(0x7fa304075050 msgr2=0x7fa304107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.015+0000 7fa30befe640 1 -- 172.21.15.78:0/3176985257 shutdown_connections 2024-05-21T15:19:33.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.015+0000 7fa30befe640 1 -- 172.21.15.78:0/3176985257 wait complete. 2024-05-21T15:19:33.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:19:33.036 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:19:33.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.199+0000 7fa03b4f7640 1 -- 172.21.15.78:0/2186777576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 msgr2=0x7fa0341026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.199+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/2186777576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 0x7fa0341026b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa030009900 tx=0x7fa03002a420 comp rx=0 tx=0).stop 2024-05-21T15:19:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.200+0000 7fa03b4f7640 1 -- 172.21.15.78:0/2186777576 shutdown_connections 2024-05-21T15:19:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.200+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/2186777576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa034103260 0x7fa03410f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.200+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/2186777576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa034108f60 0x7fa034102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.200+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/2186777576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 0x7fa0341026b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.200+0000 7fa03b4f7640 1 -- 172.21.15.78:0/2186777576 >> 172.21.15.78:0/2186777576 conn(0x7fa0340fe310 msgr2=0x7fa034100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.200+0000 7fa03b4f7640 1 -- 172.21.15.78:0/2186777576 shutdown_connections 2024-05-21T15:19:33.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.200+0000 7fa03b4f7640 1 -- 172.21.15.78:0/2186777576 wait complete. 2024-05-21T15:19:33.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa03b4f7640 1 Processor -- start 2024-05-21T15:19:33.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa03b4f7640 1 -- start start 2024-05-21T15:19:33.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa03b4f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa034103260 0x7fa034118630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa03b4f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 0x7fa034112790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa03b4f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa034108f60 0x7fa034112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa03b4f7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa034110ca0 con 0x7fa034103260 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa03b4f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa034110b20 con 0x7fa034108ac0 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa03b4f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa034110e20 con 0x7fa034108f60 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa038a92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 0x7fa034112790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa038a92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 0x7fa034112790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:46178/0 (socket says 172.21.15.78:46178) 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.201+0000 7fa038a92640 1 -- 172.21.15.78:0/748874425 learned_addr learned my addr 172.21.15.78:0/748874425 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.202+0000 7fa039293640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa034103260 0x7fa034118630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.202+0000 7fa039a94640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa034108f60 0x7fa034112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.202+0000 7fa038a92640 1 -- 172.21.15.78:0/748874425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa034108f60 msgr2=0x7fa034112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.202+0000 7fa038a92640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa034108f60 0x7fa034112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.202+0000 7fa038a92640 1 -- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa034103260 msgr2=0x7fa034118630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.202+0000 7fa038a92640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa034103260 0x7fa034118630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.202+0000 7fa038a92640 1 -- 172.21.15.78:0/748874425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0341134f0 con 0x7fa034108ac0 2024-05-21T15:19:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.203+0000 7fa038a92640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 0x7fa034112790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa024006fd0 tx=0x7fa02400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.203+0000 7fa02a7fc640 1 -- 172.21.15.78:0/748874425 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024011830 con 0x7fa034108ac0 2024-05-21T15:19:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.203+0000 7fa02a7fc640 1 -- 172.21.15.78:0/748874425 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa024016070 con 0x7fa034108ac0 2024-05-21T15:19:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.203+0000 7fa03b4f7640 1 -- 172.21.15.78:0/748874425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0341b4ad0 con 0x7fa034108ac0 2024-05-21T15:19:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.203+0000 7fa03b4f7640 1 -- 172.21.15.78:0/748874425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0341b5070 con 0x7fa034108ac0 2024-05-21T15:19:33.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.205+0000 7fa02a7fc640 1 -- 172.21.15.78:0/748874425 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02400f2e0 con 0x7fa034108ac0 2024-05-21T15:19:33.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.207+0000 7fa04239f740 1 Processor -- start 2024-05-21T15:19:33.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.207+0000 7fa04239f740 1 -- start start 2024-05-21T15:19:33.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.208+0000 7fa04239f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a3705390 0x5588a3aa3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.208+0000 7fa04239f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aa4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa04239f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3aa47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa04239f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588a3a09d00 con 0x5588a3705390 2024-05-21T15:19:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa04239f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5588a39b14c0 con 0x5588a36e46b0 2024-05-21T15:19:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa04239f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588a3a05cc0 con 0x5588a36dc4d0 2024-05-21T15:19:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa039293640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aa4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa039293640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aa4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44268/0 (socket says 172.21.15.78:44268) 2024-05-21T15:19:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa039293640 1 -- 172.21.15.78:0/1110554398 learned_addr learned my addr 172.21.15.78:0/1110554398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:33.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa039a94640 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a3705390 0x5588a3aa3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa038a92640 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3aa47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.209+0000 7fa02a7fc640 1 -- 172.21.15.78:0/748874425 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa024010380 con 0x7fa034108ac0 2024-05-21T15:19:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.210+0000 7fa039293640 1 -- 172.21.15.78:0/1110554398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 msgr2=0x5588a3aa47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.210+0000 7fa039293640 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3aa47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.210+0000 7fa039293640 1 -- 172.21.15.78:0/1110554398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a3705390 msgr2=0x5588a3aa3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.210+0000 7fa039293640 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a3705390 0x5588a3aa3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.210+0000 7fa039293640 1 -- 172.21.15.78:0/1110554398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a37e77d0 con 0x5588a36dc4d0 2024-05-21T15:19:33.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.211+0000 7fa02a7fc640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0200a2cd0 0x7fa0200a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.211+0000 7fa039293640 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aa4270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa030009900 tx=0x7fa030005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2b17a3cf919948 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.211+0000 7fa039a94640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0200a2cd0 0x7fa0200a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.211+0000 7fa02a7fc640 1 -- 172.21.15.78:0/748874425 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa0240ce8c0 con 0x7fa034108ac0 2024-05-21T15:19:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.211+0000 7fa01f7fe640 1 -- 172.21.15.78:0/1110554398 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030035040 con 0x5588a36dc4d0 2024-05-21T15:19:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.212+0000 7fa01f7fe640 1 -- 172.21.15.78:0/1110554398 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa030005520 con 0x5588a36dc4d0 2024-05-21T15:19:33.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.212+0000 7fa01f7fe640 1 -- 172.21.15.78:0/1110554398 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030003850 con 0x5588a36dc4d0 2024-05-21T15:19:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.213+0000 7fa039a94640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0200a2cd0 0x7fa0200a51e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5588a3aa0ea0 tx=0x7fa02c009500 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.213+0000 7fa04239f740 1 -- 172.21.15.78:0/1110554398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 msgr2=0x5588a3aa4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.213+0000 7fa04239f740 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aa4270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa030009900 tx=0x7fa030005f30 comp rx=0 tx=0).stop 2024-05-21T15:19:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.213+0000 7fa04239f740 1 -- 172.21.15.78:0/1110554398 shutdown_connections 2024-05-21T15:19:33.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.213+0000 7fa04239f740 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3aa47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.213+0000 7fa04239f740 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aa4270 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.213+0000 7fa04239f740 1 --2- 172.21.15.78:0/1110554398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a3705390 0x5588a3aa3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.213+0000 7fa04239f740 1 -- 172.21.15.78:0/1110554398 >> 172.21.15.78:0/1110554398 conn(0x5588a36f94e0 msgr2=0x5588a3aa04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.214+0000 7fa04239f740 1 -- 172.21.15.78:0/1110554398 shutdown_connections 2024-05-21T15:19:33.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.214+0000 7fa04239f740 1 -- 172.21.15.78:0/1110554398 wait complete. 2024-05-21T15:19:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.214+0000 7fa04239f740 1 -- start start 2024-05-21T15:19:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.214+0000 7fa04239f740 1 Processor -- start 2024-05-21T15:19:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa04239f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aafed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa04239f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ab0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa04239f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a3705390 0x5588a3aae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa04239f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5588a3ab0fc0 con 0x5588a36e46b0 2024-05-21T15:19:33.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa038a92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ab0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa039293640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aafed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa04239f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5588a39c4b90 con 0x5588a3705390 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa038a92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ab0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37074/0 (socket says 172.21.15.78:37074) 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa038a92640 1 -- 172.21.15.78:0/2490754491 learned_addr learned my addr 172.21.15.78:0/2490754491 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa039293640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aafed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44272/0 (socket says 172.21.15.78:44272) 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa04239f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588a3ab1140 con 0x5588a36dc4d0 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.215+0000 7fa039a94640 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a3705390 0x5588a3aae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.216+0000 7fa04239f740 10 client.0 _ll_register_callbacks cb 0x5588a3a00da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.216+0000 7fa038a92640 1 -- 172.21.15.78:0/2490754491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 msgr2=0x5588a3aafed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.216+0000 7fa038a92640 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aafed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.216+0000 7fa038a92640 1 -- 172.21.15.78:0/2490754491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a3705390 msgr2=0x5588a3aae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.216+0000 7fa038a92640 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a3705390 0x5588a3aae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.216+0000 7fa038a92640 1 -- 172.21.15.78:0/2490754491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a371d3f0 con 0x5588a36e46b0 2024-05-21T15:19:33.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.216+0000 7fa038a92640 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ab0410 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa024097290 tx=0x7fa02409acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fa4094c7634558 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:33.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.216+0000 7fa01cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:19:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.217+0000 7fa01cff9640 1 -- 172.21.15.78:0/2490754491 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024010670 con 0x5588a36e46b0 2024-05-21T15:19:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.217+0000 7fa04239f740 1 -- 172.21.15.78:0/2490754491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa030005520 con 0x5588a36e46b0 2024-05-21T15:19:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.217+0000 7fa01cff9640 1 -- 172.21.15.78:0/2490754491 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa0240974a0 con 0x5588a36e46b0 2024-05-21T15:19:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.218+0000 7fa01cff9640 1 -- 172.21.15.78:0/2490754491 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa02409b530 con 0x5588a36e46b0 2024-05-21T15:19:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.218+0000 7fa04239f740 10 client.16686 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.218+0000 7fa04239f740 1 -- 172.21.15.78:0/2490754491 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588a36dd0f0 con 0x5588a36e46b0 2024-05-21T15:19:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.219+0000 7fa01cff9640 1 -- 172.21.15.78:0/2490754491 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0240d4030 con 0x5588a36e46b0 2024-05-21T15:19:33.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.219+0000 7fa01cff9640 1 -- 172.21.15.78:0/2490754491 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fa02409a280 con 0x5588a36e46b0 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.219+0000 7fa04239f740 10 client.16686 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.219+0000 7fa04239f740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.219+0000 7fa04239f740 20 client.16686 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.219+0000 7fa04239f740 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5588a36c3f60 0x5588a3ac0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.219+0000 7fa04239f740 4 client.16686 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.220+0000 7fa04239f740 1 -- 172.21.15.78:0/2490754491 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa030005520 con 0x5588a36c3f60 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.220+0000 7fa039293640 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5588a36c3f60 0x5588a3ac0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.220+0000 7fa039293640 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5588a36c3f60 0x5588a3ac0f90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=debcbc5af6f4785 server_cookie=4e7d48874c55a4c7 in_seq=0 out_seq=0 2024-05-21T15:19:33.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.220+0000 7fa01cff9640 10 client.16686 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.221+0000 7fa01cff9640 1 -- 172.21.15.78:0/2490754491 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fa030005520 con 0x5588a36c3f60 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.221+0000 7fa01cff9640 10 client.16686 handle_command_reply: tid=0 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.222+0000 7fa04239f740 1 client.16686 shutdown 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.222+0000 7fa04239f740 1 -- 172.21.15.78:0/2490754491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 msgr2=0x5588a3ab0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.222+0000 7fa04239f740 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ab0410 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa024097290 tx=0x7fa02409acf0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa04239f740 1 -- 172.21.15.78:0/2490754491 shutdown_connections 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa04239f740 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5588a36c3f60 0x5588a3ac0f90 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa04239f740 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a3705390 0x5588a3aae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa04239f740 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ab0410 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa01cff9640 0 client.16686 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:33.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa04239f740 1 --2- 172.21.15.78:0/2490754491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36dc4d0 0x5588a3aafed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa04239f740 1 -- 172.21.15.78:0/2490754491 >> 172.21.15.78:0/2490754491 conn(0x5588a36f94e0 msgr2=0x5588a3aa5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa04239f740 1 -- 172.21.15.78:0/2490754491 shutdown_connections 2024-05-21T15:19:33.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.223+0000 7fa04239f740 1 -- 172.21.15.78:0/2490754491 wait complete. 2024-05-21T15:19:33.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.224+0000 7fa04239f740 20 client.16686 trim_cache size 0 max 16384 2024-05-21T15:19:33.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.253+0000 7fa04239f740 1 Processor -- start 2024-05-21T15:19:33.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.253+0000 7fa04239f740 1 -- start start 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.253+0000 7fa04239f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3aa6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.253+0000 7fa04239f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36dc4d0 0x5588a3ac6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.253+0000 7fa04239f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ac6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.253+0000 7fa04239f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341bd930 con 0x5588a36e46b0 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.253+0000 7fa04239f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341bf620 con 0x5588a36dc4d0 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.253+0000 7fa04239f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341be000 con 0x5588a36c3f60 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa038a92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36dc4d0 0x5588a3ac6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa039293640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3aa6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa039293640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3aa6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44280/0 (socket says 172.21.15.78:44280) 2024-05-21T15:19:33.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa039293640 1 -- 172.21.15.78:0/364149555 learned_addr learned my addr 172.21.15.78:0/364149555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa039a94640 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ac6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa038a92640 1 -- 172.21.15.78:0/364149555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 msgr2=0x5588a3aa6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa038a92640 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3aa6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa038a92640 1 -- 172.21.15.78:0/364149555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 msgr2=0x5588a3ac6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa038a92640 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ac6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa038a92640 1 -- 172.21.15.78:0/364149555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a3a9a830 con 0x5588a36dc4d0 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa039a94640 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ac6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa039293640 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3aa6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.254+0000 7fa038a92640 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36dc4d0 0x5588a3ac6750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa024097800 tx=0x7fa024099340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17ad852ca38164c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.255+0000 7fa01effd640 1 -- 172.21.15.78:0/364149555 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024010670 con 0x5588a36dc4d0 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.255+0000 7fa01effd640 1 -- 172.21.15.78:0/364149555 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0240d4040 con 0x5588a36dc4d0 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.255+0000 7fa01effd640 1 -- 172.21.15.78:0/364149555 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0240974a0 con 0x5588a36dc4d0 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 -- 172.21.15.78:0/364149555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36dc4d0 msgr2=0x5588a3ac6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36dc4d0 0x5588a3ac6750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa024097800 tx=0x7fa024099340 comp rx=0 tx=0).stop 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 -- 172.21.15.78:0/364149555 shutdown_connections 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36dc4d0 0x5588a3ac6750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36e46b0 0x5588a3ac6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 --2- 172.21.15.78:0/364149555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3aa6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 -- 172.21.15.78:0/364149555 >> 172.21.15.78:0/364149555 conn(0x5588a36f94e0 msgr2=0x5588a3aa5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 -- 172.21.15.78:0/364149555 shutdown_connections 2024-05-21T15:19:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.256+0000 7fa04239f740 1 -- 172.21.15.78:0/364149555 wait complete. 2024-05-21T15:19:33.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.257+0000 7fa04239f740 1 -- start start 2024-05-21T15:19:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.257+0000 7fa04239f740 1 Processor -- start 2024-05-21T15:19:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.257+0000 7fa04239f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3ab3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.257+0000 7fa04239f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36dc4d0 0x5588a3ab3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa04239f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3ab3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa04239f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341bee90 con 0x5588a36dc4d0 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa04239f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341be790 con 0x5588a36e46b0 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa04239f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0341bb910 con 0x5588a36c3f60 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa04239f740 10 client.0 _ll_register_callbacks cb 0x7fa024097140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa039a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3ab3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa039a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3ab3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:46200/0 (socket says 172.21.15.78:46200) 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa039a94640 1 -- 172.21.15.78:0/2693971985 learned_addr learned my addr 172.21.15.78:0/2693971985 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa038a92640 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36dc4d0 0x5588a3ab3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa039a94640 1 -- 172.21.15.78:0/2693971985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 msgr2=0x5588a3ab3240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa039a94640 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3ab3240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa039a94640 1 -- 172.21.15.78:0/2693971985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36dc4d0 msgr2=0x5588a3ab3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:33.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa039a94640 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36dc4d0 0x5588a3ab3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.258+0000 7fa039a94640 1 -- 172.21.15.78:0/2693971985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a3b27460 con 0x5588a36e46b0 2024-05-21T15:19:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.259+0000 7fa039a94640 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3ab3cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005fd0 tx=0x7fa02c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374a85ab1df1e071 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.259+0000 7fa01cff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.259+0000 7fa01cff9640 1 -- 172.21.15.78:0/2693971985 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c02e060 con 0x5588a36e46b0 2024-05-21T15:19:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.259+0000 7fa01cff9640 1 -- 172.21.15.78:0/2693971985 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa02c00fc70 con 0x5588a36e46b0 2024-05-21T15:19:33.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.260+0000 7fa04239f740 1 -- 172.21.15.78:0/2693971985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0240d4040 con 0x5588a36e46b0 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.260+0000 7fa01cff9640 1 -- 172.21.15.78:0/2693971985 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa02c01e430 con 0x5588a36e46b0 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.260+0000 7fa04239f740 10 client.26165 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.260+0000 7fa04239f740 1 -- 172.21.15.78:0/2693971985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588a3b28730 con 0x5588a36e46b0 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.260+0000 7fa01cff9640 1 -- 172.21.15.78:0/2693971985 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa02c01eab0 con 0x5588a36e46b0 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.261+0000 7fa01cff9640 1 -- 172.21.15.78:0/2693971985 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fa02c01d700 con 0x5588a36e46b0 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.261+0000 7fa04239f740 10 client.26165 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.261+0000 7fa04239f740 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.261+0000 7fa04239f740 20 client.26165 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.261+0000 7fa04239f740 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5588a3705390 0x5588a3b2af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.261+0000 7fa04239f740 4 client.26165 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.261+0000 7fa04239f740 1 -- 172.21.15.78:0/2693971985 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa0240d4040 con 0x5588a3705390 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.261+0000 7fa039293640 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5588a3705390 0x5588a3b2af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa01cff9640 10 client.26165 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa039293640 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5588a3705390 0x5588a3b2af40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6e221b5d4a4e4a server_cookie=1a87f60f3e376bd9 in_seq=0 out_seq=0 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa01cff9640 1 -- 172.21.15.78:0/2693971985 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fa0240d4040 con 0x5588a3705390 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa01cff9640 10 client.26165 handle_command_reply: tid=0 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa04239f740 1 client.26165 shutdown 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa04239f740 1 -- 172.21.15.78:0/2693971985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 msgr2=0x5588a3ab3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa04239f740 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3ab3cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005fd0 tx=0x7fa02c004a40 comp rx=0 tx=0).stop 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa04239f740 1 -- 172.21.15.78:0/2693971985 shutdown_connections 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.262+0000 7fa04239f740 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5588a3705390 0x5588a3b2af40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.263+0000 7fa04239f740 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5588a36e46b0 0x5588a3ab3cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.263+0000 7fa04239f740 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588a36dc4d0 0x5588a3ab3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.263+0000 7fa04239f740 1 --2- 172.21.15.78:0/2693971985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a36c3f60 0x5588a3ab3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.263+0000 7fa04239f740 1 -- 172.21.15.78:0/2693971985 >> 172.21.15.78:0/2693971985 conn(0x5588a36f94e0 msgr2=0x5588a3aad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.263+0000 7fa01cff9640 0 client.26165 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.263+0000 7fa04239f740 1 -- 172.21.15.78:0/2693971985 shutdown_connections 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.263+0000 7fa04239f740 1 -- 172.21.15.78:0/2693971985 wait complete. 2024-05-21T15:19:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.263+0000 7fa04239f740 20 client.26165 trim_cache size 0 max 16384 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "565139b8-07e0-480a-9d81-c69e3ad35f70": { 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "565139b8-07e0-480a-9d81-c69e3ad35f70", 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:19:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:19:33.271 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.265+0000 7fa03b4f7640 1 -- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0200a2cd0 msgr2=0x7fa0200a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0200a2cd0 0x7fa0200a51e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5588a3aa0ea0 tx=0x7fa02c009500 comp rx=0 tx=0).stop 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 -- 172.21.15.78:0/748874425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 msgr2=0x7fa034112790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 0x7fa034112790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa024006fd0 tx=0x7fa02400b8e0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 -- 172.21.15.78:0/748874425 shutdown_connections 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa034108ac0 0x7fa034112790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa0200a2cd0 0x7fa0200a51e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa034108f60 0x7fa034112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 --2- 172.21.15.78:0/748874425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa034103260 0x7fa034118630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 -- 172.21.15.78:0/748874425 >> 172.21.15.78:0/748874425 conn(0x7fa0340fe310 msgr2=0x7fa0340fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 -- 172.21.15.78:0/748874425 shutdown_connections 2024-05-21T15:19:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:19:33.266+0000 7fa03b4f7640 1 -- 172.21.15.78:0/748874425 wait complete. 2024-05-21T15:19:33.286 INFO:tasks.cephfs.filesystem:scrub status for tag:565139b8-07e0-480a-9d81-c69e3ad35f70 - {'path': '/', 'tag': '565139b8-07e0-480a-9d81-c69e3ad35f70', 'options': 'recursive,force'} 2024-05-21T15:19:33.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:33 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:19:33.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:33 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:33 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:33 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:33 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:33.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:33 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:33.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:33 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:19:33.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:33 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:33.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:33 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:33.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:33 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:33.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:33 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:33.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:33 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:33.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:33 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:19:33.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:33 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:33.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:33 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:33.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:33 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:33.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:33 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:33.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:33 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: pgmap v107: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 6.34k op/s 2024-05-21T15:19:34.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:34.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:34.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:34.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:19:34.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:34 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: pgmap v107: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 6.34k op/s 2024-05-21T15:19:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:19:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:34 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: pgmap v107: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 6.34k op/s 2024-05-21T15:19:34.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:34.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:34.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:34.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:19:34.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:34.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:34 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:19:36.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:36 smithi078 ceph-mon[31997]: pgmap v108: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 6.34k op/s 2024-05-21T15:19:36.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:36 smithi084 ceph-mon[36334]: pgmap v108: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 6.34k op/s 2024-05-21T15:19:36.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:36 smithi172 ceph-mon[36402]: pgmap v108: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 25 KiB/s rd, 33 MiB/s wr, 6.34k op/s 2024-05-21T15:19:38.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:38 smithi078 ceph-mon[31997]: pgmap v109: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 67 KiB/s rd, 46 MiB/s wr, 8.97k op/s 2024-05-21T15:19:38.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:38 smithi084 ceph-mon[36334]: pgmap v109: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 67 KiB/s rd, 46 MiB/s wr, 8.97k op/s 2024-05-21T15:19:38.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:38 smithi172 ceph-mon[36402]: pgmap v109: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 67 KiB/s rd, 46 MiB/s wr, 8.97k op/s 2024-05-21T15:19:40.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:40 smithi084 ceph-mon[36334]: pgmap v110: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 62 KiB/s rd, 33 MiB/s wr, 6.56k op/s 2024-05-21T15:19:40.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:40 smithi172 ceph-mon[36402]: pgmap v110: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 62 KiB/s rd, 33 MiB/s wr, 6.56k op/s 2024-05-21T15:19:40.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:40 smithi078 ceph-mon[31997]: pgmap v110: 97 pgs: 97 active+clean; 15 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 62 KiB/s rd, 33 MiB/s wr, 6.56k op/s 2024-05-21T15:19:42.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:42 smithi084 ceph-mon[36334]: pgmap v111: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 73 KiB/s rd, 46 MiB/s wr, 9.46k op/s 2024-05-21T15:19:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:42 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:42 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:42.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:42 smithi172 ceph-mon[36402]: pgmap v111: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 73 KiB/s rd, 46 MiB/s wr, 9.46k op/s 2024-05-21T15:19:42.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:42 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:42.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:42 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:42 smithi078 ceph-mon[31997]: pgmap v111: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 73 KiB/s rd, 46 MiB/s wr, 9.46k op/s 2024-05-21T15:19:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:42 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:42 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:43.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:43 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:43.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:43 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:43.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:43 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:19:43.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:43 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:43.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:43 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:43.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:43 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:19:43.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:43 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:43.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:43 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:43.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:43 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:19:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 -- 172.21.15.78:0/1944482516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b30102170 msgr2=0x7f3b3010e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 --2- 172.21.15.78:0/1944482516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b30102170 0x7f3b3010e4a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009be0 tx=0x7f3b2802a6c0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 -- 172.21.15.78:0/1944482516 shutdown_connections 2024-05-21T15:19:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 --2- 172.21.15.78:0/1944482516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b30102170 0x7f3b3010e4a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 --2- 172.21.15.78:0/1944482516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b30107eb0 0x7f3b30101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 --2- 172.21.15.78:0/1944482516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b30107a10 0x7f3b301015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 -- 172.21.15.78:0/1944482516 >> 172.21.15.78:0/1944482516 conn(0x7f3b30077160 msgr2=0x7f3b300775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 -- 172.21.15.78:0/1944482516 shutdown_connections 2024-05-21T15:19:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.896+0000 7f3b36056640 1 -- 172.21.15.78:0/1944482516 wait complete. 2024-05-21T15:19:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.897+0000 7f3b36056640 1 Processor -- start 2024-05-21T15:19:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b36056640 1 -- start start 2024-05-21T15:19:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b36056640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b30102170 0x7f3b301975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b36056640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b30107a10 0x7f3b30197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b36056640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b30107eb0 0x7f3b3019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b36056640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30110c20 con 0x7f3b30107eb0 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b36056640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30110aa0 con 0x7f3b30107a10 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b36056640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30110da0 con 0x7f3b30102170 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b2f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b30102170 0x7f3b301975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.898+0000 7f3b2effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b30107a10 0x7f3b30197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b30107a10 0x7f3b30197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:43704/0 (socket says 172.21.15.78:43704) 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2effd640 1 -- 172.21.15.78:0/1011801794 learned_addr learned my addr 172.21.15.78:0/1011801794 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b30107eb0 0x7f3b3019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2effd640 1 -- 172.21.15.78:0/1011801794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b30102170 msgr2=0x7f3b301975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2effd640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b30102170 0x7f3b301975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2effd640 1 -- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b30107eb0 msgr2=0x7f3b3019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2effd640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b30107eb0 0x7f3b3019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2effd640 1 -- 172.21.15.78:0/1011801794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3019c400 con 0x7f3b30107a10 2024-05-21T15:19:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.899+0000 7f3b2effd640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b30107a10 0x7f3b30197b00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000d750 tx=0x7f3b2000dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.900+0000 7f3b2cff9640 1 -- 172.21.15.78:0/1011801794 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20017070 con 0x7f3b30107a10 2024-05-21T15:19:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.900+0000 7f3b2cff9640 1 -- 172.21.15.78:0/1011801794 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b20009c30 con 0x7f3b30107a10 2024-05-21T15:19:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.900+0000 7f3b36056640 1 -- 172.21.15.78:0/1011801794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3019c6f0 con 0x7f3b30107a10 2024-05-21T15:19:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.900+0000 7f3b36056640 1 -- 172.21.15.78:0/1011801794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3019cbd0 con 0x7f3b30107a10 2024-05-21T15:19:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.900+0000 7f3b2cff9640 1 -- 172.21.15.78:0/1011801794 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000f620 con 0x7f3b30107a10 2024-05-21T15:19:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b2cff9640 1 -- 172.21.15.78:0/1011801794 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3b2001a070 con 0x7f3b30107a10 2024-05-21T15:19:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b3cfd2740 1 Processor -- start 2024-05-21T15:19:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b3cfd2740 1 -- start start 2024-05-21T15:19:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958b8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e9581d340 con 0x559e954f84d0 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559e957c5290 con 0x559e955191b0 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.904+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e95819300 con 0x559e954f02f0 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958b8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:43708/0 (socket says 172.21.15.78:43708) 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2ffff640 1 -- 172.21.15.78:0/2528371109 learned_addr learned my addr 172.21.15.78:0/2528371109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2ffff640 1 -- 172.21.15.78:0/2528371109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 msgr2=0x559e958b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2cff9640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b180a5000 0x7f3b180a7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2ffff640 1 -- 172.21.15.78:0/2528371109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 msgr2=0x559e958b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2ffff640 1 -- 172.21.15.78:0/2528371109 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e957e98e0 con 0x559e955191b0 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2effd640 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.905+0000 7f3b2cff9640 1 -- 172.21.15.78:0/1011801794 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3b200cf280 con 0x7f3b30107a10 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.906+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958b8080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3b280092f0 tx=0x7f3b28005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f8350a9ee45bb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.906+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b180a5000 0x7f3b180a7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.906+0000 7f3b15ffb640 1 -- 172.21.15.78:0/2528371109 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28042030 con 0x559e955191b0 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.907+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b180a5000 0x7f3b180a7510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3b28045000 tx=0x7f3b28004080 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.907+0000 7f3b15ffb640 1 -- 172.21.15.78:0/2528371109 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b28037040 con 0x559e955191b0 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.907+0000 7f3b15ffb640 1 -- 172.21.15.78:0/2528371109 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28036070 con 0x559e955191b0 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.908+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2528371109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 msgr2=0x559e958b8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.908+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958b8080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3b280092f0 tx=0x7f3b28005c60 comp rx=0 tx=0).stop 2024-05-21T15:19:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.908+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2528371109 shutdown_connections 2024-05-21T15:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.908+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.908+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.908+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/2528371109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958b8080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.908+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2528371109 >> 172.21.15.78:0/2528371109 conn(0x559e9550d300 msgr2=0x559e958b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.909+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2528371109 shutdown_connections 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.909+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2528371109 wait complete. 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.910+0000 7f3b3cfd2740 1 -- start start 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.910+0000 7f3b3cfd2740 1 Processor -- start 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958c4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559e958c5150 con 0x559e954f84d0 2024-05-21T15:19:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559e957d8790 con 0x559e955191b0 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e958c52d0 con 0x559e954f02f0 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b2effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b2ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b2effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54758/0 (socket says 172.21.15.78:54758) 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b2effd640 1 -- 172.21.15.78:0/285400441 learned_addr learned my addr 172.21.15.78:0/285400441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.911+0000 7f3b3cfd2740 10 client.0 _ll_register_callbacks cb 0x559e95813a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b2ffff640 1 -- 172.21.15.78:0/285400441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 msgr2=0x559e958c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b2ffff640 1 -- 172.21.15.78:0/285400441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 msgr2=0x559e958c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b2ffff640 1 -- 172.21.15.78:0/285400441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e95531210 con 0x559e955191b0 2024-05-21T15:19:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b2ffff640 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958c2620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009420 tx=0x7f3b28046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a09b2f67d623b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b0effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b0effd640 1 -- 172.21.15.78:0/285400441 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28042040 con 0x559e955191b0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/285400441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b28037040 con 0x559e955191b0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.912+0000 7f3b0effd640 1 -- 172.21.15.78:0/285400441 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3b28003970 con 0x559e955191b0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b2effd640 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b0effd640 1 -- 172.21.15.78:0/285400441 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3b28003b70 con 0x559e955191b0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b3cfd2740 10 client.26177 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/285400441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559e955fd620 con 0x559e955191b0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b0effd640 1 -- 172.21.15.78:0/285400441 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b28037040 con 0x559e955191b0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b0effd640 1 -- 172.21.15.78:0/285400441 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f3b2803c040 con 0x559e955191b0 2024-05-21T15:19:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b3cfd2740 10 client.26177 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b3cfd2740 10 client.26177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b3cfd2740 20 client.26177 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559e954d7d80 0x559e958d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b3cfd2740 4 client.26177 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.913+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/285400441 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e9582aa70 con 0x559e954d7d80 2024-05-21T15:19:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.914+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559e954d7d80 0x559e958d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.916+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559e954d7d80 0x559e958d4810 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c834cc91e3f5360e server_cookie=9122ece95356586a in_seq=0 out_seq=0 2024-05-21T15:19:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.916+0000 7f3b0effd640 10 client.26177 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.918+0000 7f3b0effd640 1 -- 172.21.15.78:0/285400441 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e9582aa70 con 0x559e954d7d80 2024-05-21T15:19:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.918+0000 7f3b0effd640 10 client.26177 handle_command_reply: tid=0 2024-05-21T15:19:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.918+0000 7f3b3cfd2740 1 client.26177 shutdown 2024-05-21T15:19:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.919+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/285400441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 msgr2=0x559e958c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.919+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958c2620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009420 tx=0x7f3b28046000 comp rx=0 tx=0).stop 2024-05-21T15:19:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.919+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/285400441 shutdown_connections 2024-05-21T15:19:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.919+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559e954d7d80 0x559e958d4810 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.919+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e955191b0 0x559e958c2620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.919+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.919+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/285400441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.919+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/285400441 >> 172.21.15.78:0/285400441 conn(0x559e9550d300 msgr2=0x559e958b7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.920+0000 7f3b0effd640 0 client.26177 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.921+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/285400441 shutdown_connections 2024-05-21T15:19:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.922+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/285400441 wait complete. 2024-05-21T15:19:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.922+0000 7f3b3cfd2740 20 client.26177 trim_cache size 0 max 16384 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.947+0000 7f3b3cfd2740 1 Processor -- start 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b3cfd2740 1 -- start start 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301aca60 con 0x559e954f84d0 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301ae750 con 0x559e9550d300 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301ad130 con 0x559e954f02f0 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b2f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b2f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:43738/0 (socket says 172.21.15.78:43738) 2024-05-21T15:19:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b2f7fe640 1 -- 172.21.15.78:0/2906705844 learned_addr learned my addr 172.21.15.78:0/2906705844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.948+0000 7f3b2effd640 1 --2- 172.21.15.78:0/2906705844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b2f7fe640 1 -- 172.21.15.78:0/2906705844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 msgr2=0x559e958da260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/2906705844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958da260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b2f7fe640 1 -- 172.21.15.78:0/2906705844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 msgr2=0x559e958b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/2906705844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b2f7fe640 1 -- 172.21.15.78:0/2906705844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e958b9230 con 0x559e9550d300 2024-05-21T15:19:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/2906705844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958b4900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x559e958b5b70 tx=0x7f3b24005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ad8b162a56a5367 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b157fa640 1 -- 172.21.15.78:0/2906705844 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24026070 con 0x559e9550d300 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b157fa640 1 -- 172.21.15.78:0/2906705844 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b24004230 con 0x559e9550d300 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.949+0000 7f3b157fa640 1 -- 172.21.15.78:0/2906705844 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b240204b0 con 0x559e9550d300 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2906705844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 msgr2=0x559e958b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/2906705844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958b4900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x559e958b5b70 tx=0x7f3b24005f90 comp rx=0 tx=0).stop 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2906705844 shutdown_connections 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/2906705844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/2906705844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958da260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/2906705844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958b4900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2906705844 >> 172.21.15.78:0/2906705844 conn(0x559e958cf580 msgr2=0x559e958b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2906705844 shutdown_connections 2024-05-21T15:19:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.951+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/2906705844 wait complete. 2024-05-21T15:19:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.952+0000 7f3b3cfd2740 1 -- start start 2024-05-21T15:19:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.952+0000 7f3b3cfd2740 1 Processor -- start 2024-05-21T15:19:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.952+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.952+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.952+0000 7f3b3cfd2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.952+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301adfc0 con 0x559e954f84d0 2024-05-21T15:19:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.952+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301ad8c0 con 0x559e9550d300 2024-05-21T15:19:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.952+0000 7f3b3cfd2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301aaa40 con 0x559e954f02f0 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b3cfd2740 10 client.0 _ll_register_callbacks cb 0x7f3b200d2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b2f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b2f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:34918/0 (socket says 172.21.15.78:34918) 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b2f7fe640 1 -- 172.21.15.78:0/1057783311 learned_addr learned my addr 172.21.15.78:0/1057783311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b2f7fe640 1 -- 172.21.15.78:0/1057783311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 msgr2=0x559e958c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b2f7fe640 1 -- 172.21.15.78:0/1057783311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 msgr2=0x559e958cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.953+0000 7f3b2f7fe640 1 -- 172.21.15.78:0/1057783311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e958c8370 con 0x559e954f02f0 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.954+0000 7f3b2f7fe640 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958ca730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b24000c00 tx=0x7f3b24005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89cdd254282262e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.954+0000 7f3b0effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:19:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.954+0000 7f3b0effd640 1 -- 172.21.15.78:0/1057783311 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24026040 con 0x559e954f02f0 2024-05-21T15:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.954+0000 7f3b0effd640 1 -- 172.21.15.78:0/1057783311 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3b24005030 con 0x559e954f02f0 2024-05-21T15:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.954+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/1057783311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b24004230 con 0x559e954f02f0 2024-05-21T15:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b0effd640 1 -- 172.21.15.78:0/1057783311 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3b24005230 con 0x559e954f02f0 2024-05-21T15:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b3cfd2740 10 client.26167 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/1057783311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559e958d4f80 con 0x559e954f02f0 2024-05-21T15:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b0effd640 1 -- 172.21.15.78:0/1057783311 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b240053d0 con 0x559e954f02f0 2024-05-21T15:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b0effd640 1 -- 172.21.15.78:0/1057783311 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f3b24032030 con 0x559e954f02f0 2024-05-21T15:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b3cfd2740 10 client.26167 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b3cfd2740 10 client.26167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b3cfd2740 20 client.26167 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559e955191b0 0x559e958d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b3cfd2740 4 client.26167 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.955+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/1057783311 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3b24004230 con 0x559e955191b0 2024-05-21T15:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.956+0000 7f3b2effd640 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559e955191b0 0x559e958d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.956+0000 7f3b0effd640 10 client.26167 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.956+0000 7f3b2effd640 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559e955191b0 0x559e958d77b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b1534404e02f5 server_cookie=c5c9a8a45d9a06c6 in_seq=0 out_seq=0 2024-05-21T15:19:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.993+0000 7f3b0effd640 1 -- 172.21.15.78:0/1057783311 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f3b24004230 con 0x559e955191b0 2024-05-21T15:19:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.993+0000 7f3b0effd640 10 client.26167 handle_command_reply: tid=0 2024-05-21T15:19:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.993+0000 7f3b3cfd2740 1 client.26167 shutdown 2024-05-21T15:19:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/1057783311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 msgr2=0x559e958ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958ca730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b24000c00 tx=0x7f3b24005b00 comp rx=0 tx=0).stop 2024-05-21T15:19:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/1057783311 shutdown_connections 2024-05-21T15:19:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559e955191b0 0x559e958d77b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559e9550d300 0x559e958c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559e954f84d0 0x559e958cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b3cfd2740 1 --2- 172.21.15.78:0/1057783311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e954f02f0 0x559e958ca730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b0effd640 0 client.26167 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.994+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/1057783311 >> 172.21.15.78:0/1057783311 conn(0x559e958cf580 msgr2=0x559e958bbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.995+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/1057783311 shutdown_connections 2024-05-21T15:19:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.995+0000 7f3b3cfd2740 1 -- 172.21.15.78:0/1057783311 wait complete. 2024-05-21T15:19:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.995+0000 7f3b3cfd2740 20 client.26167 trim_cache size 0 max 16384 2024-05-21T15:19:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:19:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:19:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:19:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:19:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.996+0000 7f3b36056640 1 -- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b180a5000 msgr2=0x7f3b180a7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.996+0000 7f3b36056640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b180a5000 0x7f3b180a7510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3b28045000 tx=0x7f3b28004080 comp rx=0 tx=0).stop 2024-05-21T15:19:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.996+0000 7f3b36056640 1 -- 172.21.15.78:0/1011801794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b30107a10 msgr2=0x7f3b30197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.996+0000 7f3b36056640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b30107a10 0x7f3b30197b00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000d750 tx=0x7f3b2000dc10 comp rx=0 tx=0).stop 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.996+0000 7f3b36056640 1 -- 172.21.15.78:0/1011801794 shutdown_connections 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.996+0000 7f3b36056640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b30107a10 0x7f3b30197b00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.996+0000 7f3b36056640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3b180a5000 0x7f3b180a7510 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.997+0000 7f3b36056640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b30107eb0 0x7f3b3019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.997+0000 7f3b36056640 1 --2- 172.21.15.78:0/1011801794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b30102170 0x7f3b301975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.997+0000 7f3b36056640 1 -- 172.21.15.78:0/1011801794 >> 172.21.15.78:0/1011801794 conn(0x7f3b30077160 msgr2=0x7f3b300ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.997+0000 7f3b36056640 1 -- 172.21.15.78:0/1011801794 shutdown_connections 2024-05-21T15:19:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:43.997+0000 7f3b36056640 1 -- 172.21.15.78:0/1011801794 wait complete. 2024-05-21T15:19:44.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:44 smithi084 ceph-mon[36334]: pgmap v112: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 5.68k op/s 2024-05-21T15:19:44.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:44 smithi172 ceph-mon[36402]: pgmap v112: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 5.68k op/s 2024-05-21T15:19:44.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:44 smithi078 ceph-mon[31997]: pgmap v112: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 5.68k op/s 2024-05-21T15:19:46.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:46 smithi084 ceph-mon[36334]: pgmap v113: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 5.68k op/s 2024-05-21T15:19:46.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:46 smithi172 ceph-mon[36402]: pgmap v113: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 5.68k op/s 2024-05-21T15:19:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:46 smithi078 ceph-mon[31997]: pgmap v113: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 5.68k op/s 2024-05-21T15:19:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:48 smithi084 ceph-mon[36334]: pgmap v114: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 93 KiB/s rd, 51 MiB/s wr, 10.93k op/s 2024-05-21T15:19:48.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:48 smithi172 ceph-mon[36402]: pgmap v114: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 93 KiB/s rd, 51 MiB/s wr, 10.93k op/s 2024-05-21T15:19:48.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:48 smithi078 ceph-mon[31997]: pgmap v114: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 93 KiB/s rd, 51 MiB/s wr, 10.93k op/s 2024-05-21T15:19:50.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:50 smithi084 ceph-mon[36334]: pgmap v115: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 38 MiB/s wr, 8.14k op/s 2024-05-21T15:19:50.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:50 smithi172 ceph-mon[36402]: pgmap v115: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 38 MiB/s wr, 8.14k op/s 2024-05-21T15:19:50.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:50 smithi078 ceph-mon[31997]: pgmap v115: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 38 MiB/s wr, 8.14k op/s 2024-05-21T15:19:52.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:52 smithi172 ceph-mon[36402]: pgmap v116: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 9.95k op/s 2024-05-21T15:19:52.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:52 smithi078 ceph-mon[31997]: pgmap v116: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 9.95k op/s 2024-05-21T15:19:52.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:52 smithi084 ceph-mon[36334]: pgmap v116: 97 pgs: 97 active+clean; 15 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 43 KiB/s rd, 46 MiB/s wr, 9.95k op/s 2024-05-21T15:19:54.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:54 smithi172 ceph-mon[36402]: pgmap v117: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 8.19k op/s 2024-05-21T15:19:54.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:54 smithi078 ceph-mon[31997]: pgmap v117: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 8.19k op/s 2024-05-21T15:19:54.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:54 smithi084 ceph-mon[36334]: pgmap v117: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 8.19k op/s 2024-05-21T15:19:56.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:56 smithi172 ceph-mon[36402]: pgmap v118: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 8.19k op/s 2024-05-21T15:19:56.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:56 smithi078 ceph-mon[31997]: pgmap v118: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 8.19k op/s 2024-05-21T15:19:56.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:56 smithi084 ceph-mon[36334]: pgmap v118: 97 pgs: 97 active+clean; 15 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 38 MiB/s wr, 8.19k op/s 2024-05-21T15:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.149+0000 7f22a9f6b640 1 -- 172.21.15.78:0/137487622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a4107a20 msgr2=0x7f22a41015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.149+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/137487622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a4107a20 0x7f22a41015d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2294009900 tx=0x7f229402a410 comp rx=0 tx=0).stop 2024-05-21T15:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.150+0000 7f22a9f6b640 1 -- 172.21.15.78:0/137487622 shutdown_connections 2024-05-21T15:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.150+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/137487622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a4102180 0x7f22a410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.150+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/137487622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22a4107ec0 0x7f22a4101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.150+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/137487622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a4107a20 0x7f22a41015d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.150+0000 7f22a9f6b640 1 -- 172.21.15.78:0/137487622 >> 172.21.15.78:0/137487622 conn(0x7f22a4077100 msgr2=0x7f22a4077540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.151+0000 7f22a9f6b640 1 -- 172.21.15.78:0/137487622 shutdown_connections 2024-05-21T15:19:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.152+0000 7f22a9f6b640 1 -- 172.21.15.78:0/137487622 wait complete. 2024-05-21T15:19:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.153+0000 7f22a9f6b640 1 Processor -- start 2024-05-21T15:19:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.153+0000 7f22a9f6b640 1 -- start start 2024-05-21T15:19:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a9f6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a4102180 0x7f22a41975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a9f6b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a4107a20 0x7f22a4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a9f6b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22a4107ec0 0x7f22a419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a9f6b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4110cb0 con 0x7f22a4102180 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a9f6b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4110b30 con 0x7f22a4107a20 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a9f6b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4110e30 con 0x7f22a4107ec0 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a4107a20 0x7f22a4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a4107a20 0x7f22a4197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:58312/0 (socket says 172.21.15.78:58312) 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a2ffd640 1 -- 172.21.15.78:0/2927204301 learned_addr learned my addr 172.21.15.78:0/2927204301 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a3fff640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22a4107ec0 0x7f22a419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.154+0000 7f22a37fe640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a4102180 0x7f22a41975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.155+0000 7f22a2ffd640 1 -- 172.21.15.78:0/2927204301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22a4107ec0 msgr2=0x7f22a419bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.155+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22a4107ec0 0x7f22a419bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.155+0000 7f22a2ffd640 1 -- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a4102180 msgr2=0x7f22a41975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.155+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a4102180 0x7f22a41975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.155+0000 7f22a2ffd640 1 -- 172.21.15.78:0/2927204301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a419c3b0 con 0x7f22a4107a20 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.155+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a4107a20 0x7f22a4197b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f229800c970 tx=0x7f229800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.156+0000 7f22a0ff9640 1 -- 172.21.15.78:0/2927204301 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298007c70 con 0x7f22a4107a20 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.156+0000 7f22a0ff9640 1 -- 172.21.15.78:0/2927204301 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2298007e10 con 0x7f22a4107a20 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.156+0000 7f22a0ff9640 1 -- 172.21.15.78:0/2927204301 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980188c0 con 0x7f22a4107a20 2024-05-21T15:19:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.156+0000 7f22a9f6b640 1 -- 172.21.15.78:0/2927204301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a419c6a0 con 0x7f22a4107a20 2024-05-21T15:19:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.156+0000 7f22a9f6b640 1 -- 172.21.15.78:0/2927204301 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a419cab0 con 0x7f22a4107a20 2024-05-21T15:19:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.159+0000 7f22b0e0c740 1 Processor -- start 2024-05-21T15:19:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.160+0000 7f22b0e0c740 1 -- start start 2024-05-21T15:19:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.160+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c661b0 0x55de87005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.160+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c3d2f0 0x55de870055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.160+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.160+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de86f6a340 con 0x55de86c3d2f0 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.160+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55de86f12290 con 0x55de86c661b0 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.160+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55de86f66300 con 0x55de86c454d0 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c3d2f0 0x55de870055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c3d2f0 0x55de870055c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45460/0 (socket says 172.21.15.78:45460) 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a37fe640 1 -- 172.21.15.78:0/490022128 learned_addr learned my addr 172.21.15.78:0/490022128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a3fff640 1 --2- 172.21.15.78:0/490022128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c661b0 0x55de87005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a37fe640 1 -- 172.21.15.78:0/490022128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 msgr2=0x55de87005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:19:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a37fe640 1 --2- 172.21.15.78:0/490022128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a37fe640 1 -- 172.21.15.78:0/490022128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c661b0 msgr2=0x55de87005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a37fe640 1 --2- 172.21.15.78:0/490022128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c661b0 0x55de87005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.161+0000 7f22a37fe640 1 -- 172.21.15.78:0/490022128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de86f368e0 con 0x55de86c3d2f0 2024-05-21T15:19:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.162+0000 7f22a37fe640 1 --2- 172.21.15.78:0/490022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c3d2f0 0x55de870055c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2294006010 tx=0x7f22940040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc918d1efd70e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.162+0000 7f22a0ff9640 1 -- 172.21.15.78:0/2927204301 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f229800f450 con 0x7f22a4107a20 2024-05-21T15:19:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.163+0000 7f22a0ff9640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f228c0a2f00 0x7f228c0a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.163+0000 7f22a0ff9640 1 -- 172.21.15.78:0/2927204301 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f22980cf4f0 con 0x7f22a4107a20 2024-05-21T15:19:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.163+0000 7f2289ffb640 1 -- 172.21.15.78:0/490022128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294035030 con 0x55de86c3d2f0 2024-05-21T15:19:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.163+0000 7f2289ffb640 1 -- 172.21.15.78:0/490022128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2294005520 con 0x55de86c3d2f0 2024-05-21T15:19:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.163+0000 7f2289ffb640 1 -- 172.21.15.78:0/490022128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22940035f0 con 0x55de86c3d2f0 2024-05-21T15:19:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.163+0000 7f22a3fff640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f228c0a2f00 0x7f228c0a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.164+0000 7f22a3fff640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f228c0a2f00 0x7f228c0a5410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55de870021f0 tx=0x7f229c009500 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.164+0000 7f22b0e0c740 1 -- 172.21.15.78:0/490022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c3d2f0 msgr2=0x55de870055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.164+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/490022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c3d2f0 0x55de870055c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2294006010 tx=0x7f22940040b0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.165+0000 7f22b0e0c740 1 -- 172.21.15.78:0/490022128 shutdown_connections 2024-05-21T15:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.165+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/490022128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87005b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.165+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/490022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c3d2f0 0x55de870055c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.165+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/490022128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c661b0 0x55de87005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.165+0000 7f22b0e0c740 1 -- 172.21.15.78:0/490022128 >> 172.21.15.78:0/490022128 conn(0x55de86c5a300 msgr2=0x55de87001830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.165+0000 7f22b0e0c740 1 -- 172.21.15.78:0/490022128 shutdown_connections 2024-05-21T15:19:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.165+0000 7f22b0e0c740 1 -- 172.21.15.78:0/490022128 wait complete. 2024-05-21T15:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.166+0000 7f22b0e0c740 1 -- start start 2024-05-21T15:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.166+0000 7f22b0e0c740 1 Processor -- start 2024-05-21T15:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de870111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de870116e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 0x55de8700f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de870111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de870111a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:58320/0 (socket says 172.21.15.78:58320) 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de870116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de870116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45476/0 (socket says 172.21.15.78:45476) 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22a2ffd640 1 -- 172.21.15.78:0/235553002 learned_addr learned my addr 172.21.15.78:0/235553002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55de87012150 con 0x55de86c454d0 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55de86f25790 con 0x55de86c3d2f0 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55de870122d0 con 0x55de86c661b0 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22b0e0c740 10 client.0 _ll_register_callbacks cb 0x55de86f60a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.167+0000 7f22a3fff640 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 0x55de8700f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22a2ffd640 1 -- 172.21.15.78:0/235553002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 msgr2=0x55de8700f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 0x55de8700f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22a2ffd640 1 -- 172.21.15.78:0/235553002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 msgr2=0x55de870111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de870111a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22a2ffd640 1 -- 172.21.15.78:0/235553002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de86c7e210 con 0x55de86c454d0 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de870116e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f229801f000 tx=0x7f229809b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26af054bbe4dd54d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f2282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f2282ffd640 1 -- 172.21.15.78:0/235553002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980b3020 con 0x55de86c454d0 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2294005520 con 0x55de86c454d0 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f2282ffd640 1 -- 172.21.15.78:0/235553002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f229809bdb0 con 0x55de86c454d0 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f2282ffd640 1 -- 172.21.15.78:0/235553002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f22980cd010 con 0x55de86c454d0 2024-05-21T15:19:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22b0e0c740 10 client.16716 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.168+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55de86d4a620 con 0x55de86c454d0 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.169+0000 7f2282ffd640 1 -- 172.21.15.78:0/235553002 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2294005520 con 0x55de86c454d0 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.169+0000 7f2282ffd640 1 -- 172.21.15.78:0/235553002 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f22980a41b0 con 0x55de86c454d0 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f22b0e0c740 10 client.16716 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f22b0e0c740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f22b0e0c740 20 client.16716 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55de86c24d80 0x55de870218c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f22b0e0c740 4 client.16716 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de86f77a70 con 0x55de86c24d80 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f22a37fe640 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55de86c24d80 0x55de870218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f2282ffd640 10 client.16716 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.170+0000 7f22a37fe640 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55de86c24d80 0x55de870218c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55e8ae76e6bf85c server_cookie=2b839324e85527a1 in_seq=0 out_seq=0 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.171+0000 7f2282ffd640 1 -- 172.21.15.78:0/235553002 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de86f77a70 con 0x55de86c24d80 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.171+0000 7f2282ffd640 10 client.16716 handle_command_reply: tid=0 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.172+0000 7f22b0e0c740 1 client.16716 shutdown 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.172+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 msgr2=0x55de870116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.172+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de870116e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f229801f000 tx=0x7f229809b550 comp rx=0 tx=0).stop 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.172+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 shutdown_connections 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.172+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55de86c24d80 0x55de870218c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.172+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 0x55de8700f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.173+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de870116e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.173+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/235553002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de870111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.173+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 >> 172.21.15.78:0/235553002 conn(0x55de86c5a300 msgr2=0x55de87006da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.173+0000 7f2282ffd640 0 client.16716 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.173+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 shutdown_connections 2024-05-21T15:19:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.175+0000 7f22b0e0c740 1 -- 172.21.15.78:0/235553002 wait complete. 2024-05-21T15:19:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.175+0000 7f22b0e0c740 20 client.16716 trim_cache size 0 max 16384 2024-05-21T15:19:57.207 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.204+0000 7f22b0e0c740 1 Processor -- start 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.204+0000 7f22b0e0c740 1 -- start start 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.204+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de8700f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.204+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87008d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.204+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c661b0 0x55de87009270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.204+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b5190 con 0x55de86c661b0 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.204+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b6e80 con 0x55de86c3d2f0 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.204+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b5860 con 0x55de86c454d0 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87008d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87008d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:39218/0 (socket says 172.21.15.78:39218) 2024-05-21T15:19:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a2ffd640 1 -- 172.21.15.78:0/2306716775 learned_addr learned my addr 172.21.15.78:0/2306716775 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a3fff640 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c661b0 0x55de87009270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a37fe640 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de8700f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a2ffd640 1 -- 172.21.15.78:0/2306716775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 msgr2=0x55de8700f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de8700f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a2ffd640 1 -- 172.21.15.78:0/2306716775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c661b0 msgr2=0x55de87009270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c661b0 0x55de87009270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.205+0000 7f22a2ffd640 1 -- 172.21.15.78:0/2306716775 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de86ffba80 con 0x55de86c454d0 2024-05-21T15:19:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.206+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87008d30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f229809b8e0 tx=0x7f229800ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c982a43919d002aa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.207+0000 7f22897fa640 1 -- 172.21.15.78:0/2306716775 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980b3070 con 0x55de86c454d0 2024-05-21T15:19:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.207+0000 7f22897fa640 1 -- 172.21.15.78:0/2306716775 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22980ba360 con 0x55de86c454d0 2024-05-21T15:19:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.207+0000 7f22897fa640 1 -- 172.21.15.78:0/2306716775 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980a4390 con 0x55de86c454d0 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.208+0000 7f22b0e0c740 1 -- 172.21.15.78:0/2306716775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 msgr2=0x55de87008d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.208+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87008d30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f229809b8e0 tx=0x7f229800ff50 comp rx=0 tx=0).stop 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.209+0000 7f22b0e0c740 1 -- 172.21.15.78:0/2306716775 shutdown_connections 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.209+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c661b0 0x55de87009270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.209+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c454d0 0x55de87008d30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.209+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/2306716775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de8700f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.209+0000 7f22b0e0c740 1 -- 172.21.15.78:0/2306716775 >> 172.21.15.78:0/2306716775 conn(0x55de86c5a300 msgr2=0x55de8700ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.209+0000 7f22b0e0c740 1 -- 172.21.15.78:0/2306716775 shutdown_connections 2024-05-21T15:19:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.209+0000 7f22b0e0c740 1 -- 172.21.15.78:0/2306716775 wait complete. 2024-05-21T15:19:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.210+0000 7f22b0e0c740 1 -- start start 2024-05-21T15:19:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.210+0000 7f22b0e0c740 1 Processor -- start 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.210+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de87017760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.210+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de87017ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.210+0000 7f22b0e0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 0x55de87084d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.210+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b66f0 con 0x55de86c454d0 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.210+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b5ff0 con 0x55de86c3d2f0 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.210+0000 7f22b0e0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41b3170 con 0x55de86c661b0 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22b0e0c740 10 client.0 _ll_register_callbacks cb 0x7f229800fd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de87017ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de87017ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45504/0 (socket says 172.21.15.78:45504) 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a2ffd640 1 -- 172.21.15.78:0/1401196875 learned_addr learned my addr 172.21.15.78:0/1401196875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:19:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a3fff640 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 0x55de87084d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a37fe640 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de87017760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a37fe640 1 -- 172.21.15.78:0/1401196875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 msgr2=0x55de87084d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a37fe640 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 0x55de87084d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a37fe640 1 -- 172.21.15.78:0/1401196875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 msgr2=0x55de87017ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a37fe640 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de87017ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.212+0000 7f22a37fe640 1 -- 172.21.15.78:0/1401196875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de86ffca60 con 0x55de86c3d2f0 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.213+0000 7f22a37fe640 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de87017760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55de87006430 tx=0x7f22940041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce53b89ffbb0a261 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.213+0000 7f2282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.213+0000 7f2282ffd640 1 -- 172.21.15.78:0/1401196875 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294055030 con 0x55de86c3d2f0 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.213+0000 7f2282ffd640 1 -- 172.21.15.78:0/1401196875 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f22940317c0 con 0x55de86c3d2f0 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.213+0000 7f22b0e0c740 1 -- 172.21.15.78:0/1401196875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22980ba360 con 0x55de86c3d2f0 2024-05-21T15:19:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.214+0000 7f2282ffd640 1 -- 172.21.15.78:0/1401196875 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2294035040 con 0x55de86c3d2f0 2024-05-21T15:19:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.214+0000 7f22b0e0c740 10 client.26198 fetch_fsmap learned FSMap version 31 2024-05-21T15:19:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.214+0000 7f22b0e0c740 1 -- 172.21.15.78:0/1401196875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55de87015420 con 0x55de86c3d2f0 2024-05-21T15:19:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.214+0000 7f2282ffd640 1 -- 172.21.15.78:0/1401196875 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22980ba360 con 0x55de86c3d2f0 2024-05-21T15:19:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.215+0000 7f2282ffd640 1 -- 172.21.15.78:0/1401196875 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f2294003c90 con 0x55de86c3d2f0 2024-05-21T15:19:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.215+0000 7f22b0e0c740 10 client.26198 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:19:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.215+0000 7f22b0e0c740 10 client.26198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:19:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.215+0000 7f22b0e0c740 20 client.26198 populate_metadata read hostname 'smithi078' 2024-05-21T15:19:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.215+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55de86bfa1c0 0x55de8709cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:19:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.215+0000 7f22b0e0c740 4 client.26198 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:19:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.215+0000 7f22b0e0c740 1 -- 172.21.15.78:0/1401196875 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de86f71dc0 con 0x55de86bfa1c0 2024-05-21T15:19:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.215+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55de86bfa1c0 0x55de8709cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:19:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.216+0000 7f22a2ffd640 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55de86bfa1c0 0x55de8709cc90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ac049f14181276 server_cookie=da70626aa6dca47c in_seq=0 out_seq=0 2024-05-21T15:19:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.216+0000 7f2282ffd640 10 client.26198 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f2282ffd640 1 -- 172.21.15.78:0/1401196875 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de86f71dc0 con 0x55de86bfa1c0 2024-05-21T15:19:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f2282ffd640 10 client.26198 handle_command_reply: tid=0 2024-05-21T15:19:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 client.26198 shutdown 2024-05-21T15:19:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 -- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 msgr2=0x55de87017760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de87017760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55de87006430 tx=0x7f22940041a0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 -- 172.21.15.78:0/1401196875 shutdown_connections 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55de86bfa1c0 0x55de8709cc90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de86c661b0 0x55de87084d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55de86c454d0 0x55de87017ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 --2- 172.21.15.78:0/1401196875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55de86c3d2f0 0x55de87017760 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 -- 172.21.15.78:0/1401196875 >> 172.21.15.78:0/1401196875 conn(0x55de86c5a300 msgr2=0x55de870073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f2282ffd640 0 client.26198 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 -- 172.21.15.78:0/1401196875 shutdown_connections 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.260+0000 7f22b0e0c740 1 -- 172.21.15.78:0/1401196875 wait complete. 2024-05-21T15:19:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.261+0000 7f22b0e0c740 20 client.26198 trim_cache size 0 max 16384 2024-05-21T15:19:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:19:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:19:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:19:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.263+0000 7f22a9f6b640 1 -- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f228c0a2f00 msgr2=0x7f228c0a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.263+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f228c0a2f00 0x7f228c0a5410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55de870021f0 tx=0x7f229c009500 comp rx=0 tx=0).stop 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.263+0000 7f22a9f6b640 1 -- 172.21.15.78:0/2927204301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a4107a20 msgr2=0x7f22a4197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.263+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a4107a20 0x7f22a4197b30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f229800c970 tx=0x7f229800ce30 comp rx=0 tx=0).stop 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.263+0000 7f22a9f6b640 1 -- 172.21.15.78:0/2927204301 shutdown_connections 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.263+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22a4107a20 0x7f22a4197b30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.263+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f228c0a2f00 0x7f228c0a5410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.264+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22a4107ec0 0x7f22a419bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.264+0000 7f22a9f6b640 1 --2- 172.21.15.78:0/2927204301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22a4102180 0x7f22a41975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:19:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.264+0000 7f22a9f6b640 1 -- 172.21.15.78:0/2927204301 >> 172.21.15.78:0/2927204301 conn(0x7f22a4077100 msgr2=0x7f22a4103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:19:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.264+0000 7f22a9f6b640 1 -- 172.21.15.78:0/2927204301 shutdown_connections 2024-05-21T15:19:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:19:57.264+0000 7f22a9f6b640 1 -- 172.21.15.78:0/2927204301 wait complete. 2024-05-21T15:19:58.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:19:58 smithi172 ceph-mon[36402]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 47 KiB/s rd, 65 MiB/s wr, 14.02k op/s 2024-05-21T15:19:58.448 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:19:58 smithi084 ceph-mon[36334]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 47 KiB/s rd, 65 MiB/s wr, 14.02k op/s 2024-05-21T15:19:58.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:19:58 smithi078 ceph-mon[31997]: pgmap v119: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 47 KiB/s rd, 65 MiB/s wr, 14.02k op/s 2024-05-21T15:20:00.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:00 smithi078 ceph-mon[31997]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 8.78k op/s 2024-05-21T15:20:00.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:00 smithi078 ceph-mon[31997]: overall HEALTH_OK 2024-05-21T15:20:00.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:00 smithi084 ceph-mon[36334]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 8.78k op/s 2024-05-21T15:20:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:00 smithi084 ceph-mon[36334]: overall HEALTH_OK 2024-05-21T15:20:00.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:00 smithi172 ceph-mon[36402]: pgmap v120: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 8.78k op/s 2024-05-21T15:20:00.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:00 smithi172 ceph-mon[36402]: overall HEALTH_OK 2024-05-21T15:20:02.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:02 smithi078 ceph-mon[31997]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 24 KiB/s rd, 43 MiB/s wr, 9.42k op/s 2024-05-21T15:20:02.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:02 smithi084 ceph-mon[36334]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 24 KiB/s rd, 43 MiB/s wr, 9.42k op/s 2024-05-21T15:20:02.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:02 smithi172 ceph-mon[36402]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 24 KiB/s rd, 43 MiB/s wr, 9.42k op/s 2024-05-21T15:20:03.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:20:03.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:20:03.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.480+0000 7f9fbe84a640 1 -- 172.21.15.78:0/1458784317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb8109060 msgr2=0x7f9fb8109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.480+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/1458784317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb8109060 0x7f9fb8109470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009930 tx=0x7f9fa802a510 comp rx=0 tx=0).stop 2024-05-21T15:20:03.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.482+0000 7f9fbe84a640 1 -- 172.21.15.78:0/1458784317 shutdown_connections 2024-05-21T15:20:03.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.482+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/1458784317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb811c0d0 0x7f9fb811e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.482+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/1458784317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fb8119770 0x7f9fb811bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.482+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/1458784317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb8109060 0x7f9fb8109470 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.482+0000 7f9fbe84a640 1 -- 172.21.15.78:0/1458784317 >> 172.21.15.78:0/1458784317 conn(0x7f9fb806d750 msgr2=0x7f9fb806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:03.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.487+0000 7f9fbe84a640 1 -- 172.21.15.78:0/1458784317 shutdown_connections 2024-05-21T15:20:03.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.488+0000 7f9fbe84a640 1 -- 172.21.15.78:0/1458784317 wait complete. 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.490+0000 7f9fbe84a640 1 Processor -- start 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.490+0000 7f9fbe84a640 1 -- start start 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.490+0000 7f9fbe84a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fb8109060 0x7f9fb8118fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.490+0000 7f9fbe84a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb811c0d0 0x7f9fb8113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.490+0000 7f9fbe84a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb8113660 0x7f9fb8113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.490+0000 7f9fbe84a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb8111570 con 0x7f9fb8109060 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.490+0000 7f9fbe84a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81113f0 con 0x7f9fb811c0d0 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.490+0000 7f9fbe84a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb81116f0 con 0x7f9fb8113660 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.491+0000 7f9fbcde7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb8113660 0x7f9fb8113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.491+0000 7f9fbcde7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb8113660 0x7f9fb8113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56560/0 (socket says 172.21.15.78:56560) 2024-05-21T15:20:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.491+0000 7f9fbcde7640 1 -- 172.21.15.78:0/2799874724 learned_addr learned my addr 172.21.15.78:0/2799874724 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.491+0000 7f9fbcde7640 1 -- 172.21.15.78:0/2799874724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb811c0d0 msgr2=0x7f9fb8113120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:20:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.491+0000 7f9fbcde7640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb811c0d0 0x7f9fb8113120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.491+0000 7f9fbcde7640 1 -- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fb8109060 msgr2=0x7f9fb8118fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T15:20:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.492+0000 7f9fbcde7640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fb8109060 0x7f9fb8118fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.492+0000 7f9fbcde7640 1 -- 172.21.15.78:0/2799874724 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb81bd470 con 0x7f9fb8113660 2024-05-21T15:20:03.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.492+0000 7f9fbcde7640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb8113660 0x7f9fb8113a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000cad0 tx=0x7f9fb000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.492+0000 7f9fb57fa640 1 -- 172.21.15.78:0/2799874724 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000ae20 con 0x7f9fb8113660 2024-05-21T15:20:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.493+0000 7f9fbe84a640 1 -- 172.21.15.78:0/2799874724 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb81bd760 con 0x7f9fb8113660 2024-05-21T15:20:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.493+0000 7f9fbe84a640 1 -- 172.21.15.78:0/2799874724 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb81bdcb0 con 0x7f9fb8113660 2024-05-21T15:20:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.493+0000 7f9fb57fa640 1 -- 172.21.15.78:0/2799874724 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb0005020 con 0x7f9fb8113660 2024-05-21T15:20:03.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.493+0000 7f9fb57fa640 1 -- 172.21.15.78:0/2799874724 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb001d060 con 0x7f9fb8113660 2024-05-21T15:20:03.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fc57c5740 1 Processor -- start 2024-05-21T15:20:03.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fc57c5740 1 -- start start 2024-05-21T15:20:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e843731d00 con 0x55e8434044d0 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e8436d94c0 con 0x55e84342d390 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e84372dcc0 con 0x55e84340c6b0 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fb57fa640 1 -- 172.21.15.78:0/2799874724 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9fb000f6f0 con 0x7f9fb8113660 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fb77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fb77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38616/0 (socket says 172.21.15.78:38616) 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.499+0000 7f9fb77fe640 1 -- 172.21.15.78:0/1455964796 learned_addr learned my addr 172.21.15.78:0/1455964796 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.500+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb77fe640 1 -- 172.21.15.78:0/1455964796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 msgr2=0x55e8437cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb77fe640 1 -- 172.21.15.78:0/1455964796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 msgr2=0x55e8437cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:03.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb77fe640 1 -- 172.21.15.78:0/1455964796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84350f7d0 con 0x55e84342d390 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437cbd30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001e60 tx=0x7f9fac009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce90f553e3744aa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.501+0000 7f9fb57fa640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9fa00a2e40 0x7f9fa00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.502+0000 7f9fb57fa640 1 -- 172.21.15.78:0/2799874724 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9fb00d0610 con 0x7f9fb8113660 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.502+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9fa00a2e40 0x7f9fa00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.502+0000 7f9f9e7fc640 1 -- 172.21.15.78:0/1455964796 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac008b80 con 0x55e84342d390 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.502+0000 7f9f9e7fc640 1 -- 172.21.15.78:0/1455964796 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fac034040 con 0x55e84342d390 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.502+0000 7f9f9e7fc640 1 -- 172.21.15.78:0/1455964796 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac004c90 con 0x55e84342d390 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.503+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1455964796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 msgr2=0x55e8437cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.503+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437cbd30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001e60 tx=0x7f9fac009da0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.504+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1455964796 shutdown_connections 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.504+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.504+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.504+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/1455964796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437cbd30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.504+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1455964796 >> 172.21.15.78:0/1455964796 conn(0x55e8434214e0 msgr2=0x55e8437c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.505+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1455964796 shutdown_connections 2024-05-21T15:20:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.506+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1455964796 wait complete. 2024-05-21T15:20:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.506+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9fa00a2e40 0x7f9fa00a5350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55e8437c9470 tx=0x7f9fa8005a00 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.506+0000 7f9fc57c5740 1 -- start start 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.506+0000 7f9fc57c5740 1 Processor -- start 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 0x55e8437d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437d8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e8437d62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8437d9020 con 0x55e84342d390 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e8436ecb90 con 0x55e8434044d0 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e8437d91a0 con 0x55e84340c6b0 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fb77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fb77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437d8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56578/0 (socket says 172.21.15.78:56578) 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fb77fe640 1 -- 172.21.15.78:0/494248954 learned_addr learned my addr 172.21.15.78:0/494248954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.507+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 0x55e8437d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.508+0000 7f9fb77fe640 1 -- 172.21.15.78:0/494248954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 msgr2=0x55e8437d7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.508+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 0x55e8437d7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.508+0000 7f9fb77fe640 1 -- 172.21.15.78:0/494248954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 msgr2=0x55e8437d62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.508+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e8437d62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.508+0000 7f9fb77fe640 1 -- 172.21.15.78:0/494248954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8434453f0 con 0x55e84340c6b0 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.508+0000 7f9fc57c5740 10 client.0 _ll_register_callbacks cb 0x55e843728da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.508+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437d8420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001a20 tx=0x7f9fac037ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7e3d0d4bbd5552 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.508+0000 7f9fc57c5740 1 -- 172.21.15.78:0/494248954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fac034040 con 0x55e84340c6b0 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.509+0000 7f9f977fe640 10 client.26188 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.509+0000 7f9f977fe640 1 -- 172.21.15.78:0/494248954 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac049060 con 0x55e84340c6b0 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.509+0000 7f9f977fe640 1 -- 172.21.15.78:0/494248954 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9fac004d20 con 0x55e84340c6b0 2024-05-21T15:20:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.509+0000 7f9f977fe640 1 -- 172.21.15.78:0/494248954 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9fac0070f0 con 0x55e84340c6b0 2024-05-21T15:20:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.513+0000 7f9fc57c5740 10 client.26188 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.513+0000 7f9fc57c5740 1 -- 172.21.15.78:0/494248954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55e8434050f0 con 0x55e84340c6b0 2024-05-21T15:20:03.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.515+0000 7f9f977fe640 1 -- 172.21.15.78:0/494248954 <== mon.1 v2:172.21.15.172:3300/0 4 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f9fac043030 con 0x55e84340c6b0 2024-05-21T15:20:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.516+0000 7f9fc57c5740 10 client.26188 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.516+0000 7f9fc57c5740 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.516+0000 7f9fc57c5740 20 client.26188 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.516+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8433ebf60 0x55e8437e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.516+0000 7f9fc57c5740 4 client.26188 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:03.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.516+0000 7f9fc57c5740 1 -- 172.21.15.78:0/494248954 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9fac034040 con 0x55e8433ebf60 2024-05-21T15:20:03.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.521+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8433ebf60 0x55e8437e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.523+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8433ebf60 0x55e8437e8550 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4126c06fa9d94ec server_cookie=2e846e6a36c8dba7 in_seq=0 out_seq=0 2024-05-21T15:20:03.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.524+0000 7f9f977fe640 10 client.26188 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:03.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.527+0000 7f9f977fe640 1 -- 172.21.15.78:0/494248954 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f9fac034040 con 0x55e8433ebf60 2024-05-21T15:20:03.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.527+0000 7f9f977fe640 10 client.26188 handle_command_reply: tid=0 2024-05-21T15:20:03.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.527+0000 7f9fc57c5740 1 client.26188 shutdown 2024-05-21T15:20:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.527+0000 7f9fc57c5740 1 -- 172.21.15.78:0/494248954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 msgr2=0x55e8437d8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.527+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437d8420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001a20 tx=0x7f9fac037ee0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 1 -- 172.21.15.78:0/494248954 shutdown_connections 2024-05-21T15:20:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8433ebf60 0x55e8437e8550 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e8437d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e84340c6b0 0x55e8437d8420 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001a20 tx=0x7f9fac037ee0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/494248954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 0x55e8437d7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 1 -- 172.21.15.78:0/494248954 >> 172.21.15.78:0/494248954 conn(0x55e8434214e0 msgr2=0x55e8437cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:03.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9f977fe640 0 client.26188 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:03.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 1 -- 172.21.15.78:0/494248954 shutdown_connections 2024-05-21T15:20:03.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 1 -- 172.21.15.78:0/494248954 wait complete. 2024-05-21T15:20:03.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.528+0000 7f9fc57c5740 20 client.26188 trim_cache size 0 max 16384 2024-05-21T15:20:03.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.559+0000 7f9fc57c5740 1 Processor -- start 2024-05-21T15:20:03.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.559+0000 7f9fc57c5740 1 -- start start 2024-05-21T15:20:03.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.559+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.559+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.559+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 0x55e8437ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa400ea00 con 0x55e8434044d0 2024-05-21T15:20:03.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa40106f0 con 0x55e84342d390 2024-05-21T15:20:03.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44600/0 (socket says 172.21.15.78:44600) 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb7fff640 1 -- 172.21.15.78:0/1857384795 learned_addr learned my addr 172.21.15.78:0/1857384795 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1857384795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa400f0d0 con 0x55e8433c13a0 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/1857384795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb7fff640 1 -- 172.21.15.78:0/1857384795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 msgr2=0x55e8437ebac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/1857384795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 0x55e8437ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb7fff640 1 -- 172.21.15.78:0/1857384795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 msgr2=0x55e8437eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/1857384795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.560+0000 7f9fb7fff640 1 -- 172.21.15.78:0/1857384795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8437ec000 con 0x55e8434044d0 2024-05-21T15:20:03.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.562+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/1857384795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437ce120 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80096c0 tx=0x7f9fa803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421572de13fc188e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:03.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.562+0000 7f9f9dffb640 1 -- 172.21.15.78:0/1857384795 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8035050 con 0x55e8434044d0 2024-05-21T15:20:03.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.562+0000 7f9f9dffb640 1 -- 172.21.15.78:0/1857384795 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fa8073020 con 0x55e8434044d0 2024-05-21T15:20:03.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.562+0000 7f9f9dffb640 1 -- 172.21.15.78:0/1857384795 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8004c90 con 0x55e8434044d0 2024-05-21T15:20:03.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1857384795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 msgr2=0x55e8437ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/1857384795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437ce120 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80096c0 tx=0x7f9fa803b000 comp rx=0 tx=0).stop 2024-05-21T15:20:03.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1857384795 shutdown_connections 2024-05-21T15:20:03.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/1857384795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e84342d390 0x55e8437eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/1857384795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 0x55e8437ebac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/1857384795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8434044d0 0x55e8437ce120 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1857384795 >> 172.21.15.78:0/1857384795 conn(0x55e8434214e0 msgr2=0x55e8437cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:03.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1857384795 shutdown_connections 2024-05-21T15:20:03.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.563+0000 7f9fc57c5740 1 -- 172.21.15.78:0/1857384795 wait complete. 2024-05-21T15:20:03.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.564+0000 7f9fc57c5740 1 -- start start 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.564+0000 7f9fc57c5740 1 Processor -- start 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.565+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 0x55e843844610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.565+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 0x55e843844b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.565+0000 7f9fc57c5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e84384f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.565+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa400ff60 con 0x55e84342d390 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.565+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa400f860 con 0x55e8434044d0 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.565+0000 7f9fc57c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa400c9e0 con 0x55e8433c13a0 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fc57c5740 10 client.0 _ll_register_callbacks cb 0x7f9fac0018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fbcde7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e84384f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fbcde7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e84384f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44608/0 (socket says 172.21.15.78:44608) 2024-05-21T15:20:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fbcde7640 1 -- 172.21.15.78:0/2479210171 learned_addr learned my addr 172.21.15.78:0/2479210171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 0x55e843844610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fb77fe640 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 0x55e843844b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fbcde7640 1 -- 172.21.15.78:0/2479210171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 msgr2=0x55e843844610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fbcde7640 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 0x55e843844610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fbcde7640 1 -- 172.21.15.78:0/2479210171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 msgr2=0x55e843844b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fbcde7640 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 0x55e843844b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.566+0000 7f9fbcde7640 1 -- 172.21.15.78:0/2479210171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e843845450 con 0x55e84342d390 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.567+0000 7f9fbcde7640 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e84384f690 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9fb009c770 tx=0x7f9fb009d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf23344cc3f35682 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.567+0000 7f9f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.567+0000 7f9f977fe640 1 -- 172.21.15.78:0/2479210171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00ce130 con 0x55e84342d390 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.567+0000 7f9f977fe640 1 -- 172.21.15.78:0/2479210171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9fb0004d70 con 0x55e84342d390 2024-05-21T15:20:03.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.567+0000 7f9fc57c5740 1 -- 172.21.15.78:0/2479210171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa8073020 con 0x55e84342d390 2024-05-21T15:20:03.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.567+0000 7f9f977fe640 1 -- 172.21.15.78:0/2479210171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9fb009bec0 con 0x55e84342d390 2024-05-21T15:20:03.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.567+0000 7f9fc57c5740 10 client.16728 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:03.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.567+0000 7f9fc57c5740 1 -- 172.21.15.78:0/2479210171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e843850c90 con 0x55e84342d390 2024-05-21T15:20:03.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.568+0000 7f9f977fe640 1 -- 172.21.15.78:0/2479210171 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb009c060 con 0x55e84342d390 2024-05-21T15:20:03.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.568+0000 7f9f977fe640 1 -- 172.21.15.78:0/2479210171 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f9fb00988e0 con 0x55e84342d390 2024-05-21T15:20:03.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.568+0000 7f9fc57c5740 10 client.16728 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:03.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.568+0000 7f9fc57c5740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:03.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.568+0000 7f9fc57c5740 20 client.16728 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:03.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.568+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e843851470 0x55e843853890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.568+0000 7f9fc57c5740 4 client.16728 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:20:03.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.568+0000 7f9fc57c5740 1 -- 172.21.15.78:0/2479210171 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9fa8073020 con 0x55e843851470 2024-05-21T15:20:03.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.569+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e843851470 0x55e843853890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.570+0000 7f9fb7fff640 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e843851470 0x55e843853890 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6bc9e9e97deb21 server_cookie=5a148fa78f759fea in_seq=0 out_seq=0 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.570+0000 7f9f977fe640 10 client.16728 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.570+0000 7f9f977fe640 1 -- 172.21.15.78:0/2479210171 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9fa8073020 con 0x55e843851470 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.570+0000 7f9f977fe640 10 client.16728 handle_command_reply: tid=0 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.570+0000 7f9fc57c5740 1 client.16728 shutdown 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 -- 172.21.15.78:0/2479210171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 msgr2=0x55e84384f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e84384f690 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9fb009c770 tx=0x7f9fb009d2e0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 -- 172.21.15.78:0/2479210171 shutdown_connections 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e843851470 0x55e843853890 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9f977fe640 0 client.16728 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8434044d0 0x55e843844b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e84342d390 0x55e84384f690 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 --2- 172.21.15.78:0/2479210171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8433c13a0 0x55e843844610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 -- 172.21.15.78:0/2479210171 >> 172.21.15.78:0/2479210171 conn(0x55e8434214e0 msgr2=0x55e8437cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 -- 172.21.15.78:0/2479210171 shutdown_connections 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 1 -- 172.21.15.78:0/2479210171 wait complete. 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.571+0000 7f9fc57c5740 20 client.16728 trim_cache size 0 max 16384 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:20:03.577 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:20:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.572+0000 7f9fbe84a640 1 -- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9fa00a2e40 msgr2=0x7f9fa00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.572+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9fa00a2e40 0x7f9fa00a5350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55e8437c9470 tx=0x7f9fa8005a00 comp rx=0 tx=0).stop 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.572+0000 7f9fbe84a640 1 -- 172.21.15.78:0/2799874724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb8113660 msgr2=0x7f9fb8113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.572+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb8113660 0x7f9fb8113a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000cad0 tx=0x7f9fb000cf90 comp rx=0 tx=0).stop 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.573+0000 7f9fbe84a640 1 -- 172.21.15.78:0/2799874724 shutdown_connections 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.573+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb811c0d0 0x7f9fb8113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.573+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9fa00a2e40 0x7f9fa00a5350 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.573+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb8113660 0x7f9fb8113a70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.573+0000 7f9fbe84a640 1 --2- 172.21.15.78:0/2799874724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9fb8109060 0x7f9fb8118fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.573+0000 7f9fbe84a640 1 -- 172.21.15.78:0/2799874724 >> 172.21.15.78:0/2799874724 conn(0x7f9fb806d750 msgr2=0x7f9fb8119b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.573+0000 7f9fbe84a640 1 -- 172.21.15.78:0/2799874724 shutdown_connections 2024-05-21T15:20:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.573+0000 7f9fbe84a640 1 -- 172.21.15.78:0/2799874724 wait complete. 2024-05-21T15:20:03.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:20:03.597 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:20:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.765+0000 7f4e55377640 1 -- 172.21.15.78:0/2574039982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 msgr2=0x7f4e5010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.765+0000 7f4e55377640 1 --2- 172.21.15.78:0/2574039982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 0x7f4e5010a180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009900 tx=0x7f4e4002a480 comp rx=0 tx=0).stop 2024-05-21T15:20:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.765+0000 7f4e55377640 1 -- 172.21.15.78:0/2574039982 shutdown_connections 2024-05-21T15:20:03.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.765+0000 7f4e55377640 1 --2- 172.21.15.78:0/2574039982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 0x7f4e5010a180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.765+0000 7f4e55377640 1 --2- 172.21.15.78:0/2574039982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e50114290 0x7f4e501166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.765+0000 7f4e55377640 1 --2- 172.21.15.78:0/2574039982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e5010f9e0 0x7f4e50109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.765+0000 7f4e55377640 1 -- 172.21.15.78:0/2574039982 >> 172.21.15.78:0/2574039982 conn(0x7f4e5006d760 msgr2=0x7f4e5006dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.766+0000 7f4e55377640 1 -- 172.21.15.78:0/2574039982 shutdown_connections 2024-05-21T15:20:03.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.766+0000 7f4e55377640 1 -- 172.21.15.78:0/2574039982 wait complete. 2024-05-21T15:20:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e55377640 1 Processor -- start 2024-05-21T15:20:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e55377640 1 -- start start 2024-05-21T15:20:03.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e55377640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 0x7f4e501a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e55377640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e5010f9e0 0x7f4e501a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e55377640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e50114290 0x7f4e501a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e55377640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e501193c0 con 0x7f4e5010f9e0 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e55377640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50119240 con 0x7f4e50109d70 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e55377640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50119540 con 0x7f4e50114290 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.767+0000 7f4e4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e50114290 0x7f4e501a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e50114290 0x7f4e501a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56622/0 (socket says 172.21.15.78:56622) 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e4f7fe640 1 -- 172.21.15.78:0/285777059 learned_addr learned my addr 172.21.15.78:0/285777059 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e4effd640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 0x7f4e501a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e47fff640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e5010f9e0 0x7f4e501a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e47fff640 1 -- 172.21.15.78:0/285777059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e50114290 msgr2=0x7f4e501a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e47fff640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e50114290 0x7f4e501a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e47fff640 1 -- 172.21.15.78:0/285777059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 msgr2=0x7f4e501a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:03.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e47fff640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 0x7f4e501a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:03.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e47fff640 1 -- 172.21.15.78:0/285777059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e501a6e30 con 0x7f4e5010f9e0 2024-05-21T15:20:03.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.768+0000 7f4e4effd640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 0x7f4e501a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:03.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.769+0000 7f4e47fff640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e5010f9e0 0x7f4e501a2530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e40032a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.769+0000 7f4e4cff9640 1 -- 172.21.15.78:0/285777059 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4003f030 con 0x7f4e5010f9e0 2024-05-21T15:20:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.769+0000 7f4e55377640 1 -- 172.21.15.78:0/285777059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e501a70c0 con 0x7f4e5010f9e0 2024-05-21T15:20:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.769+0000 7f4e55377640 1 -- 172.21.15.78:0/285777059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501a7600 con 0x7f4e5010f9e0 2024-05-21T15:20:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.770+0000 7f4e4cff9640 1 -- 172.21.15.78:0/285777059 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e40032e50 con 0x7f4e5010f9e0 2024-05-21T15:20:03.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.770+0000 7f4e4cff9640 1 -- 172.21.15.78:0/285777059 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40035070 con 0x7f4e5010f9e0 2024-05-21T15:20:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.774+0000 7f4e4cff9640 1 -- 172.21.15.78:0/285777059 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4e40042030 con 0x7f4e5010f9e0 2024-05-21T15:20:03.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.774+0000 7f4e45ffb640 1 -- 172.21.15.78:0/285777059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e50068cf0 con 0x7f4e5010f9e0 2024-05-21T15:20:03.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.775+0000 7f4e4cff9640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4e3c0a2e00 0x7f4e3c0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:03.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.775+0000 7f4e4cff9640 1 -- 172.21.15.78:0/285777059 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f4e400ef7f0 con 0x7f4e5010f9e0 2024-05-21T15:20:03.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.777+0000 7f4e4effd640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4e3c0a2e00 0x7f4e3c0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:03.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.778+0000 7f4e4effd640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4e3c0a2e00 0x7f4e3c0a5310 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e380058c0 tx=0x7f4e38005850 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:03.783+0000 7f4e4cff9640 1 -- 172.21.15.78:0/285777059 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e400bc350 con 0x7f4e5010f9e0 2024-05-21T15:20:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.177+0000 7f4e45ffb640 1 -- 172.21.15.78:0/285777059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e501a3320 con 0x7f4e5010f9e0 2024-05-21T15:20:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.178+0000 7f4e4cff9640 1 -- 172.21.15.78:0/285777059 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11940 (secure 0 0 0) 0x7f4e400bbc50 con 0x7f4e5010f9e0 2024-05-21T15:20:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:20:04.182 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:18:57.755829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527,24526,24502,24419,24448]},"id":1}]} 2024-05-21T15:20:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-05-21T15:20:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.182+0000 7f4e45ffb640 1 -- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4e3c0a2e00 msgr2=0x7f4e3c0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.182+0000 7f4e45ffb640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4e3c0a2e00 0x7f4e3c0a5310 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e380058c0 tx=0x7f4e38005850 comp rx=0 tx=0).stop 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.182+0000 7f4e45ffb640 1 -- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e5010f9e0 msgr2=0x7f4e501a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.182+0000 7f4e45ffb640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e5010f9e0 0x7f4e501a2530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e40032a50 comp rx=0 tx=0).stop 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.183+0000 7f4e45ffb640 1 -- 172.21.15.78:0/285777059 shutdown_connections 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.183+0000 7f4e45ffb640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4e3c0a2e00 0x7f4e3c0a5310 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.183+0000 7f4e45ffb640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e50114290 0x7f4e501a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.183+0000 7f4e45ffb640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e5010f9e0 0x7f4e501a2530 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.183+0000 7f4e45ffb640 1 --2- 172.21.15.78:0/285777059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e50109d70 0x7f4e501a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.183+0000 7f4e45ffb640 1 -- 172.21.15.78:0/285777059 >> 172.21.15.78:0/285777059 conn(0x7f4e5006d760 msgr2=0x7f4e501101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.184+0000 7f4e45ffb640 1 -- 172.21.15.78:0/285777059 shutdown_connections 2024-05-21T15:20:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.184+0000 7f4e45ffb640 1 -- 172.21.15.78:0/285777059 wait complete. 2024-05-21T15:20:04.207 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:20:04.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.375+0000 7f7ea3b57640 1 -- 172.21.15.78:0/2299626148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9c101000 msgr2=0x7f7e9c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.375+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/2299626148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9c101000 0x7f7e9c101410 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009900 tx=0x7f7e8c02a410 comp rx=0 tx=0).stop 2024-05-21T15:20:04.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.375+0000 7f7ea3b57640 1 -- 172.21.15.78:0/2299626148 shutdown_connections 2024-05-21T15:20:04.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.375+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/2299626148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c10d500 0x7f7e9c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.375+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/2299626148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9c101000 0x7f7e9c101410 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.375+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/2299626148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e9c108b50 0x7f7e9c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.375+0000 7f7ea3b57640 1 -- 172.21.15.78:0/2299626148 >> 172.21.15.78:0/2299626148 conn(0x7f7e9c076830 msgr2=0x7f7e9c076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.376+0000 7f7ea3b57640 1 -- 172.21.15.78:0/2299626148 shutdown_connections 2024-05-21T15:20:04.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.376+0000 7f7ea3b57640 1 -- 172.21.15.78:0/2299626148 wait complete. 2024-05-21T15:20:04.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.377+0000 7f7ea3b57640 1 Processor -- start 2024-05-21T15:20:04.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.377+0000 7f7ea3b57640 1 -- start start 2024-05-21T15:20:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.377+0000 7f7ea3b57640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c101000 0x7f7e9c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea3b57640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e9c108b50 0x7f7e9c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea3b57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9c10d500 0x7f7e9c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea18f3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c101000 0x7f7e9c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea18f3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c101000 0x7f7e9c1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56632/0 (socket says 172.21.15.78:56632) 2024-05-21T15:20:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea18f3640 1 -- 172.21.15.78:0/4148100777 learned_addr learned my addr 172.21.15.78:0/4148100777 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9c10d500 0x7f7e9c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c112e10 con 0x7f7e9c10d500 2024-05-21T15:20:04.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c112c90 con 0x7f7e9c108b50 2024-05-21T15:20:04.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e9c108b50 0x7f7e9c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c112f90 con 0x7f7e9c101000 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea18f3640 1 -- 172.21.15.78:0/4148100777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e9c108b50 msgr2=0x7f7e9c199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea18f3640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e9c108b50 0x7f7e9c199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea18f3640 1 -- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9c10d500 msgr2=0x7f7e9c19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.378+0000 7f7ea18f3640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9c10d500 0x7f7e9c19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.379+0000 7f7ea18f3640 1 -- 172.21.15.78:0/4148100777 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c19e610 con 0x7f7e9c101000 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.379+0000 7f7ea18f3640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c101000 0x7f7e9c1997b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800e9f0 tx=0x7f7e9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.379+0000 7f7e92ffd640 1 -- 172.21.15.78:0/4148100777 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800ccc0 con 0x7f7e9c101000 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.379+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c19e900 con 0x7f7e9c101000 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.379+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c19ee10 con 0x7f7e9c101000 2024-05-21T15:20:04.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.379+0000 7f7e92ffd640 1 -- 172.21.15.78:0/4148100777 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e98021910 con 0x7f7e9c101000 2024-05-21T15:20:04.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.380+0000 7f7e92ffd640 1 -- 172.21.15.78:0/4148100777 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980108d0 con 0x7f7e9c101000 2024-05-21T15:20:04.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.383+0000 7f7eaaa04740 1 Processor -- start 2024-05-21T15:20:04.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7eaaa04740 1 -- start start 2024-05-21T15:20:04.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d49302400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f394d0 0x563d49300b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d49301150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563d49266c40 con 0x563d48f62390 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563d4920e580 con 0x563d48f416b0 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563d49262c00 con 0x563d48f394d0 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7e92ffd640 1 -- 172.21.15.78:0/4148100777 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7e98021430 con 0x7f7e9c101000 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7ea20f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d49302400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7ea20f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d49302400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44658/0 (socket says 172.21.15.78:44658) 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.384+0000 7f7ea20f4640 1 -- 172.21.15.78:0/250413265 learned_addr learned my addr 172.21.15.78:0/250413265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.385+0000 7f7ea18f3640 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f394d0 0x563d49300b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.385+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d49301150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.385+0000 7f7ea20f4640 1 -- 172.21.15.78:0/250413265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f394d0 msgr2=0x563d49300b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.385+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f394d0 0x563d49300b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.386+0000 7f7ea20f4640 1 -- 172.21.15.78:0/250413265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 msgr2=0x563d49301150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.386+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d49301150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.386+0000 7f7ea20f4640 1 -- 172.21.15.78:0/250413265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d48f7a3f0 con 0x563d48f62390 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.386+0000 7f7ea18f3640 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f394d0 0x563d49300b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.386+0000 7f7e92ffd640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7e880a2d80 0x7f7e880a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.386+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d49302400 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009f90 tx=0x7f7e940118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a351250abb6bc93f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.386+0000 7f7e92ffd640 1 -- 172.21.15.78:0/4148100777 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7e980cf8f0 con 0x7f7e9c101000 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.386+0000 7f7ea18f3640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7e880a2d80 0x7f7e880a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.387+0000 7f7e87fff640 1 -- 172.21.15.78:0/250413265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94004d20 con 0x563d48f62390 2024-05-21T15:20:04.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.387+0000 7f7e87fff640 1 -- 172.21.15.78:0/250413265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e94004050 con 0x563d48f62390 2024-05-21T15:20:04.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.387+0000 7f7e87fff640 1 -- 172.21.15.78:0/250413265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94005260 con 0x563d48f62390 2024-05-21T15:20:04.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7ea18f3640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7e880a2d80 0x7f7e880a5290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x563d49307c30 tx=0x7f7e98098100 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 -- 172.21.15.78:0/250413265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 msgr2=0x563d49302400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d49302400 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009f90 tx=0x7f7e940118d0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 -- 172.21.15.78:0/250413265 shutdown_connections 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d49301150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f394d0 0x563d49300b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/250413265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d49302400 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 -- 172.21.15.78:0/250413265 >> 172.21.15.78:0/250413265 conn(0x563d48f564e0 msgr2=0x563d492fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 -- 172.21.15.78:0/250413265 shutdown_connections 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.388+0000 7f7eaaa04740 1 -- 172.21.15.78:0/250413265 wait complete. 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.389+0000 7f7eaaa04740 1 -- start start 2024-05-21T15:20:04.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.389+0000 7f7eaaa04740 1 Processor -- start 2024-05-21T15:20:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.389+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f394d0 0x563d49313ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f416b0 0x563d49314430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d4931fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563d493168b0 con 0x563d48f62390 2024-05-21T15:20:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f394d0 0x563d49313ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f394d0 0x563d49313ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38712/0 (socket says 172.21.15.78:38712) 2024-05-21T15:20:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 -- 172.21.15.78:0/437572841 learned_addr learned my addr 172.21.15.78:0/437572841 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563d49221c40 con 0x563d48f394d0 2024-05-21T15:20:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563d49316a30 con 0x563d48f416b0 2024-05-21T15:20:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d4931fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7eaaa04740 10 client.0 _ll_register_callbacks cb 0x563d48f79be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 -- 172.21.15.78:0/437572841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f416b0 msgr2=0x563d49314430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f416b0 0x563d49314430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 -- 172.21.15.78:0/437572841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f394d0 msgr2=0x563d49313ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f394d0 0x563d49313ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.390+0000 7f7ea20f4640 1 -- 172.21.15.78:0/437572841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d490447d0 con 0x563d48f62390 2024-05-21T15:20:04.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.391+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d4931fdc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7e9401b040 tx=0x7f7e94005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5bc3de45c27fd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.391+0000 7f7e857fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:04.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.391+0000 7f7e857fa640 1 -- 172.21.15.78:0/437572841 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9402d060 con 0x563d48f62390 2024-05-21T15:20:04.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.391+0000 7f7e857fa640 1 -- 172.21.15.78:0/437572841 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7e94004a40 con 0x563d48f62390 2024-05-21T15:20:04.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.391+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e94004050 con 0x563d48f62390 2024-05-21T15:20:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.392+0000 7f7e857fa640 1 -- 172.21.15.78:0/437572841 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e94004c40 con 0x563d48f62390 2024-05-21T15:20:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.392+0000 7f7eaaa04740 10 client.16746 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.392+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d49322fc0 con 0x563d48f62390 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.392+0000 7f7e857fa640 1 -- 172.21.15.78:0/437572841 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e94004050 con 0x563d48f62390 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.393+0000 7f7e857fa640 1 -- 172.21.15.78:0/437572841 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f7e9400f4e0 con 0x563d48f62390 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.393+0000 7f7eaaa04740 10 client.16746 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.393+0000 7f7eaaa04740 10 client.16746 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.393+0000 7f7eaaa04740 20 client.16746 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.393+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563d48f20f60 0x563d49326320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.393+0000 7f7eaaa04740 4 client.16746 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.393+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d49274370 con 0x563d48f20f60 2024-05-21T15:20:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.393+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563d48f20f60 0x563d49326320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.394+0000 7f7e857fa640 10 client.16746 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:20:04.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.394+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563d48f20f60 0x563d49326320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0d690f04a33678b server_cookie=4b14f08b6b263e70 in_seq=0 out_seq=0 2024-05-21T15:20:04.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.395+0000 7f7e857fa640 1 -- 172.21.15.78:0/437572841 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d49274370 con 0x563d48f20f60 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.395+0000 7f7e857fa640 10 client.16746 handle_command_reply: tid=0 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.395+0000 7f7eaaa04740 1 client.16746 shutdown 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 msgr2=0x563d4931fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d4931fdc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7e9401b040 tx=0x7f7e94005fd0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 shutdown_connections 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563d48f20f60 0x563d49326320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f62390 0x563d4931fdc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f416b0 0x563d49314430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/437572841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f394d0 0x563d49313ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 >> 172.21.15.78:0/437572841 conn(0x563d48f564e0 msgr2=0x563d492fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7e857fa640 0 client.16746 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 shutdown_connections 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 1 -- 172.21.15.78:0/437572841 wait complete. 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.396+0000 7f7eaaa04740 20 client.16746 trim_cache size 0 max 16384 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.421+0000 7f7eaaa04740 1 Processor -- start 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.421+0000 7f7eaaa04740 1 -- start start 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d49313890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d493011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 0x563d4931f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000ea10 con 0x563d48f394d0 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80010700 con 0x563d48f416b0 2024-05-21T15:20:04.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000f0e0 con 0x563d48f62390 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea20f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d49313890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea20f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d49313890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44682/0 (socket says 172.21.15.78:44682) 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea20f4640 1 -- 172.21.15.78:0/3617934124 learned_addr learned my addr 172.21.15.78:0/3617934124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d493011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 0x563d4931f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea20f4640 1 -- 172.21.15.78:0/3617934124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 msgr2=0x563d4931f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 0x563d4931f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea20f4640 1 -- 172.21.15.78:0/3617934124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 msgr2=0x563d493011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.422+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d493011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.423+0000 7f7ea20f4640 1 -- 172.21.15.78:0/3617934124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d49302620 con 0x563d48f394d0 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.423+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d49313890 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400f8e0 tx=0x7f7e9400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc99d3e05be682c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.424+0000 7f7e877fe640 1 -- 172.21.15.78:0/3617934124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9402d040 con 0x563d48f394d0 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.424+0000 7f7e877fe640 1 -- 172.21.15.78:0/3617934124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e94004000 con 0x563d48f394d0 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.424+0000 7f7e877fe640 1 -- 172.21.15.78:0/3617934124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9405c480 con 0x563d48f394d0 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.425+0000 7f7eaaa04740 1 -- 172.21.15.78:0/3617934124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 msgr2=0x563d49313890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.425+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d49313890 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400f8e0 tx=0x7f7e9400ff70 comp rx=0 tx=0).stop 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.425+0000 7f7eaaa04740 1 -- 172.21.15.78:0/3617934124 shutdown_connections 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.426+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d493011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.426+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 0x563d4931f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.426+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/3617934124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d49313890 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.426+0000 7f7eaaa04740 1 -- 172.21.15.78:0/3617934124 >> 172.21.15.78:0/3617934124 conn(0x563d48f564e0 msgr2=0x563d49300ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.426+0000 7f7eaaa04740 1 -- 172.21.15.78:0/3617934124 shutdown_connections 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.426+0000 7f7eaaa04740 1 -- 172.21.15.78:0/3617934124 wait complete. 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.426+0000 7f7eaaa04740 1 -- start start 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.426+0000 7f7eaaa04740 1 Processor -- start 2024-05-21T15:20:04.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d4931a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d4931aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7eaaa04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 0x563d4931b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7eaaa04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000ff70 con 0x563d48f394d0 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7ea20f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d4931a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7ea20f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d4931a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44690/0 (socket says 172.21.15.78:44690) 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7ea20f4640 1 -- 172.21.15.78:0/812095491 learned_addr learned my addr 172.21.15.78:0/812095491 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000f870 con 0x563d48f416b0 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000c9f0 con 0x563d48f62390 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d4931aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.427+0000 7f7eaaa04740 10 client.0 _ll_register_callbacks cb 0x7f7e94009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 0x563d4931b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7ea10f2640 1 -- 172.21.15.78:0/812095491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 msgr2=0x563d4931b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 0x563d4931b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7ea10f2640 1 -- 172.21.15.78:0/812095491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 msgr2=0x563d4931a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d4931a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7ea10f2640 1 -- 172.21.15.78:0/812095491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d49307c60 con 0x563d48f416b0 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7ea10f2640 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d4931aea0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006010 tx=0x7f7e8c004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a277f34c5c1916 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7e857fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7e857fa640 1 -- 172.21.15.78:0/812095491 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c035040 con 0x563d48f416b0 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e94004000 con 0x563d48f416b0 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.428+0000 7f7e857fa640 1 -- 172.21.15.78:0/812095491 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7e8c0317b0 con 0x563d48f416b0 2024-05-21T15:20:04.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.429+0000 7f7e857fa640 1 -- 172.21.15.78:0/812095491 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7e8c0319b0 con 0x563d48f416b0 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.429+0000 7f7eaaa04740 10 client.26234 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.429+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d49385460 con 0x563d48f416b0 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.429+0000 7f7e857fa640 1 -- 172.21.15.78:0/812095491 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e94004000 con 0x563d48f416b0 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.429+0000 7f7e857fa640 1 -- 172.21.15.78:0/812095491 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f7e8c044ce0 con 0x563d48f416b0 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.429+0000 7f7eaaa04740 10 client.26234 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.429+0000 7f7eaaa04740 10 client.26234 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.430+0000 7f7eaaa04740 20 client.26234 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.430+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563d48ef63a0 0x563d49387c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.430+0000 7f7eaaa04740 4 client.26234 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.430+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d4926e6c0 con 0x563d48ef63a0 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.430+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563d48ef63a0 0x563d49387c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.432+0000 7f7ea20f4640 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563d48ef63a0 0x563d49387c20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=54cc958f33ad0790 server_cookie=b440eba9d35ee33a in_seq=0 out_seq=0 2024-05-21T15:20:04.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.432+0000 7f7e857fa640 10 client.26234 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.434+0000 7f7e857fa640 1 -- 172.21.15.78:0/812095491 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d4926e6c0 con 0x563d48ef63a0 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.434+0000 7f7e857fa640 10 client.26234 handle_command_reply: tid=0 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.434+0000 7f7eaaa04740 1 client.26234 shutdown 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.435+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 msgr2=0x563d4931aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.435+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d4931aea0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006010 tx=0x7f7e8c004250 comp rx=0 tx=0).stop 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.436+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 shutdown_connections 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.436+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563d48ef63a0 0x563d49387c20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.436+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d48f416b0 0x563d4931aea0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.436+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d48f62390 0x563d4931b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.436+0000 7f7eaaa04740 1 --2- 172.21.15.78:0/812095491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563d48f394d0 0x563d4931a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.436+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 >> 172.21.15.78:0/812095491 conn(0x563d48f564e0 msgr2=0x563d493049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.437+0000 7f7e857fa640 0 client.26234 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.437+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 shutdown_connections 2024-05-21T15:20:04.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.437+0000 7f7eaaa04740 1 -- 172.21.15.78:0/812095491 wait complete. 2024-05-21T15:20:04.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.438+0000 7f7eaaa04740 20 client.26234 trim_cache size 0 max 16384 2024-05-21T15:20:04.554 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:04.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7e880a2d80 msgr2=0x7f7e880a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7e880a2d80 0x7f7e880a5290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x563d49307c30 tx=0x7f7e98098100 comp rx=0 tx=0).stop 2024-05-21T15:20:04.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c101000 msgr2=0x7f7e9c1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c101000 0x7f7e9c1997b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800e9f0 tx=0x7f7e9800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 shutdown_connections 2024-05-21T15:20:04.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e9c108b50 0x7f7e9c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7e880a2d80 0x7f7e880a5290 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e9c10d500 0x7f7e9c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 --2- 172.21.15.78:0/4148100777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c101000 0x7f7e9c1997b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.439+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 >> 172.21.15.78:0/4148100777 conn(0x7f7e9c076830 msgr2=0x7f7e9c1094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.440+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 shutdown_connections 2024-05-21T15:20:04.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.440+0000 7f7ea3b57640 1 -- 172.21.15.78:0/4148100777 wait complete. 2024-05-21T15:20:04.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:04 smithi078 ceph-mon[31997]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 10.49k op/s 2024-05-21T15:20:04.556 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:04 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/285777059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:20:04.557 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:20:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:04 smithi084 ceph-mon[36334]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 10.49k op/s 2024-05-21T15:20:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:04 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/285777059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:20:04.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:04 smithi172 ceph-mon[36402]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 32 KiB/s rd, 48 MiB/s wr, 10.49k op/s 2024-05-21T15:20:04.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:04 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/285777059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:20:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.737+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/3424828662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe0109060 msgr2=0x7f5fe0109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.737+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/3424828662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe0109060 0x7f5fe0109470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009860 tx=0x7f5fd002a3d0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.738+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/3424828662 shutdown_connections 2024-05-21T15:20:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.738+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/3424828662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fe0119770 0x7f5fe011bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.738+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/3424828662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe011c0d0 0x7f5fe011e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.738+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/3424828662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe0109060 0x7f5fe0109470 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.738+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/3424828662 >> 172.21.15.78:0/3424828662 conn(0x7f5fe006d750 msgr2=0x7f5fe006dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.739+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/3424828662 shutdown_connections 2024-05-21T15:20:04.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.740+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/3424828662 wait complete. 2024-05-21T15:20:04.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.741+0000 7f5fe6ec3640 1 Processor -- start 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.741+0000 7f5fe6ec3640 1 -- start start 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.741+0000 7f5fe6ec3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe0109060 0x7f5fe01191d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.741+0000 7f5fe6ec3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fe0119770 0x7f5fe01122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.741+0000 7f5fe6ec3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe011c0d0 0x7f5fe0112990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.742+0000 7f5fe6ec3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0121c60 con 0x7f5fe011c0d0 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.742+0000 7f5fe6ec3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0121ae0 con 0x7f5fe0119770 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.742+0000 7f5fe6ec3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0121de0 con 0x7f5fe0109060 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.742+0000 7f5fe4c5f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe0109060 0x7f5fe01191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.742+0000 7f5fe5460640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe011c0d0 0x7f5fe0112990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.742+0000 7f5fe5460640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe011c0d0 0x7f5fe0112990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44712/0 (socket says 172.21.15.78:44712) 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.742+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fe0119770 0x7f5fe01122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.742+0000 7f5fe5460640 1 -- 172.21.15.78:0/171074177 learned_addr learned my addr 172.21.15.78:0/171074177 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fe4c5f640 1 -- 172.21.15.78:0/171074177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fe0119770 msgr2=0x7f5fe01122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fe0119770 0x7f5fe01122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fe4c5f640 1 -- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe011c0d0 msgr2=0x7f5fe0112990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe011c0d0 0x7f5fe0112990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fe4c5f640 1 -- 172.21.15.78:0/171074177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe0116cf0 con 0x7f5fe0109060 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe0109060 0x7f5fe01191d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009780 tx=0x7f5fd00309d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fddffb640 1 -- 172.21.15.78:0/171074177 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0040070 con 0x7f5fe0109060 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fddffb640 1 -- 172.21.15.78:0/171074177 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd0005550 con 0x7f5fe0109060 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.743+0000 7f5fddffb640 1 -- 172.21.15.78:0/171074177 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0004cc0 con 0x7f5fe0109060 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.744+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/171074177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe0116f80 con 0x7f5fe0109060 2024-05-21T15:20:04.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.744+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/171074177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe0117420 con 0x7f5fe0109060 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.747+0000 7f5fedd67740 1 Processor -- start 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.747+0000 7f5fedd67740 1 -- start start 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd114390 0x5612fd4b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fedd67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612fd418c40 con 0x5612fd0f36b0 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fedd67740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5612fd3c0580 con 0x5612fd114390 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fedd67740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5612fd414c00 con 0x5612fd0eb4d0 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fe5460640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd114390 0x5612fd4b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fddffb640 1 -- 172.21.15.78:0/171074177 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5fd0039040 con 0x7f5fe0109060 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fe5460640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd114390 0x5612fd4b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38742/0 (socket says 172.21.15.78:38742) 2024-05-21T15:20:04.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.748+0000 7f5fe5460640 1 -- 172.21.15.78:0/1282435211 learned_addr learned my addr 172.21.15.78:0/1282435211 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fdffff640 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fe5460640 1 -- 172.21.15.78:0/1282435211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 msgr2=0x5612fd4b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fe5460640 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fe5460640 1 -- 172.21.15.78:0/1282435211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 msgr2=0x5612fd4b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fe5460640 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fe5460640 1 -- 172.21.15.78:0/1282435211 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612fd12c3f0 con 0x5612fd114390 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fe5460640 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd114390 0x5612fd4b2d20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009d70 tx=0x7f5fd4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce5bab328f4a157 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.749+0000 7f5fc6ffd640 1 -- 172.21.15.78:0/1282435211 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd401e070 con 0x5612fd114390 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.750+0000 7f5fc6ffd640 1 -- 172.21.15.78:0/1282435211 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd4019b10 con 0x5612fd114390 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.750+0000 7f5fc6ffd640 1 -- 172.21.15.78:0/1282435211 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400ead0 con 0x5612fd114390 2024-05-21T15:20:04.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.751+0000 7f5fedd67740 1 -- 172.21.15.78:0/1282435211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd114390 msgr2=0x5612fd4b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.751+0000 7f5fedd67740 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd114390 0x5612fd4b2d20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009d70 tx=0x7f5fd4011810 comp rx=0 tx=0).stop 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.752+0000 7f5fddffb640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5fc80a2dc0 0x7f5fc80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.752+0000 7f5fe5460640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5fc80a2dc0 0x7f5fc80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.752+0000 7f5fddffb640 1 -- 172.21.15.78:0/171074177 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f5fd00f0840 con 0x7f5fe0109060 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.752+0000 7f5fedd67740 1 -- 172.21.15.78:0/1282435211 shutdown_connections 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.752+0000 7f5fedd67740 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.752+0000 7f5fedd67740 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.752+0000 7f5fedd67740 1 --2- 172.21.15.78:0/1282435211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd114390 0x5612fd4b2d20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.753+0000 7f5fedd67740 1 -- 172.21.15.78:0/1282435211 >> 172.21.15.78:0/1282435211 conn(0x5612fd1084e0 msgr2=0x5612fd4af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.753+0000 7f5fedd67740 1 -- 172.21.15.78:0/1282435211 shutdown_connections 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.753+0000 7f5fe5460640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5fc80a2dc0 0x7f5fc80a52d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009d70 tx=0x7f5fd400ec70 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.753+0000 7f5fedd67740 1 -- 172.21.15.78:0/1282435211 wait complete. 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.753+0000 7f5fedd67740 1 -- start start 2024-05-21T15:20:04.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.753+0000 7f5fedd67740 1 Processor -- start 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.754+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4bd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.754+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.754+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd114390 0x5612fd4c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fe4c5f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fe4c5f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4bd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56696/0 (socket says 172.21.15.78:56696) 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fe4c5f640 1 -- 172.21.15.78:0/527974317 learned_addr learned my addr 172.21.15.78:0/527974317 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fe5460640 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd114390 0x5612fd4c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fdffff640 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612fd4c0010 con 0x5612fd114390 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5612fd3d3c40 con 0x5612fd0f36b0 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5612fd4c0190 con 0x5612fd0eb4d0 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.755+0000 7f5fedd67740 10 client.0 _ll_register_callbacks cb 0x5612fd12bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:04.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.756+0000 7f5fdffff640 1 -- 172.21.15.78:0/527974317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 msgr2=0x5612fd4bd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.756+0000 7f5fdffff640 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4bd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.756+0000 7f5fdffff640 1 -- 172.21.15.78:0/527974317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd114390 msgr2=0x5612fd4c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.756+0000 7f5fdffff640 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd114390 0x5612fd4c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.756+0000 7f5fdffff640 1 -- 172.21.15.78:0/527974317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612fd1f67d0 con 0x5612fd0f36b0 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.756+0000 7f5fdffff640 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4bd7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800abc0 tx=0x7f5fd8007680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54d755bab7519d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.756+0000 7f5fbffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.757+0000 7f5fbffff640 1 -- 172.21.15.78:0/527974317 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd801b060 con 0x5612fd0f36b0 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.757+0000 7f5fbffff640 1 -- 172.21.15.78:0/527974317 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f5fd8016640 con 0x5612fd0f36b0 2024-05-21T15:20:04.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.757+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd4019b10 con 0x5612fd0f36b0 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.757+0000 7f5fbffff640 1 -- 172.21.15.78:0/527974317 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5fd8007e50 con 0x5612fd0f36b0 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.757+0000 7f5fedd67740 10 client.26243 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.757+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612fd4cc4f0 con 0x5612fd0f36b0 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.758+0000 7f5fbffff640 1 -- 172.21.15.78:0/527974317 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd4019b10 con 0x5612fd0f36b0 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.759+0000 7f5fbffff640 1 -- 172.21.15.78:0/527974317 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f5fd8015910 con 0x5612fd0f36b0 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.759+0000 7f5fedd67740 10 client.26243 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.759+0000 7f5fedd67740 10 client.26243 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.759+0000 7f5fedd67740 20 client.26243 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.759+0000 7f5fedd67740 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5612fd0d2f60 0x5612fd4cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.759+0000 7f5fedd67740 4 client.26243 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:04.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.759+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612fd426370 con 0x5612fd0d2f60 2024-05-21T15:20:04.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.759+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5612fd0d2f60 0x5612fd4cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.760+0000 7f5fbffff640 10 client.26243 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:20:04.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.761+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5612fd0d2f60 0x5612fd4cf990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdfb709cb5520683 server_cookie=99483c2bc5ae517e in_seq=0 out_seq=0 2024-05-21T15:20:04.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.763+0000 7f5fbffff640 1 -- 172.21.15.78:0/527974317 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612fd426370 con 0x5612fd0d2f60 2024-05-21T15:20:04.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.763+0000 7f5fbffff640 10 client.26243 handle_command_reply: tid=0 2024-05-21T15:20:04.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.763+0000 7f5fedd67740 1 client.26243 shutdown 2024-05-21T15:20:04.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.763+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 msgr2=0x5612fd4bd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.763+0000 7f5fedd67740 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4bd7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800abc0 tx=0x7f5fd8007680 comp rx=0 tx=0).stop 2024-05-21T15:20:04.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 shutdown_connections 2024-05-21T15:20:04.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5612fd0d2f60 0x5612fd4cf990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fbffff640 0 client.26243 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:20:04.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4bd7c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd114390 0x5612fd4c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 1 --2- 172.21.15.78:0/527974317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4bd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 >> 172.21.15.78:0/527974317 conn(0x5612fd1084e0 msgr2=0x5612fd4b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 shutdown_connections 2024-05-21T15:20:04.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 1 -- 172.21.15.78:0/527974317 wait complete. 2024-05-21T15:20:04.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.764+0000 7f5fedd67740 20 client.26243 trim_cache size 0 max 16384 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.788+0000 7f5fedd67740 1 Processor -- start 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.788+0000 7f5fedd67740 1 -- start start 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0d2f60 0x5612fd4bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4d5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fedd67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01c6240 con 0x5612fd0f36b0 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fedd67740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01c7f30 con 0x5612fd0eb4d0 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fedd67740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01c6910 con 0x5612fd0d2f60 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fe4c5f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0d2f60 0x5612fd4bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fe5460640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38768/0 (socket says 172.21.15.78:38768) 2024-05-21T15:20:04.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fe5460640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4d5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44740/0 (socket says 172.21.15.78:44740) 2024-05-21T15:20:04.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.789+0000 7f5fdffff640 1 -- 172.21.15.78:0/1350403661 learned_addr learned my addr 172.21.15.78:0/1350403661 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.790+0000 7f5fe5460640 1 -- 172.21.15.78:0/1350403661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0d2f60 msgr2=0x5612fd4bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.790+0000 7f5fe5460640 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0d2f60 0x5612fd4bc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.790+0000 7f5fe5460640 1 -- 172.21.15.78:0/1350403661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0eb4d0 msgr2=0x5612fd4c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.790+0000 7f5fe5460640 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.790+0000 7f5fe5460640 1 -- 172.21.15.78:0/1350403661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612fd4bff50 con 0x5612fd0f36b0 2024-05-21T15:20:04.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.790+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0d2f60 0x5612fd4bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:04.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.790+0000 7f5fdffff640 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:04.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.790+0000 7f5fe5460640 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4d5150 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40095a0 tx=0x7f5fd4025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee749d289a62799 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.791+0000 7f5fc67fc640 1 -- 172.21.15.78:0/1350403661 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd401e040 con 0x5612fd0f36b0 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.791+0000 7f5fc67fc640 1 -- 172.21.15.78:0/1350403661 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd400ed40 con 0x5612fd0f36b0 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.791+0000 7f5fc67fc640 1 -- 172.21.15.78:0/1350403661 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4028070 con 0x5612fd0f36b0 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.792+0000 7f5fedd67740 1 -- 172.21.15.78:0/1350403661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 msgr2=0x5612fd4d5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.792+0000 7f5fedd67740 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4d5150 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40095a0 tx=0x7f5fd4025000 comp rx=0 tx=0).stop 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.792+0000 7f5fedd67740 1 -- 172.21.15.78:0/1350403661 shutdown_connections 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.792+0000 7f5fedd67740 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.792+0000 7f5fedd67740 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0f36b0 0x5612fd4d5150 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.792+0000 7f5fedd67740 1 --2- 172.21.15.78:0/1350403661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0d2f60 0x5612fd4bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.792+0000 7f5fedd67740 1 -- 172.21.15.78:0/1350403661 >> 172.21.15.78:0/1350403661 conn(0x5612fd1084e0 msgr2=0x5612fd4bb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.792+0000 7f5fedd67740 1 -- 172.21.15.78:0/1350403661 shutdown_connections 2024-05-21T15:20:04.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.793+0000 7f5fedd67740 1 -- 172.21.15.78:0/1350403661 wait complete. 2024-05-21T15:20:04.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.794+0000 7f5fedd67740 1 -- start start 2024-05-21T15:20:04.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.794+0000 7f5fedd67740 1 Processor -- start 2024-05-21T15:20:04.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.795+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0d2f60 0x5612fd4c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.795+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.795+0000 7f5fedd67740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.795+0000 7f5fedd67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01c77a0 con 0x5612fd0d2f60 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.795+0000 7f5fedd67740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01c70a0 con 0x5612fd0f36b0 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.795+0000 7f5fedd67740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01c4220 con 0x5612fd0eb4d0 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.795+0000 7f5fedd67740 10 client.0 _ll_register_callbacks cb 0x7f5fd8009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fe4c5f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0d2f60 0x5612fd4c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fe4c5f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0d2f60 0x5612fd4c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44746/0 (socket says 172.21.15.78:44746) 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fe4c5f640 1 -- 172.21.15.78:0/3929090833 learned_addr learned my addr 172.21.15.78:0/3929090833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fdffff640 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fe5460640 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fdffff640 1 -- 172.21.15.78:0/3929090833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 msgr2=0x5612fd4c11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fdffff640 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4c11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fdffff640 1 -- 172.21.15.78:0/3929090833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0d2f60 msgr2=0x5612fd4c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:04.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fdffff640 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0d2f60 0x5612fd4c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fdffff640 1 -- 172.21.15.78:0/3929090833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612fd4c1820 con 0x5612fd0eb4d0 2024-05-21T15:20:04.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.796+0000 7f5fdffff640 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c0c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5612fd4bb890 tx=0x7f5fd800aaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f772d47e49f1f806 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:04.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.797+0000 7f5fbffff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:20:04.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.797+0000 7f5fbffff640 1 -- 172.21.15.78:0/3929090833 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd802d020 con 0x5612fd0eb4d0 2024-05-21T15:20:04.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.797+0000 7f5fbffff640 1 -- 172.21.15.78:0/3929090833 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f5fd801f710 con 0x5612fd0eb4d0 2024-05-21T15:20:04.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.797+0000 7f5fedd67740 1 -- 172.21.15.78:0/3929090833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd400ed40 con 0x5612fd0eb4d0 2024-05-21T15:20:04.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.797+0000 7f5fbffff640 1 -- 172.21.15.78:0/3929090833 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5fd8025080 con 0x5612fd0eb4d0 2024-05-21T15:20:04.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.797+0000 7f5fedd67740 10 client.26224 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:04.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.797+0000 7f5fedd67740 1 -- 172.21.15.78:0/3929090833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612fd537b00 con 0x5612fd0eb4d0 2024-05-21T15:20:04.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fbffff640 1 -- 172.21.15.78:0/3929090833 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd400ed40 con 0x5612fd0eb4d0 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fbffff640 1 -- 172.21.15.78:0/3929090833 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f5fd8029440 con 0x5612fd0eb4d0 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fedd67740 10 client.26224 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fedd67740 10 client.26224 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fedd67740 20 client.26224 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fedd67740 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5612fd114390 0x5612fd53a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fedd67740 4 client.26224 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fedd67740 1 -- 172.21.15.78:0/3929090833 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612fd4206c0 con 0x5612fd114390 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.798+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5612fd114390 0x5612fd53a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.799+0000 7f5fbffff640 10 client.26224 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.800+0000 7f5fe4c5f640 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5612fd114390 0x5612fd53a290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3bc715602780a58 server_cookie=718faabd5808e4 in_seq=0 out_seq=0 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.800+0000 7f5fbffff640 1 -- 172.21.15.78:0/3929090833 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612fd4206c0 con 0x5612fd114390 2024-05-21T15:20:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.800+0000 7f5fbffff640 10 client.26224 handle_command_reply: tid=0 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.801+0000 7f5fedd67740 1 client.26224 shutdown 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.801+0000 7f5fedd67740 1 -- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 msgr2=0x5612fd4c0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.801+0000 7f5fedd67740 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c0c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x5612fd4bb890 tx=0x7f5fd800aaa0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.802+0000 7f5fedd67740 1 -- 172.21.15.78:0/3929090833 shutdown_connections 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.802+0000 7f5fedd67740 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5612fd114390 0x5612fd53a290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.802+0000 7f5fedd67740 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5612fd0f36b0 0x5612fd4c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.802+0000 7f5fbffff640 0 client.26224 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.802+0000 7f5fedd67740 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5612fd0eb4d0 0x5612fd4c0c70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.802+0000 7f5fedd67740 1 --2- 172.21.15.78:0/3929090833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5612fd0d2f60 0x5612fd4c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.802+0000 7f5fedd67740 1 -- 172.21.15.78:0/3929090833 >> 172.21.15.78:0/3929090833 conn(0x5612fd1084e0 msgr2=0x5612fd4b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.803+0000 7f5fedd67740 1 -- 172.21.15.78:0/3929090833 shutdown_connections 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.803+0000 7f5fedd67740 1 -- 172.21.15.78:0/3929090833 wait complete. 2024-05-21T15:20:04.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.803+0000 7f5fedd67740 20 client.26224 trim_cache size 0 max 16384 2024-05-21T15:20:04.810 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5fc80a2dc0 msgr2=0x7f5fc80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5fc80a2dc0 0x7f5fc80a52d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009d70 tx=0x7f5fd400ec70 comp rx=0 tx=0).stop 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/171074177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe0109060 msgr2=0x7f5fe01191d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe0109060 0x7f5fe01191d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009780 tx=0x7f5fd00309d0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/171074177 shutdown_connections 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fe0119770 0x7f5fe01122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5fc80a2dc0 0x7f5fc80a52d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fe011c0d0 0x7f5fe0112990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 --2- 172.21.15.78:0/171074177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fe0109060 0x7f5fe01191d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/171074177 >> 172.21.15.78:0/171074177 conn(0x7f5fe006d750 msgr2=0x7f5fe0119b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/171074177 shutdown_connections 2024-05-21T15:20:04.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:04.804+0000 7f5fe6ec3640 1 -- 172.21.15.78:0/171074177 wait complete. 2024-05-21T15:20:04.829 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:20:05.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.007+0000 7f8073d3e640 1 -- 172.21.15.78:0/2620035883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 msgr2=0x7f806c111820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.007+0000 7f8073d3e640 1 --2- 172.21.15.78:0/2620035883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 0x7f806c111820 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8064009be0 tx=0x7f806402a6e0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.007+0000 7f8073d3e640 1 -- 172.21.15.78:0/2620035883 shutdown_connections 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.007+0000 7f8073d3e640 1 --2- 172.21.15.78:0/2620035883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c10b1a0 0x7f806c104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.007+0000 7f8073d3e640 1 --2- 172.21.15.78:0/2620035883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 0x7f806c111820 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.007+0000 7f8073d3e640 1 --2- 172.21.15.78:0/2620035883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f806c10ad00 0x7f806c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.007+0000 7f8073d3e640 1 -- 172.21.15.78:0/2620035883 >> 172.21.15.78:0/2620035883 conn(0x7f806c1003a0 msgr2=0x7f806c1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.007+0000 7f8073d3e640 1 -- 172.21.15.78:0/2620035883 shutdown_connections 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.008+0000 7f8073d3e640 1 -- 172.21.15.78:0/2620035883 wait complete. 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.009+0000 7f8073d3e640 1 Processor -- start 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.009+0000 7f8073d3e640 1 -- start start 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.009+0000 7f8073d3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 0x7f806c1a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.009+0000 7f8073d3e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c10ad00 0x7f806c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.009+0000 7f8073d3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f806c10b1a0 0x7f806c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8073d3e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f806c115070 con 0x7f806c105450 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8073d3e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f806c114ef0 con 0x7f806c10ad00 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8073d3e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1151f0 con 0x7f806c10b1a0 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8071ada640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 0x7f806c1a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8071ada640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 0x7f806c1a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44760/0 (socket says 172.21.15.78:44760) 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8071ada640 1 -- 172.21.15.78:0/634384818 learned_addr learned my addr 172.21.15.78:0/634384818 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f80712d9640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c10ad00 0x7f806c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f80722db640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f806c10b1a0 0x7f806c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8071ada640 1 -- 172.21.15.78:0/634384818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f806c10b1a0 msgr2=0x7f806c19b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8071ada640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f806c10b1a0 0x7f806c19b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8071ada640 1 -- 172.21.15.78:0/634384818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c10ad00 msgr2=0x7f806c1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8071ada640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c10ad00 0x7f806c1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.010+0000 7f8071ada640 1 -- 172.21.15.78:0/634384818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c19bf10 con 0x7f806c105450 2024-05-21T15:20:05.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.011+0000 7f8071ada640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 0x7f806c1a1600 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f805c00e9f0 tx=0x7f805c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.011+0000 7f8062ffd640 1 -- 172.21.15.78:0/634384818 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c00ccc0 con 0x7f806c105450 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.011+0000 7f8062ffd640 1 -- 172.21.15.78:0/634384818 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f805c010910 con 0x7f806c105450 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.011+0000 7f8073d3e640 1 -- 172.21.15.78:0/634384818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c19c200 con 0x7f806c105450 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.011+0000 7f8062ffd640 1 -- 172.21.15.78:0/634384818 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c018960 con 0x7f806c105450 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.011+0000 7f8073d3e640 1 -- 172.21.15.78:0/634384818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c1a8450 con 0x7f806c105450 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.014+0000 7f807abee740 1 Processor -- start 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f807abee740 1 -- start start 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f8062ffd640 1 -- 172.21.15.78:0/634384818 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f805c018b00 con 0x7f806c105450 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f807abee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b54d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f807abee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f807abee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37956b0 0x558bf3b557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f807abee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558bf3abac40 con 0x558bf378d4d0 2024-05-21T15:20:05.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f807abee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558bf3a62580 con 0x558bf37b6390 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f807abee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558bf3ab6c00 con 0x558bf37956b0 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f80722db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b54d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f8071ada640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f80722db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b54d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38790/0 (socket says 172.21.15.78:38790) 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f80722db640 1 -- 172.21.15.78:0/3614779198 learned_addr learned my addr 172.21.15.78:0/3614779198 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.015+0000 7f80712d9640 1 --2- 172.21.15.78:0/3614779198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37956b0 0x558bf3b557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.016+0000 7f80722db640 1 -- 172.21.15.78:0/3614779198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37956b0 msgr2=0x558bf3b557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.016+0000 7f80722db640 1 --2- 172.21.15.78:0/3614779198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37956b0 0x558bf3b557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.016+0000 7f80722db640 1 -- 172.21.15.78:0/3614779198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 msgr2=0x558bf3b55260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.016+0000 7f80722db640 1 --2- 172.21.15.78:0/3614779198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b55260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.016+0000 7f80722db640 1 -- 172.21.15.78:0/3614779198 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bf37ce3f0 con 0x558bf37b6390 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.016+0000 7f80722db640 1 --2- 172.21.15.78:0/3614779198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b54d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8064009be0 tx=0x7f8064030ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faffd53b7814e94c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.017+0000 7f8062ffd640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f80580a2e00 0x7f80580a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.017+0000 7f8057fff640 1 -- 172.21.15.78:0/3614779198 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064042030 con 0x558bf37b6390 2024-05-21T15:20:05.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.017+0000 7f8057fff640 1 -- 172.21.15.78:0/3614779198 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8064037040 con 0x558bf37b6390 2024-05-21T15:20:05.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.017+0000 7f8057fff640 1 -- 172.21.15.78:0/3614779198 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064036070 con 0x558bf37b6390 2024-05-21T15:20:05.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.017+0000 7f8062ffd640 1 -- 172.21.15.78:0/634384818 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f805c0cf750 con 0x7f806c105450 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.018+0000 7f8071ada640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f80580a2e00 0x7f80580a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.018+0000 7f807abee740 1 -- 172.21.15.78:0/3614779198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 msgr2=0x558bf3b54d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.018+0000 7f807abee740 1 --2- 172.21.15.78:0/3614779198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b54d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8064009be0 tx=0x7f8064030ac0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.018+0000 7f807abee740 1 -- 172.21.15.78:0/3614779198 shutdown_connections 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.019+0000 7f807abee740 1 --2- 172.21.15.78:0/3614779198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37956b0 0x558bf3b557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.019+0000 7f807abee740 1 --2- 172.21.15.78:0/3614779198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b55260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.019+0000 7f807abee740 1 --2- 172.21.15.78:0/3614779198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b54d20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.019+0000 7f807abee740 1 -- 172.21.15.78:0/3614779198 >> 172.21.15.78:0/3614779198 conn(0x558bf37aa4e0 msgr2=0x558bf3b514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.020+0000 7f8071ada640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f80580a2e00 0x7f80580a5310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x558bf3b52810 tx=0x7f805c0d5530 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.020+0000 7f807abee740 1 -- 172.21.15.78:0/3614779198 shutdown_connections 2024-05-21T15:20:05.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.021+0000 7f807abee740 1 -- 172.21.15.78:0/3614779198 wait complete. 2024-05-21T15:20:05.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.021+0000 7f807abee740 1 -- start start 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.023+0000 7f807abee740 1 Processor -- start 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f807abee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b60e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f807abee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37956b0 0x558bf3b613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f807abee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b5f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f80722db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b60e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f80722db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b60e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44776/0 (socket says 172.21.15.78:44776) 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f80722db640 1 -- 172.21.15.78:0/2651313330 learned_addr learned my addr 172.21.15.78:0/2651313330 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558bf3b61fb0 con 0x558bf378d4d0 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558bf3a75c40 con 0x558bf37956b0 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558bf3b62130 con 0x558bf37b6390 2024-05-21T15:20:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f807abee740 10 client.0 _ll_register_callbacks cb 0x558bf37cdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.024+0000 7f80722db640 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b5f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.025+0000 7f80712d9640 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37956b0 0x558bf3b613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.025+0000 7f80712d9640 1 -- 172.21.15.78:0/2651313330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 msgr2=0x558bf3b5f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.025+0000 7f80712d9640 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b5f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.025+0000 7f80712d9640 1 -- 172.21.15.78:0/2651313330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 msgr2=0x558bf3b60e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.025+0000 7f80712d9640 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b60e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.025+0000 7f80712d9640 1 -- 172.21.15.78:0/2651313330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bf38987d0 con 0x558bf37956b0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.025+0000 7f80712d9640 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37956b0 0x558bf3b613c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f806c109f70 tx=0x7f80680339e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d164b4ab0fa38b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.025+0000 7f80557fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.026+0000 7f80557fa640 1 -- 172.21.15.78:0/2651313330 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068035680 con 0x558bf37956b0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.026+0000 7f80557fa640 1 -- 172.21.15.78:0/2651313330 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8068035820 con 0x558bf37956b0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.026+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8064037040 con 0x558bf37956b0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.026+0000 7f80557fa640 1 -- 172.21.15.78:0/2651313330 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8068035a40 con 0x558bf37956b0 2024-05-21T15:20:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.026+0000 7f807abee740 10 client.26258 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.026+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558bf3b64740 con 0x558bf37956b0 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.026+0000 7f80557fa640 1 -- 172.21.15.78:0/2651313330 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8064037040 con 0x558bf37956b0 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.027+0000 7f80557fa640 1 -- 172.21.15.78:0/2651313330 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f8068049420 con 0x558bf37956b0 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.027+0000 7f807abee740 10 client.26258 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.027+0000 7f807abee740 10 client.26258 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.027+0000 7f807abee740 20 client.26258 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.027+0000 7f807abee740 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x558bf3774f60 0x558bf3b681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.027+0000 7f807abee740 4 client.26258 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:05.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.027+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bf3ac8370 con 0x558bf3774f60 2024-05-21T15:20:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.028+0000 7f80722db640 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x558bf3774f60 0x558bf3b681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.029+0000 7f80722db640 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x558bf3774f60 0x558bf3b681a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbdee40bfd237114 server_cookie=75c33664ef456af in_seq=0 out_seq=0 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.029+0000 7f80557fa640 10 client.26258 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.030+0000 7f80557fa640 1 -- 172.21.15.78:0/2651313330 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558bf3ac8370 con 0x558bf3774f60 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.030+0000 7f80557fa640 10 client.26258 handle_command_reply: tid=0 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.030+0000 7f807abee740 1 client.26258 shutdown 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37956b0 msgr2=0x558bf3b613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f807abee740 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37956b0 0x558bf3b613c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f806c109f70 tx=0x7f80680339e0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 shutdown_connections 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f807abee740 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x558bf3774f60 0x558bf3b681a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f807abee740 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37956b0 0x558bf3b613c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f807abee740 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b5f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f807abee740 1 --2- 172.21.15.78:0/2651313330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf378d4d0 0x558bf3b60e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 >> 172.21.15.78:0/2651313330 conn(0x558bf37aa4e0 msgr2=0x558bf3b56a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.031+0000 7f80557fa640 0 client.26258 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:20:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.032+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 shutdown_connections 2024-05-21T15:20:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.032+0000 7f807abee740 1 -- 172.21.15.78:0/2651313330 wait complete. 2024-05-21T15:20:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.032+0000 7f807abee740 20 client.26258 trim_cache size 0 max 16384 2024-05-21T15:20:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.055+0000 7f807abee740 1 Processor -- start 2024-05-21T15:20:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f807abee740 1 -- start start 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f807abee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf378d4d0 0x558bf3b590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f807abee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b79db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f807abee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b7a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f807abee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1acb90 con 0x558bf37956b0 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f807abee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1ae880 con 0x558bf378d4d0 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f807abee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1ad260 con 0x558bf37b6390 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f80712d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b79db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.056+0000 7f80722db640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b7a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80712d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b79db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44780/0 (socket says 172.21.15.78:44780) 2024-05-21T15:20:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80722db640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b7a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56776/0 (socket says 172.21.15.78:56776) 2024-05-21T15:20:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80712d9640 1 -- 172.21.15.78:0/1428933914 learned_addr learned my addr 172.21.15.78:0/1428933914 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80722db640 1 --2- 172.21.15.78:0/1428933914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf378d4d0 0x558bf3b590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80712d9640 1 -- 172.21.15.78:0/1428933914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 msgr2=0x558bf3b7a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80712d9640 1 --2- 172.21.15.78:0/1428933914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b7a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80712d9640 1 -- 172.21.15.78:0/1428933914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf378d4d0 msgr2=0x558bf3b590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80712d9640 1 --2- 172.21.15.78:0/1428933914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf378d4d0 0x558bf3b590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80712d9640 1 -- 172.21.15.78:0/1428933914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bf3775190 con 0x558bf37956b0 2024-05-21T15:20:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80712d9640 1 --2- 172.21.15.78:0/1428933914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b79db0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8068009a60 tx=0x7f8068009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c787d0dfad281c43 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.057+0000 7f80577fe640 1 -- 172.21.15.78:0/1428933914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068049620 con 0x558bf37956b0 2024-05-21T15:20:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.058+0000 7f80577fe640 1 -- 172.21.15.78:0/1428933914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806804e070 con 0x558bf37956b0 2024-05-21T15:20:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.058+0000 7f80577fe640 1 -- 172.21.15.78:0/1428933914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068035e40 con 0x558bf37956b0 2024-05-21T15:20:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.059+0000 7f807abee740 1 -- 172.21.15.78:0/1428933914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 msgr2=0x558bf3b79db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.059+0000 7f807abee740 1 --2- 172.21.15.78:0/1428933914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b79db0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8068009a60 tx=0x7f8068009a90 comp rx=0 tx=0).stop 2024-05-21T15:20:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.059+0000 7f807abee740 1 -- 172.21.15.78:0/1428933914 shutdown_connections 2024-05-21T15:20:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.059+0000 7f807abee740 1 --2- 172.21.15.78:0/1428933914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf37b6390 0x558bf3b7a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.059+0000 7f807abee740 1 --2- 172.21.15.78:0/1428933914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b79db0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.059+0000 7f807abee740 1 --2- 172.21.15.78:0/1428933914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf378d4d0 0x558bf3b590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.059+0000 7f807abee740 1 -- 172.21.15.78:0/1428933914 >> 172.21.15.78:0/1428933914 conn(0x558bf37aa4e0 msgr2=0x558bf3b589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.060+0000 7f807abee740 1 -- 172.21.15.78:0/1428933914 shutdown_connections 2024-05-21T15:20:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.060+0000 7f807abee740 1 -- 172.21.15.78:0/1428933914 wait complete. 2024-05-21T15:20:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.061+0000 7f807abee740 1 -- start start 2024-05-21T15:20:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.061+0000 7f807abee740 1 Processor -- start 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f807abee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf378d4d0 0x558bf3b6e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f807abee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b6a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f807abee740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b6ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f807abee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1ae0f0 con 0x558bf37956b0 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f807abee740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1ad9f0 con 0x558bf37b6390 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f807abee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1aab90 con 0x558bf378d4d0 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f807abee740 10 client.0 _ll_register_callbacks cb 0x7f80680094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f80722db640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf378d4d0 0x558bf3b6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f80712d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b6a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f80722db640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf378d4d0 0x558bf3b6e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56792/0 (socket says 172.21.15.78:56792) 2024-05-21T15:20:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f80722db640 1 -- 172.21.15.78:0/4090753178 learned_addr learned my addr 172.21.15.78:0/4090753178 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f80712d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b6a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44796/0 (socket says 172.21.15.78:44796) 2024-05-21T15:20:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.062+0000 7f80722db640 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b6ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80722db640 1 -- 172.21.15.78:0/4090753178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 msgr2=0x558bf3b6ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80722db640 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b6ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80722db640 1 -- 172.21.15.78:0/4090753178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 msgr2=0x558bf3b6a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80722db640 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b6a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80722db640 1 -- 172.21.15.78:0/4090753178 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bf3b6b490 con 0x558bf378d4d0 2024-05-21T15:20:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80712d9640 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b6a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80722db640 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf378d4d0 0x558bf3b6e580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f806402a720 tx=0x7f8064003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6b6e1764669965 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80557fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:20:05.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f80557fa640 1 -- 172.21.15.78:0/4090753178 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064042040 con 0x558bf378d4d0 2024-05-21T15:20:05.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.063+0000 7f807abee740 1 -- 172.21.15.78:0/4090753178 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f806804e070 con 0x558bf378d4d0 2024-05-21T15:20:05.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.064+0000 7f80557fa640 1 -- 172.21.15.78:0/4090753178 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8064030660 con 0x558bf378d4d0 2024-05-21T15:20:05.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.064+0000 7f80557fa640 1 -- 172.21.15.78:0/4090753178 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8064036040 con 0x558bf378d4d0 2024-05-21T15:20:05.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.064+0000 7f807abee740 10 client.26242 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:05.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.064+0000 7f807abee740 1 -- 172.21.15.78:0/4090753178 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558bf3bd97a0 con 0x558bf378d4d0 2024-05-21T15:20:05.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.065+0000 7f80557fa640 1 -- 172.21.15.78:0/4090753178 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f806804e070 con 0x558bf378d4d0 2024-05-21T15:20:05.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.065+0000 7f80557fa640 1 -- 172.21.15.78:0/4090753178 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f806403c040 con 0x558bf378d4d0 2024-05-21T15:20:05.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.065+0000 7f807abee740 10 client.26242 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:05.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.065+0000 7f807abee740 10 client.26242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:20:05.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.065+0000 7f807abee740 20 client.26242 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:05.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.065+0000 7f807abee740 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x558bf374a3a0 0x558bf3bdbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.065+0000 7f807abee740 4 client.26242 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:05.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.065+0000 7f807abee740 1 -- 172.21.15.78:0/4090753178 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bf3ac26c0 con 0x558bf374a3a0 2024-05-21T15:20:05.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.066+0000 7f80712d9640 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x558bf374a3a0 0x558bf3bdbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.067+0000 7f80712d9640 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x558bf374a3a0 0x558bf3bdbae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19092c9e6e7c5ca6 server_cookie=e385a62a28ad41a8 in_seq=0 out_seq=0 2024-05-21T15:20:05.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.067+0000 7f80557fa640 10 client.26242 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:20:05.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.068+0000 7f80557fa640 1 -- 172.21.15.78:0/4090753178 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bf3ac26c0 con 0x558bf374a3a0 2024-05-21T15:20:05.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.068+0000 7f80557fa640 10 client.26242 handle_command_reply: tid=0 2024-05-21T15:20:05.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.068+0000 7f807abee740 1 client.26242 shutdown 2024-05-21T15:20:05.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.069+0000 7f807abee740 1 -- 172.21.15.78:0/4090753178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf378d4d0 msgr2=0x558bf3b6e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.069+0000 7f807abee740 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf378d4d0 0x558bf3b6e580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f806402a720 tx=0x7f8064003970 comp rx=0 tx=0).stop 2024-05-21T15:20:05.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.069+0000 7f807abee740 1 -- 172.21.15.78:0/4090753178 shutdown_connections 2024-05-21T15:20:05.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.069+0000 7f807abee740 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x558bf374a3a0 0x558bf3bdbae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.069+0000 7f807abee740 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bf37b6390 0x558bf3b6ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.069+0000 7f80557fa640 0 client.26242 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:20:05.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.069+0000 7f807abee740 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558bf37956b0 0x558bf3b6a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.070+0000 7f807abee740 1 --2- 172.21.15.78:0/4090753178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bf378d4d0 0x558bf3b6e580 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.070+0000 7f807abee740 1 -- 172.21.15.78:0/4090753178 >> 172.21.15.78:0/4090753178 conn(0x558bf37aa4e0 msgr2=0x558bf3b5e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.072+0000 7f807abee740 1 -- 172.21.15.78:0/4090753178 shutdown_connections 2024-05-21T15:20:05.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.072+0000 7f807abee740 1 -- 172.21.15.78:0/4090753178 wait complete. 2024-05-21T15:20:05.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.073+0000 7f807abee740 20 client.26242 trim_cache size 0 max 16384 2024-05-21T15:20:05.080 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.076+0000 7f8073d3e640 1 -- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f80580a2e00 msgr2=0x7f80580a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.076+0000 7f8073d3e640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f80580a2e00 0x7f80580a5310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x558bf3b52810 tx=0x7f805c0d5530 comp rx=0 tx=0).stop 2024-05-21T15:20:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.076+0000 7f8073d3e640 1 -- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 msgr2=0x7f806c1a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.076+0000 7f8073d3e640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 0x7f806c1a1600 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f805c00e9f0 tx=0x7f805c00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.076+0000 7f8073d3e640 1 -- 172.21.15.78:0/634384818 shutdown_connections 2024-05-21T15:20:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.076+0000 7f8073d3e640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806c10ad00 0x7f806c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.076+0000 7f8073d3e640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f80580a2e00 0x7f80580a5310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.076+0000 7f8073d3e640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f806c10b1a0 0x7f806c19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.077+0000 7f8073d3e640 1 --2- 172.21.15.78:0/634384818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f806c105450 0x7f806c1a1600 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.077+0000 7f8073d3e640 1 -- 172.21.15.78:0/634384818 >> 172.21.15.78:0/634384818 conn(0x7f806c1003a0 msgr2=0x7f806c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.077+0000 7f8073d3e640 1 -- 172.21.15.78:0/634384818 shutdown_connections 2024-05-21T15:20:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.077+0000 7f8073d3e640 1 -- 172.21.15.78:0/634384818 wait complete. 2024-05-21T15:20:05.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:20:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.296+0000 7faa71826640 1 -- 172.21.15.78:0/1682991479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 msgr2=0x7faa6c109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.296+0000 7faa71826640 1 --2- 172.21.15.78:0/1682991479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 0x7faa6c109770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa60009900 tx=0x7faa6002a440 comp rx=0 tx=0).stop 2024-05-21T15:20:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.297+0000 7faa71826640 1 -- 172.21.15.78:0/1682991479 shutdown_connections 2024-05-21T15:20:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.297+0000 7faa71826640 1 --2- 172.21.15.78:0/1682991479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa6c10e3b0 0x7faa6c115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.297+0000 7faa71826640 1 --2- 172.21.15.78:0/1682991479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa6c10d8f0 0x7faa6c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.297+0000 7faa71826640 1 --2- 172.21.15.78:0/1682991479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 0x7faa6c109770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.297+0000 7faa71826640 1 -- 172.21.15.78:0/1682991479 >> 172.21.15.78:0/1682991479 conn(0x7faa6c075fa0 msgr2=0x7faa6c076400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.300+0000 7faa71826640 1 -- 172.21.15.78:0/1682991479 shutdown_connections 2024-05-21T15:20:05.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.300+0000 7faa71826640 1 -- 172.21.15.78:0/1682991479 wait complete. 2024-05-21T15:20:05.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.301+0000 7faa71826640 1 Processor -- start 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.305+0000 7faa71826640 1 -- start start 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.305+0000 7faa71826640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 0x7faa6c083680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.305+0000 7faa71826640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa6c10d8f0 0x7faa6c07c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.305+0000 7faa71826640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa6c10e3b0 0x7faa6c07cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.305+0000 7faa71826640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1194c0 con 0x7faa6c10d8f0 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.305+0000 7faa71826640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c119340 con 0x7faa6c109360 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.305+0000 7faa71826640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c119640 con 0x7faa6c10e3b0 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.306+0000 7faa6affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 0x7faa6c083680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.306+0000 7faa6b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa6c10e3b0 0x7faa6c07cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.306+0000 7faa6b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa6c10e3b0 0x7faa6c07cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56824/0 (socket says 172.21.15.78:56824) 2024-05-21T15:20:05.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.306+0000 7faa6b7fe640 1 -- 172.21.15.78:0/851068149 learned_addr learned my addr 172.21.15.78:0/851068149 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.309+0000 7faa6a7fc640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa6c10d8f0 0x7faa6c07c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.309+0000 7faa6affd640 1 -- 172.21.15.78:0/851068149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa6c10e3b0 msgr2=0x7faa6c07cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.309+0000 7faa6affd640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa6c10e3b0 0x7faa6c07cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.309+0000 7faa6affd640 1 -- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa6c10d8f0 msgr2=0x7faa6c07c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.309+0000 7faa6affd640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa6c10d8f0 0x7faa6c07c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.309+0000 7faa6affd640 1 -- 172.21.15.78:0/851068149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c07d5d0 con 0x7faa6c109360 2024-05-21T15:20:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.311+0000 7faa6affd640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 0x7faa6c083680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa6002aa30 tx=0x7faa60003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.312+0000 7faa70824640 1 -- 172.21.15.78:0/851068149 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6003f070 con 0x7faa6c109360 2024-05-21T15:20:05.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.312+0000 7faa71826640 1 -- 172.21.15.78:0/851068149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c081260 con 0x7faa6c109360 2024-05-21T15:20:05.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.312+0000 7faa71826640 1 -- 172.21.15.78:0/851068149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c081720 con 0x7faa6c109360 2024-05-21T15:20:05.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.312+0000 7faa70824640 1 -- 172.21.15.78:0/851068149 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa60031e30 con 0x7faa6c109360 2024-05-21T15:20:05.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.313+0000 7faa70824640 1 -- 172.21.15.78:0/851068149 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60035030 con 0x7faa6c109360 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.320+0000 7faa787aa740 1 Processor -- start 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.320+0000 7faa787aa740 1 -- start start 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.320+0000 7faa70824640 1 -- 172.21.15.78:0/851068149 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faa60042070 con 0x7faa6c109360 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.321+0000 7faa70824640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faa540a2f00 0x7faa540a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.321+0000 7faa787aa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a776390 0x55674ab14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.321+0000 7faa787aa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a74d4d0 0x55674ab150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.321+0000 7faa787aa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab15630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.321+0000 7faa787aa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55674aa7ac40 con 0x55674a776390 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.321+0000 7faa787aa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55674aa22580 con 0x55674a74d4d0 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.321+0000 7faa787aa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55674aa76c00 con 0x55674a7556b0 2024-05-21T15:20:05.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a776390 0x55674ab14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a776390 0x55674ab14bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44824/0 (socket says 172.21.15.78:44824) 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 -- 172.21.15.78:0/3812708248 learned_addr learned my addr 172.21.15.78:0/3812708248 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6b7fe640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faa540a2f00 0x7faa540a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6b7fe640 1 --2- 172.21.15.78:0/3812708248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab15630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 -- 172.21.15.78:0/3812708248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 msgr2=0x55674ab15630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 --2- 172.21.15.78:0/3812708248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab15630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 -- 172.21.15.78:0/3812708248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a74d4d0 msgr2=0x55674ab150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 --2- 172.21.15.78:0/3812708248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a74d4d0 0x55674ab150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 -- 172.21.15.78:0/3812708248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55674a78e3f0 con 0x55674a776390 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.322+0000 7faa6affd640 1 --2- 172.21.15.78:0/3812708248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a776390 0x55674ab14bb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faa600ed120 tx=0x7faa600b7b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a5fd84deb70fb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.323+0000 7faa70824640 1 -- 172.21.15.78:0/851068149 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7faa600efb50 con 0x7faa6c109360 2024-05-21T15:20:05.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.326+0000 7faa6b7fe640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faa540a2f00 0x7faa540a5410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faa640099c0 tx=0x7faa64009710 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.328+0000 7faa51ffb640 1 -- 172.21.15.78:0/3812708248 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600bbaf0 con 0x55674a776390 2024-05-21T15:20:05.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.328+0000 7faa51ffb640 1 -- 172.21.15.78:0/3812708248 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa600bc6b0 con 0x55674a776390 2024-05-21T15:20:05.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.328+0000 7faa51ffb640 1 -- 172.21.15.78:0/3812708248 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600d5e90 con 0x55674a776390 2024-05-21T15:20:05.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.329+0000 7faa787aa740 1 -- 172.21.15.78:0/3812708248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a776390 msgr2=0x55674ab14bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.329+0000 7faa787aa740 1 --2- 172.21.15.78:0/3812708248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a776390 0x55674ab14bb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faa600ed120 tx=0x7faa600b7b20 comp rx=0 tx=0).stop 2024-05-21T15:20:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.333+0000 7faa787aa740 1 -- 172.21.15.78:0/3812708248 shutdown_connections 2024-05-21T15:20:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.333+0000 7faa787aa740 1 --2- 172.21.15.78:0/3812708248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a74d4d0 0x55674ab150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.333+0000 7faa787aa740 1 --2- 172.21.15.78:0/3812708248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab15630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.333+0000 7faa787aa740 1 --2- 172.21.15.78:0/3812708248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a776390 0x55674ab14bb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.333+0000 7faa787aa740 1 -- 172.21.15.78:0/3812708248 >> 172.21.15.78:0/3812708248 conn(0x55674a76a4e0 msgr2=0x55674ab114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.334+0000 7faa787aa740 1 -- 172.21.15.78:0/3812708248 shutdown_connections 2024-05-21T15:20:05.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.335+0000 7faa787aa740 1 -- 172.21.15.78:0/3812708248 wait complete. 2024-05-21T15:20:05.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.336+0000 7faa787aa740 1 -- start start 2024-05-21T15:20:05.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.336+0000 7faa787aa740 1 Processor -- start 2024-05-21T15:20:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.338+0000 7faa787aa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a74d4d0 0x55674ab24050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.338+0000 7faa787aa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab24590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.338+0000 7faa787aa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a776390 0x55674ab1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.338+0000 7faa787aa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55674aa7ac40 con 0x55674a7556b0 2024-05-21T15:20:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.338+0000 7faa787aa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55674aa35c40 con 0x55674a74d4d0 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.338+0000 7faa787aa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55674ab17b40 con 0x55674a776390 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.338+0000 7faa787aa740 10 client.0 _ll_register_callbacks cb 0x55674a78dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.339+0000 7faa6b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a776390 0x55674ab1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.339+0000 7faa6b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a776390 0x55674ab1f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56830/0 (socket says 172.21.15.78:56830) 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.339+0000 7faa6b7fe640 1 -- 172.21.15.78:0/307690099 learned_addr learned my addr 172.21.15.78:0/307690099 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.339+0000 7faa6b7fe640 1 -- 172.21.15.78:0/307690099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a74d4d0 msgr2=0x55674ab24050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.339+0000 7faa6b7fe640 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a74d4d0 0x55674ab24050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.339+0000 7faa6b7fe640 1 -- 172.21.15.78:0/307690099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 msgr2=0x55674ab24590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.339+0000 7faa6b7fe640 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab24590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.339+0000 7faa6b7fe640 1 -- 172.21.15.78:0/307690099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674a8587d0 con 0x55674a776390 2024-05-21T15:20:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.345+0000 7faa6b7fe640 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a776390 0x55674ab1f430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faa6400b010 tx=0x7faa64004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25507bbaf8951ea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.345+0000 7faa4affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:20:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.349+0000 7faa4affd640 1 -- 172.21.15.78:0/307690099 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6402a060 con 0x55674a776390 2024-05-21T15:20:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.349+0000 7faa787aa740 1 -- 172.21.15.78:0/307690099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa600bc6b0 con 0x55674a776390 2024-05-21T15:20:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.350+0000 7faa4affd640 1 -- 172.21.15.78:0/307690099 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7faa6401ac00 con 0x55674a776390 2024-05-21T15:20:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.354+0000 7faa4affd640 1 -- 172.21.15.78:0/307690099 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa6401ae00 con 0x55674a776390 2024-05-21T15:20:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.354+0000 7faa787aa740 10 client.26251 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.354+0000 7faa787aa740 1 -- 172.21.15.78:0/307690099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55674ab2e1a0 con 0x55674a776390 2024-05-21T15:20:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.356+0000 7faa4affd640 1 -- 172.21.15.78:0/307690099 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa600bc6b0 con 0x55674a776390 2024-05-21T15:20:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.357+0000 7faa4affd640 1 -- 172.21.15.78:0/307690099 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7faa64024030 con 0x55674a776390 2024-05-21T15:20:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.357+0000 7faa787aa740 10 client.26251 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.357+0000 7faa787aa740 10 client.26251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:20:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.357+0000 7faa787aa740 20 client.26251 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.357+0000 7faa787aa740 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55674a734f60 0x55674ab31c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.357+0000 7faa787aa740 4 client.26251 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.357+0000 7faa787aa740 1 -- 172.21.15.78:0/307690099 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55674aa88370 con 0x55674a734f60 2024-05-21T15:20:05.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.361+0000 7faa6affd640 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55674a734f60 0x55674ab31c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.367+0000 7faa6affd640 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55674a734f60 0x55674ab31c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fda89c6a9ae21db server_cookie=230ac468a3a23c7 in_seq=0 out_seq=0 2024-05-21T15:20:05.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.367+0000 7faa4affd640 10 client.26251 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:20:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.370+0000 7faa4affd640 1 -- 172.21.15.78:0/307690099 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55674aa88370 con 0x55674a734f60 2024-05-21T15:20:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.370+0000 7faa4affd640 10 client.26251 handle_command_reply: tid=0 2024-05-21T15:20:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.370+0000 7faa787aa740 1 client.26251 shutdown 2024-05-21T15:20:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.372+0000 7faa787aa740 1 -- 172.21.15.78:0/307690099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a776390 msgr2=0x55674ab1f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.372+0000 7faa787aa740 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a776390 0x55674ab1f430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faa6400b010 tx=0x7faa64004a40 comp rx=0 tx=0).stop 2024-05-21T15:20:05.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.375+0000 7faa787aa740 1 -- 172.21.15.78:0/307690099 shutdown_connections 2024-05-21T15:20:05.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.375+0000 7faa787aa740 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55674a734f60 0x55674ab31c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.375+0000 7faa787aa740 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a776390 0x55674ab1f430 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.375+0000 7faa787aa740 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab24590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.375+0000 7faa787aa740 1 --2- 172.21.15.78:0/307690099 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a74d4d0 0x55674ab24050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.375+0000 7faa787aa740 1 -- 172.21.15.78:0/307690099 >> 172.21.15.78:0/307690099 conn(0x55674a76a4e0 msgr2=0x55674ab16900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.378+0000 7faa4affd640 0 client.26251 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:20:05.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.382+0000 7faa787aa740 1 -- 172.21.15.78:0/307690099 shutdown_connections 2024-05-21T15:20:05.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.384+0000 7faa787aa740 1 -- 172.21.15.78:0/307690099 wait complete. 2024-05-21T15:20:05.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.384+0000 7faa787aa740 20 client.26251 trim_cache size 0 max 16384 2024-05-21T15:20:05.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.417+0000 7faa787aa740 1 Processor -- start 2024-05-21T15:20:05.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.417+0000 7faa787aa740 1 -- start start 2024-05-21T15:20:05.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.417+0000 7faa787aa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a74d4d0 0x55674ab114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.417+0000 7faa787aa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.417+0000 7faa787aa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 0x55674ab37e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.417+0000 7faa787aa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa5800e8d0 con 0x55674a74d4d0 2024-05-21T15:20:05.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.417+0000 7faa787aa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faa580105c0 con 0x55674a776390 2024-05-21T15:20:05.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.417+0000 7faa787aa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faa5800efa0 con 0x55674a7556b0 2024-05-21T15:20:05.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.419+0000 7faa6affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a74d4d0 0x55674ab114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.419+0000 7faa6a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.420+0000 7faa6a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56842/0 (socket says 172.21.15.78:56842) 2024-05-21T15:20:05.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.420+0000 7faa6a7fc640 1 -- 172.21.15.78:0/3892619405 learned_addr learned my addr 172.21.15.78:0/3892619405 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.420+0000 7faa6b7fe640 1 --2- 172.21.15.78:0/3892619405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 0x55674ab37e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.421+0000 7faa6a7fc640 1 -- 172.21.15.78:0/3892619405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 msgr2=0x55674ab37e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.421+0000 7faa6a7fc640 1 --2- 172.21.15.78:0/3892619405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 0x55674ab37e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.421+0000 7faa6a7fc640 1 -- 172.21.15.78:0/3892619405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a74d4d0 msgr2=0x55674ab114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.421+0000 7faa6a7fc640 1 --2- 172.21.15.78:0/3892619405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a74d4d0 0x55674ab114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.421+0000 7faa6a7fc640 1 -- 172.21.15.78:0/3892619405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55674ab1d980 con 0x55674a7556b0 2024-05-21T15:20:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.425+0000 7faa6a7fc640 1 --2- 172.21.15.78:0/3892619405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab239b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa6c06b3a0 tx=0x7faa5c00af60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a7c8a74830d188 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.427+0000 7faa517fa640 1 -- 172.21.15.78:0/3892619405 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c005020 con 0x55674a7556b0 2024-05-21T15:20:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.427+0000 7faa517fa640 1 -- 172.21.15.78:0/3892619405 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5c0056a0 con 0x55674a7556b0 2024-05-21T15:20:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.427+0000 7faa517fa640 1 -- 172.21.15.78:0/3892619405 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c018730 con 0x55674a7556b0 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.428+0000 7faa787aa740 1 -- 172.21.15.78:0/3892619405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 msgr2=0x55674ab239b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.428+0000 7faa787aa740 1 --2- 172.21.15.78:0/3892619405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab239b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa6c06b3a0 tx=0x7faa5c00af60 comp rx=0 tx=0).stop 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.428+0000 7faa787aa740 1 -- 172.21.15.78:0/3892619405 shutdown_connections 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.428+0000 7faa787aa740 1 --2- 172.21.15.78:0/3892619405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 0x55674ab37e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.428+0000 7faa787aa740 1 --2- 172.21.15.78:0/3892619405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a7556b0 0x55674ab239b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.428+0000 7faa787aa740 1 --2- 172.21.15.78:0/3892619405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a74d4d0 0x55674ab114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.428+0000 7faa787aa740 1 -- 172.21.15.78:0/3892619405 >> 172.21.15.78:0/3892619405 conn(0x55674a76a4e0 msgr2=0x55674ab10d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.429+0000 7faa787aa740 1 -- 172.21.15.78:0/3892619405 shutdown_connections 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.429+0000 7faa787aa740 1 -- 172.21.15.78:0/3892619405 wait complete. 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.429+0000 7faa787aa740 1 -- start start 2024-05-21T15:20:05.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.430+0000 7faa787aa740 1 Processor -- start 2024-05-21T15:20:05.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.430+0000 7faa787aa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a74d4d0 0x55674ab2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.430+0000 7faa787aa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab2b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.430+0000 7faa787aa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 0x55674ab257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.430+0000 7faa787aa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa5800fe30 con 0x55674a7556b0 2024-05-21T15:20:05.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.430+0000 7faa787aa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faa5800f730 con 0x55674a776390 2024-05-21T15:20:05.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.430+0000 7faa787aa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faa5800c960 con 0x55674a74d4d0 2024-05-21T15:20:05.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.430+0000 7faa787aa740 10 client.0 _ll_register_callbacks cb 0x7faa64013380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:05.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.431+0000 7faa6a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab2b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.431+0000 7faa6a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab2b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44850/0 (socket says 172.21.15.78:44850) 2024-05-21T15:20:05.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.431+0000 7faa6a7fc640 1 -- 172.21.15.78:0/937836158 learned_addr learned my addr 172.21.15.78:0/937836158 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.432+0000 7faa6a7fc640 1 -- 172.21.15.78:0/937836158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a74d4d0 msgr2=0x55674ab2b3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:20:05.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.432+0000 7faa6a7fc640 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a74d4d0 0x55674ab2b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.432+0000 7faa6a7fc640 1 -- 172.21.15.78:0/937836158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 msgr2=0x55674ab257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.432+0000 7faa6a7fc640 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 0x55674ab257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.432+0000 7faa6a7fc640 1 -- 172.21.15.78:0/937836158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674ab25fd0 con 0x55674a7556b0 2024-05-21T15:20:05.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.432+0000 7faa6a7fc640 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab2b950 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7faa5c004e10 tx=0x7faa5c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437e95ad493016d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.432+0000 7faa4affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.436+0000 7faa4affd640 1 -- 172.21.15.78:0/937836158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c027070 con 0x55674a7556b0 2024-05-21T15:20:05.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.436+0000 7faa787aa740 1 -- 172.21.15.78:0/937836158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5c0056a0 con 0x55674a7556b0 2024-05-21T15:20:05.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.436+0000 7faa4affd640 1 -- 172.21.15.78:0/937836158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7faa5c005230 con 0x55674a7556b0 2024-05-21T15:20:05.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.439+0000 7faa4affd640 1 -- 172.21.15.78:0/937836158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa5c030050 con 0x55674a7556b0 2024-05-21T15:20:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.442+0000 7faa787aa740 10 client.16800 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.442+0000 7faa787aa740 1 -- 172.21.15.78:0/937836158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55674ab994d0 con 0x55674a7556b0 2024-05-21T15:20:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.445+0000 7faa4affd640 1 -- 172.21.15.78:0/937836158 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5c0056a0 con 0x55674a7556b0 2024-05-21T15:20:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.445+0000 7faa4affd640 1 -- 172.21.15.78:0/937836158 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7faa5c0039d0 con 0x55674a7556b0 2024-05-21T15:20:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.447+0000 7faa787aa740 10 client.16800 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.447+0000 7faa787aa740 10 client.16800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:20:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.447+0000 7faa787aa740 20 client.16800 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.447+0000 7faa787aa740 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55674a70a3a0 0x55674ab9bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.447+0000 7faa787aa740 4 client.16800 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.447+0000 7faa787aa740 1 -- 172.21.15.78:0/937836158 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55674aa826c0 con 0x55674a70a3a0 2024-05-21T15:20:05.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.451+0000 7faa6affd640 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55674a70a3a0 0x55674ab9bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.464+0000 7faa6affd640 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55674a70a3a0 0x55674ab9bbf0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9805402fa15f855b server_cookie=176a61eeb4ddf5c2 in_seq=0 out_seq=0 2024-05-21T15:20:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.465+0000 7faa4affd640 10 client.16800 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:20:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.466+0000 7faa4affd640 1 -- 172.21.15.78:0/937836158 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55674aa826c0 con 0x55674a70a3a0 2024-05-21T15:20:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.466+0000 7faa4affd640 10 client.16800 handle_command_reply: tid=0 2024-05-21T15:20:05.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.467+0000 7faa787aa740 1 client.16800 shutdown 2024-05-21T15:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.472+0000 7faa787aa740 1 -- 172.21.15.78:0/937836158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 msgr2=0x55674ab2b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.472+0000 7faa787aa740 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab2b950 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7faa5c004e10 tx=0x7faa5c0048f0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.476+0000 7faa787aa740 1 -- 172.21.15.78:0/937836158 shutdown_connections 2024-05-21T15:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.476+0000 7faa787aa740 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55674a70a3a0 0x55674ab9bbf0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.476+0000 7faa787aa740 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55674a776390 0x55674ab257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.476+0000 7faa787aa740 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55674a7556b0 0x55674ab2b950 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.476+0000 7faa787aa740 1 --2- 172.21.15.78:0/937836158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55674a74d4d0 0x55674ab2b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.476+0000 7faa787aa740 1 -- 172.21.15.78:0/937836158 >> 172.21.15.78:0/937836158 conn(0x55674a76a4e0 msgr2=0x55674ab1ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.478+0000 7faa4affd640 0 client.16800 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.478+0000 7faa787aa740 1 -- 172.21.15.78:0/937836158 shutdown_connections 2024-05-21T15:20:05.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.479+0000 7faa787aa740 1 -- 172.21.15.78:0/937836158 wait complete. 2024-05-21T15:20:05.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.480+0000 7faa787aa740 20 client.16800 trim_cache size 0 max 16384 2024-05-21T15:20:05.487 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 -- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faa540a2f00 msgr2=0x7faa540a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faa540a2f00 0x7faa540a5410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faa640099c0 tx=0x7faa64009710 comp rx=0 tx=0).stop 2024-05-21T15:20:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 -- 172.21.15.78:0/851068149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 msgr2=0x7faa6c083680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 0x7faa6c083680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa6002aa30 tx=0x7faa60003cd0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 -- 172.21.15.78:0/851068149 shutdown_connections 2024-05-21T15:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faa540a2f00 0x7faa540a5410 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa6c10e3b0 0x7faa6c07cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa6c10d8f0 0x7faa6c07c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 --2- 172.21.15.78:0/851068149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c109360 0x7faa6c083680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.486+0000 7faa71826640 1 -- 172.21.15.78:0/851068149 >> 172.21.15.78:0/851068149 conn(0x7faa6c075fa0 msgr2=0x7faa6c0796b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.487+0000 7faa71826640 1 -- 172.21.15.78:0/851068149 shutdown_connections 2024-05-21T15:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.487+0000 7faa71826640 1 -- 172.21.15.78:0/851068149 wait complete. 2024-05-21T15:20:05.512 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:20:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.732+0000 7f13dfae9640 1 -- 172.21.15.78:0/2568424278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d8119770 msgr2=0x7f13d811bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.732+0000 7f13dfae9640 1 --2- 172.21.15.78:0/2568424278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d8119770 0x7f13d811bb90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f13c801a940 tx=0x7f13c803b500 comp rx=0 tx=0).stop 2024-05-21T15:20:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.734+0000 7f13dfae9640 1 -- 172.21.15.78:0/2568424278 shutdown_connections 2024-05-21T15:20:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.734+0000 7f13dfae9640 1 --2- 172.21.15.78:0/2568424278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d8119770 0x7f13d811bb90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.734+0000 7f13dfae9640 1 --2- 172.21.15.78:0/2568424278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d811c0d0 0x7f13d811e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.734+0000 7f13dfae9640 1 --2- 172.21.15.78:0/2568424278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d8109060 0x7f13d8109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.734+0000 7f13dfae9640 1 -- 172.21.15.78:0/2568424278 >> 172.21.15.78:0/2568424278 conn(0x7f13d806d750 msgr2=0x7f13d806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.735+0000 7f13dfae9640 1 -- 172.21.15.78:0/2568424278 shutdown_connections 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.736+0000 7f13dfae9640 1 -- 172.21.15.78:0/2568424278 wait complete. 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.736+0000 7f13dfae9640 1 Processor -- start 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13dfae9640 1 -- start start 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13dfae9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d8109060 0x7f13d81153b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13dfae9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8119770 0x7f13d81158f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13dfae9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d811c0d0 0x7f13d8115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13dfae9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8113940 con 0x7f13d8119770 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13dfae9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81137c0 con 0x7f13d8109060 2024-05-21T15:20:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13dfae9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8113ac0 con 0x7f13d811c0d0 2024-05-21T15:20:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13de086640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d811c0d0 0x7f13d8115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13de086640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d811c0d0 0x7f13d8115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56860/0 (socket says 172.21.15.78:56860) 2024-05-21T15:20:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.737+0000 7f13de086640 1 -- 172.21.15.78:0/1923719929 learned_addr learned my addr 172.21.15.78:0/1923719929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.738+0000 7f13dd885640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d8109060 0x7f13d81153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.738+0000 7f13de086640 1 -- 172.21.15.78:0/1923719929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d8109060 msgr2=0x7f13d81153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.738+0000 7f13de086640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d8109060 0x7f13d81153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.738+0000 7f13de086640 1 -- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8119770 msgr2=0x7f13d81158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.738+0000 7f13de086640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8119770 0x7f13d81158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.738+0000 7f13de086640 1 -- 172.21.15.78:0/1923719929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d8079db0 con 0x7f13d811c0d0 2024-05-21T15:20:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.738+0000 7f13de086640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d811c0d0 0x7f13d8115e30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f13d000eb10 tx=0x7f13d000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.740+0000 7f13ceffd640 1 -- 172.21.15.78:0/1923719929 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0004b20 con 0x7f13d811c0d0 2024-05-21T15:20:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.740+0000 7f13dfae9640 1 -- 172.21.15.78:0/1923719929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d807a0a0 con 0x7f13d811c0d0 2024-05-21T15:20:05.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.740+0000 7f13dfae9640 1 -- 172.21.15.78:0/1923719929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d807a5f0 con 0x7f13d811c0d0 2024-05-21T15:20:05.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.741+0000 7f13ceffd640 1 -- 172.21.15.78:0/1923719929 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d0004cc0 con 0x7f13d811c0d0 2024-05-21T15:20:05.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.741+0000 7f13ceffd640 1 -- 172.21.15.78:0/1923719929 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00107b0 con 0x7f13d811c0d0 2024-05-21T15:20:05.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.746+0000 7f13ceffd640 1 -- 172.21.15.78:0/1923719929 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f13d00109f0 con 0x7f13d811c0d0 2024-05-21T15:20:05.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.747+0000 7f13ceffd640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13c40a2f10 0x7f13c40a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.747+0000 7f13ceffd640 1 -- 172.21.15.78:0/1923719929 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f13d00cfe50 con 0x7f13d811c0d0 2024-05-21T15:20:05.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.749+0000 7f13e6992740 1 Processor -- start 2024-05-21T15:20:05.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.750+0000 7f13e6992740 1 -- start start 2024-05-21T15:20:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.750+0000 7f13e6992740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1ac1e390 0x55ea1afbcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.750+0000 7f13e6992740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1abf54d0 0x55ea1afbd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.750+0000 7f13e6992740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afbd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.751+0000 7f13e6992740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea1af22c40 con 0x55ea1ac1e390 2024-05-21T15:20:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.751+0000 7f13e6992740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ea1aeca580 con 0x55ea1abfd6b0 2024-05-21T15:20:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.751+0000 7f13e6992740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ea1af1ec00 con 0x55ea1abf54d0 2024-05-21T15:20:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.752+0000 7f13de086640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afbd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.753+0000 7f13de086640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afbd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38900/0 (socket says 172.21.15.78:38900) 2024-05-21T15:20:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.753+0000 7f13de086640 1 -- 172.21.15.78:0/3681252800 learned_addr learned my addr 172.21.15.78:0/3681252800 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.753+0000 7f13de086640 1 -- 172.21.15.78:0/3681252800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1abf54d0 msgr2=0x55ea1afbd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.753+0000 7f13de086640 1 --2- 172.21.15.78:0/3681252800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1abf54d0 0x55ea1afbd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.753+0000 7f13de086640 1 -- 172.21.15.78:0/3681252800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1ac1e390 msgr2=0x55ea1afbcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.753+0000 7f13de086640 1 --2- 172.21.15.78:0/3681252800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1ac1e390 0x55ea1afbcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.753+0000 7f13de086640 1 -- 172.21.15.78:0/3681252800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1ac363f0 con 0x55ea1abfd6b0 2024-05-21T15:20:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.754+0000 7f13de086640 1 --2- 172.21.15.78:0/3681252800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afbd7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f13d0098710 tx=0x7f13d0098680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85b00fa9bd1e653 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.755+0000 7f13dd885640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13c40a2f10 0x7f13c40a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.759+0000 7f13c3fff640 1 -- 172.21.15.78:0/3681252800 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00b5270 con 0x55ea1abfd6b0 2024-05-21T15:20:05.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.759+0000 7f13c3fff640 1 -- 172.21.15.78:0/3681252800 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d009c2b0 con 0x55ea1abfd6b0 2024-05-21T15:20:05.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.759+0000 7f13c3fff640 1 -- 172.21.15.78:0/3681252800 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00a4a90 con 0x55ea1abfd6b0 2024-05-21T15:20:05.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.760+0000 7f13e6992740 1 -- 172.21.15.78:0/3681252800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 msgr2=0x55ea1afbd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.760+0000 7f13e6992740 1 --2- 172.21.15.78:0/3681252800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afbd7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f13d0098710 tx=0x7f13d0098680 comp rx=0 tx=0).stop 2024-05-21T15:20:05.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.762+0000 7f13e6992740 1 -- 172.21.15.78:0/3681252800 shutdown_connections 2024-05-21T15:20:05.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.762+0000 7f13e6992740 1 --2- 172.21.15.78:0/3681252800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afbd7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.762+0000 7f13e6992740 1 --2- 172.21.15.78:0/3681252800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1abf54d0 0x55ea1afbd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.762+0000 7f13e6992740 1 --2- 172.21.15.78:0/3681252800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1ac1e390 0x55ea1afbcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.762+0000 7f13e6992740 1 -- 172.21.15.78:0/3681252800 >> 172.21.15.78:0/3681252800 conn(0x55ea1ac124e0 msgr2=0x55ea1afb94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.763+0000 7f13e6992740 1 -- 172.21.15.78:0/3681252800 shutdown_connections 2024-05-21T15:20:05.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.765+0000 7f13e6992740 1 -- 172.21.15.78:0/3681252800 wait complete. 2024-05-21T15:20:05.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.765+0000 7f13e6992740 1 -- start start 2024-05-21T15:20:05.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.766+0000 7f13e6992740 1 Processor -- start 2024-05-21T15:20:05.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.766+0000 7f13dd885640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13c40a2f10 0x7f13c40a5420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55ea1afb9e90 tx=0x7f13d400b5b0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.766+0000 7f13e6992740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afc7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.766+0000 7f13e6992740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afc7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.766+0000 7f13e6992740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afd3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.767+0000 7f13e6992740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ea1afca010 con 0x55ea1abf54d0 2024-05-21T15:20:05.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.767+0000 7f13e6992740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ea1aeddc40 con 0x55ea1abfd6b0 2024-05-21T15:20:05.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.767+0000 7f13e6992740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ea1afca190 con 0x55ea1ac1e390 2024-05-21T15:20:05.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.767+0000 7f13e6992740 10 client.0 _ll_register_callbacks cb 0x55ea1ac35be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:05.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.767+0000 7f13dd084640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afc7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.767+0000 7f13dd084640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afc7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38914/0 (socket says 172.21.15.78:38914) 2024-05-21T15:20:05.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.767+0000 7f13dd084640 1 -- 172.21.15.78:0/3741865780 learned_addr learned my addr 172.21.15.78:0/3741865780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.767+0000 7f13de086640 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afd3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.768+0000 7f13dd084640 1 -- 172.21.15.78:0/3741865780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 msgr2=0x55ea1afd3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.768+0000 7f13dd084640 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afd3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.768+0000 7f13dd084640 1 -- 172.21.15.78:0/3741865780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 msgr2=0x55ea1afc7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.768+0000 7f13dd084640 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afc7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.768+0000 7f13dd084640 1 -- 172.21.15.78:0/3741865780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1ad007d0 con 0x55ea1abfd6b0 2024-05-21T15:20:05.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.775+0000 7f13dd084640 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afc7840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13c801a860 tx=0x7f13c801a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19867b17320e33b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.777+0000 7f13c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:20:05.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.780+0000 7f13c17fa640 1 -- 172.21.15.78:0/3741865780 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8046070 con 0x55ea1abfd6b0 2024-05-21T15:20:05.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.780+0000 7f13c17fa640 1 -- 172.21.15.78:0/3741865780 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f13c8040db0 con 0x55ea1abfd6b0 2024-05-21T15:20:05.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.780+0000 7f13e6992740 1 -- 172.21.15.78:0/3741865780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d009c2b0 con 0x55ea1abfd6b0 2024-05-21T15:20:05.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.791+0000 7f13c17fa640 1 -- 172.21.15.78:0/3741865780 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f13c8004050 con 0x55ea1abfd6b0 2024-05-21T15:20:05.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.795+0000 7f13e6992740 10 client.26285 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:05.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.795+0000 7f13e6992740 1 -- 172.21.15.78:0/3741865780 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ea1afd6760 con 0x55ea1abfd6b0 2024-05-21T15:20:05.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.798+0000 7f13c17fa640 1 -- 172.21.15.78:0/3741865780 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d009c2b0 con 0x55ea1abfd6b0 2024-05-21T15:20:05.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.799+0000 7f13c17fa640 1 -- 172.21.15.78:0/3741865780 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f13c803bce0 con 0x55ea1abfd6b0 2024-05-21T15:20:05.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.800+0000 7f13e6992740 10 client.26285 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:05.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.800+0000 7f13e6992740 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:05.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.800+0000 7f13e6992740 20 client.26285 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:05.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.800+0000 7f13e6992740 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ea1abdcf60 0x55ea1afd9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.800+0000 7f13e6992740 4 client.26285 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:05.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.800+0000 7f13e6992740 1 -- 172.21.15.78:0/3741865780 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea1af30370 con 0x55ea1abdcf60 2024-05-21T15:20:05.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.801+0000 7f13dd885640 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ea1abdcf60 0x55ea1afd9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.806+0000 7f13dd885640 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ea1abdcf60 0x55ea1afd9d10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cbc4c99a9c57a2 server_cookie=f22ed76f79cc1f49 in_seq=0 out_seq=0 2024-05-21T15:20:05.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.806+0000 7f13c17fa640 10 client.26285 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:05.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.808+0000 7f13c17fa640 1 -- 172.21.15.78:0/3741865780 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea1af30370 con 0x55ea1abdcf60 2024-05-21T15:20:05.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.808+0000 7f13c17fa640 10 client.26285 handle_command_reply: tid=0 2024-05-21T15:20:05.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.810+0000 7f13e6992740 1 client.26285 shutdown 2024-05-21T15:20:05.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.815+0000 7f13e6992740 1 -- 172.21.15.78:0/3741865780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 msgr2=0x55ea1afc7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.815+0000 7f13e6992740 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afc7840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13c801a860 tx=0x7f13c801a570 comp rx=0 tx=0).stop 2024-05-21T15:20:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.816+0000 7f13e6992740 1 -- 172.21.15.78:0/3741865780 shutdown_connections 2024-05-21T15:20:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.816+0000 7f13e6992740 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ea1abdcf60 0x55ea1afd9d10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.816+0000 7f13e6992740 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afc7840 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.816+0000 7f13e6992740 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afd3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.816+0000 7f13e6992740 1 --2- 172.21.15.78:0/3741865780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afc7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.816+0000 7f13e6992740 1 -- 172.21.15.78:0/3741865780 >> 172.21.15.78:0/3741865780 conn(0x55ea1ac124e0 msgr2=0x55ea1afbea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.816+0000 7f13c17fa640 0 client.26285 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:05.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.818+0000 7f13e6992740 1 -- 172.21.15.78:0/3741865780 shutdown_connections 2024-05-21T15:20:05.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.818+0000 7f13e6992740 1 -- 172.21.15.78:0/3741865780 wait complete. 2024-05-21T15:20:05.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.819+0000 7f13e6992740 20 client.26285 trim_cache size 0 max 16384 2024-05-21T15:20:05.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.858+0000 7f13e6992740 1 Processor -- start 2024-05-21T15:20:05.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.861+0000 7f13e6992740 1 -- start start 2024-05-21T15:20:05.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.861+0000 7f13e6992740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afc60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.862+0000 7f13e6992740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afd3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.862+0000 7f13e6992740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afc09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.862+0000 7f13e6992740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc00e880 con 0x55ea1abf54d0 2024-05-21T15:20:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.862+0000 7f13e6992740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc010570 con 0x55ea1abfd6b0 2024-05-21T15:20:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.862+0000 7f13e6992740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc00ef50 con 0x55ea1ac1e390 2024-05-21T15:20:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.862+0000 7f13de086640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afc09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.862+0000 7f13de086640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afc09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56900/0 (socket says 172.21.15.78:56900) 2024-05-21T15:20:05.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.862+0000 7f13de086640 1 -- 172.21.15.78:0/3017396904 learned_addr learned my addr 172.21.15.78:0/3017396904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.864+0000 7f13dd084640 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afd3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.864+0000 7f13dd885640 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afc60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.864+0000 7f13de086640 1 -- 172.21.15.78:0/3017396904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 msgr2=0x55ea1afd3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.864+0000 7f13de086640 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afd3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.864+0000 7f13de086640 1 -- 172.21.15.78:0/3017396904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 msgr2=0x55ea1afc60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.864+0000 7f13de086640 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afc60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.864+0000 7f13de086640 1 -- 172.21.15.78:0/3017396904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1afc99d0 con 0x55ea1ac1e390 2024-05-21T15:20:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.865+0000 7f13de086640 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afc09d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f13d00049d0 tx=0x7f13d0005b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39add1349e4e5d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.866+0000 7f13c37fe640 1 -- 172.21.15.78:0/3017396904 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00a6070 con 0x55ea1ac1e390 2024-05-21T15:20:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.866+0000 7f13c37fe640 1 -- 172.21.15.78:0/3017396904 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d00b5240 con 0x55ea1ac1e390 2024-05-21T15:20:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.866+0000 7f13c37fe640 1 -- 172.21.15.78:0/3017396904 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00a4cf0 con 0x55ea1ac1e390 2024-05-21T15:20:05.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.867+0000 7f13dd885640 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afc60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.867+0000 7f13e6992740 1 -- 172.21.15.78:0/3017396904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 msgr2=0x55ea1afc09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.867+0000 7f13e6992740 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afc09d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f13d00049d0 tx=0x7f13d0005b60 comp rx=0 tx=0).stop 2024-05-21T15:20:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.869+0000 7f13e6992740 1 -- 172.21.15.78:0/3017396904 shutdown_connections 2024-05-21T15:20:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.869+0000 7f13e6992740 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afd3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.869+0000 7f13e6992740 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afc09d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.869+0000 7f13e6992740 1 --2- 172.21.15.78:0/3017396904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afc60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.869+0000 7f13e6992740 1 -- 172.21.15.78:0/3017396904 >> 172.21.15.78:0/3017396904 conn(0x55ea1ac124e0 msgr2=0x55ea1afc59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.870+0000 7f13e6992740 1 -- 172.21.15.78:0/3017396904 shutdown_connections 2024-05-21T15:20:05.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.870+0000 7f13e6992740 1 -- 172.21.15.78:0/3017396904 wait complete. 2024-05-21T15:20:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.871+0000 7f13e6992740 1 -- start start 2024-05-21T15:20:05.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.871+0000 7f13e6992740 1 Processor -- start 2024-05-21T15:20:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.872+0000 7f13e6992740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.872+0000 7f13e6992740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.873+0000 7f13e6992740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afcaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.873+0000 7f13e6992740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc00fde0 con 0x55ea1abf54d0 2024-05-21T15:20:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.873+0000 7f13e6992740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc00f6e0 con 0x55ea1abfd6b0 2024-05-21T15:20:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.873+0000 7f13e6992740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc00c910 con 0x55ea1ac1e390 2024-05-21T15:20:05.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.873+0000 7f13e6992740 10 client.0 _ll_register_callbacks cb 0x7f13c8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:05.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.874+0000 7f13de086640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afcaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.875+0000 7f13de086640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afcaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56910/0 (socket says 172.21.15.78:56910) 2024-05-21T15:20:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.875+0000 7f13de086640 1 -- 172.21.15.78:0/2847574282 learned_addr learned my addr 172.21.15.78:0/2847574282 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.875+0000 7f13dd084640 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.875+0000 7f13de086640 1 -- 172.21.15.78:0/2847574282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 msgr2=0x55ea1afca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.875+0000 7f13de086640 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.875+0000 7f13de086640 1 -- 172.21.15.78:0/2847574282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 msgr2=0x55ea1afca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.875+0000 7f13de086640 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.875+0000 7f13de086640 1 -- 172.21.15.78:0/2847574282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1b040180 con 0x55ea1ac1e390 2024-05-21T15:20:05.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.878+0000 7f13de086640 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afcaec0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13d009c170 tx=0x7f13d00a4cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e6403e5b6f7b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:05.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.878+0000 7f13c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:20:05.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.879+0000 7f13c17fa640 1 -- 172.21.15.78:0/2847574282 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00a6070 con 0x55ea1ac1e390 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.879+0000 7f13e6992740 1 -- 172.21.15.78:0/2847574282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d00b5240 con 0x55ea1ac1e390 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.879+0000 7f13c17fa640 1 -- 172.21.15.78:0/2847574282 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f13d00b5040 con 0x55ea1ac1e390 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.880+0000 7f13c17fa640 1 -- 172.21.15.78:0/2847574282 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f13d00a0660 con 0x55ea1ac1e390 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.880+0000 7f13e6992740 10 client.26269 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.880+0000 7f13e6992740 1 -- 172.21.15.78:0/2847574282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ea1b041370 con 0x55ea1ac1e390 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.880+0000 7f13c17fa640 1 -- 172.21.15.78:0/2847574282 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d00b5240 con 0x55ea1ac1e390 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.880+0000 7f13c17fa640 1 -- 172.21.15.78:0/2847574282 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f13d0010ce0 con 0x55ea1ac1e390 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.881+0000 7f13e6992740 10 client.26269 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.881+0000 7f13e6992740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.881+0000 7f13e6992740 20 client.26269 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.881+0000 7f13e6992740 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ea1abb23a0 0x55ea1b043970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.881+0000 7f13e6992740 4 client.26269 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:05.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.881+0000 7f13e6992740 1 -- 172.21.15.78:0/2847574282 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea1af2a6c0 con 0x55ea1abb23a0 2024-05-21T15:20:05.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.881+0000 7f13dd885640 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ea1abb23a0 0x55ea1b043970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.885+0000 7f13dd885640 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ea1abb23a0 0x55ea1b043970 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c9d1cc54b23d101 server_cookie=de18e5c0f0c7a118 in_seq=0 out_seq=0 2024-05-21T15:20:05.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.886+0000 7f13c17fa640 10 client.26269 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.887+0000 7f13c17fa640 1 -- 172.21.15.78:0/2847574282 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea1af2a6c0 con 0x55ea1abb23a0 2024-05-21T15:20:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.887+0000 7f13c17fa640 10 client.26269 handle_command_reply: tid=0 2024-05-21T15:20:05.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.887+0000 7f13e6992740 1 client.26269 shutdown 2024-05-21T15:20:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.891+0000 7f13e6992740 1 -- 172.21.15.78:0/2847574282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 msgr2=0x55ea1afcaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.891+0000 7f13e6992740 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afcaec0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13d009c170 tx=0x7f13d00a4cc0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.895+0000 7f13e6992740 1 -- 172.21.15.78:0/2847574282 shutdown_connections 2024-05-21T15:20:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.895+0000 7f13e6992740 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ea1abb23a0 0x55ea1b043970 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.895+0000 7f13e6992740 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ea1abfd6b0 0x55ea1afca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.895+0000 7f13e6992740 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea1ac1e390 0x55ea1afcaec0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.895+0000 7f13e6992740 1 --2- 172.21.15.78:0/2847574282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ea1abf54d0 0x55ea1afca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.895+0000 7f13e6992740 1 -- 172.21.15.78:0/2847574282 >> 172.21.15.78:0/2847574282 conn(0x55ea1ac124e0 msgr2=0x55ea1afc87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.895+0000 7f13c17fa640 0 client.26269 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:05.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.899+0000 7f13e6992740 1 -- 172.21.15.78:0/2847574282 shutdown_connections 2024-05-21T15:20:05.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.899+0000 7f13e6992740 1 -- 172.21.15.78:0/2847574282 wait complete. 2024-05-21T15:20:05.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.900+0000 7f13e6992740 20 client.26269 trim_cache size 0 max 16384 2024-05-21T15:20:05.907 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:05.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 -- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13c40a2f10 msgr2=0x7f13c40a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13c40a2f10 0x7f13c40a5420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55ea1afb9e90 tx=0x7f13d400b5b0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 -- 172.21.15.78:0/1923719929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d811c0d0 msgr2=0x7f13d8115e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d811c0d0 0x7f13d8115e30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f13d000eb10 tx=0x7f13d000c2d0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 -- 172.21.15.78:0/1923719929 shutdown_connections 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f13c40a2f10 0x7f13c40a5420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d811c0d0 0x7f13d8115e30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8119770 0x7f13d81158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 --2- 172.21.15.78:0/1923719929 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13d8109060 0x7f13d81153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 -- 172.21.15.78:0/1923719929 >> 172.21.15.78:0/1923719929 conn(0x7f13d806d750 msgr2=0x7f13d8119b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 -- 172.21.15.78:0/1923719929 shutdown_connections 2024-05-21T15:20:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:05.908+0000 7f13dfae9640 1 -- 172.21.15.78:0/1923719929 wait complete. 2024-05-21T15:20:06.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:06 smithi078 ceph-mon[31997]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 32 KiB/s rd, 43 MiB/s wr, 9.36k op/s 2024-05-21T15:20:06.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:06 smithi084 ceph-mon[36334]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 32 KiB/s rd, 43 MiB/s wr, 9.36k op/s 2024-05-21T15:20:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:06 smithi172 ceph-mon[36402]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 32 KiB/s rd, 43 MiB/s wr, 9.36k op/s 2024-05-21T15:20:06.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:20:06.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:20:06.932 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:20:07.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.114+0000 7fac84a59640 1 -- 172.21.15.78:0/2343387534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 msgr2=0x7fac7809b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.114+0000 7fac84a59640 1 --2- 172.21.15.78:0/2343387534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 0x7fac7809b200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac70009860 tx=0x7fac7002a3d0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.115+0000 7fac84a59640 1 -- 172.21.15.78:0/2343387534 shutdown_connections 2024-05-21T15:20:07.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.115+0000 7fac84a59640 1 --2- 172.21.15.78:0/2343387534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 0x7fac7809b200 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.115+0000 7fac84a59640 1 --2- 172.21.15.78:0/2343387534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac7809a380 0x7fac7809a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.115+0000 7fac84a59640 1 --2- 172.21.15.78:0/2343387534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac78095930 0x7fac78095d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.115+0000 7fac84a59640 1 -- 172.21.15.78:0/2343387534 >> 172.21.15.78:0/2343387534 conn(0x7fac7800afe0 msgr2=0x7fac7800b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.115+0000 7fac84a59640 1 -- 172.21.15.78:0/2343387534 shutdown_connections 2024-05-21T15:20:07.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.115+0000 7fac84a59640 1 -- 172.21.15.78:0/2343387534 wait complete. 2024-05-21T15:20:07.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.117+0000 7fac84a59640 1 Processor -- start 2024-05-21T15:20:07.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac84a59640 1 -- start start 2024-05-21T15:20:07.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac84a59640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac78095930 0x7fac780a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac84a59640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac7809a380 0x7fac780a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac84a59640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 0x7fac780a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac84a59640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac7809e330 con 0x7fac7809a380 2024-05-21T15:20:07.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac84a59640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fac7809e1b0 con 0x7fac78095930 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac84a59640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fac7809e4b0 con 0x7fac7809ad20 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac7ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac78095930 0x7fac780a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.118+0000 7fac7f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 0x7fac780a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 0x7fac780a6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56924/0 (socket says 172.21.15.78:56924) 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7f59e640 1 -- 172.21.15.78:0/739240833 learned_addr learned my addr 172.21.15.78:0/739240833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7e59c640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac7809a380 0x7fac780a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7f59e640 1 -- 172.21.15.78:0/739240833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac78095930 msgr2=0x7fac780a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7f59e640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac78095930 0x7fac780a9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7f59e640 1 -- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac7809a380 msgr2=0x7fac780a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7f59e640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac7809a380 0x7fac780a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7f59e640 1 -- 172.21.15.78:0/739240833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac781376d0 con 0x7fac7809ad20 2024-05-21T15:20:07.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.119+0000 7fac7e59c640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac7809a380 0x7fac780a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:07.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.120+0000 7fac7f59e640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 0x7fac780a6360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac70009980 tx=0x7fac70030890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.120+0000 7fac67fff640 1 -- 172.21.15.78:0/739240833 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70004440 con 0x7fac7809ad20 2024-05-21T15:20:07.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.120+0000 7fac67fff640 1 -- 172.21.15.78:0/739240833 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac700055a0 con 0x7fac7809ad20 2024-05-21T15:20:07.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.120+0000 7fac67fff640 1 -- 172.21.15.78:0/739240833 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac700049d0 con 0x7fac7809ad20 2024-05-21T15:20:07.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.120+0000 7fac84a59640 1 -- 172.21.15.78:0/739240833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac781378a0 con 0x7fac7809ad20 2024-05-21T15:20:07.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.121+0000 7fac84a59640 1 -- 172.21.15.78:0/739240833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac78137c50 con 0x7fac7809ad20 2024-05-21T15:20:07.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.124+0000 7fac8c1dd740 1 Processor -- start 2024-05-21T15:20:07.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.125+0000 7fac8c1dd740 1 -- start start 2024-05-21T15:20:07.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.127+0000 7fac67fff640 1 -- 172.21.15.78:0/739240833 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fac70039040 con 0x7fac7809ad20 2024-05-21T15:20:07.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 0x5561cbbab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbabd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5561cbb10990 con 0x5561cb7e3550 2024-05-21T15:20:07.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5561cbab85b0 con 0x5561cb7eb730 2024-05-21T15:20:07.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5561cbb0c950 con 0x5561cb80c410 2024-05-21T15:20:07.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac7e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 0x5561cbbab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac7ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbabd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac67fff640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fac680a2e00 0x7fac680a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac7ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbabd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44924/0 (socket says 172.21.15.78:44924) 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac7ed9d640 1 -- 172.21.15.78:0/127366217 learned_addr learned my addr 172.21.15.78:0/127366217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac7e59c640 1 --2- 172.21.15.78:0/127366217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.128+0000 7fac7f59e640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fac680a2e00 0x7fac680a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.129+0000 7fac7e59c640 1 -- 172.21.15.78:0/127366217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 msgr2=0x5561cbbab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.129+0000 7fac7e59c640 1 --2- 172.21.15.78:0/127366217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 0x5561cbbab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.129+0000 7fac7e59c640 1 -- 172.21.15.78:0/127366217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 msgr2=0x5561cbbabd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.129+0000 7fac7e59c640 1 --2- 172.21.15.78:0/127366217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbabd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.129+0000 7fac7e59c640 1 -- 172.21.15.78:0/127366217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cb8ec7e0 con 0x5561cb7eb730 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.129+0000 7fac67fff640 1 -- 172.21.15.78:0/739240833 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fac700f01d0 con 0x7fac7809ad20 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.129+0000 7fac7f59e640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fac680a2e00 0x7fac680a5310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac700bc330 tx=0x7fac700b88d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.130+0000 7fac7e59c640 1 --2- 172.21.15.78:0/127366217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbac2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5561cbba8990 tx=0x7fac74005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4b84c242c3fc8ae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.130+0000 7fac657fa640 1 -- 172.21.15.78:0/127366217 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74016070 con 0x5561cb7eb730 2024-05-21T15:20:07.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.130+0000 7fac657fa640 1 -- 172.21.15.78:0/127366217 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac7400dbf0 con 0x5561cb7eb730 2024-05-21T15:20:07.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.130+0000 7fac657fa640 1 -- 172.21.15.78:0/127366217 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74003970 con 0x5561cb7eb730 2024-05-21T15:20:07.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.132+0000 7fac8c1dd740 1 -- 172.21.15.78:0/127366217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 msgr2=0x5561cbbac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.132+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/127366217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbac2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5561cbba8990 tx=0x7fac74005f90 comp rx=0 tx=0).stop 2024-05-21T15:20:07.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.133+0000 7fac8c1dd740 1 -- 172.21.15.78:0/127366217 shutdown_connections 2024-05-21T15:20:07.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.133+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/127366217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbac2a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.133+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/127366217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbabd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.133+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/127366217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 0x5561cbbab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.133+0000 7fac8c1dd740 1 -- 172.21.15.78:0/127366217 >> 172.21.15.78:0/127366217 conn(0x5561cb800560 msgr2=0x5561cbba7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.133+0000 7fac8c1dd740 1 -- 172.21.15.78:0/127366217 shutdown_connections 2024-05-21T15:20:07.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.133+0000 7fac8c1dd740 1 -- 172.21.15.78:0/127366217 wait complete. 2024-05-21T15:20:07.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.134+0000 7fac8c1dd740 1 -- start start 2024-05-21T15:20:07.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.134+0000 7fac8c1dd740 1 Processor -- start 2024-05-21T15:20:07.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.134+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbb79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.134+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbb7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.134+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 0x5561cbbb5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.134+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5561cbbb8b70 con 0x5561cb7e3550 2024-05-21T15:20:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.134+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5561cbacbb40 con 0x5561cb7eb730 2024-05-21T15:20:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.134+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5561cbbb8cf0 con 0x5561cb80c410 2024-05-21T15:20:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac8c1dd740 10 client.0 _ll_register_callbacks cb 0x5561cbac00f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac7ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbb7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac7ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbb7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38966/0 (socket says 172.21.15.78:38966) 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac7ed9d640 1 -- 172.21.15.78:0/1243866141 learned_addr learned my addr 172.21.15.78:0/1243866141 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac7ed9d640 1 -- 172.21.15.78:0/1243866141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 msgr2=0x5561cbbb5e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac7ed9d640 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 0x5561cbbb5e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac7ed9d640 1 -- 172.21.15.78:0/1243866141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 msgr2=0x5561cbbb79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac7ed9d640 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbb79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.135+0000 7fac7ed9d640 1 -- 172.21.15.78:0/1243866141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cb94cf70 con 0x5561cb7eb730 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.136+0000 7fac7ed9d640 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbb7f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fac800675f0 tx=0x7fac8006bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c26c99b3c619a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.136+0000 7fac62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.136+0000 7fac62ff5640 1 -- 172.21.15.78:0/1243866141 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80073320 con 0x5561cb7eb730 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.136+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1243866141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7400dbf0 con 0x5561cb7eb730 2024-05-21T15:20:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.136+0000 7fac62ff5640 1 -- 172.21.15.78:0/1243866141 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fac8006a050 con 0x5561cb7eb730 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.136+0000 7fac62ff5640 1 -- 172.21.15.78:0/1243866141 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fac800734c0 con 0x5561cb7eb730 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.136+0000 7fac8c1dd740 10 client.26303 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.136+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1243866141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561cb7e4170 con 0x5561cb7eb730 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac62ff5640 1 -- 172.21.15.78:0/1243866141 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac7400dbf0 con 0x5561cb7eb730 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac62ff5640 1 -- 172.21.15.78:0/1243866141 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fac8006aa30 con 0x5561cb7eb730 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac8c1dd740 10 client.26303 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac8c1dd740 10 client.26303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac8c1dd740 20 client.26303 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5561cb7cb1f0 0x5561cbbc87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac8c1dd740 4 client.26303 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1243866141 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561cbb1e0c0 con 0x5561cb7cb1f0 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.137+0000 7fac7e59c640 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5561cb7cb1f0 0x5561cbbc87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.138+0000 7fac62ff5640 10 client.26303 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.138+0000 7fac7e59c640 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5561cb7cb1f0 0x5561cbbc87f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39579d0168abe50d server_cookie=25d72cad0c101b1f in_seq=0 out_seq=0 2024-05-21T15:20:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.139+0000 7fac62ff5640 1 -- 172.21.15.78:0/1243866141 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561cbb1e0c0 con 0x5561cb7cb1f0 2024-05-21T15:20:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.139+0000 7fac62ff5640 10 client.26303 handle_command_reply: tid=0 2024-05-21T15:20:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.139+0000 7fac8c1dd740 1 client.26303 shutdown 2024-05-21T15:20:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.139+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1243866141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 msgr2=0x5561cbbb7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.139+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbb7f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fac800675f0 tx=0x7fac8006bf90 comp rx=0 tx=0).stop 2024-05-21T15:20:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1243866141 shutdown_connections 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5561cb7cb1f0 0x5561cbbc87f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbb7f30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb80c410 0x5561cbbb5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac62ff5640 0 client.26303 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1243866141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbb79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1243866141 >> 172.21.15.78:0/1243866141 conn(0x5561cb800560 msgr2=0x5561cbbad540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1243866141 shutdown_connections 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1243866141 wait complete. 2024-05-21T15:20:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.140+0000 7fac8c1dd740 20 client.26303 trim_cache size 0 max 16384 2024-05-21T15:20:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.173+0000 7fac8c1dd740 1 Processor -- start 2024-05-21T15:20:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.173+0000 7fac8c1dd740 1 -- start start 2024-05-21T15:20:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.173+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7cb1f0 0x5561cbba80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.173+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbcdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.173+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.173+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac781408c0 con 0x5561cb7e3550 2024-05-21T15:20:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.173+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fac781425b0 con 0x5561cb7eb730 2024-05-21T15:20:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.173+0000 7fac8c1dd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fac78140f90 con 0x5561cb7cb1f0 2024-05-21T15:20:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7cb1f0 0x5561cbba80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7cb1f0 0x5561cbba80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56952/0 (socket says 172.21.15.78:56952) 2024-05-21T15:20:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 -- 172.21.15.78:0/1415885625 learned_addr learned my addr 172.21.15.78:0/1415885625 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7ed9d640 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbcdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 -- 172.21.15.78:0/1415885625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 msgr2=0x5561cbbce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 -- 172.21.15.78:0/1415885625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 msgr2=0x5561cbbcdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbcdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 -- 172.21.15.78:0/1415885625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cbbc8dc0 con 0x5561cb7cb1f0 2024-05-21T15:20:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.174+0000 7fac7e59c640 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7cb1f0 0x5561cbba80a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac74012570 tx=0x7fac740124e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df14dc1ea8aef347 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.175+0000 7fac64ff9640 1 -- 172.21.15.78:0/1415885625 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74016040 con 0x5561cb7cb1f0 2024-05-21T15:20:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.175+0000 7fac64ff9640 1 -- 172.21.15.78:0/1415885625 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac740055b0 con 0x5561cb7cb1f0 2024-05-21T15:20:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.175+0000 7fac64ff9640 1 -- 172.21.15.78:0/1415885625 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74003850 con 0x5561cb7cb1f0 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.176+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1415885625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7cb1f0 msgr2=0x5561cbba80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.176+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7cb1f0 0x5561cbba80a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac74012570 tx=0x7fac740124e0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.176+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1415885625 shutdown_connections 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.176+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7eb730 0x5561cbbce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.177+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.177+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/1415885625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7cb1f0 0x5561cbba80a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.177+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1415885625 >> 172.21.15.78:0/1415885625 conn(0x5561cb800560 msgr2=0x5561cbba7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.177+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1415885625 shutdown_connections 2024-05-21T15:20:07.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.177+0000 7fac8c1dd740 1 -- 172.21.15.78:0/1415885625 wait complete. 2024-05-21T15:20:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.177+0000 7fac8c1dd740 1 -- start start 2024-05-21T15:20:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.177+0000 7fac8c1dd740 1 Processor -- start 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7cb1f0 0x5561cbbb9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac8c1dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbb9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac7e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7cb1f0 0x5561cbbb9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac7e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7cb1f0 0x5561cbbb9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38980/0 (socket says 172.21.15.78:38980) 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac7e59c640 1 -- 172.21.15.78:0/148161548 learned_addr learned my addr 172.21.15.78:0/148161548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7eb730 0x5561cbc28b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac78141e20 con 0x5561cb7e3550 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac7ed9d640 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbb9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fac78141720 con 0x5561cb7cb1f0 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.178+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fac7813e8a0 con 0x5561cb7eb730 2024-05-21T15:20:07.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac8c1dd740 10 client.0 _ll_register_callbacks cb 0x7fac8006b4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac7e59c640 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7eb730 0x5561cbc28b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac7e59c640 1 -- 172.21.15.78:0/148161548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7eb730 msgr2=0x5561cbc28b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac7e59c640 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7eb730 0x5561cbc28b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac7e59c640 1 -- 172.21.15.78:0/148161548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 msgr2=0x5561cbbb9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac7e59c640 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbb9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac7e59c640 1 -- 172.21.15.78:0/148161548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cbc290d0 con 0x5561cb7cb1f0 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac7e59c640 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7cb1f0 0x5561cbbb9580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fac74038010 tx=0x7fac74005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9573268a200426bb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.179+0000 7fac62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.180+0000 7fac62ff5640 1 -- 172.21.15.78:0/148161548 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74048060 con 0x5561cb7cb1f0 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.180+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac740055b0 con 0x5561cb7cb1f0 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.180+0000 7fac62ff5640 1 -- 172.21.15.78:0/148161548 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fac74012250 con 0x5561cb7cb1f0 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.180+0000 7fac62ff5640 1 -- 172.21.15.78:0/148161548 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fac74016040 con 0x5561cb7cb1f0 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.180+0000 7fac8c1dd740 10 client.26309 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.180+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561cbc303b0 con 0x5561cb7cb1f0 2024-05-21T15:20:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.180+0000 7fac62ff5640 1 -- 172.21.15.78:0/148161548 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac740055b0 con 0x5561cb7cb1f0 2024-05-21T15:20:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.181+0000 7fac62ff5640 1 -- 172.21.15.78:0/148161548 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7fac74003780 con 0x5561cb7cb1f0 2024-05-21T15:20:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.181+0000 7fac8c1dd740 10 client.26309 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.181+0000 7fac8c1dd740 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.181+0000 7fac8c1dd740 20 client.26309 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.181+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5561cb80c410 0x5561cbc32990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.181+0000 7fac8c1dd740 4 client.26309 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:20:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.181+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5561cbb18410 con 0x5561cb80c410 2024-05-21T15:20:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.181+0000 7fac7ed9d640 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5561cb80c410 0x5561cbc32990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.182+0000 7fac7ed9d640 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5561cb80c410 0x5561cbc32990 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89456bdc3471772 server_cookie=b26375b675ff2e6d in_seq=0 out_seq=0 2024-05-21T15:20:07.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.182+0000 7fac62ff5640 10 client.26309 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.183+0000 7fac62ff5640 1 -- 172.21.15.78:0/148161548 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5561cbb18410 con 0x5561cb80c410 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.184+0000 7fac62ff5640 10 client.26309 handle_command_reply: tid=0 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.184+0000 7fac8c1dd740 1 client.26309 shutdown 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.184+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7cb1f0 msgr2=0x5561cbbb9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.184+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7cb1f0 0x5561cbbb9580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fac74038010 tx=0x7fac74005760 comp rx=0 tx=0).stop 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.184+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 shutdown_connections 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.184+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5561cb80c410 0x5561cbc32990 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.184+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561cb7eb730 0x5561cbc28b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.184+0000 7fac62ff5640 0 client.26309 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.185+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5561cb7e3550 0x5561cbbb9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.185+0000 7fac8c1dd740 1 --2- 172.21.15.78:0/148161548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5561cb7cb1f0 0x5561cbbb9580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.185+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 >> 172.21.15.78:0/148161548 conn(0x5561cb800560 msgr2=0x5561cbbab7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.185+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 shutdown_connections 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.185+0000 7fac8c1dd740 1 -- 172.21.15.78:0/148161548 wait complete. 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.185+0000 7fac8c1dd740 20 client.26309 trim_cache size 0 max 16384 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "93d6fced-908b-491a-b38a-dd1d959cf29e", 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:20:07.189 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 -- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fac680a2e00 msgr2=0x7fac680a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fac680a2e00 0x7fac680a5310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac700bc330 tx=0x7fac700b88d0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 -- 172.21.15.78:0/739240833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 msgr2=0x7fac780a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 0x7fac780a6360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac70009980 tx=0x7fac70030890 comp rx=0 tx=0).stop 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 -- 172.21.15.78:0/739240833 shutdown_connections 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fac680a2e00 0x7fac680a5310 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7809ad20 0x7fac780a6360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac7809a380 0x7fac780a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 --2- 172.21.15.78:0/739240833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac78095930 0x7fac780a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.187+0000 7fac84a59640 1 -- 172.21.15.78:0/739240833 >> 172.21.15.78:0/739240833 conn(0x7fac7800afe0 msgr2=0x7fac78093260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.188+0000 7fac84a59640 1 -- 172.21.15.78:0/739240833 shutdown_connections 2024-05-21T15:20:07.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.188+0000 7fac84a59640 1 -- 172.21.15.78:0/739240833 wait complete. 2024-05-21T15:20:07.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:20:07.209 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:20:07.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.393+0000 7f9564837640 1 -- 172.21.15.78:0/2098571207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9560101aa0 msgr2=0x7f9560101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.393+0000 7f9564837640 1 --2- 172.21.15.78:0/2098571207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9560101aa0 0x7f9560101eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f955800b9f0 tx=0x7f955802c0a0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.393+0000 7f9564837640 1 -- 172.21.15.78:0/2098571207 shutdown_connections 2024-05-21T15:20:07.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.393+0000 7f9564837640 1 --2- 172.21.15.78:0/2098571207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956010c330 0x7f956010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.393+0000 7f9564837640 1 --2- 172.21.15.78:0/2098571207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9560101aa0 0x7f9560101eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.393+0000 7f9564837640 1 --2- 172.21.15.78:0/2098571207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95601079d0 0x7f9560101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.393+0000 7f9564837640 1 -- 172.21.15.78:0/2098571207 >> 172.21.15.78:0/2098571207 conn(0x7f95600770b0 msgr2=0x7f95600774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.394+0000 7f9564837640 1 -- 172.21.15.78:0/2098571207 shutdown_connections 2024-05-21T15:20:07.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.394+0000 7f9564837640 1 -- 172.21.15.78:0/2098571207 wait complete. 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f9564837640 1 Processor -- start 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f9564837640 1 -- start start 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f9564837640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9560101aa0 0x7f9560197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f9564837640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95601079d0 0x7f9560197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f9564837640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956010c330 0x7f956019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f9564837640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110be0 con 0x7f9560101aa0 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f9564837640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110a60 con 0x7f956010c330 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f9564837640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9560110d60 con 0x7f95601079d0 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956010c330 0x7f956019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f955dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95601079d0 0x7f9560197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956010c330 0x7f956019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:38990/0 (socket says 172.21.15.78:38990) 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f955dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95601079d0 0x7f9560197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:56986/0 (socket says 172.21.15.78:56986) 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f955ed9d640 1 -- 172.21.15.78:0/1461094063 learned_addr learned my addr 172.21.15.78:0/1461094063 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.395+0000 7f955e59c640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9560101aa0 0x7f9560197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.396+0000 7f955dd9b640 1 -- 172.21.15.78:0/1461094063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956010c330 msgr2=0x7f956019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.396+0000 7f955dd9b640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956010c330 0x7f956019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.396+0000 7f955dd9b640 1 -- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9560101aa0 msgr2=0x7f9560197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.396+0000 7f955dd9b640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9560101aa0 0x7f9560197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.396+0000 7f955dd9b640 1 -- 172.21.15.78:0/1461094063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956019c3e0 con 0x7f95601079d0 2024-05-21T15:20:07.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.396+0000 7f955dd9b640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95601079d0 0x7f9560197ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f955802c5c0 tx=0x7f9558009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.396+0000 7f954f7fe640 1 -- 172.21.15.78:0/1461094063 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558041030 con 0x7f95601079d0 2024-05-21T15:20:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.397+0000 7f954f7fe640 1 -- 172.21.15.78:0/1461094063 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95580035f0 con 0x7f95601079d0 2024-05-21T15:20:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.397+0000 7f9564837640 1 -- 172.21.15.78:0/1461094063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956019c670 con 0x7f95601079d0 2024-05-21T15:20:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.397+0000 7f954f7fe640 1 -- 172.21.15.78:0/1461094063 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558039040 con 0x7f95601079d0 2024-05-21T15:20:07.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.397+0000 7f9564837640 1 -- 172.21.15.78:0/1461094063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956019cbb0 con 0x7f95601079d0 2024-05-21T15:20:07.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.400+0000 7f956b7b5740 1 Processor -- start 2024-05-21T15:20:07.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.400+0000 7f956b7b5740 1 -- start start 2024-05-21T15:20:07.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.400+0000 7f954f7fe640 1 -- 172.21.15.78:0/1461094063 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9558044070 con 0x7f95601079d0 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.400+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3eb8390 0x558ff4256d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e8f4d0 0x558ff4257270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff42577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f956b7b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558ff41bcd00 con 0x558ff3eb8390 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f956b7b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558ff41644c0 con 0x558ff3e8f4d0 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f956b7b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558ff41b8cc0 con 0x558ff3e976b0 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3eb8390 0x558ff4256d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3eb8390 0x558ff4256d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44978/0 (socket says 172.21.15.78:44978) 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955ed9d640 1 -- 172.21.15.78:0/99309742 learned_addr learned my addr 172.21.15.78:0/99309742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955e59c640 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e8f4d0 0x558ff4257270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955dd9b640 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff42577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955e59c640 1 -- 172.21.15.78:0/99309742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 msgr2=0x558ff42577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955e59c640 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff42577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955e59c640 1 -- 172.21.15.78:0/99309742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3eb8390 msgr2=0x558ff4256d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955e59c640 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3eb8390 0x558ff4256d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955e59c640 1 -- 172.21.15.78:0/99309742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff3f9a7d0 con 0x558ff3e8f4d0 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.401+0000 7f955e59c640 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e8f4d0 0x558ff4257270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9554005fd0 tx=0x7f9554011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=604655a4d2299a0c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.402+0000 7f955dd9b640 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff42577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.402+0000 7f954f7fe640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95480a2d80 0x7f95480a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.402+0000 7f954cff9640 1 -- 172.21.15.78:0/99309742 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540042f0 con 0x558ff3e8f4d0 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.402+0000 7f954cff9640 1 -- 172.21.15.78:0/99309742 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9554004490 con 0x558ff3e8f4d0 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.402+0000 7f954cff9640 1 -- 172.21.15.78:0/99309742 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554019a40 con 0x558ff3e8f4d0 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.402+0000 7f954f7fe640 1 -- 172.21.15.78:0/1461094063 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f95580f0a20 con 0x7f95601079d0 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.402+0000 7f955ed9d640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95480a2d80 0x7f95480a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 -- 172.21.15.78:0/99309742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e8f4d0 msgr2=0x558ff4257270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e8f4d0 0x558ff4257270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9554005fd0 tx=0x7f9554011440 comp rx=0 tx=0).stop 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 -- 172.21.15.78:0/99309742 shutdown_connections 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e8f4d0 0x558ff4257270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff42577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 --2- 172.21.15.78:0/99309742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3eb8390 0x558ff4256d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 -- 172.21.15.78:0/99309742 >> 172.21.15.78:0/99309742 conn(0x558ff3eac4e0 msgr2=0x558ff42534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f955ed9d640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95480a2d80 0x7f95480a5290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9550009fd0 tx=0x7f9550009500 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 -- 172.21.15.78:0/99309742 shutdown_connections 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.403+0000 7f956b7b5740 1 -- 172.21.15.78:0/99309742 wait complete. 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 1 -- start start 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 1 Processor -- start 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e8f4d0 0x558ff4262d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e976b0 0x558ff4263290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3eb8390 0x558ff42611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558ff4268490 con 0x558ff3e976b0 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558ff4177b90 con 0x558ff3eb8390 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558ff4268610 con 0x558ff3e8f4d0 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f956b7b5740 10 client.0 _ll_register_callbacks cb 0x558ff41b3da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f955e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e8f4d0 0x558ff4262d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f955dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e976b0 0x558ff4263290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f955e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e8f4d0 0x558ff4262d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:57010/0 (socket says 172.21.15.78:57010) 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.404+0000 7f955e59c640 1 -- 172.21.15.78:0/3962988448 learned_addr learned my addr 172.21.15.78:0/3962988448 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f955ed9d640 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3eb8390 0x558ff42611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f955e59c640 1 -- 172.21.15.78:0/3962988448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3eb8390 msgr2=0x558ff42611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f955e59c640 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3eb8390 0x558ff42611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f955e59c640 1 -- 172.21.15.78:0/3962988448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e976b0 msgr2=0x558ff4263290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f955e59c640 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e976b0 0x558ff4263290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f955e59c640 1 -- 172.21.15.78:0/3962988448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff3ed03f0 con 0x558ff3e8f4d0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f955dd9b640 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e976b0 0x558ff4263290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f955e59c640 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e8f4d0 0x558ff4262d50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9554000c00 tx=0x7f95540052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e1ab0957c41607 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f9545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f9545ffb640 1 -- 172.21.15.78:0/3962988448 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554028020 con 0x558ff3e8f4d0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f9545ffb640 1 -- 172.21.15.78:0/3962988448 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f95540197c0 con 0x558ff3e8f4d0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.405+0000 7f956b7b5740 1 -- 172.21.15.78:0/3962988448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9554004490 con 0x558ff3e8f4d0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f9545ffb640 1 -- 172.21.15.78:0/3962988448 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9554033050 con 0x558ff3e8f4d0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f956b7b5740 10 client.26293 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f956b7b5740 1 -- 172.21.15.78:0/3962988448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558ff3e900f0 con 0x558ff3e8f4d0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f9545ffb640 1 -- 172.21.15.78:0/3962988448 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9554004490 con 0x558ff3e8f4d0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f9545ffb640 1 -- 172.21.15.78:0/3962988448 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f9554018070 con 0x558ff3e8f4d0 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f956b7b5740 10 client.26293 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:07.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f956b7b5740 10 client.26293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:07.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f956b7b5740 20 client.26293 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:07.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.406+0000 7f956b7b5740 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558ff3e76f60 0x558ff4277ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.407+0000 7f956b7b5740 4 client.26293 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:07.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.407+0000 7f956b7b5740 1 -- 172.21.15.78:0/3962988448 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff41ca430 con 0x558ff3e76f60 2024-05-21T15:20:07.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.407+0000 7f955dd9b640 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558ff3e76f60 0x558ff4277ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.408+0000 7f955dd9b640 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558ff3e76f60 0x558ff4277ec0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e68b6cf5ff7dea server_cookie=a8842c74d8f73138 in_seq=0 out_seq=0 2024-05-21T15:20:07.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.408+0000 7f9545ffb640 10 client.26293 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:07.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.410+0000 7f9545ffb640 1 -- 172.21.15.78:0/3962988448 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ff41ca430 con 0x558ff3e76f60 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.410+0000 7f9545ffb640 10 client.26293 handle_command_reply: tid=0 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 client.26293 shutdown 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 -- 172.21.15.78:0/3962988448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e8f4d0 msgr2=0x558ff4262d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e8f4d0 0x558ff4262d50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9554000c00 tx=0x7f95540052f0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 -- 172.21.15.78:0/3962988448 shutdown_connections 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558ff3e76f60 0x558ff4277ec0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f9545ffb640 0 client.26293 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3eb8390 0x558ff42611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e976b0 0x558ff4263290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 --2- 172.21.15.78:0/3962988448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e8f4d0 0x558ff4262d50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.411+0000 7f956b7b5740 1 -- 172.21.15.78:0/3962988448 >> 172.21.15.78:0/3962988448 conn(0x558ff3eac4e0 msgr2=0x558ff4256b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.412+0000 7f956b7b5740 1 -- 172.21.15.78:0/3962988448 shutdown_connections 2024-05-21T15:20:07.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.412+0000 7f956b7b5740 1 -- 172.21.15.78:0/3962988448 wait complete. 2024-05-21T15:20:07.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.413+0000 7f956b7b5740 20 client.26293 trim_cache size 0 max 16384 2024-05-21T15:20:07.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f956b7b5740 1 Processor -- start 2024-05-21T15:20:07.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f956b7b5740 1 -- start start 2024-05-21T15:20:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff4257ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff427d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff427dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f956b7b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95601ac990 con 0x558ff3e8f4d0 2024-05-21T15:20:07.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f956b7b5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95601ae680 con 0x558ff3e76f60 2024-05-21T15:20:07.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f956b7b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95601ad060 con 0x558ff3e976b0 2024-05-21T15:20:07.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.450+0000 7f955e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff4257ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955e59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff4257ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:39016/0 (socket says 172.21.15.78:39016) 2024-05-21T15:20:07.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955e59c640 1 -- 172.21.15.78:0/2327471554 learned_addr learned my addr 172.21.15.78:0/2327471554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff427d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955dd9b640 1 -- 172.21.15.78:0/2327471554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 msgr2=0x558ff427dbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:20:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955dd9b640 1 --2- 172.21.15.78:0/2327471554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff427dbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955dd9b640 1 -- 172.21.15.78:0/2327471554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 msgr2=0x558ff4257ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955dd9b640 1 --2- 172.21.15.78:0/2327471554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff4257ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955dd9b640 1 -- 172.21.15.78:0/2327471554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff425f740 con 0x558ff3e8f4d0 2024-05-21T15:20:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.451+0000 7f955dd9b640 1 --2- 172.21.15.78:0/2327471554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff427d680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f95580ee220 tx=0x7f9558005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4330693e31da7170 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.455+0000 7f9547fff640 1 -- 172.21.15.78:0/2327471554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580bd080 con 0x558ff3e8f4d0 2024-05-21T15:20:07.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.455+0000 7f9547fff640 1 -- 172.21.15.78:0/2327471554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95580c1430 con 0x558ff3e8f4d0 2024-05-21T15:20:07.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.455+0000 7f9547fff640 1 -- 172.21.15.78:0/2327471554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580b96f0 con 0x558ff3e8f4d0 2024-05-21T15:20:07.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 -- 172.21.15.78:0/2327471554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 msgr2=0x558ff427d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2327471554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff427d680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f95580ee220 tx=0x7f9558005f90 comp rx=0 tx=0).stop 2024-05-21T15:20:07.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 -- 172.21.15.78:0/2327471554 shutdown_connections 2024-05-21T15:20:07.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2327471554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff427dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2327471554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff427d680 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2327471554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff4257ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 -- 172.21.15.78:0/2327471554 >> 172.21.15.78:0/2327471554 conn(0x558ff3eac4e0 msgr2=0x558ff425a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 -- 172.21.15.78:0/2327471554 shutdown_connections 2024-05-21T15:20:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.456+0000 7f956b7b5740 1 -- 172.21.15.78:0/2327471554 wait complete. 2024-05-21T15:20:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.457+0000 7f956b7b5740 1 -- start start 2024-05-21T15:20:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.457+0000 7f956b7b5740 1 Processor -- start 2024-05-21T15:20:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.458+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff426a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.458+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff426ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.458+0000 7f956b7b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff426b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.458+0000 7f956b7b5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95601adef0 con 0x558ff3e8f4d0 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.458+0000 7f955dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff426ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.458+0000 7f955dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff426ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45000/0 (socket says 172.21.15.78:45000) 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.458+0000 7f955dd9b640 1 -- 172.21.15.78:0/2417776347 learned_addr learned my addr 172.21.15.78:0/2417776347 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.458+0000 7f955ed9d640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff426b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95601ad7f0 con 0x558ff3e76f60 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95601aa970 con 0x558ff3e976b0 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f956b7b5740 10 client.0 _ll_register_callbacks cb 0x7f9554009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f955e59c640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff426a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f955ed9d640 1 -- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 msgr2=0x558ff426a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f955ed9d640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff426a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f955ed9d640 1 -- 172.21.15.78:0/2417776347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 msgr2=0x558ff426ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f955ed9d640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff426ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.459+0000 7f955ed9d640 1 -- 172.21.15.78:0/2417776347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff42de430 con 0x558ff3e976b0 2024-05-21T15:20:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.460+0000 7f955ed9d640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff426b180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95500049d0 tx=0x7f9550004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d5d281328888e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.460+0000 7f9545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:20:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.460+0000 7f955e59c640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff426a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.460+0000 7f955dd9b640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff426ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.460+0000 7f9545ffb640 1 -- 172.21.15.78:0/2417776347 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550011a20 con 0x558ff3e976b0 2024-05-21T15:20:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.460+0000 7f9545ffb640 1 -- 172.21.15.78:0/2417776347 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9550018030 con 0x558ff3e976b0 2024-05-21T15:20:07.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.460+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95580c1430 con 0x558ff3e976b0 2024-05-21T15:20:07.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.461+0000 7f9545ffb640 1 -- 172.21.15.78:0/2417776347 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9550011bc0 con 0x558ff3e976b0 2024-05-21T15:20:07.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.461+0000 7f956b7b5740 10 client.26296 fetch_fsmap learned FSMap version 31 2024-05-21T15:20:07.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.461+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558ff42df700 con 0x558ff3e976b0 2024-05-21T15:20:07.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.462+0000 7f9545ffb640 1 -- 172.21.15.78:0/2417776347 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95580c1430 con 0x558ff3e976b0 2024-05-21T15:20:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.463+0000 7f9545ffb640 1 -- 172.21.15.78:0/2417776347 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8390+0+0 (secure 0 0 0) 0x7f955001f420 con 0x558ff3e976b0 2024-05-21T15:20:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.463+0000 7f956b7b5740 10 client.26296 fetch_fsmap finished waiting for FSMap version 31 2024-05-21T15:20:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.463+0000 7f956b7b5740 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.463+0000 7f956b7b5740 20 client.26296 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.463+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558ff3eb8390 0x558ff42e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.463+0000 7f956b7b5740 4 client.26296 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:20:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.463+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ff41c4780 con 0x558ff3eb8390 2024-05-21T15:20:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.464+0000 7f955e59c640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558ff3eb8390 0x558ff42e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.464+0000 7f955e59c640 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558ff3eb8390 0x558ff42e1ed0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e3eec7cd16d916 server_cookie=9f631a01c8f6f99d in_seq=0 out_seq=0 2024-05-21T15:20:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.465+0000 7f9545ffb640 10 client.26296 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:07.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.465+0000 7f9545ffb640 1 -- 172.21.15.78:0/2417776347 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558ff41c4780 con 0x558ff3eb8390 2024-05-21T15:20:07.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.465+0000 7f9545ffb640 10 client.26296 handle_command_reply: tid=0 2024-05-21T15:20:07.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.465+0000 7f956b7b5740 1 client.26296 shutdown 2024-05-21T15:20:07.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.465+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 msgr2=0x558ff426b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.465+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff426b180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95500049d0 tx=0x7f9550004ab0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 shutdown_connections 2024-05-21T15:20:07.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558ff3eb8390 0x558ff42e1ed0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3e976b0 0x558ff426b180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f9545ffb640 0 client.26296 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:07.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ff3e8f4d0 0x558ff426ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 1 --2- 172.21.15.78:0/2417776347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ff3e76f60 0x558ff426a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 >> 172.21.15.78:0/2417776347 conn(0x558ff3eac4e0 msgr2=0x558ff4260a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 shutdown_connections 2024-05-21T15:20:07.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 1 -- 172.21.15.78:0/2417776347 wait complete. 2024-05-21T15:20:07.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.466+0000 7f956b7b5740 20 client.26296 trim_cache size 0 max 16384 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout: "93d6fced-908b-491a-b38a-dd1d959cf29e": { 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "93d6fced-908b-491a-b38a-dd1d959cf29e", 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:20:07.471 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:20:07.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.467+0000 7f9564837640 1 -- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95480a2d80 msgr2=0x7f95480a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.467+0000 7f9564837640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95480a2d80 0x7f95480a5290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9550009fd0 tx=0x7f9550009500 comp rx=0 tx=0).stop 2024-05-21T15:20:07.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.467+0000 7f9564837640 1 -- 172.21.15.78:0/1461094063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95601079d0 msgr2=0x7f9560197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:07.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.467+0000 7f9564837640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95601079d0 0x7f9560197ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f955802c5c0 tx=0x7f9558009b20 comp rx=0 tx=0).stop 2024-05-21T15:20:07.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.468+0000 7f9564837640 1 -- 172.21.15.78:0/1461094063 shutdown_connections 2024-05-21T15:20:07.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.468+0000 7f9564837640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f956010c330 0x7f956019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.468+0000 7f9564837640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95480a2d80 0x7f95480a5290 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.468+0000 7f9564837640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95601079d0 0x7f9560197ae0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.468+0000 7f9564837640 1 --2- 172.21.15.78:0/1461094063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9560101aa0 0x7f9560197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:07.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.468+0000 7f9564837640 1 -- 172.21.15.78:0/1461094063 >> 172.21.15.78:0/1461094063 conn(0x7f95600770b0 msgr2=0x7f9560103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:07.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.468+0000 7f9564837640 1 -- 172.21.15.78:0/1461094063 shutdown_connections 2024-05-21T15:20:07.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:07.468+0000 7f9564837640 1 -- 172.21.15.78:0/1461094063 wait complete. 2024-05-21T15:20:07.489 INFO:tasks.cephfs.filesystem:scrub status for tag:93d6fced-908b-491a-b38a-dd1d959cf29e - {'path': '/', 'tag': '93d6fced-908b-491a-b38a-dd1d959cf29e', 'options': 'recursive,force'} 2024-05-21T15:20:08.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 54 KiB/s rd, 58 MiB/s wr, 12.78k op/s 2024-05-21T15:20:08.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:20:08.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:08.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:08.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:08.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:08.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:08.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:08.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.600 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 54 KiB/s rd, 58 MiB/s wr, 12.78k op/s 2024-05-21T15:20:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:20:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:08.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:08.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:08.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:08.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 54 KiB/s rd, 58 MiB/s wr, 12.78k op/s 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.456+0000 7f7cadd77640 1 -- 172.21.15.78:0/2992123158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ca8107f40 msgr2=0x7f7ca81021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.456+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2992123158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ca8107f40 0x7f7ca81021e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009900 tx=0x7f7c9802a420 comp rx=0 tx=0).stop 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.456+0000 7f7cadd77640 1 -- 172.21.15.78:0/2992123158 shutdown_connections 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.456+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2992123158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca8102850 0x7f7ca810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.456+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2992123158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ca8107f40 0x7f7ca81021e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.456+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2992123158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca8107aa0 0x7f7ca8101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.456+0000 7f7cadd77640 1 -- 172.21.15.78:0/2992123158 >> 172.21.15.78:0/2992123158 conn(0x7f7ca8076860 msgr2=0x7f7ca8076ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.459+0000 7f7cadd77640 1 -- 172.21.15.78:0/2992123158 shutdown_connections 2024-05-21T15:20:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.459+0000 7f7cadd77640 1 -- 172.21.15.78:0/2992123158 wait complete. 2024-05-21T15:20:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.460+0000 7f7cadd77640 1 Processor -- start 2024-05-21T15:20:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.460+0000 7f7cadd77640 1 -- start start 2024-05-21T15:20:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.461+0000 7f7cadd77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca8102850 0x7f7ca819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.461+0000 7f7cadd77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.461+0000 7f7cadd77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca8107f40 0x7f7ca81a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.461+0000 7f7cadd77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8110d20 con 0x7f7ca8107f40 2024-05-21T15:20:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.461+0000 7f7cadd77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8110ba0 con 0x7f7ca8102850 2024-05-21T15:20:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.461+0000 7f7cadd77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8110ea0 con 0x7f7ca8107aa0 2024-05-21T15:20:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.462+0000 7f7ca77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca8102850 0x7f7ca819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.462+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca8107f40 0x7f7ca81a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.463+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca8107f40 0x7f7ca81a2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43980/0 (socket says 172.21.15.78:43980) 2024-05-21T15:20:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.463+0000 7f7ca77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca8102850 0x7f7ca819c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49108/0 (socket says 172.21.15.78:49108) 2024-05-21T15:20:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.463+0000 7f7ca77fe640 1 -- 172.21.15.78:0/2774027233 learned_addr learned my addr 172.21.15.78:0/2774027233 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.464+0000 7f7ca6ffd640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.465+0000 7f7ca77fe640 1 -- 172.21.15.78:0/2774027233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ca8107aa0 msgr2=0x7f7ca819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.465+0000 7f7ca77fe640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.465+0000 7f7ca77fe640 1 -- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca8107f40 msgr2=0x7f7ca81a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.465+0000 7f7ca77fe640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca8107f40 0x7f7ca81a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.465+0000 7f7ca77fe640 1 -- 172.21.15.78:0/2774027233 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca81a2810 con 0x7f7ca8102850 2024-05-21T15:20:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.466+0000 7f7ca77fe640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca8102850 0x7f7ca819c300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0058c0 tx=0x7f7c9c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.467+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/2774027233 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c009e50 con 0x7f7ca8102850 2024-05-21T15:20:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.467+0000 7f7cadd77640 1 -- 172.21.15.78:0/2774027233 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca81a2b00 con 0x7f7ca8102850 2024-05-21T15:20:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.468+0000 7f7cadd77640 1 -- 172.21.15.78:0/2774027233 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca81977f0 con 0x7f7ca8102850 2024-05-21T15:20:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.468+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/2774027233 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c00ed90 con 0x7f7ca8102850 2024-05-21T15:20:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.468+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/2774027233 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004a00 con 0x7f7ca8102850 2024-05-21T15:20:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.474+0000 7f7cb4c27740 1 Processor -- start 2024-05-21T15:20:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.474+0000 7f7cb4c27740 1 -- start start 2024-05-21T15:20:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.474+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dc9e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.475+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8d62f0 0x55ff0dc9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.475+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8de4d0 0x55ff0dc9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.475+0000 7f7cb4c27740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ff0dc03340 con 0x55ff0d8d62f0 2024-05-21T15:20:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.475+0000 7f7cb4c27740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ff0dbab290 con 0x55ff0d8de4d0 2024-05-21T15:20:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.475+0000 7f7cb4c27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff0dbff300 con 0x55ff0d8ff1b0 2024-05-21T15:20:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.475+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dc9e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.475+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dc9e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:39246/0 (socket says 172.21.15.78:39246) 2024-05-21T15:20:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.475+0000 7f7ca7fff640 1 -- 172.21.15.78:0/1417235833 learned_addr learned my addr 172.21.15.78:0/1417235833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.478+0000 7f7ca6ffd640 1 --2- 172.21.15.78:0/1417235833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8de4d0 0x55ff0dc9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.479+0000 7f7ca7fff640 1 -- 172.21.15.78:0/1417235833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8de4d0 msgr2=0x55ff0dc9eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.479+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1417235833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8de4d0 0x55ff0dc9eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.479+0000 7f7ca7fff640 1 -- 172.21.15.78:0/1417235833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8d62f0 msgr2=0x55ff0dc9e5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T15:20:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.479+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1417235833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8d62f0 0x55ff0dc9e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.479+0000 7f7ca7fff640 1 -- 172.21.15.78:0/1417235833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0dbcf8e0 con 0x55ff0d8ff1b0 2024-05-21T15:20:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.481+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/2774027233 <== mon.2 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c9c01b020 con 0x7f7ca8102850 2024-05-21T15:20:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.482+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1417235833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dc9e080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009f80 tx=0x7f7ca0010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea3601dd56946342 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.482+0000 7f7ca4ff9640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7c900a2fd0 0x7f7c900a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.482+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/2774027233 <== mon.2 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7c9c0ceb80 con 0x7f7ca8102850 2024-05-21T15:20:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.484+0000 7f7c8dffb640 1 -- 172.21.15.78:0/1417235833 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0003b70 con 0x55ff0d8ff1b0 2024-05-21T15:20:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.484+0000 7f7c8dffb640 1 -- 172.21.15.78:0/1417235833 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca0003d10 con 0x55ff0d8ff1b0 2024-05-21T15:20:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.484+0000 7f7c8dffb640 1 -- 172.21.15.78:0/1417235833 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0011b30 con 0x55ff0d8ff1b0 2024-05-21T15:20:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.484+0000 7f7ca77fe640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7c900a2fd0 0x7f7c900a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.486+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1417235833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 msgr2=0x55ff0dc9e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.486+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1417235833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dc9e080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009f80 tx=0x7f7ca0010860 comp rx=0 tx=0).stop 2024-05-21T15:20:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.488+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1417235833 shutdown_connections 2024-05-21T15:20:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.488+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1417235833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8de4d0 0x55ff0dc9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.488+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1417235833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8d62f0 0x55ff0dc9e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.488+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1417235833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dc9e080 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.488+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1417235833 >> 172.21.15.78:0/1417235833 conn(0x55ff0d8f3300 msgr2=0x55ff0dc9a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.491+0000 7f7ca77fe640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7c900a2fd0 0x7f7c900a54e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55ff0dc9bb70 tx=0x7f7c9c004730 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.495+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1417235833 shutdown_connections 2024-05-21T15:20:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.495+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1417235833 wait complete. 2024-05-21T15:20:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.495+0000 7f7cb4c27740 1 -- start start 2024-05-21T15:20:10.499 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.496+0000 7f7cb4c27740 1 Processor -- start 2024-05-21T15:20:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dcaa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcaa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dca8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7cb4c27740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ff0dcab1b0 con 0x55ff0d8de4d0 2024-05-21T15:20:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7cb4c27740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ff0dbbe790 con 0x55ff0d8d62f0 2024-05-21T15:20:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7cb4c27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff0dcab330 con 0x55ff0d8ff1b0 2024-05-21T15:20:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7cb4c27740 10 client.0 _ll_register_callbacks cb 0x55ff0dbf9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7ca6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcaa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7ca6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcaa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43992/0 (socket says 172.21.15.78:43992) 2024-05-21T15:20:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.497+0000 7f7ca6ffd640 1 -- 172.21.15.78:0/1378334024 learned_addr learned my addr 172.21.15.78:0/1378334024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.498+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dca8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.498+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dcaa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.499+0000 7f7ca6ffd640 1 -- 172.21.15.78:0/1378334024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 msgr2=0x55ff0dca8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.499+0000 7f7ca6ffd640 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dca8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.499+0000 7f7ca6ffd640 1 -- 172.21.15.78:0/1378334024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 msgr2=0x55ff0dcaa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.499+0000 7f7ca6ffd640 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dcaa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.499+0000 7f7ca6ffd640 1 -- 172.21.15.78:0/1378334024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff0d917210 con 0x55ff0d8de4d0 2024-05-21T15:20:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.503+0000 7f7ca6ffd640 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcaa600 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7c98000c00 tx=0x7f7c980095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49131b8c66a02dab server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.503+0000 7f7c86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.504+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1378334024 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98004230 con 0x55ff0d8de4d0 2024-05-21T15:20:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.505+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1378334024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca0003d10 con 0x55ff0d8de4d0 2024-05-21T15:20:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.505+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1378334024 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7c980043d0 con 0x55ff0d8de4d0 2024-05-21T15:20:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.513+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1378334024 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7c98004e70 con 0x55ff0d8de4d0 2024-05-21T15:20:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.513+0000 7f7cb4c27740 10 client.16845 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.513+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1378334024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ff0d9e3620 con 0x55ff0d8de4d0 2024-05-21T15:20:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.515+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1378334024 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca0003d10 con 0x55ff0d8de4d0 2024-05-21T15:20:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.515+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1378334024 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7c98003b20 con 0x55ff0d8de4d0 2024-05-21T15:20:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.517+0000 7f7cb4c27740 10 client.16845 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.517+0000 7f7cb4c27740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.517+0000 7f7cb4c27740 20 client.16845 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.517+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ff0d8bdd80 0x55ff0dcba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.517+0000 7f7cb4c27740 4 client.16845 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.517+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1378334024 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff0dc10a70 con 0x55ff0d8bdd80 2024-05-21T15:20:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.518+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ff0d8bdd80 0x55ff0dcba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.525+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ff0d8bdd80 0x55ff0dcba980 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f693d6076d4583bb server_cookie=d2da4b12a787939e in_seq=0 out_seq=0 2024-05-21T15:20:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.526+0000 7f7c86ffd640 10 client.16845 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.527+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1378334024 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff0dc10a70 con 0x55ff0d8bdd80 2024-05-21T15:20:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.527+0000 7f7c86ffd640 10 client.16845 handle_command_reply: tid=0 2024-05-21T15:20:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.527+0000 7f7cb4c27740 1 client.16845 shutdown 2024-05-21T15:20:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.533+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1378334024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 msgr2=0x55ff0dcaa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.533+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcaa600 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7c98000c00 tx=0x7f7c980095a0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.533+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1378334024 shutdown_connections 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.533+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ff0d8bdd80 0x55ff0dcba980 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.533+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dca8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.533+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcaa600 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.533+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1378334024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dcaa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.533+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1378334024 >> 172.21.15.78:0/1378334024 conn(0x55ff0d8f3300 msgr2=0x55ff0dc9deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.534+0000 7f7c86ffd640 0 client.16845 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.534+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1378334024 shutdown_connections 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.534+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1378334024 wait complete. 2024-05-21T15:20:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.534+0000 7f7cb4c27740 20 client.16845 trim_cache size 0 max 16384 2024-05-21T15:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.564+0000 7f7cb4c27740 1 Processor -- start 2024-05-21T15:20:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.564+0000 7f7cb4c27740 1 -- start start 2024-05-21T15:20:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.565+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dca8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.565+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcc0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.565+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dcc06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.565+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dca8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.565+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dca8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49138/0 (socket says 172.21.15.78:49138) 2024-05-21T15:20:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.565+0000 7f7ca6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcc0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.565+0000 7f7ca6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcc0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:39270/0 (socket says 172.21.15.78:39270) 2024-05-21T15:20:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.565+0000 7f7ca7fff640 1 -- 172.21.15.78:0/3786582176 learned_addr learned my addr 172.21.15.78:0/3786582176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.567+0000 7f7cb4c27740 1 -- 172.21.15.78:0/3786582176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400ea60 con 0x55ff0d8ff1b0 2024-05-21T15:20:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.567+0000 7f7cb4c27740 1 -- 172.21.15.78:0/3786582176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94010720 con 0x55ff0d8d62f0 2024-05-21T15:20:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.567+0000 7f7cb4c27740 1 -- 172.21.15.78:0/3786582176 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400f130 con 0x55ff0d8de4d0 2024-05-21T15:20:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.572+0000 7f7ca7fff640 1 -- 172.21.15.78:0/3786582176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8de4d0 msgr2=0x55ff0dcc0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.572+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/3786582176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcc0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.572+0000 7f7ca7fff640 1 -- 172.21.15.78:0/3786582176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8ff1b0 msgr2=0x55ff0dcc06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.572+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/3786582176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dcc06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.572+0000 7f7ca7fff640 1 -- 172.21.15.78:0/3786582176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0dc9f430 con 0x55ff0d8d62f0 2024-05-21T15:20:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.573+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/3786582176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dca8c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00116a0 tx=0x7f7ca00116d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583a6171e0bee20d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.574+0000 7f7c8d7fa640 1 -- 172.21.15.78:0/3786582176 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0010d60 con 0x55ff0d8d62f0 2024-05-21T15:20:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.575+0000 7f7c8d7fa640 1 -- 172.21.15.78:0/3786582176 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ca0011740 con 0x55ff0d8d62f0 2024-05-21T15:20:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.575+0000 7f7c8d7fa640 1 -- 172.21.15.78:0/3786582176 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0005280 con 0x55ff0d8d62f0 2024-05-21T15:20:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.576+0000 7f7cb4c27740 1 -- 172.21.15.78:0/3786582176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 msgr2=0x55ff0dca8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.576+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/3786582176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dca8c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00116a0 tx=0x7f7ca00116d0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.582+0000 7f7cb4c27740 1 -- 172.21.15.78:0/3786582176 shutdown_connections 2024-05-21T15:20:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.582+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/3786582176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dcc06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.582+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/3786582176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8de4d0 0x55ff0dcc0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.582+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/3786582176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8d62f0 0x55ff0dca8c60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.582+0000 7f7cb4c27740 1 -- 172.21.15.78:0/3786582176 >> 172.21.15.78:0/3786582176 conn(0x55ff0d8f3300 msgr2=0x55ff0dca8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.584+0000 7f7cb4c27740 1 -- 172.21.15.78:0/3786582176 shutdown_connections 2024-05-21T15:20:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.586+0000 7f7cb4c27740 1 -- 172.21.15.78:0/3786582176 wait complete. 2024-05-21T15:20:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.586+0000 7f7cb4c27740 1 -- start start 2024-05-21T15:20:10.589 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.587+0000 7f7cb4c27740 1 Processor -- start 2024-05-21T15:20:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.587+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8d62f0 0x55ff0dd1b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.588+0000 7f7cb4c27740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dd1b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.588+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8d62f0 0x55ff0dd1b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.588+0000 7f7ca7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8d62f0 0x55ff0dd1b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:39274/0 (socket says 172.21.15.78:39274) 2024-05-21T15:20:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.588+0000 7f7ca7fff640 1 -- 172.21.15.78:0/1888803011 learned_addr learned my addr 172.21.15.78:0/1888803011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:10.591 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.588+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dcafa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.588+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400ff90 con 0x55ff0d8de4d0 2024-05-21T15:20:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.588+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400f8c0 con 0x55ff0d8ff1b0 2024-05-21T15:20:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.588+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9400ca40 con 0x55ff0d8d62f0 2024-05-21T15:20:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7cb4c27740 10 client.0 _ll_register_callbacks cb 0x7f7ca00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dcafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7ca7fff640 1 -- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8ff1b0 msgr2=0x55ff0dcafa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dcafa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7ca7fff640 1 -- 172.21.15.78:0/1888803011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 msgr2=0x55ff0dd1b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dd1b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7ca7fff640 1 -- 172.21.15.78:0/1888803011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff0dcb00e0 con 0x55ff0d8d62f0 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca0011740 con 0x55ff0d8d62f0 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7ca7fff640 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8d62f0 0x55ff0dd1b160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009560 tx=0x7f7ca0005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a269742aeca07f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.589+0000 7f7c86ffd640 10 client.26311 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:20:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.590+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1888803011 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0011af0 con 0x55ff0d8d62f0 2024-05-21T15:20:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.593+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1888803011 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7ca0011c90 con 0x55ff0d8d62f0 2024-05-21T15:20:10.596 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.593+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1888803011 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ca0004e40 con 0x55ff0d8d62f0 2024-05-21T15:20:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.594+0000 7f7cb4c27740 10 client.26311 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.595+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55ff0dd226d0 con 0x55ff0d8d62f0 2024-05-21T15:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.595+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1888803011 <== mon.1 v2:172.21.15.172:3300/0 4 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7ca0029040 con 0x55ff0d8d62f0 2024-05-21T15:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.596+0000 7f7cb4c27740 10 client.26311 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.596+0000 7f7cb4c27740 10 client.26311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.596+0000 7f7cb4c27740 20 client.26311 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.596+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ff0d8931c0 0x55ff0dd24e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.596+0000 7f7cb4c27740 4 client.26311 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-21T15:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.596+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7ca0011740 con 0x55ff0d8931c0 2024-05-21T15:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.596+0000 7f7ca6ffd640 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ff0d8931c0 0x55ff0dd24e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:10.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.597+0000 7f7ca6ffd640 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ff0d8931c0 0x55ff0dd24e60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27bbf8262c51d90a server_cookie=429c3a0e32df3cf8 in_seq=0 out_seq=0 2024-05-21T15:20:10.600 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:10.597+0000 7f7c86ffd640 10 client.26311 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:10.607 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:10 smithi078 ceph-mon[31997]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 6.94k op/s 2024-05-21T15:20:10.608 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:10 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:20:10.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:10 smithi084 ceph-mon[36334]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 6.94k op/s 2024-05-21T15:20:10.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:10 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:20:10.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:10 smithi172 ceph-mon[36402]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 39 KiB/s rd, 31 MiB/s wr, 6.94k op/s 2024-05-21T15:20:10.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:10 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:20:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.579+0000 7f7c86ffd640 1 -- 172.21.15.78:0/1888803011 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7ca0011740 con 0x55ff0d8931c0 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.579+0000 7f7c86ffd640 10 client.26311 handle_command_reply: tid=0 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 client.26311 shutdown 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8d62f0 msgr2=0x55ff0dd1b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8d62f0 0x55ff0dd1b160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009560 tx=0x7f7ca0005c80 comp rx=0 tx=0).stop 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 shutdown_connections 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55ff0d8931c0 0x55ff0dd24e60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ff0d8ff1b0 0x55ff0dcafa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ff0d8de4d0 0x55ff0dd1b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 --2- 172.21.15.78:0/1888803011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0d8d62f0 0x55ff0dd1b160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 >> 172.21.15.78:0/1888803011 conn(0x55ff0d8f3300 msgr2=0x55ff0dca1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7c86ffd640 0 client.26311 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 shutdown_connections 2024-05-21T15:20:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.580+0000 7f7cb4c27740 1 -- 172.21.15.78:0/1888803011 wait complete. 2024-05-21T15:20:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.581+0000 7f7cb4c27740 20 client.26311 trim_cache size 0 max 16384 2024-05-21T15:20:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:{ 2024-05-21T15:20:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "message": "", 2024-05-21T15:20:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout: "return_code": 0 2024-05-21T15:20:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stdout:} 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.581+0000 7f7cadd77640 1 -- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7c900a2fd0 msgr2=0x7f7c900a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7c900a2fd0 0x7f7c900a54e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55ff0dc9bb70 tx=0x7f7c9c004730 comp rx=0 tx=0).stop 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 -- 172.21.15.78:0/2774027233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca8102850 msgr2=0x7f7ca819c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca8102850 0x7f7ca819c300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0058c0 tx=0x7f7c9c009c60 comp rx=0 tx=0).stop 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 -- 172.21.15.78:0/2774027233 shutdown_connections 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7c900a2fd0 0x7f7c900a54e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ca8107f40 0x7f7ca81a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 --2- 172.21.15.78:0/2774027233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ca8102850 0x7f7ca819c300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 -- 172.21.15.78:0/2774027233 >> 172.21.15.78:0/2774027233 conn(0x7f7ca8076860 msgr2=0x7f7ca80ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 -- 172.21.15.78:0/2774027233 shutdown_connections 2024-05-21T15:20:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi078.front.sepia.ceph.com.smithi078.stderr:2024-05-21T15:20:11.582+0000 7f7cadd77640 1 -- 172.21.15.78:0/2774027233 wait complete. 2024-05-21T15:20:12.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:12 smithi078 ceph-mon[31997]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 6.96k op/s 2024-05-21T15:20:12.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:12 smithi084 ceph-mon[36334]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 6.96k op/s 2024-05-21T15:20:12.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:12 smithi172 ceph-mon[36402]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 6.96k op/s 2024-05-21T15:20:13.815 INFO:tasks.workunit.client.0.smithi078.stdout:Wrote -1 instead of 4096 bytes. 2024-05-21T15:20:13.816 INFO:tasks.workunit.client.0.smithi078.stdout:Probably out of disk space 2024-05-21T15:20:13.816 INFO:tasks.workunit.client.0.smithi078.stderr:write: Resource temporarily unavailable 2024-05-21T15:20:13.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:20:13.818 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-05-21T15:20:13.818 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-21T15:20:14.144 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi078 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2b507c724ef6c8295ee0a35d185cdab167fd61d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-05-21T15:20:14.369 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9ed1b325c7e74cbba4e902807f8afc89 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi078 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2b507c724ef6c8295ee0a35d185cdab167fd61d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-05-21T15:20:14.372 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-21T15:20:14.381 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:20:14.400 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-21T15:20:14.411 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-21T15:20:14.411 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T15:20:14.411 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:20:14.412 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-21T15:20:14.462 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:14 smithi078 ceph-mon[31997]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 87 KiB/s rd, 39 MiB/s wr, 8.95k op/s 2024-05-21T15:20:14.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-21T15:20:14.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:14 smithi084 ceph-mon[36334]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 87 KiB/s rd, 39 MiB/s wr, 8.95k op/s 2024-05-21T15:20:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:14 smithi172 ceph-mon[36402]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 87 KiB/s rd, 39 MiB/s wr, 8.95k op/s 2024-05-21T15:20:16.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:16 smithi078 ceph-mon[31997]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 75 KiB/s rd, 26 MiB/s wr, 6.07k op/s 2024-05-21T15:20:16.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:16 smithi084 ceph-mon[36334]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 75 KiB/s rd, 26 MiB/s wr, 6.07k op/s 2024-05-21T15:20:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:16 smithi172 ceph-mon[36402]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 75 KiB/s rd, 26 MiB/s wr, 6.07k op/s 2024-05-21T15:20:17.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:17 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:17.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:17 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:17.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:17 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:17.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:17 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:17 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:17 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:18.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:18 smithi078 ceph-mon[31997]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 89 KiB/s rd, 33 MiB/s wr, 7.67k op/s 2024-05-21T15:20:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:18 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:18 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:18.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:18 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:18.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:18 smithi084 ceph-mon[36334]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 89 KiB/s rd, 33 MiB/s wr, 7.67k op/s 2024-05-21T15:20:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:18 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:18 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:18.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:18 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:18 smithi172 ceph-mon[36402]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 89 KiB/s rd, 33 MiB/s wr, 7.67k op/s 2024-05-21T15:20:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:18 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:18 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:18 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:20.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:20 smithi078 ceph-mon[31997]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 67 KiB/s rd, 18 MiB/s wr, 4.25k op/s 2024-05-21T15:20:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:20 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:20:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:20 smithi084 ceph-mon[36334]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 67 KiB/s rd, 18 MiB/s wr, 4.25k op/s 2024-05-21T15:20:20.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:20 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:20:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:20 smithi172 ceph-mon[36402]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 67 KiB/s rd, 18 MiB/s wr, 4.25k op/s 2024-05-21T15:20:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:20 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:20:21.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:21 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:20:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:21 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:20:21.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:21 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:20:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:21 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:20:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:21 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:20:21.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:21 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:20:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:21 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:20:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:21 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:20:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:21 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:20:22.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:22 smithi078 ceph-mon[31997]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 68 KiB/s rd, 18 MiB/s wr, 4.26k op/s 2024-05-21T15:20:22.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:22 smithi084 ceph-mon[36334]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 68 KiB/s rd, 18 MiB/s wr, 4.26k op/s 2024-05-21T15:20:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:22 smithi172 ceph-mon[36402]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 68 KiB/s rd, 18 MiB/s wr, 4.26k op/s 2024-05-21T15:20:24.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:24 smithi078 ceph-mon[31997]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 72 KiB/s rd, 19 MiB/s wr, 4.44k op/s 2024-05-21T15:20:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:24 smithi084 ceph-mon[36334]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 72 KiB/s rd, 19 MiB/s wr, 4.44k op/s 2024-05-21T15:20:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:24 smithi172 ceph-mon[36402]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 72 KiB/s rd, 19 MiB/s wr, 4.44k op/s 2024-05-21T15:20:26.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:26 smithi078 ceph-mon[31997]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 1.81k op/s 2024-05-21T15:20:26.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:26 smithi084 ceph-mon[36334]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 1.81k op/s 2024-05-21T15:20:26.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:26 smithi172 ceph-mon[36402]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 1.81k op/s 2024-05-21T15:20:28.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:28 smithi078 ceph-mon[31997]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-05-21T15:20:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:28 smithi084 ceph-mon[36334]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-05-21T15:20:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:28 smithi172 ceph-mon[36402]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 2.92k op/s 2024-05-21T15:20:30.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:30 smithi078 ceph-mon[31997]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 1.31k op/s 2024-05-21T15:20:30.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:30 smithi084 ceph-mon[36334]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 1.31k op/s 2024-05-21T15:20:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:30 smithi172 ceph-mon[36402]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 1.31k op/s 2024-05-21T15:20:32.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:32 smithi078 ceph-mon[31997]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 1.33k op/s 2024-05-21T15:20:32.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:32 smithi084 ceph-mon[36334]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 1.33k op/s 2024-05-21T15:20:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:32 smithi172 ceph-mon[36402]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 1.33k op/s 2024-05-21T15:20:34.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:34 smithi078 ceph-mon[31997]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 2.45k op/s 2024-05-21T15:20:34.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:34 smithi084 ceph-mon[36334]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 2.45k op/s 2024-05-21T15:20:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:34 smithi172 ceph-mon[36402]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 2.45k op/s 2024-05-21T15:20:36.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:36 smithi078 ceph-mon[31997]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 2.25k op/s 2024-05-21T15:20:36.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:36 smithi084 ceph-mon[36334]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 2.25k op/s 2024-05-21T15:20:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:36 smithi172 ceph-mon[36402]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 9.4 MiB/s wr, 2.25k op/s 2024-05-21T15:20:37.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:20:37.491 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:20:37.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 -- 172.21.15.78:0/1779263289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 msgr2=0x7f6930104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 --2- 172.21.15.78:0/1779263289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 0x7f6930104000 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f692c009900 tx=0x7f692c02a410 comp rx=0 tx=0).stop 2024-05-21T15:20:37.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 -- 172.21.15.78:0/1779263289 shutdown_connections 2024-05-21T15:20:37.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 --2- 172.21.15.78:0/1779263289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f693010a650 0x7f693010ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 --2- 172.21.15.78:0/1779263289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f693010d040 0x7f693010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 --2- 172.21.15.78:0/1779263289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 0x7f6930104000 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 -- 172.21.15.78:0/1779263289 >> 172.21.15.78:0/1779263289 conn(0x7f69300fd210 msgr2=0x7f69300ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:37.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 -- 172.21.15.78:0/1779263289 shutdown_connections 2024-05-21T15:20:37.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.618+0000 7f6937702640 1 -- 172.21.15.78:0/1779263289 wait complete. 2024-05-21T15:20:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6937702640 1 Processor -- start 2024-05-21T15:20:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6937702640 1 -- start start 2024-05-21T15:20:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6937702640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 0x7f69301997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6937702640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f693010a650 0x7f6930199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6937702640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f693010d040 0x7f693019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6937702640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6930112df0 con 0x7f6930103bf0 2024-05-21T15:20:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6937702640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6930112c70 con 0x7f693010d040 2024-05-21T15:20:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6937702640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6930112f70 con 0x7f693010a650 2024-05-21T15:20:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f693549e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 0x7f69301997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f693549e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 0x7f69301997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50750/0 (socket says 172.21.15.78:50750) 2024-05-21T15:20:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f693549e640 1 -- 172.21.15.78:0/474610187 learned_addr learned my addr 172.21.15.78:0/474610187 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.619+0000 7f6934c9d640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f693010a650 0x7f6930199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f6935c9f640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f693010d040 0x7f693019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f693549e640 1 -- 172.21.15.78:0/474610187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f693010a650 msgr2=0x7f6930199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f693549e640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f693010a650 0x7f6930199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f693549e640 1 -- 172.21.15.78:0/474610187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f693010d040 msgr2=0x7f693019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f693549e640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f693010d040 0x7f693019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f693549e640 1 -- 172.21.15.78:0/474610187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693019e630 con 0x7f6930103bf0 2024-05-21T15:20:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f693549e640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 0x7f69301997d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f692c006010 tx=0x7f692c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f69267fc640 1 -- 172.21.15.78:0/474610187 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c03f030 con 0x7f6930103bf0 2024-05-21T15:20:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f69267fc640 1 -- 172.21.15.78:0/474610187 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c004440 con 0x7f6930103bf0 2024-05-21T15:20:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f6937702640 1 -- 172.21.15.78:0/474610187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693019e8c0 con 0x7f6930103bf0 2024-05-21T15:20:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f69267fc640 1 -- 172.21.15.78:0/474610187 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c035070 con 0x7f6930103bf0 2024-05-21T15:20:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.620+0000 7f6937702640 1 -- 172.21.15.78:0/474610187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693019edb0 con 0x7f6930103bf0 2024-05-21T15:20:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.623+0000 7f69267fc640 1 -- 172.21.15.78:0/474610187 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f692c042030 con 0x7f6930103bf0 2024-05-21T15:20:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.623+0000 7f693e5af740 1 Processor -- start 2024-05-21T15:20:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.623+0000 7f693e5af740 1 -- start start 2024-05-21T15:20:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f693e5af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d6fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f693e5af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f693e5af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f693e5af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557499cd5d00 con 0x5574999d1390 2024-05-21T15:20:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f693e5af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557499c7d4c0 con 0x5574999a84d0 2024-05-21T15:20:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f693e5af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557499cd1cc0 con 0x5574999b06b0 2024-05-21T15:20:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d6fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f69267fc640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f691c0a2d00 0x7f691c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d6fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50758/0 (socket says 172.21.15.78:50758) 2024-05-21T15:20:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 -- 172.21.15.78:0/87659704 learned_addr learned my addr 172.21.15.78:0/87659704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f693549e640 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6934c9d640 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f691c0a2d00 0x7f691c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f69267fc640 1 -- 172.21.15.78:0/474610187 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f692c0ef860 con 0x7f6930103bf0 2024-05-21T15:20:37.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 -- 172.21.15.78:0/87659704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 msgr2=0x557499d707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:37.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 -- 172.21.15.78:0/87659704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 msgr2=0x557499d70270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:37.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d70270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 -- 172.21.15.78:0/87659704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557499ab37d0 con 0x5574999d1390 2024-05-21T15:20:37.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.624+0000 7f6935c9f640 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d6fd30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6928005c40 tx=0x7f6928013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413ad15df424a708 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:37.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.625+0000 7f691b7fe640 1 -- 172.21.15.78:0/87659704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928022020 con 0x5574999d1390 2024-05-21T15:20:37.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.625+0000 7f691b7fe640 1 -- 172.21.15.78:0/87659704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69280134e0 con 0x5574999d1390 2024-05-21T15:20:37.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.625+0000 7f691b7fe640 1 -- 172.21.15.78:0/87659704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928010a10 con 0x5574999d1390 2024-05-21T15:20:37.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 -- 172.21.15.78:0/87659704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 msgr2=0x557499d6fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f6935c9f640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f691c0a2d00 0x7f691c0a5210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6928023000 tx=0x7f6928021000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:37.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d6fd30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6928005c40 tx=0x7f6928013f50 comp rx=0 tx=0).stop 2024-05-21T15:20:37.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 -- 172.21.15.78:0/87659704 shutdown_connections 2024-05-21T15:20:37.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d70270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 --2- 172.21.15.78:0/87659704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d6fd30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 -- 172.21.15.78:0/87659704 >> 172.21.15.78:0/87659704 conn(0x5574999c54e0 msgr2=0x557499d6c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:37.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 -- 172.21.15.78:0/87659704 shutdown_connections 2024-05-21T15:20:37.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 -- 172.21.15.78:0/87659704 wait complete. 2024-05-21T15:20:37.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 -- start start 2024-05-21T15:20:37.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.626+0000 7f693e5af740 1 Processor -- start 2024-05-21T15:20:37.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693e5af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999a84d0 0x557499d7be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693e5af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d7c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693e5af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999d1390 0x557499d7a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693e5af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557499d7cfc0 con 0x5574999a84d0 2024-05-21T15:20:37.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693e5af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557499c90b90 con 0x5574999d1390 2024-05-21T15:20:37.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693e5af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557499d7d140 con 0x5574999b06b0 2024-05-21T15:20:37.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693e5af740 10 client.0 _ll_register_callbacks cb 0x557499cccda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:37.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999a84d0 0x557499d7be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999a84d0 0x557499d7be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50766/0 (socket says 172.21.15.78:50766) 2024-05-21T15:20:37.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 -- 172.21.15.78:0/705207907 learned_addr learned my addr 172.21.15.78:0/705207907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:37.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f6935c9f640 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999d1390 0x557499d7a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f6934c9d640 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d7c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 -- 172.21.15.78:0/705207907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 msgr2=0x557499d7c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:37.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d7c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 -- 172.21.15.78:0/705207907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999d1390 msgr2=0x557499d7a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:37.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999d1390 0x557499d7a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 -- 172.21.15.78:0/705207907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574999e93f0 con 0x5574999a84d0 2024-05-21T15:20:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693549e640 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999a84d0 0x557499d7be90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f692c0bbd60 tx=0x7f692c0bb910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15aebba5ae8887e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f6918ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f6918ff9640 1 -- 172.21.15.78:0/705207907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c049040 con 0x5574999a84d0 2024-05-21T15:20:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f6918ff9640 1 -- 172.21.15.78:0/705207907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f692c0b8440 con 0x5574999a84d0 2024-05-21T15:20:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.627+0000 7f693e5af740 1 -- 172.21.15.78:0/705207907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557499ce3430 con 0x5574999a84d0 2024-05-21T15:20:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f6918ff9640 1 -- 172.21.15.78:0/705207907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f692c0c00d0 con 0x5574999a84d0 2024-05-21T15:20:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f693e5af740 10 client.16857 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f693e5af740 1 -- 172.21.15.78:0/705207907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5574999a90f0 con 0x5574999a84d0 2024-05-21T15:20:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f6918ff9640 1 -- 172.21.15.78:0/705207907 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557499ce3430 con 0x5574999a84d0 2024-05-21T15:20:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f6918ff9640 1 -- 172.21.15.78:0/705207907 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f692c0bc0c0 con 0x5574999a84d0 2024-05-21T15:20:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f693e5af740 10 client.16857 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f693e5af740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f693e5af740 20 client.16857 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f693e5af740 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55749998ff60 0x557499d8cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f693e5af740 4 client.16857 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f693e5af740 1 -- 172.21.15.78:0/705207907 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557499cdd780 con 0x55749998ff60 2024-05-21T15:20:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.628+0000 7f6934c9d640 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55749998ff60 0x557499d8cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.629+0000 7f6918ff9640 10 client.16857 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.629+0000 7f6934c9d640 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55749998ff60 0x557499d8cb60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963b8ff367e39c88 server_cookie=e134b863139baec2 in_seq=0 out_seq=0 2024-05-21T15:20:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.629+0000 7f6918ff9640 1 -- 172.21.15.78:0/705207907 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557499cdd780 con 0x55749998ff60 2024-05-21T15:20:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.629+0000 7f6918ff9640 10 client.16857 handle_command_reply: tid=0 2024-05-21T15:20:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.629+0000 7f693e5af740 1 client.16857 shutdown 2024-05-21T15:20:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 -- 172.21.15.78:0/705207907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999a84d0 msgr2=0x557499d7be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999a84d0 0x557499d7be90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f692c0bbd60 tx=0x7f692c0bb910 comp rx=0 tx=0).stop 2024-05-21T15:20:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 -- 172.21.15.78:0/705207907 shutdown_connections 2024-05-21T15:20:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55749998ff60 0x557499d8cb60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999d1390 0x557499d7a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f6918ff9640 0 client.16857 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d7c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 --2- 172.21.15.78:0/705207907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999a84d0 0x557499d7be90 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 -- 172.21.15.78:0/705207907 >> 172.21.15.78:0/705207907 conn(0x5574999c54e0 msgr2=0x557499d71a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 -- 172.21.15.78:0/705207907 shutdown_connections 2024-05-21T15:20:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 1 -- 172.21.15.78:0/705207907 wait complete. 2024-05-21T15:20:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.630+0000 7f693e5af740 20 client.16857 trim_cache size 0 max 16384 2024-05-21T15:20:37.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.647+0000 7f693e5af740 1 Processor -- start 2024-05-21T15:20:37.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.647+0000 7f693e5af740 1 -- start start 2024-05-21T15:20:37.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f693e5af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d79250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f693e5af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d74100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f693e5af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d92360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f693e5af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69301aec20 con 0x5574999d1390 2024-05-21T15:20:37.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f693e5af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69301b08e0 con 0x5574999a84d0 2024-05-21T15:20:37.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f693e5af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69301af2f0 con 0x5574999b06b0 2024-05-21T15:20:37.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d92360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d92360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50778/0 (socket says 172.21.15.78:50778) 2024-05-21T15:20:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 -- 172.21.15.78:0/2731619357 learned_addr learned my addr 172.21.15.78:0/2731619357 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6934c9d640 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d74100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f693549e640 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d79250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 -- 172.21.15.78:0/2731619357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 msgr2=0x557499d74100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d74100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 -- 172.21.15.78:0/2731619357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 msgr2=0x557499d79250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d79250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 -- 172.21.15.78:0/2731619357 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557499d78860 con 0x5574999d1390 2024-05-21T15:20:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f693549e640 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d79250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f6935c9f640 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d92360 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6928012fd0 tx=0x7f69280106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f60cb9a1a2e1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.648+0000 7f691affd640 1 -- 172.21.15.78:0/2731619357 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928022070 con 0x5574999d1390 2024-05-21T15:20:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f691affd640 1 -- 172.21.15.78:0/2731619357 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6928010da0 con 0x5574999d1390 2024-05-21T15:20:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f691affd640 1 -- 172.21.15.78:0/2731619357 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69280109b0 con 0x5574999d1390 2024-05-21T15:20:37.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 -- 172.21.15.78:0/2731619357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 msgr2=0x557499d92360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d92360 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6928012fd0 tx=0x7f69280106e0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 -- 172.21.15.78:0/2731619357 shutdown_connections 2024-05-21T15:20:37.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d92360 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d74100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 --2- 172.21.15.78:0/2731619357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d79250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 -- 172.21.15.78:0/2731619357 >> 172.21.15.78:0/2731619357 conn(0x5574999c54e0 msgr2=0x557499d739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:37.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 -- 172.21.15.78:0/2731619357 shutdown_connections 2024-05-21T15:20:37.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.649+0000 7f693e5af740 1 -- 172.21.15.78:0/2731619357 wait complete. 2024-05-21T15:20:37.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 1 -- start start 2024-05-21T15:20:37.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 1 Processor -- start 2024-05-21T15:20:37.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d87000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d84370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69301b0150 con 0x5574999d1390 2024-05-21T15:20:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69301afa80 con 0x5574999a84d0 2024-05-21T15:20:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69301acc00 con 0x5574999b06b0 2024-05-21T15:20:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f693e5af740 10 client.0 _ll_register_callbacks cb 0x7f692c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f6935c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f6935c9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d848b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50790/0 (socket says 172.21.15.78:50790) 2024-05-21T15:20:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.650+0000 7f6935c9f640 1 -- 172.21.15.78:0/2299820214 learned_addr learned my addr 172.21.15.78:0/2299820214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f693549e640 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d87000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6934c9d640 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d84370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6935c9f640 1 -- 172.21.15.78:0/2299820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 msgr2=0x557499d84370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6935c9f640 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d84370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6935c9f640 1 -- 172.21.15.78:0/2299820214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 msgr2=0x557499d87000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6935c9f640 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d87000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6935c9f640 1 -- 172.21.15.78:0/2299820214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557499d85040 con 0x5574999d1390 2024-05-21T15:20:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6934c9d640 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d84370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6935c9f640 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d848b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6928013990 tx=0x7f6928012390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ba79d0ea3d8054 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6918ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6918ff9640 1 -- 172.21.15.78:0/2299820214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928022070 con 0x5574999d1390 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6918ff9640 1 -- 172.21.15.78:0/2299820214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f692800fde0 con 0x5574999d1390 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f693e5af740 1 -- 172.21.15.78:0/2299820214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6928010da0 con 0x5574999d1390 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6918ff9640 1 -- 172.21.15.78:0/2299820214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6928010960 con 0x5574999d1390 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f693e5af740 10 client.16863 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f693e5af740 1 -- 172.21.15.78:0/2299820214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557499df4df0 con 0x5574999d1390 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.651+0000 7f6918ff9640 1 -- 172.21.15.78:0/2299820214 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6928010da0 con 0x5574999d1390 2024-05-21T15:20:37.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f6918ff9640 1 -- 172.21.15.78:0/2299820214 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f692803a610 con 0x5574999d1390 2024-05-21T15:20:37.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693e5af740 10 client.16863 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:37.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693e5af740 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:37.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693e5af740 20 client.16863 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:37.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693e5af740 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5574999653a0 0x557499df7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693e5af740 4 client.16863 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:20:37.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693e5af740 1 -- 172.21.15.78:0/2299820214 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557499c5a6f0 con 0x5574999653a0 2024-05-21T15:20:37.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693549e640 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5574999653a0 0x557499df7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f6918ff9640 10 client.16863 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693549e640 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5574999653a0 0x557499df7330 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cf4c55b9fc18d23 server_cookie=3e74ba5ec91d3fd3 in_seq=0 out_seq=0 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f6918ff9640 1 -- 172.21.15.78:0/2299820214 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557499c5a6f0 con 0x5574999653a0 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f6918ff9640 10 client.16863 handle_command_reply: tid=0 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.652+0000 7f693e5af740 1 client.16863 shutdown 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 -- 172.21.15.78:0/2299820214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 msgr2=0x557499d848b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d848b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6928013990 tx=0x7f6928012390 comp rx=0 tx=0).stop 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 -- 172.21.15.78:0/2299820214 shutdown_connections 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5574999653a0 0x557499df7330 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5574999d1390 0x557499d848b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574999b06b0 0x557499d84370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6918ff9640 0 client.16863 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:37.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 --2- 172.21.15.78:0/2299820214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5574999a84d0 0x557499d87000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 -- 172.21.15.78:0/2299820214 >> 172.21.15.78:0/2299820214 conn(0x5574999c54e0 msgr2=0x557499d7b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:37.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 -- 172.21.15.78:0/2299820214 shutdown_connections 2024-05-21T15:20:37.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 1 -- 172.21.15.78:0/2299820214 wait complete. 2024-05-21T15:20:37.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f693e5af740 20 client.16863 trim_cache size 0 max 16384 2024-05-21T15:20:37.669 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:20:37.669 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:20:37.669 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:20:37.669 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:20:37.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 -- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f691c0a2d00 msgr2=0x7f691c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f691c0a2d00 0x7f691c0a5210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6928023000 tx=0x7f6928021000 comp rx=0 tx=0).stop 2024-05-21T15:20:37.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 -- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 msgr2=0x7f69301997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 0x7f69301997d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f692c006010 tx=0x7f692c004040 comp rx=0 tx=0).stop 2024-05-21T15:20:37.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 -- 172.21.15.78:0/474610187 shutdown_connections 2024-05-21T15:20:37.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f693010d040 0x7f693019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f691c0a2d00 0x7f691c0a5210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f693010a650 0x7f6930199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 --2- 172.21.15.78:0/474610187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6930103bf0 0x7f69301997d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.653+0000 7f6937702640 1 -- 172.21.15.78:0/474610187 >> 172.21.15.78:0/474610187 conn(0x7f69300fd210 msgr2=0x7f693010d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:37.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.654+0000 7f6937702640 1 -- 172.21.15.78:0/474610187 shutdown_connections 2024-05-21T15:20:37.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.654+0000 7f6937702640 1 -- 172.21.15.78:0/474610187 wait complete. 2024-05-21T15:20:37.672 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:20:37.673 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:20:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 -- 172.21.15.78:0/433242584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 msgr2=0x7f63f010f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/433242584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 0x7f63f010f600 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009900 tx=0x7f63e802a410 comp rx=0 tx=0).stop 2024-05-21T15:20:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 -- 172.21.15.78:0/433242584 shutdown_connections 2024-05-21T15:20:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/433242584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63f0108f80 0x7f63f0102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/433242584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 0x7f63f010f600 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/433242584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f0108ae0 0x7f63f0102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 -- 172.21.15.78:0/433242584 >> 172.21.15.78:0/433242584 conn(0x7f63f00fe1c0 msgr2=0x7f63f0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 -- 172.21.15.78:0/433242584 shutdown_connections 2024-05-21T15:20:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.783+0000 7f63f4ce9640 1 -- 172.21.15.78:0/433242584 wait complete. 2024-05-21T15:20:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63f4ce9640 1 Processor -- start 2024-05-21T15:20:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63f4ce9640 1 -- start start 2024-05-21T15:20:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63f4ce9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 0x7f63f0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63f4ce9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f0108ae0 0x7f63f0199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63f4ce9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63f0108f80 0x7f63f019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63f4ce9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0112e70 con 0x7f63f0103230 2024-05-21T15:20:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63f4ce9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0112cf0 con 0x7f63f0108f80 2024-05-21T15:20:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63f4ce9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63f0112ff0 con 0x7f63f0108ae0 2024-05-21T15:20:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 0x7f63f0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 0x7f63f0199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50800/0 (socket says 172.21.15.78:50800) 2024-05-21T15:20:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 -- 172.21.15.78:0/1012497421 learned_addr learned my addr 172.21.15.78:0/1012497421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63edd9b640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f0108ae0 0x7f63f0199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63eed9d640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63f0108f80 0x7f63f019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 -- 172.21.15.78:0/1012497421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f0108ae0 msgr2=0x7f63f0199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f0108ae0 0x7f63f0199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 -- 172.21.15.78:0/1012497421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63f0108f80 msgr2=0x7f63f019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63f0108f80 0x7f63f019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 -- 172.21.15.78:0/1012497421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f019e5d0 con 0x7f63f0103230 2024-05-21T15:20:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.784+0000 7f63ee59c640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 0x7f63f0199770 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f63e000dbf0 tx=0x7f63e000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.785+0000 7f63df7fe640 1 -- 172.21.15.78:0/1012497421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0017070 con 0x7f63f0103230 2024-05-21T15:20:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.785+0000 7f63df7fe640 1 -- 172.21.15.78:0/1012497421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e0003e50 con 0x7f63f0103230 2024-05-21T15:20:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.785+0000 7f63df7fe640 1 -- 172.21.15.78:0/1012497421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e00106c0 con 0x7f63f0103230 2024-05-21T15:20:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.785+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f019e8c0 con 0x7f63f0103230 2024-05-21T15:20:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.785+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f01a6240 con 0x7f63f0103230 2024-05-21T15:20:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.786+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f0069140 con 0x7f63f0103230 2024-05-21T15:20:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.787+0000 7f63df7fe640 1 -- 172.21.15.78:0/1012497421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f63e0003970 con 0x7f63f0103230 2024-05-21T15:20:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.788+0000 7f63df7fe640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f63d80a2d00 0x7f63d80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.788+0000 7f63edd9b640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f63d80a2d00 0x7f63d80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.788+0000 7f63df7fe640 1 -- 172.21.15.78:0/1012497421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f63e00cf740 con 0x7f63f0103230 2024-05-21T15:20:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.788+0000 7f63edd9b640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f63d80a2d00 0x7f63d80a5210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63f0106b00 tx=0x7f63e4005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:37.791+0000 7f63df7fe640 1 -- 172.21.15.78:0/1012497421 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f63e009c300 con 0x7f63f0103230 2024-05-21T15:20:38.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.006+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63f010abf0 con 0x7f63f0103230 2024-05-21T15:20:38.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.007+0000 7f63df7fe640 1 -- 172.21.15.78:0/1012497421 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f63e009bc00 con 0x7f63f0103230 2024-05-21T15:20:38.010 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:20:38.010 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:20:08.349454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24448,24419,24502,24527,24526]},"id":1}]} 2024-05-21T15:20:38.010 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 32 2024-05-21T15:20:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.008+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f63d80a2d00 msgr2=0x7f63d80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.008+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f63d80a2d00 0x7f63d80a5210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63f0106b00 tx=0x7f63e4005f90 comp rx=0 tx=0).stop 2024-05-21T15:20:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 msgr2=0x7f63f0199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 0x7f63f0199770 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f63e000dbf0 tx=0x7f63e000b630 comp rx=0 tx=0).stop 2024-05-21T15:20:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 shutdown_connections 2024-05-21T15:20:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63f0108f80 0x7f63f019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f63d80a2d00 0x7f63d80a5210 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63f0108ae0 0x7f63f0199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 --2- 172.21.15.78:0/1012497421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63f0103230 0x7f63f0199770 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 >> 172.21.15.78:0/1012497421 conn(0x7f63f00fe1c0 msgr2=0x7f63f0105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 shutdown_connections 2024-05-21T15:20:38.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.009+0000 7f63f4ce9640 1 -- 172.21.15.78:0/1012497421 wait complete. 2024-05-21T15:20:38.023 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:20:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 -- 172.21.15.78:0/3630567145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f151410c330 msgr2=0x7f151410e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 --2- 172.21.15.78:0/3630567145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f151410c330 0x7f151410e750 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f150c009ca0 tx=0x7f150c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 -- 172.21.15.78:0/3630567145 shutdown_connections 2024-05-21T15:20:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 --2- 172.21.15.78:0/3630567145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1514101aa0 0x7f1514101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 --2- 172.21.15.78:0/3630567145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f151410c330 0x7f151410e750 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 --2- 172.21.15.78:0/3630567145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15141079d0 0x7f1514101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 -- 172.21.15.78:0/3630567145 >> 172.21.15.78:0/3630567145 conn(0x7f15140770b0 msgr2=0x7f15140774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 -- 172.21.15.78:0/3630567145 shutdown_connections 2024-05-21T15:20:38.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.133+0000 7f151b5ce640 1 -- 172.21.15.78:0/3630567145 wait complete. 2024-05-21T15:20:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151b5ce640 1 Processor -- start 2024-05-21T15:20:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151b5ce640 1 -- start start 2024-05-21T15:20:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151b5ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1514101aa0 0x7f151419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151b5ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15141079d0 0x7f151419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151b5ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f151410c330 0x7f151419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151b5ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1514111d00 con 0x7f1514101aa0 2024-05-21T15:20:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151b5ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1514111b80 con 0x7f15141079d0 2024-05-21T15:20:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151b5ce640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1514111e80 con 0x7f151410c330 2024-05-21T15:20:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151936a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1514101aa0 0x7f151419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151936a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1514101aa0 0x7f151419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50814/0 (socket says 172.21.15.78:50814) 2024-05-21T15:20:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.134+0000 7f151936a640 1 -- 172.21.15.78:0/2751276703 learned_addr learned my addr 172.21.15.78:0/2751276703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f1518b69640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15141079d0 0x7f151419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f1519b6b640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f151410c330 0x7f151419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f151936a640 1 -- 172.21.15.78:0/2751276703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f151410c330 msgr2=0x7f151419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f151936a640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f151410c330 0x7f151419b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f151936a640 1 -- 172.21.15.78:0/2751276703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15141079d0 msgr2=0x7f151419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f151936a640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15141079d0 0x7f151419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f151936a640 1 -- 172.21.15.78:0/2751276703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15141a5960 con 0x7f1514101aa0 2024-05-21T15:20:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f1518b69640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15141079d0 0x7f151419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f151936a640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1514101aa0 0x7f151419a620 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f151000d740 tx=0x7f151000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f150a7fc640 1 -- 172.21.15.78:0/2751276703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510017070 con 0x7f1514101aa0 2024-05-21T15:20:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f150a7fc640 1 -- 172.21.15.78:0/2751276703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f151000de50 con 0x7f1514101aa0 2024-05-21T15:20:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f151b5ce640 1 -- 172.21.15.78:0/2751276703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15141a5c50 con 0x7f1514101aa0 2024-05-21T15:20:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f150a7fc640 1 -- 172.21.15.78:0/2751276703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510004e00 con 0x7f1514101aa0 2024-05-21T15:20:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.135+0000 7f151b5ce640 1 -- 172.21.15.78:0/2751276703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141a6150 con 0x7f1514101aa0 2024-05-21T15:20:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1522478740 1 Processor -- start 2024-05-21T15:20:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1522478740 1 -- start start 2024-05-21T15:20:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f150a7fc640 1 -- 172.21.15.78:0/2751276703 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1510010460 con 0x7f1514101aa0 2024-05-21T15:20:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1522478740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52788d390 0x55f527c2bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1522478740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c2c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1522478740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c2c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1522478740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f527b91c40 con 0x55f52788d390 2024-05-21T15:20:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1522478740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f527b39580 con 0x55f52786c6b0 2024-05-21T15:20:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1522478740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f527b8dc00 con 0x55f5278644d0 2024-05-21T15:20:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1519b6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52788d390 0x55f527c2bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1519b6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52788d390 0x55f527c2bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50830/0 (socket says 172.21.15.78:50830) 2024-05-21T15:20:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1519b6b640 1 -- 172.21.15.78:0/1883788364 learned_addr learned my addr 172.21.15.78:0/1883788364 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f151936a640 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c2c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1518b69640 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c2c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1519b6b640 1 -- 172.21.15.78:0/1883788364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 msgr2=0x55f527c2c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1519b6b640 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c2c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1519b6b640 1 -- 172.21.15.78:0/1883788364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 msgr2=0x55f527c2c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1519b6b640 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c2c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1519b6b640 1 -- 172.21.15.78:0/1883788364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5278a53f0 con 0x55f52788d390 2024-05-21T15:20:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f151936a640 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c2c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:38.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f1518b69640 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c2c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.138+0000 7f150a7fc640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15000a2d00 0x7f15000a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.139+0000 7f151936a640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15000a2d00 0x7f15000a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.139+0000 7f1519b6b640 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52788d390 0x55f527c2bd20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f150c009bf0 tx=0x7f150c004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1e1b0c948cec35 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.139+0000 7f150a7fc640 1 -- 172.21.15.78:0/2751276703 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f15100ceee0 con 0x7f1514101aa0 2024-05-21T15:20:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.139+0000 7f14ff7fe640 1 -- 172.21.15.78:0/1883788364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c031bb0 con 0x55f52788d390 2024-05-21T15:20:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.139+0000 7f14ff7fe640 1 -- 172.21.15.78:0/1883788364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c036070 con 0x55f52788d390 2024-05-21T15:20:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.139+0000 7f14ff7fe640 1 -- 172.21.15.78:0/1883788364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c004040 con 0x55f52788d390 2024-05-21T15:20:38.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 -- 172.21.15.78:0/1883788364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52788d390 msgr2=0x55f527c2bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f151936a640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15000a2d00 0x7f15000a5210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55f527c29810 tx=0x7f15100048a0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52788d390 0x55f527c2bd20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f150c009bf0 tx=0x7f150c004aa0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 -- 172.21.15.78:0/1883788364 shutdown_connections 2024-05-21T15:20:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c2c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c2c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 --2- 172.21.15.78:0/1883788364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52788d390 0x55f527c2bd20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 -- 172.21.15.78:0/1883788364 >> 172.21.15.78:0/1883788364 conn(0x55f5278814e0 msgr2=0x55f527c284d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 -- 172.21.15.78:0/1883788364 shutdown_connections 2024-05-21T15:20:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 -- 172.21.15.78:0/1883788364 wait complete. 2024-05-21T15:20:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 -- start start 2024-05-21T15:20:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.140+0000 7f1522478740 1 Processor -- start 2024-05-21T15:20:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1522478740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5278644d0 0x55f527c37d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1522478740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1522478740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f52788d390 0x55f527c36230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1522478740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f527c38fb0 con 0x55f5278644d0 2024-05-21T15:20:38.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1522478740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f527b4cc40 con 0x55f52786c6b0 2024-05-21T15:20:38.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1522478740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f527c39130 con 0x55f52788d390 2024-05-21T15:20:38.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1522478740 10 client.0 _ll_register_callbacks cb 0x55f5278a4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5278644d0 0x55f527c37d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5278644d0 0x55f527c37d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50838/0 (socket says 172.21.15.78:50838) 2024-05-21T15:20:38.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 -- 172.21.15.78:0/3203906831 learned_addr learned my addr 172.21.15.78:0/3203906831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1518b69640 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f52788d390 0x55f527c36230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 -- 172.21.15.78:0/3203906831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f52788d390 msgr2=0x55f527c36230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f52788d390 0x55f527c36230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 -- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 msgr2=0x55f527c382a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c382a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 -- 172.21.15.78:0/3203906831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52796f7d0 con 0x55f5278644d0 2024-05-21T15:20:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1518b69640 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:38.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1519b6b640 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5278644d0 0x55f527c37d60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f150c02f540 tx=0x7f150c031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28262e6ca4cb9d58 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f14fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f14fcff9640 1 -- 172.21.15.78:0/3203906831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c003b70 con 0x55f5278644d0 2024-05-21T15:20:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f14fcff9640 1 -- 172.21.15.78:0/3203906831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f150c003d10 con 0x55f5278644d0 2024-05-21T15:20:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.141+0000 7f1522478740 1 -- 172.21.15.78:0/3203906831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150c036070 con 0x55f5278644d0 2024-05-21T15:20:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f14fcff9640 1 -- 172.21.15.78:0/3203906831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f150c004040 con 0x55f5278644d0 2024-05-21T15:20:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1522478740 10 client.16881 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1522478740 1 -- 172.21.15.78:0/3203906831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f527c456f0 con 0x55f5278644d0 2024-05-21T15:20:38.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f14fcff9640 1 -- 172.21.15.78:0/3203906831 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c036070 con 0x55f5278644d0 2024-05-21T15:20:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f14fcff9640 1 -- 172.21.15.78:0/3203906831 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f150c02f9e0 con 0x55f5278644d0 2024-05-21T15:20:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1522478740 10 client.16881 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1522478740 10 client.16881 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:20:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1522478740 20 client.16881 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1522478740 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55f52784bf60 0x55f527c48a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1522478740 4 client.16881 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:38.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1522478740 1 -- 172.21.15.78:0/3203906831 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f527b9f370 con 0x55f52784bf60 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.142+0000 7f1518b69640 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55f52784bf60 0x55f527c48a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.143+0000 7f14fcff9640 10 client.16881 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.143+0000 7f1518b69640 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55f52784bf60 0x55f527c48a50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d6e3a21e99ea0db server_cookie=aaf6077650f246a9 in_seq=0 out_seq=0 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f14fcff9640 1 -- 172.21.15.78:0/3203906831 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f527b9f370 con 0x55f52784bf60 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f14fcff9640 10 client.16881 handle_command_reply: tid=0 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 client.16881 shutdown 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 -- 172.21.15.78:0/3203906831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5278644d0 msgr2=0x55f527c37d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5278644d0 0x55f527c37d60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f150c02f540 tx=0x7f150c031810 comp rx=0 tx=0).stop 2024-05-21T15:20:38.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 -- 172.21.15.78:0/3203906831 shutdown_connections 2024-05-21T15:20:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55f52784bf60 0x55f527c48a50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52786c6b0 0x55f527c382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f52788d390 0x55f527c36230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 --2- 172.21.15.78:0/3203906831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5278644d0 0x55f527c37d60 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f14fcff9640 0 client.16881 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:20:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 -- 172.21.15.78:0/3203906831 >> 172.21.15.78:0/3203906831 conn(0x55f5278814e0 msgr2=0x55f527c2da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 -- 172.21.15.78:0/3203906831 shutdown_connections 2024-05-21T15:20:38.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 1 -- 172.21.15.78:0/3203906831 wait complete. 2024-05-21T15:20:38.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.144+0000 7f1522478740 20 client.16881 trim_cache size 0 max 16384 2024-05-21T15:20:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.157+0000 7f1522478740 1 Processor -- start 2024-05-21T15:20:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1522478740 1 -- start start 2024-05-21T15:20:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1522478740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c350e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1522478740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527c4ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1522478740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1522478740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15141aea90 con 0x55f52786c6b0 2024-05-21T15:20:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1522478740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f15141b0750 con 0x55f52788d390 2024-05-21T15:20:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1522478740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15141af160 con 0x55f5278644d0 2024-05-21T15:20:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527c4ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527c4ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50854/0 (socket says 172.21.15.78:50854) 2024-05-21T15:20:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 -- 172.21.15.78:0/19250747 learned_addr learned my addr 172.21.15.78:0/19250747 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1519b6b640 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1519b6b640 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 -- 172.21.15.78:0/19250747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 msgr2=0x55f527c350e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c350e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 -- 172.21.15.78:0/19250747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 msgr2=0x55f527c4bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 -- 172.21.15.78:0/19250747 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f527c4c500 con 0x55f52786c6b0 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.158+0000 7f1518b69640 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527c4ba80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f150400efd0 tx=0x7f150400e370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad4454460ce7079 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.159+0000 7f14feffd640 1 -- 172.21.15.78:0/19250747 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150401e070 con 0x55f52786c6b0 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.159+0000 7f14feffd640 1 -- 172.21.15.78:0/19250747 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150401a9a0 con 0x55f52786c6b0 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.159+0000 7f14feffd640 1 -- 172.21.15.78:0/19250747 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040103d0 con 0x55f52786c6b0 2024-05-21T15:20:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.159+0000 7f1522478740 1 -- 172.21.15.78:0/19250747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 msgr2=0x55f527c4ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.159+0000 7f1522478740 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527c4ba80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f150400efd0 tx=0x7f150400e370 comp rx=0 tx=0).stop 2024-05-21T15:20:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 -- 172.21.15.78:0/19250747 shutdown_connections 2024-05-21T15:20:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527c4ba80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 --2- 172.21.15.78:0/19250747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527c350e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 -- 172.21.15.78:0/19250747 >> 172.21.15.78:0/19250747 conn(0x55f5278814e0 msgr2=0x55f527c349d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 -- 172.21.15.78:0/19250747 shutdown_connections 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 -- 172.21.15.78:0/19250747 wait complete. 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 -- start start 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 Processor -- start 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527ca9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527ca9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15141affc0 con 0x55f52786c6b0 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f15141af8f0 con 0x55f52788d390 2024-05-21T15:20:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15141aca70 con 0x55f5278644d0 2024-05-21T15:20:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.160+0000 7f1522478740 10 client.0 _ll_register_callbacks cb 0x7f150c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527ca9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527ca9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50868/0 (socket says 172.21.15.78:50868) 2024-05-21T15:20:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 -- 172.21.15.78:0/812220037 learned_addr learned my addr 172.21.15.78:0/812220037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1519b6b640 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 -- 172.21.15.78:0/812220037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 msgr2=0x55f527ca9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527ca9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 -- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 msgr2=0x55f527c4b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 -- 172.21.15.78:0/812220037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f527c4b7e0 con 0x55f52786c6b0 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1519b6b640 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1518b69640 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527ca9a50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1504005d00 tx=0x7f150400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a50bedc8d02aff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f14fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f14fcff9640 1 -- 172.21.15.78:0/812220037 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504030060 con 0x55f52786c6b0 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f14fcff9640 1 -- 172.21.15.78:0/812220037 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f150401a2c0 con 0x55f52786c6b0 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1522478740 1 -- 172.21.15.78:0/812220037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150401a9a0 con 0x55f52786c6b0 2024-05-21T15:20:38.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f14fcff9640 1 -- 172.21.15.78:0/812220037 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f150401e050 con 0x55f52786c6b0 2024-05-21T15:20:38.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1522478740 10 client.16887 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f1522478740 1 -- 172.21.15.78:0/812220037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f527cb0aa0 con 0x55f52786c6b0 2024-05-21T15:20:38.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.161+0000 7f14fcff9640 1 -- 172.21.15.78:0/812220037 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150401a9a0 con 0x55f52786c6b0 2024-05-21T15:20:38.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f14fcff9640 1 -- 172.21.15.78:0/812220037 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f150402a030 con 0x55f52786c6b0 2024-05-21T15:20:38.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1522478740 10 client.16887 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1522478740 10 client.16887 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:20:38.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1522478740 20 client.16887 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1522478740 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55f5278213a0 0x55f527cb3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1522478740 4 client.16887 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:38.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1522478740 1 -- 172.21.15.78:0/812220037 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f527b996c0 con 0x55f5278213a0 2024-05-21T15:20:38.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1519b6b640 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55f5278213a0 0x55f527cb3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f14fcff9640 10 client.16887 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:20:38.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1519b6b640 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55f5278213a0 0x55f527cb3180 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a4966e068de6413 server_cookie=90f6269cdcebe09f in_seq=0 out_seq=0 2024-05-21T15:20:38.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f14fcff9640 1 -- 172.21.15.78:0/812220037 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f527b996c0 con 0x55f5278213a0 2024-05-21T15:20:38.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f14fcff9640 10 client.16887 handle_command_reply: tid=0 2024-05-21T15:20:38.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.162+0000 7f1522478740 1 client.16887 shutdown 2024-05-21T15:20:38.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 -- 172.21.15.78:0/812220037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 msgr2=0x55f527ca9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527ca9a50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1504005d00 tx=0x7f150400e440 comp rx=0 tx=0).stop 2024-05-21T15:20:38.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 -- 172.21.15.78:0/812220037 shutdown_connections 2024-05-21T15:20:38.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55f5278213a0 0x55f527cb3180 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f52788d390 0x55f527c4b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f14fcff9640 0 client.16887 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:20:38.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f52786c6b0 0x55f527ca9a50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 --2- 172.21.15.78:0/812220037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5278644d0 0x55f527ca9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 -- 172.21.15.78:0/812220037 >> 172.21.15.78:0/812220037 conn(0x55f5278814e0 msgr2=0x55f527c2bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 -- 172.21.15.78:0/812220037 shutdown_connections 2024-05-21T15:20:38.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 1 -- 172.21.15.78:0/812220037 wait complete. 2024-05-21T15:20:38.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f1522478740 20 client.16887 trim_cache size 0 max 16384 2024-05-21T15:20:38.178 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:38.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 -- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15000a2d00 msgr2=0x7f15000a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15000a2d00 0x7f15000a5210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55f527c29810 tx=0x7f15100048a0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 -- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1514101aa0 msgr2=0x7f151419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1514101aa0 0x7f151419a620 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f151000d740 tx=0x7f151000dc00 comp rx=0 tx=0).stop 2024-05-21T15:20:38.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 -- 172.21.15.78:0/2751276703 shutdown_connections 2024-05-21T15:20:38.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15141079d0 0x7f151419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15000a2d00 0x7f15000a5210 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f151410c330 0x7f151419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 --2- 172.21.15.78:0/2751276703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1514101aa0 0x7f151419a620 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.163+0000 7f151b5ce640 1 -- 172.21.15.78:0/2751276703 >> 172.21.15.78:0/2751276703 conn(0x7f15140770b0 msgr2=0x7f1514103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.164+0000 7f151b5ce640 1 -- 172.21.15.78:0/2751276703 shutdown_connections 2024-05-21T15:20:38.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.164+0000 7f151b5ce640 1 -- 172.21.15.78:0/2751276703 wait complete. 2024-05-21T15:20:38.181 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:20:38.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.291+0000 7f36f43f5640 1 -- 172.21.15.78:0/2841236911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 msgr2=0x7f36ec10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.291+0000 7f36f43f5640 1 --2- 172.21.15.78:0/2841236911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 0x7f36ec10e780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009ca0 tx=0x7f36e402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 -- 172.21.15.78:0/2841236911 shutdown_connections 2024-05-21T15:20:38.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 --2- 172.21.15.78:0/2841236911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36ec101b20 0x7f36ec101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 --2- 172.21.15.78:0/2841236911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 0x7f36ec10e780 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 --2- 172.21.15.78:0/2841236911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36ec107a00 0x7f36ec1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 -- 172.21.15.78:0/2841236911 >> 172.21.15.78:0/2841236911 conn(0x7f36ec0770c0 msgr2=0x7f36ec077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 -- 172.21.15.78:0/2841236911 shutdown_connections 2024-05-21T15:20:38.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 -- 172.21.15.78:0/2841236911 wait complete. 2024-05-21T15:20:38.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 Processor -- start 2024-05-21T15:20:38.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.292+0000 7f36f43f5640 1 -- start start 2024-05-21T15:20:38.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f43f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36ec101b20 0x7f36ec19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f43f5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36ec107a00 0x7f36ec19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f43f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 0x7f36ec19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f43f5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec111d30 con 0x7f36ec10c360 2024-05-21T15:20:38.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f43f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec111bb0 con 0x7f36ec101b20 2024-05-21T15:20:38.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f43f5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec111eb0 con 0x7f36ec107a00 2024-05-21T15:20:38.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 0x7f36ec19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 0x7f36ec19b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50884/0 (socket says 172.21.15.78:50884) 2024-05-21T15:20:38.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 -- 172.21.15.78:0/648024623 learned_addr learned my addr 172.21.15.78:0/648024623 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2191640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36ec101b20 0x7f36ec19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f1990640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36ec107a00 0x7f36ec19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 -- 172.21.15.78:0/648024623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36ec107a00 msgr2=0x7f36ec19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36ec107a00 0x7f36ec19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 -- 172.21.15.78:0/648024623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36ec101b20 msgr2=0x7f36ec19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36ec101b20 0x7f36ec19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 -- 172.21.15.78:0/648024623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec1a58f0 con 0x7f36ec10c360 2024-05-21T15:20:38.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2191640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36ec101b20 0x7f36ec19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:38.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f2992640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 0x7f36ec19b050 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f36e4031a20 tx=0x7f36e4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36e37fe640 1 -- 172.21.15.78:0/648024623 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003c30 con 0x7f36ec10c360 2024-05-21T15:20:38.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36e37fe640 1 -- 172.21.15.78:0/648024623 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e4003dd0 con 0x7f36ec10c360 2024-05-21T15:20:38.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f43f5640 1 -- 172.21.15.78:0/648024623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec1a5b80 con 0x7f36ec10c360 2024-05-21T15:20:38.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36e37fe640 1 -- 172.21.15.78:0/648024623 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e402fcf0 con 0x7f36ec10c360 2024-05-21T15:20:38.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.293+0000 7f36f43f5640 1 -- 172.21.15.78:0/648024623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a60d0 con 0x7f36ec10c360 2024-05-21T15:20:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.296+0000 7f36e37fe640 1 -- 172.21.15.78:0/648024623 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f36e4004050 con 0x7f36ec10c360 2024-05-21T15:20:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.296+0000 7f36fb372740 1 Processor -- start 2024-05-21T15:20:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.296+0000 7f36fb372740 1 -- start start 2024-05-21T15:20:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36fb372740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db30390 0x56365deced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36fb372740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365decf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36fb372740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365decf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36fb372740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56365de34c40 con 0x56365db0f6b0 2024-05-21T15:20:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36fb372740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56365dddc580 con 0x56365db074d0 2024-05-21T15:20:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36fb372740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56365de30c00 con 0x56365db30390 2024-05-21T15:20:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36e37fe640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f36d80a2d00 0x7f36d80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365decf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365decf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50894/0 (socket says 172.21.15.78:50894) 2024-05-21T15:20:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 -- 172.21.15.78:0/1856039262 learned_addr learned my addr 172.21.15.78:0/1856039262 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f2992640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f36d80a2d00 0x7f36d80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db30390 0x56365deced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f2191640 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365decf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36e37fe640 1 -- 172.21.15.78:0/648024623 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f36e40f0800 con 0x7f36ec10c360 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 -- 172.21.15.78:0/1856039262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db30390 msgr2=0x56365deced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db30390 0x56365deced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 -- 172.21.15.78:0/1856039262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 msgr2=0x56365decf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365decf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f1990640 1 -- 172.21.15.78:0/1856039262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365db483f0 con 0x56365db0f6b0 2024-05-21T15:20:38.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.297+0000 7f36f2191640 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365decf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.298+0000 7f36f1990640 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365decf7a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005fd0 tx=0x7f36dc00e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136b078d61cdecf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.298+0000 7f36f2992640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f36d80a2d00 0x7f36d80a5210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f36e40ee4f0 tx=0x7f36e4037c40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.298+0000 7f36e0bfe640 1 -- 172.21.15.78:0/1856039262 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc005220 con 0x56365db0f6b0 2024-05-21T15:20:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.298+0000 7f36e0bfe640 1 -- 172.21.15.78:0/1856039262 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc00e8d0 con 0x56365db0f6b0 2024-05-21T15:20:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.298+0000 7f36e0bfe640 1 -- 172.21.15.78:0/1856039262 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00b990 con 0x56365db0f6b0 2024-05-21T15:20:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 -- 172.21.15.78:0/1856039262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 msgr2=0x56365decf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365decf7a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005fd0 tx=0x7f36dc00e3d0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 -- 172.21.15.78:0/1856039262 shutdown_connections 2024-05-21T15:20:38.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365decf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365decf7a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 --2- 172.21.15.78:0/1856039262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db30390 0x56365deced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 -- 172.21.15.78:0/1856039262 >> 172.21.15.78:0/1856039262 conn(0x56365db244e0 msgr2=0x56365decb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 -- 172.21.15.78:0/1856039262 shutdown_connections 2024-05-21T15:20:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 -- 172.21.15.78:0/1856039262 wait complete. 2024-05-21T15:20:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 -- start start 2024-05-21T15:20:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 Processor -- start 2024-05-21T15:20:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.299+0000 7f36fb372740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365ded93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 0x56365ded98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db30390 0x56365dedf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56365dedc010 con 0x56365db30390 2024-05-21T15:20:38.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56365ddefc40 con 0x56365db074d0 2024-05-21T15:20:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56365dedc190 con 0x56365db0f6b0 2024-05-21T15:20:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 10 client.0 _ll_register_callbacks cb 0x56365db47be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db30390 0x56365dedf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db30390 0x56365dedf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50898/0 (socket says 172.21.15.78:50898) 2024-05-21T15:20:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 -- 172.21.15.78:0/217289784 learned_addr learned my addr 172.21.15.78:0/217289784 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f2191640 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 0x56365ded98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365ded93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 -- 172.21.15.78:0/217289784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 msgr2=0x56365ded98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 0x56365ded98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 -- 172.21.15.78:0/217289784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 msgr2=0x56365ded93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365ded93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 -- 172.21.15.78:0/217289784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365dc127d0 con 0x56365db30390 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36f1990640 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db30390 0x56365dedf580 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005bc0 tx=0x7f36dc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6de07da5a6e37b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36d5ffb640 1 -- 172.21.15.78:0/217289784 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc004dd0 con 0x56365db30390 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36d5ffb640 1 -- 172.21.15.78:0/217289784 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f36dc00b630 con 0x56365db30390 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 1 -- 172.21.15.78:0/217289784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc00e8d0 con 0x56365db30390 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36d5ffb640 1 -- 172.21.15.78:0/217289784 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36dc034050 con 0x56365db30390 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 10 client.16899 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.300+0000 7f36fb372740 1 -- 172.21.15.78:0/217289784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56365dee8850 con 0x56365db30390 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36d5ffb640 1 -- 172.21.15.78:0/217289784 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc00e8d0 con 0x56365db30390 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36d5ffb640 1 -- 172.21.15.78:0/217289784 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f36dc01c020 con 0x56365db30390 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36fb372740 10 client.16899 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36fb372740 10 client.16899 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36fb372740 20 client.16899 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36fb372740 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x56365daeef60 0x56365deebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36fb372740 4 client.16899 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36fb372740 1 -- 172.21.15.78:0/217289784 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56365de42370 con 0x56365daeef60 2024-05-21T15:20:38.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.301+0000 7f36f2191640 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x56365daeef60 0x56365deebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.302+0000 7f36d5ffb640 10 client.16899 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:20:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.302+0000 7f36f2191640 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x56365daeef60 0x56365deebbb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f2b8e27ac56a5b9 server_cookie=c05c7aa1ed7f6cfc in_seq=0 out_seq=0 2024-05-21T15:20:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.302+0000 7f36d5ffb640 1 -- 172.21.15.78:0/217289784 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56365de42370 con 0x56365daeef60 2024-05-21T15:20:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.302+0000 7f36d5ffb640 10 client.16899 handle_command_reply: tid=0 2024-05-21T15:20:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.302+0000 7f36fb372740 1 client.16899 shutdown 2024-05-21T15:20:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.302+0000 7f36fb372740 1 -- 172.21.15.78:0/217289784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db30390 msgr2=0x56365dedf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.302+0000 7f36fb372740 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db30390 0x56365dedf580 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005bc0 tx=0x7f36dc001e50 comp rx=0 tx=0).stop 2024-05-21T15:20:38.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 1 -- 172.21.15.78:0/217289784 shutdown_connections 2024-05-21T15:20:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x56365daeef60 0x56365deebbb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db30390 0x56365dedf580 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 0x56365ded98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 1 --2- 172.21.15.78:0/217289784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365ded93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 1 -- 172.21.15.78:0/217289784 >> 172.21.15.78:0/217289784 conn(0x56365db244e0 msgr2=0x56365ded0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36d5ffb640 0 client.16899 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:20:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 1 -- 172.21.15.78:0/217289784 shutdown_connections 2024-05-21T15:20:38.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 1 -- 172.21.15.78:0/217289784 wait complete. 2024-05-21T15:20:38.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.303+0000 7f36fb372740 20 client.16899 trim_cache size 0 max 16384 2024-05-21T15:20:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.316+0000 7f36fb372740 1 Processor -- start 2024-05-21T15:20:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.316+0000 7f36fb372740 1 -- start start 2024-05-21T15:20:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36fb372740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db244e0 0x56365ded8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36fb372740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365ded29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36fb372740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 0x56365ded2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36fb372740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1aec50 con 0x56365db244e0 2024-05-21T15:20:38.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36fb372740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b0910 con 0x56365db074d0 2024-05-21T15:20:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36fb372740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1af320 con 0x56365db0f6b0 2024-05-21T15:20:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db244e0 0x56365ded8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db244e0 0x56365ded8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50912/0 (socket says 172.21.15.78:50912) 2024-05-21T15:20:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 -- 172.21.15.78:0/1314033649 learned_addr learned my addr 172.21.15.78:0/1314033649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f2191640 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365ded29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 0x56365ded2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 -- 172.21.15.78:0/1314033649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 msgr2=0x56365ded2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 0x56365ded2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 -- 172.21.15.78:0/1314033649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 msgr2=0x56365ded29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365ded29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 -- 172.21.15.78:0/1314033649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365deec010 con 0x56365db244e0 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36f1990640 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db244e0 0x56365ded8940 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00e7e0 tx=0x7f36dc012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24dc21e34d22a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36d7fff640 1 -- 172.21.15.78:0/1314033649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc03c020 con 0x56365db244e0 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36d7fff640 1 -- 172.21.15.78:0/1314033649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc0124d0 con 0x56365db244e0 2024-05-21T15:20:38.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.317+0000 7f36d7fff640 1 -- 172.21.15.78:0/1314033649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc01e450 con 0x56365db244e0 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 -- 172.21.15.78:0/1314033649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db244e0 msgr2=0x56365ded8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db244e0 0x56365ded8940 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00e7e0 tx=0x7f36dc012730 comp rx=0 tx=0).stop 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 -- 172.21.15.78:0/1314033649 shutdown_connections 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db074d0 0x56365ded29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db0f6b0 0x56365ded2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 --2- 172.21.15.78:0/1314033649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db244e0 0x56365ded8940 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 -- 172.21.15.78:0/1314033649 >> 172.21.15.78:0/1314033649 conn(0x56365ded1370 msgr2=0x56365decf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 -- 172.21.15.78:0/1314033649 shutdown_connections 2024-05-21T15:20:38.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.318+0000 7f36fb372740 1 -- 172.21.15.78:0/1314033649 wait complete. 2024-05-21T15:20:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 1 -- start start 2024-05-21T15:20:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 1 Processor -- start 2024-05-21T15:20:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db074d0 0x56365dee58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365dee5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db244e0 0x56365dee0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b0180 con 0x56365db0f6b0 2024-05-21T15:20:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1afab0 con 0x56365db244e0 2024-05-21T15:20:38.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1acc30 con 0x56365db074d0 2024-05-21T15:20:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36fb372740 10 client.0 _ll_register_callbacks cb 0x7f36dc00edd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f2191640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365dee5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f2191640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365dee5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50922/0 (socket says 172.21.15.78:50922) 2024-05-21T15:20:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f2191640 1 -- 172.21.15.78:0/4090722311 learned_addr learned my addr 172.21.15.78:0/4090722311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f1990640 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db074d0 0x56365dee58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f2191640 1 -- 172.21.15.78:0/4090722311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db074d0 msgr2=0x56365dee58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f2191640 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db074d0 0x56365dee58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f2191640 1 -- 172.21.15.78:0/4090722311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db244e0 msgr2=0x56365dee0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f2191640 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db244e0 0x56365dee0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.319+0000 7f36f2191640 1 -- 172.21.15.78:0/4090722311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365dee1410 con 0x56365db0f6b0 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36f2191640 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365dee5e00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f36e800bce0 tx=0x7f36e801b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6568591ab5c4016c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36d5ffb640 1 -- 172.21.15.78:0/4090722311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8012aa0 con 0x56365db0f6b0 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36d5ffb640 1 -- 172.21.15.78:0/4090722311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f36e8012c40 con 0x56365db0f6b0 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 1 -- 172.21.15.78:0/4090722311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc0124d0 con 0x56365db0f6b0 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36d5ffb640 1 -- 172.21.15.78:0/4090722311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36e8012e40 con 0x56365db0f6b0 2024-05-21T15:20:38.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 10 client.16905 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 1 -- 172.21.15.78:0/4090722311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56365df53810 con 0x56365db0f6b0 2024-05-21T15:20:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36d5ffb640 1 -- 172.21.15.78:0/4090722311 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc0124d0 con 0x56365db0f6b0 2024-05-21T15:20:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36d5ffb640 1 -- 172.21.15.78:0/4090722311 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f36e802c6e0 con 0x56365db0f6b0 2024-05-21T15:20:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 10 client.16905 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 10 client.16905 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:20:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 20 client.16905 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x56365db30390 0x56365df55fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 4 client.16905 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.320+0000 7f36fb372740 1 -- 172.21.15.78:0/4090722311 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56365de3c6c0 con 0x56365db30390 2024-05-21T15:20:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36f1990640 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x56365db30390 0x56365df55fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36d5ffb640 10 client.16905 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:20:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36f1990640 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x56365db30390 0x56365df55fa0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3499ebcbf8602da server_cookie=5f2ab9175f11ef82 in_seq=0 out_seq=0 2024-05-21T15:20:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36d5ffb640 1 -- 172.21.15.78:0/4090722311 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56365de3c6c0 con 0x56365db30390 2024-05-21T15:20:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36d5ffb640 10 client.16905 handle_command_reply: tid=0 2024-05-21T15:20:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36fb372740 1 client.16905 shutdown 2024-05-21T15:20:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36fb372740 1 -- 172.21.15.78:0/4090722311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 msgr2=0x56365dee5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36fb372740 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365dee5e00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f36e800bce0 tx=0x7f36e801b6c0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36fb372740 1 -- 172.21.15.78:0/4090722311 shutdown_connections 2024-05-21T15:20:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.321+0000 7f36fb372740 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x56365db30390 0x56365df55fa0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36fb372740 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56365db244e0 0x56365dee0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36d5ffb640 0 client.16905 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:20:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36fb372740 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56365db0f6b0 0x56365dee5e00 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36fb372740 1 --2- 172.21.15.78:0/4090722311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56365db074d0 0x56365dee58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36fb372740 1 -- 172.21.15.78:0/4090722311 >> 172.21.15.78:0/4090722311 conn(0x56365ded1370 msgr2=0x56365decf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36fb372740 1 -- 172.21.15.78:0/4090722311 shutdown_connections 2024-05-21T15:20:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36fb372740 1 -- 172.21.15.78:0/4090722311 wait complete. 2024-05-21T15:20:38.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36fb372740 20 client.16905 trim_cache size 0 max 16384 2024-05-21T15:20:38.343 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 -- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f36d80a2d00 msgr2=0x7f36d80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f36d80a2d00 0x7f36d80a5210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f36e40ee4f0 tx=0x7f36e4037c40 comp rx=0 tx=0).stop 2024-05-21T15:20:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 -- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 msgr2=0x7f36ec19b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 0x7f36ec19b050 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f36e4031a20 tx=0x7f36e4031a50 comp rx=0 tx=0).stop 2024-05-21T15:20:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 -- 172.21.15.78:0/648024623 shutdown_connections 2024-05-21T15:20:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f36d80a2d00 0x7f36d80a5210 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ec10c360 0x7f36ec19b050 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36ec107a00 0x7f36ec19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 --2- 172.21.15.78:0/648024623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36ec101b20 0x7f36ec19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 -- 172.21.15.78:0/648024623 >> 172.21.15.78:0/648024623 conn(0x7f36ec0770c0 msgr2=0x7f36ec103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 -- 172.21.15.78:0/648024623 shutdown_connections 2024-05-21T15:20:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.322+0000 7f36f43f5640 1 -- 172.21.15.78:0/648024623 wait complete. 2024-05-21T15:20:38.347 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:20:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 -- 172.21.15.78:0/1286718455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474108b70 msgr2=0x7fb474100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 --2- 172.21.15.78:0/1286718455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474108b70 0x7fb474100ae0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb468009900 tx=0x7fb46802a410 comp rx=0 tx=0).stop 2024-05-21T15:20:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 -- 172.21.15.78:0/1286718455 shutdown_connections 2024-05-21T15:20:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 --2- 172.21.15.78:0/1286718455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb474101020 0x7fb474101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 --2- 172.21.15.78:0/1286718455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47410d520 0x7fb47410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 --2- 172.21.15.78:0/1286718455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474108b70 0x7fb474100ae0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 -- 172.21.15.78:0/1286718455 >> 172.21.15.78:0/1286718455 conn(0x7fb4740768d0 msgr2=0x7fb474076d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 -- 172.21.15.78:0/1286718455 shutdown_connections 2024-05-21T15:20:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.513+0000 7fb47b02b640 1 -- 172.21.15.78:0/1286718455 wait complete. 2024-05-21T15:20:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb47b02b640 1 Processor -- start 2024-05-21T15:20:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb47b02b640 1 -- start start 2024-05-21T15:20:38.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb47b02b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474101020 0x7fb47419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb47b02b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb474108b70 0x7fb47419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb47b02b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47410d520 0x7fb474199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb47b02b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb474112e30 con 0x7fb474101020 2024-05-21T15:20:38.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb47b02b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb474112cb0 con 0x7fb474108b70 2024-05-21T15:20:38.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb47b02b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb474112fb0 con 0x7fb47410d520 2024-05-21T15:20:38.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb478dc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474101020 0x7fb47419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb478dc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474101020 0x7fb47419f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50944/0 (socket says 172.21.15.78:50944) 2024-05-21T15:20:38.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.514+0000 7fb478dc7640 1 -- 172.21.15.78:0/495936511 learned_addr learned my addr 172.21.15.78:0/495936511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb473fff640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb474108b70 0x7fb47419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb478dc7640 1 -- 172.21.15.78:0/495936511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47410d520 msgr2=0x7fb474199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb478dc7640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47410d520 0x7fb474199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb478dc7640 1 -- 172.21.15.78:0/495936511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb474108b70 msgr2=0x7fb47419fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb478dc7640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb474108b70 0x7fb47419fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb478dc7640 1 -- 172.21.15.78:0/495936511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb474199d80 con 0x7fb474101020 2024-05-21T15:20:38.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb478dc7640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474101020 0x7fb47419f4d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb468005940 tx=0x7fb4680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb471ffb640 1 -- 172.21.15.78:0/495936511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46803f030 con 0x7fb474101020 2024-05-21T15:20:38.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb471ffb640 1 -- 172.21.15.78:0/495936511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb468003ce0 con 0x7fb474101020 2024-05-21T15:20:38.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb47b02b640 1 -- 172.21.15.78:0/495936511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb474199fb0 con 0x7fb474101020 2024-05-21T15:20:38.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb471ffb640 1 -- 172.21.15.78:0/495936511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468035070 con 0x7fb474101020 2024-05-21T15:20:38.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.515+0000 7fb47b02b640 1 -- 172.21.15.78:0/495936511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47419a4a0 con 0x7fb474101020 2024-05-21T15:20:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb471ffb640 1 -- 172.21.15.78:0/495936511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb468042030 con 0x7fb474101020 2024-05-21T15:20:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb481faf740 1 Processor -- start 2024-05-21T15:20:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb481faf740 1 -- start start 2024-05-21T15:20:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb481faf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403ca390 0x564d40768d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb481faf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403a14d0 0x564d40769260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb481faf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 0x564d407697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb481faf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d406cec40 con 0x564d403ca390 2024-05-21T15:20:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb481faf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d40676580 con 0x564d403a96b0 2024-05-21T15:20:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb481faf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d406cac00 con 0x564d403a14d0 2024-05-21T15:20:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb471ffb640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb4640a2d00 0x7fb4640a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb4795c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403ca390 0x564d40768d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb4795c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403ca390 0x564d40768d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50960/0 (socket says 172.21.15.78:50960) 2024-05-21T15:20:38.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb4795c8640 1 -- 172.21.15.78:0/2364933094 learned_addr learned my addr 172.21.15.78:0/2364933094 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb4795c8640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb4640a2d00 0x7fb4640a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.518+0000 7fb473fff640 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 0x564d407697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb471ffb640 1 -- 172.21.15.78:0/495936511 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb4680efad0 con 0x7fb474101020 2024-05-21T15:20:38.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb478dc7640 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403a14d0 0x564d40769260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb4795c8640 1 -- 172.21.15.78:0/2364933094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403a14d0 msgr2=0x564d40769260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb4795c8640 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403a14d0 0x564d40769260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb4795c8640 1 -- 172.21.15.78:0/2364933094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 msgr2=0x564d407697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb4795c8640 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 0x564d407697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb4795c8640 1 -- 172.21.15.78:0/2364933094 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d403e23f0 con 0x564d403ca390 2024-05-21T15:20:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb478dc7640 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403a14d0 0x564d40769260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb4795c8640 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403ca390 0x564d40768d20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb46c012c60 tx=0x7fb46c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b6d1b4de29ff56 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb4795c8640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb4640a2d00 0x7fb4640a5210 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fd0 tx=0x7fb46c01a000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb45effd640 1 -- 172.21.15.78:0/2364933094 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c020070 con 0x564d403ca390 2024-05-21T15:20:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb45effd640 1 -- 172.21.15.78:0/2364933094 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46c019ab0 con 0x564d403ca390 2024-05-21T15:20:38.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.519+0000 7fb45effd640 1 -- 172.21.15.78:0/2364933094 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c004020 con 0x564d403ca390 2024-05-21T15:20:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 -- 172.21.15.78:0/2364933094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403ca390 msgr2=0x564d40768d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403ca390 0x564d40768d20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb46c012c60 tx=0x7fb46c00f730 comp rx=0 tx=0).stop 2024-05-21T15:20:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 -- 172.21.15.78:0/2364933094 shutdown_connections 2024-05-21T15:20:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 0x564d407697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403a14d0 0x564d40769260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 --2- 172.21.15.78:0/2364933094 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403ca390 0x564d40768d20 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 -- 172.21.15.78:0/2364933094 >> 172.21.15.78:0/2364933094 conn(0x564d403be4e0 msgr2=0x564d407654d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 -- 172.21.15.78:0/2364933094 shutdown_connections 2024-05-21T15:20:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 -- 172.21.15.78:0/2364933094 wait complete. 2024-05-21T15:20:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 -- start start 2024-05-21T15:20:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.520+0000 7fb481faf740 1 Processor -- start 2024-05-21T15:20:38.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb481faf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a14d0 0x564d40774d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb481faf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 0x564d407752b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb481faf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403ca390 0x564d40773280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb481faf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d40775fb0 con 0x564d403a14d0 2024-05-21T15:20:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb481faf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d40689c40 con 0x564d403a96b0 2024-05-21T15:20:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb481faf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d40776130 con 0x564d403ca390 2024-05-21T15:20:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb481faf740 10 client.0 _ll_register_callbacks cb 0x564d403e1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a14d0 0x564d40774d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a14d0 0x564d40774d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50972/0 (socket says 172.21.15.78:50972) 2024-05-21T15:20:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 -- 172.21.15.78:0/2339596328 learned_addr learned my addr 172.21.15.78:0/2339596328 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb473fff640 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 0x564d407752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb4795c8640 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403ca390 0x564d40773280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 -- 172.21.15.78:0/2339596328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403ca390 msgr2=0x564d40773280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403ca390 0x564d40773280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 -- 172.21.15.78:0/2339596328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 msgr2=0x564d407752b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 0x564d407752b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 -- 172.21.15.78:0/2339596328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d404ac7d0 con 0x564d403a14d0 2024-05-21T15:20:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb478dc7640 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a14d0 0x564d40774d70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb4680b8960 tx=0x7fb4680b82f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fb15f196d1b42a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.521+0000 7fb453fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb453fff640 1 -- 172.21.15.78:0/2339596328 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468049040 con 0x564d403a14d0 2024-05-21T15:20:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb453fff640 1 -- 172.21.15.78:0/2339596328 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb4680b8600 con 0x564d403a14d0 2024-05-21T15:20:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 1 -- 172.21.15.78:0/2339596328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb46c019ab0 con 0x564d403a14d0 2024-05-21T15:20:38.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb453fff640 1 -- 172.21.15.78:0/2339596328 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb4680dc070 con 0x564d403a14d0 2024-05-21T15:20:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 10 client.16917 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 1 -- 172.21.15.78:0/2339596328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d40786ac0 con 0x564d403a14d0 2024-05-21T15:20:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb453fff640 1 -- 172.21.15.78:0/2339596328 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46c019ab0 con 0x564d403a14d0 2024-05-21T15:20:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb453fff640 1 -- 172.21.15.78:0/2339596328 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb46803bc50 con 0x564d403a14d0 2024-05-21T15:20:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 10 client.16917 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 10 client.16917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:20:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 20 client.16917 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564d40388f60 0x564d4078a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 4 client.16917 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb481faf740 1 -- 172.21.15.78:0/2339596328 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d406dc370 con 0x564d40388f60 2024-05-21T15:20:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.522+0000 7fb473fff640 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564d40388f60 0x564d4078a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.523+0000 7fb453fff640 10 client.16917 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:20:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.523+0000 7fb473fff640 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564d40388f60 0x564d4078a090 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53c740ebadd2a31b server_cookie=39f9384149d48633 in_seq=0 out_seq=0 2024-05-21T15:20:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb453fff640 1 -- 172.21.15.78:0/2339596328 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d406dc370 con 0x564d40388f60 2024-05-21T15:20:38.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb453fff640 10 client.16917 handle_command_reply: tid=0 2024-05-21T15:20:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 client.16917 shutdown 2024-05-21T15:20:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 -- 172.21.15.78:0/2339596328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a14d0 msgr2=0x564d40774d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a14d0 0x564d40774d70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb4680b8960 tx=0x7fb4680b82f0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 -- 172.21.15.78:0/2339596328 shutdown_connections 2024-05-21T15:20:38.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564d40388f60 0x564d4078a090 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a96b0 0x564d407752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb453fff640 0 client.16917 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:20:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d403ca390 0x564d40773280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 --2- 172.21.15.78:0/2339596328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a14d0 0x564d40774d70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 -- 172.21.15.78:0/2339596328 >> 172.21.15.78:0/2339596328 conn(0x564d403be4e0 msgr2=0x564d4076aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 -- 172.21.15.78:0/2339596328 shutdown_connections 2024-05-21T15:20:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 1 -- 172.21.15.78:0/2339596328 wait complete. 2024-05-21T15:20:38.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.524+0000 7fb481faf740 20 client.16917 trim_cache size 0 max 16384 2024-05-21T15:20:38.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb481faf740 1 Processor -- start 2024-05-21T15:20:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb481faf740 1 -- start start 2024-05-21T15:20:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb481faf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 0x564d4076b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb481faf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d4078f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb481faf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d4078fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb481faf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741aca80 con 0x564d403a96b0 2024-05-21T15:20:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb481faf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741ae740 con 0x564d403a14d0 2024-05-21T15:20:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb481faf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741ad150 con 0x564d40388f60 2024-05-21T15:20:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d4078fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d4078fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50980/0 (socket says 172.21.15.78:50980) 2024-05-21T15:20:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 -- 172.21.15.78:0/1960863228 learned_addr learned my addr 172.21.15.78:0/1960863228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb473fff640 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d4078f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb478dc7640 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 0x564d4076b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 -- 172.21.15.78:0/1960863228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 msgr2=0x564d4076b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 0x564d4076b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 -- 172.21.15.78:0/1960863228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 msgr2=0x564d4078f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d4078f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 -- 172.21.15.78:0/1960863228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d407717d0 con 0x564d403a96b0 2024-05-21T15:20:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb473fff640 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d4078f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.538+0000 7fb4795c8640 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d4078fdd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0127e0 tx=0x7fb46c012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b573b2ecaa1bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb45e7fc640 1 -- 172.21.15.78:0/1960863228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c020070 con 0x564d403a96b0 2024-05-21T15:20:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb45e7fc640 1 -- 172.21.15.78:0/1960863228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46c004d20 con 0x564d403a96b0 2024-05-21T15:20:38.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb45e7fc640 1 -- 172.21.15.78:0/1960863228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c005020 con 0x564d403a96b0 2024-05-21T15:20:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 -- 172.21.15.78:0/1960863228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 msgr2=0x564d4078fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d4078fdd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0127e0 tx=0x7fb46c012810 comp rx=0 tx=0).stop 2024-05-21T15:20:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 -- 172.21.15.78:0/1960863228 shutdown_connections 2024-05-21T15:20:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d4078f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d4078fdd0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 --2- 172.21.15.78:0/1960863228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 0x564d4076b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 -- 172.21.15.78:0/1960863228 >> 172.21.15.78:0/1960863228 conn(0x564d403be4e0 msgr2=0x564d4076a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 -- 172.21.15.78:0/1960863228 shutdown_connections 2024-05-21T15:20:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.539+0000 7fb481faf740 1 -- 172.21.15.78:0/1960863228 wait complete. 2024-05-21T15:20:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 1 -- start start 2024-05-21T15:20:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 1 Processor -- start 2024-05-21T15:20:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 0x564d40778790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d40778cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d407ea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741adfb0 con 0x564d403a96b0 2024-05-21T15:20:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741ad8e0 con 0x564d403a14d0 2024-05-21T15:20:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741aaa60 con 0x564d40388f60 2024-05-21T15:20:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb481faf740 10 client.0 _ll_register_callbacks cb 0x7fb468005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb4795c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d407ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb4795c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d407ea410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50994/0 (socket says 172.21.15.78:50994) 2024-05-21T15:20:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb4795c8640 1 -- 172.21.15.78:0/4094213947 learned_addr learned my addr 172.21.15.78:0/4094213947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.540+0000 7fb478dc7640 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 0x564d40778790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb473fff640 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d40778cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb4795c8640 1 -- 172.21.15.78:0/4094213947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 msgr2=0x564d40778790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb4795c8640 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 0x564d40778790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb4795c8640 1 -- 172.21.15.78:0/4094213947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 msgr2=0x564d40778cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb4795c8640 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d40778cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb4795c8640 1 -- 172.21.15.78:0/4094213947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d407ea950 con 0x564d403a96b0 2024-05-21T15:20:38.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb473fff640 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d40778cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:38.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb4795c8640 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d407ea410 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb46c014010 tx=0x7fb46c019ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a566211b127919a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb453fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb453fff640 1 -- 172.21.15.78:0/4094213947 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c020070 con 0x564d403a96b0 2024-05-21T15:20:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb453fff640 1 -- 172.21.15.78:0/4094213947 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb46c003b80 con 0x564d403a96b0 2024-05-21T15:20:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb481faf740 1 -- 172.21.15.78:0/4094213947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb46c004d20 con 0x564d403a96b0 2024-05-21T15:20:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb453fff640 1 -- 172.21.15.78:0/4094213947 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb46c003f30 con 0x564d403a96b0 2024-05-21T15:20:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb481faf740 10 client.16923 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb481faf740 1 -- 172.21.15.78:0/4094213947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d407f1c80 con 0x564d403a96b0 2024-05-21T15:20:38.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.541+0000 7fb453fff640 1 -- 172.21.15.78:0/4094213947 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46c004d20 con 0x564d403a96b0 2024-05-21T15:20:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb453fff640 1 -- 172.21.15.78:0/4094213947 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb46c03d030 con 0x564d403a96b0 2024-05-21T15:20:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb481faf740 10 client.16923 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb481faf740 10 client.16923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:20:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb481faf740 20 client.16923 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb481faf740 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564d403ca390 0x564d407f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb481faf740 4 client.16923 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb481faf740 1 -- 172.21.15.78:0/4094213947 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d406d66c0 con 0x564d403ca390 2024-05-21T15:20:38.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb478dc7640 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564d403ca390 0x564d407f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb453fff640 10 client.16923 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:20:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb478dc7640 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564d403ca390 0x564d407f4310 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8820c4601dfa89d server_cookie=2964f92722afa808 in_seq=0 out_seq=0 2024-05-21T15:20:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb453fff640 1 -- 172.21.15.78:0/4094213947 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d406d66c0 con 0x564d403ca390 2024-05-21T15:20:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb453fff640 10 client.16923 handle_command_reply: tid=0 2024-05-21T15:20:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.542+0000 7fb481faf740 1 client.16923 shutdown 2024-05-21T15:20:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 -- 172.21.15.78:0/4094213947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 msgr2=0x564d407ea410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d407ea410 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb46c014010 tx=0x7fb46c019ce0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 -- 172.21.15.78:0/4094213947 shutdown_connections 2024-05-21T15:20:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x564d403ca390 0x564d407f4310 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d403a14d0 0x564d40778cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d403a96b0 0x564d407ea410 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb453fff640 0 client.16923 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:20:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 --2- 172.21.15.78:0/4094213947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d40388f60 0x564d40778790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 -- 172.21.15.78:0/4094213947 >> 172.21.15.78:0/4094213947 conn(0x564d403be4e0 msgr2=0x564d40774800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 -- 172.21.15.78:0/4094213947 shutdown_connections 2024-05-21T15:20:38.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 1 -- 172.21.15.78:0/4094213947 wait complete. 2024-05-21T15:20:38.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb481faf740 20 client.16923 trim_cache size 0 max 16384 2024-05-21T15:20:38.560 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 -- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb4640a2d00 msgr2=0x7fb4640a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb4640a2d00 0x7fb4640a5210 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fd0 tx=0x7fb46c01a000 comp rx=0 tx=0).stop 2024-05-21T15:20:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 -- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474101020 msgr2=0x7fb47419f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474101020 0x7fb47419f4d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb468005940 tx=0x7fb4680042f0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 -- 172.21.15.78:0/495936511 shutdown_connections 2024-05-21T15:20:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb474108b70 0x7fb47419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb4640a2d00 0x7fb4640a5210 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47410d520 0x7fb474199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 --2- 172.21.15.78:0/495936511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb474101020 0x7fb47419f4d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 -- 172.21.15.78:0/495936511 >> 172.21.15.78:0/495936511 conn(0x7fb4740768d0 msgr2=0x7fb4741094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.543+0000 7fb47b02b640 1 -- 172.21.15.78:0/495936511 shutdown_connections 2024-05-21T15:20:38.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.544+0000 7fb47b02b640 1 -- 172.21.15.78:0/495936511 wait complete. 2024-05-21T15:20:38.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:38 smithi078 ceph-mon[31997]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 2.88k op/s 2024-05-21T15:20:38.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:38 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1012497421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:20:38.564 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:20:38.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:38 smithi084 ceph-mon[36334]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 2.88k op/s 2024-05-21T15:20:38.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:38 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1012497421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:20:38.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 -- 172.21.15.78:0/986920489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c1079d0 msgr2=0x7f831c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 --2- 172.21.15.78:0/986920489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c1079d0 0x7f831c101560 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8318009ca0 tx=0x7f831802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 -- 172.21.15.78:0/986920489 shutdown_connections 2024-05-21T15:20:38.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 --2- 172.21.15.78:0/986920489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f831c10c330 0x7f831c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 --2- 172.21.15.78:0/986920489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f831c101aa0 0x7f831c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 --2- 172.21.15.78:0/986920489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c1079d0 0x7f831c101560 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 -- 172.21.15.78:0/986920489 >> 172.21.15.78:0/986920489 conn(0x7f831c0770b0 msgr2=0x7f831c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 -- 172.21.15.78:0/986920489 shutdown_connections 2024-05-21T15:20:38.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.675+0000 7f8323984640 1 -- 172.21.15.78:0/986920489 wait complete. 2024-05-21T15:20:38.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8323984640 1 Processor -- start 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8323984640 1 -- start start 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8323984640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101aa0 0x7f831c10f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8323984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f831c1079d0 0x7f831c10f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8323984640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f831c10c330 0x7f831c10fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8323984640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f831c112e00 con 0x7f831c101aa0 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8323984640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f831c112c80 con 0x7f831c10c330 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8323984640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f831c112f80 con 0x7f831c1079d0 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321720640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101aa0 0x7f831c10f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321720640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101aa0 0x7f831c10f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52528/0 (socket says 172.21.15.78:52528) 2024-05-21T15:20:38.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321720640 1 -- 172.21.15.78:0/3254879824 learned_addr learned my addr 172.21.15.78:0/3254879824 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321f21640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f831c10c330 0x7f831c10fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8320f1f640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f831c1079d0 0x7f831c10f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321720640 1 -- 172.21.15.78:0/3254879824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f831c1079d0 msgr2=0x7f831c10f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321720640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f831c1079d0 0x7f831c10f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321720640 1 -- 172.21.15.78:0/3254879824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f831c10c330 msgr2=0x7f831c10fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321720640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f831c10c330 0x7f831c10fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.676+0000 7f8321720640 1 -- 172.21.15.78:0/3254879824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1a7ae0 con 0x7f831c101aa0 2024-05-21T15:20:38.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.677+0000 7f8321720640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101aa0 0x7f831c10f190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8318000f80 tx=0x7f83180319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.677+0000 7f83127fc640 1 -- 172.21.15.78:0/3254879824 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831802fa20 con 0x7f831c101aa0 2024-05-21T15:20:38.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.677+0000 7f83127fc640 1 -- 172.21.15.78:0/3254879824 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831802fbc0 con 0x7f831c101aa0 2024-05-21T15:20:38.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.677+0000 7f8323984640 1 -- 172.21.15.78:0/3254879824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1a7d70 con 0x7f831c101aa0 2024-05-21T15:20:38.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.677+0000 7f83127fc640 1 -- 172.21.15.78:0/3254879824 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318046800 con 0x7f831c101aa0 2024-05-21T15:20:38.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.677+0000 7f8323984640 1 -- 172.21.15.78:0/3254879824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1a8280 con 0x7f831c101aa0 2024-05-21T15:20:38.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:38 smithi172 ceph-mon[36402]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 2.88k op/s 2024-05-21T15:20:38.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:38 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1012497421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:20:38.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f83127fc640 1 -- 172.21.15.78:0/3254879824 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83180469a0 con 0x7f831c101aa0 2024-05-21T15:20:38.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f832a82a740 1 Processor -- start 2024-05-21T15:20:38.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f832a82a740 1 -- start start 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f832a82a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1004d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f832a82a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c1005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f832a82a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c10057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f832a82a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5646c0f6ac40 con 0x5646c0c3d4d0 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f832a82a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5646c0f12580 con 0x5646c0c66390 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.680+0000 7f832a82a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5646c0f66c00 con 0x5646c0c456b0 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c1005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c1005260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52544/0 (socket says 172.21.15.78:52544) 2024-05-21T15:20:38.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8320f1f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c10057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 -- 172.21.15.78:0/762355751 learned_addr learned my addr 172.21.15.78:0/762355751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321f21640 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1004d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f83127fc640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83080a2d80 0x7f83080a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 -- 172.21.15.78:0/762355751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 msgr2=0x5646c10057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c10057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 -- 172.21.15.78:0/762355751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 msgr2=0x5646c1004d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1004d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 -- 172.21.15.78:0/762355751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c0c7e3f0 con 0x5646c0c3d4d0 2024-05-21T15:20:38.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8320f1f640 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c10057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:38.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321f21640 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1004d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:38.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321f21640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83080a2d80 0x7f83080a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f83127fc640 1 -- 172.21.15.78:0/3254879824 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f83180f0720 con 0x7f831c101aa0 2024-05-21T15:20:38.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321720640 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c1005260 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f83180ede90 tx=0x7f8318087900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c14745f7577d6be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f8321f21640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83080a2d80 0x7f83080a5290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8314004650 tx=0x7f83140046c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f83077fe640 1 -- 172.21.15.78:0/762355751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180bcb50 con 0x5646c0c3d4d0 2024-05-21T15:20:38.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f83077fe640 1 -- 172.21.15.78:0/762355751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318087e10 con 0x5646c0c3d4d0 2024-05-21T15:20:38.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.681+0000 7f83077fe640 1 -- 172.21.15.78:0/762355751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180bc6e0 con 0x5646c0c3d4d0 2024-05-21T15:20:38.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 -- 172.21.15.78:0/762355751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 msgr2=0x5646c1005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c1005260 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f83180ede90 tx=0x7f8318087900 comp rx=0 tx=0).stop 2024-05-21T15:20:38.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 -- 172.21.15.78:0/762355751 shutdown_connections 2024-05-21T15:20:38.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c10057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c1005260 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 --2- 172.21.15.78:0/762355751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1004d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 -- 172.21.15.78:0/762355751 >> 172.21.15.78:0/762355751 conn(0x5646c0c5a4e0 msgr2=0x5646c10014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 -- 172.21.15.78:0/762355751 shutdown_connections 2024-05-21T15:20:38.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.682+0000 7f832a82a740 1 -- 172.21.15.78:0/762355751 wait complete. 2024-05-21T15:20:38.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 1 -- start start 2024-05-21T15:20:38.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 1 Processor -- start 2024-05-21T15:20:38.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c3d4d0 0x5646c100f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c456b0 0x5646c100f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1015160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5646c1011e30 con 0x5646c0c66390 2024-05-21T15:20:38.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5646c0f25c40 con 0x5646c0c456b0 2024-05-21T15:20:38.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5646c1011fb0 con 0x5646c0c3d4d0 2024-05-21T15:20:38.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f832a82a740 10 client.0 _ll_register_callbacks cb 0x5646c0c7dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f8321f21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1015160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f8321f21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1015160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52548/0 (socket says 172.21.15.78:52548) 2024-05-21T15:20:38.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.683+0000 7f8321f21640 1 -- 172.21.15.78:0/607989159 learned_addr learned my addr 172.21.15.78:0/607989159 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8321720640 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c3d4d0 0x5646c100f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8320f1f640 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c456b0 0x5646c100f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8321f21640 1 -- 172.21.15.78:0/607989159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c3d4d0 msgr2=0x5646c100f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8321f21640 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c3d4d0 0x5646c100f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8321f21640 1 -- 172.21.15.78:0/607989159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c456b0 msgr2=0x5646c100f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8321f21640 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c456b0 0x5646c100f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8321f21640 1 -- 172.21.15.78:0/607989159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c0d487d0 con 0x5646c0c66390 2024-05-21T15:20:38.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8321f21640 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1015160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8314003ef0 tx=0x7f8314004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bd61a311b6f6d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8304ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8304ff9640 1 -- 172.21.15.78:0/607989159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314010840 con 0x5646c0c66390 2024-05-21T15:20:38.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8304ff9640 1 -- 172.21.15.78:0/607989159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f83140109e0 con 0x5646c0c66390 2024-05-21T15:20:38.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f832a82a740 1 -- 172.21.15.78:0/607989159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8318087e10 con 0x5646c0c66390 2024-05-21T15:20:38.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8304ff9640 1 -- 172.21.15.78:0/607989159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8314010be0 con 0x5646c0c66390 2024-05-21T15:20:38.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f832a82a740 10 client.16935 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f832a82a740 1 -- 172.21.15.78:0/607989159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5646c101e4c0 con 0x5646c0c66390 2024-05-21T15:20:38.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.684+0000 7f8304ff9640 1 -- 172.21.15.78:0/607989159 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318087e10 con 0x5646c0c66390 2024-05-21T15:20:38.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f8304ff9640 1 -- 172.21.15.78:0/607989159 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8314028420 con 0x5646c0c66390 2024-05-21T15:20:38.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f832a82a740 10 client.16935 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f832a82a740 10 client.16935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:20:38.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f832a82a740 20 client.16935 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f832a82a740 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5646c0c24f60 0x5646c1021a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f832a82a740 4 client.16935 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f832a82a740 1 -- 172.21.15.78:0/607989159 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c0f78370 con 0x5646c0c24f60 2024-05-21T15:20:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f8321720640 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5646c0c24f60 0x5646c1021a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f8304ff9640 10 client.16935 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:20:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.685+0000 7f8321720640 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5646c0c24f60 0x5646c1021a40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6b72be28c6b662d server_cookie=99590c6ce51bc759 in_seq=0 out_seq=0 2024-05-21T15:20:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f8304ff9640 1 -- 172.21.15.78:0/607989159 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646c0f78370 con 0x5646c0c24f60 2024-05-21T15:20:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f8304ff9640 10 client.16935 handle_command_reply: tid=0 2024-05-21T15:20:38.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 client.16935 shutdown 2024-05-21T15:20:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 -- 172.21.15.78:0/607989159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 msgr2=0x5646c1015160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1015160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8314003ef0 tx=0x7f8314004730 comp rx=0 tx=0).stop 2024-05-21T15:20:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 -- 172.21.15.78:0/607989159 shutdown_connections 2024-05-21T15:20:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5646c0c24f60 0x5646c1021a40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c456b0 0x5646c100f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1015160 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 --2- 172.21.15.78:0/607989159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c3d4d0 0x5646c100f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f8304ff9640 0 client.16935 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:20:38.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 -- 172.21.15.78:0/607989159 >> 172.21.15.78:0/607989159 conn(0x5646c0c5a4e0 msgr2=0x5646c1006a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.686+0000 7f832a82a740 1 -- 172.21.15.78:0/607989159 shutdown_connections 2024-05-21T15:20:38.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.687+0000 7f832a82a740 1 -- 172.21.15.78:0/607989159 wait complete. 2024-05-21T15:20:38.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.687+0000 7f832a82a740 20 client.16935 trim_cache size 0 max 16384 2024-05-21T15:20:38.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f832a82a740 1 Processor -- start 2024-05-21T15:20:38.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f832a82a740 1 -- start start 2024-05-21T15:20:38.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f832a82a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c100e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f832a82a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c10067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f832a82a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1010f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f832a82a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1b0c60 con 0x5646c0c3d4d0 2024-05-21T15:20:38.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f832a82a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1b2920 con 0x5646c0c66390 2024-05-21T15:20:38.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f832a82a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1b1330 con 0x5646c0c456b0 2024-05-21T15:20:38.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f8321720640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c100e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f8321720640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c100e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52558/0 (socket says 172.21.15.78:52558) 2024-05-21T15:20:38.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.700+0000 7f8321720640 1 -- 172.21.15.78:0/3331699823 learned_addr learned my addr 172.21.15.78:0/3331699823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8321720640 1 -- 172.21.15.78:0/3331699823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 msgr2=0x5646c10067c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8321720640 1 --2- 172.21.15.78:0/3331699823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c10067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8321720640 1 -- 172.21.15.78:0/3331699823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 msgr2=0x5646c1010f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8321f21640 1 --2- 172.21.15.78:0/3331699823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1010f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8321720640 1 --2- 172.21.15.78:0/3331699823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1010f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8321720640 1 -- 172.21.15.78:0/3331699823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c100d6a0 con 0x5646c0c3d4d0 2024-05-21T15:20:38.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8321720640 1 --2- 172.21.15.78:0/3331699823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c100e320 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8318005e30 tx=0x7f831802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f32521d8497c0c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8306ffd640 1 -- 172.21.15.78:0/3331699823 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180bc8c0 con 0x5646c0c3d4d0 2024-05-21T15:20:38.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8306ffd640 1 -- 172.21.15.78:0/3331699823 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318036070 con 0x5646c0c3d4d0 2024-05-21T15:20:38.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.701+0000 7f8306ffd640 1 -- 172.21.15.78:0/3331699823 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180c5960 con 0x5646c0c3d4d0 2024-05-21T15:20:38.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 -- 172.21.15.78:0/3331699823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 msgr2=0x5646c100e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 --2- 172.21.15.78:0/3331699823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c100e320 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8318005e30 tx=0x7f831802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 -- 172.21.15.78:0/3331699823 shutdown_connections 2024-05-21T15:20:38.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 --2- 172.21.15.78:0/3331699823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c66390 0x5646c1010f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 --2- 172.21.15.78:0/3331699823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c10067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 --2- 172.21.15.78:0/3331699823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c3d4d0 0x5646c100e320 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 -- 172.21.15.78:0/3331699823 >> 172.21.15.78:0/3331699823 conn(0x5646c0c5a4e0 msgr2=0x5646c1010870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 -- 172.21.15.78:0/3331699823 shutdown_connections 2024-05-21T15:20:38.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 -- 172.21.15.78:0/3331699823 wait complete. 2024-05-21T15:20:38.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 -- start start 2024-05-21T15:20:38.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.702+0000 7f832a82a740 1 Processor -- start 2024-05-21T15:20:38.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f832a82a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c3d4d0 0x5646c10820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f832a82a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c1082610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f832a82a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1016dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f832a82a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1b2190 con 0x5646c0c66390 2024-05-21T15:20:38.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f832a82a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1b1ac0 con 0x5646c0c3d4d0 2024-05-21T15:20:38.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f832a82a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1aec40 con 0x5646c0c456b0 2024-05-21T15:20:38.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f832a82a740 10 client.0 _ll_register_callbacks cb 0x7f831400b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1016dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1016dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52574/0 (socket says 172.21.15.78:52574) 2024-05-21T15:20:38.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 -- 172.21.15.78:0/1988168119 learned_addr learned my addr 172.21.15.78:0/1988168119 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321720640 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c3d4d0 0x5646c10820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8320f1f640 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c1082610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 -- 172.21.15.78:0/1988168119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 msgr2=0x5646c1082610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c1082610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 -- 172.21.15.78:0/1988168119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c3d4d0 msgr2=0x5646c10820b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c3d4d0 0x5646c10820b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 -- 172.21.15.78:0/1988168119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c1017430 con 0x5646c0c66390 2024-05-21T15:20:38.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321720640 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c3d4d0 0x5646c10820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:38.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8321f21640 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1016dc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83140288e0 tx=0x7f8314028f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a869009e6a4071 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8304ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8304ff9640 1 -- 172.21.15.78:0/1988168119 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314034070 con 0x5646c0c66390 2024-05-21T15:20:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f8304ff9640 1 -- 172.21.15.78:0/1988168119 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8314010640 con 0x5646c0c66390 2024-05-21T15:20:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.703+0000 7f832a82a740 1 -- 172.21.15.78:0/1988168119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8318036070 con 0x5646c0c66390 2024-05-21T15:20:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f8304ff9640 1 -- 172.21.15.78:0/1988168119 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8314028620 con 0x5646c0c66390 2024-05-21T15:20:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f832a82a740 10 client.16941 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f832a82a740 1 -- 172.21.15.78:0/1988168119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5646c1089720 con 0x5646c0c66390 2024-05-21T15:20:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f8304ff9640 1 -- 172.21.15.78:0/1988168119 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318036070 con 0x5646c0c66390 2024-05-21T15:20:38.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f8304ff9640 1 -- 172.21.15.78:0/1988168119 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8314040cb0 con 0x5646c0c66390 2024-05-21T15:20:38.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f832a82a740 10 client.16941 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f832a82a740 10 client.16941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:20:38.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f832a82a740 20 client.16941 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f832a82a740 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5646c0bfa3a0 0x5646c108be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f832a82a740 4 client.16941 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:38.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f832a82a740 1 -- 172.21.15.78:0/1988168119 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646c0f726c0 con 0x5646c0bfa3a0 2024-05-21T15:20:38.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.704+0000 7f8321720640 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5646c0bfa3a0 0x5646c108be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f8304ff9640 10 client.16941 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:20:38.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f8321720640 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5646c0bfa3a0 0x5646c108be40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2da1dc6784935e86 server_cookie=19217714639c44db in_seq=0 out_seq=0 2024-05-21T15:20:38.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f8304ff9640 1 -- 172.21.15.78:0/1988168119 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646c0f726c0 con 0x5646c0bfa3a0 2024-05-21T15:20:38.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f8304ff9640 10 client.16941 handle_command_reply: tid=0 2024-05-21T15:20:38.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 client.16941 shutdown 2024-05-21T15:20:38.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 -- 172.21.15.78:0/1988168119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 msgr2=0x5646c1016dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1016dc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83140288e0 tx=0x7f8314028f20 comp rx=0 tx=0).stop 2024-05-21T15:20:38.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 -- 172.21.15.78:0/1988168119 shutdown_connections 2024-05-21T15:20:38.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5646c0bfa3a0 0x5646c108be40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5646c0c66390 0x5646c1016dc0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f8304ff9640 0 client.16941 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:20:38.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646c0c456b0 0x5646c1082610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 --2- 172.21.15.78:0/1988168119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c0c3d4d0 0x5646c10820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 -- 172.21.15.78:0/1988168119 >> 172.21.15.78:0/1988168119 conn(0x5646c0c5a4e0 msgr2=0x5646c100e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 -- 172.21.15.78:0/1988168119 shutdown_connections 2024-05-21T15:20:38.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 1 -- 172.21.15.78:0/1988168119 wait complete. 2024-05-21T15:20:38.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.705+0000 7f832a82a740 20 client.16941 trim_cache size 0 max 16384 2024-05-21T15:20:38.722 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:38.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 -- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83080a2d80 msgr2=0x7f83080a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83080a2d80 0x7f83080a5290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8314004650 tx=0x7f83140046c0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 -- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101aa0 msgr2=0x7f831c10f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101aa0 0x7f831c10f190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8318000f80 tx=0x7f83180319f0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 -- 172.21.15.78:0/3254879824 shutdown_connections 2024-05-21T15:20:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f831c10c330 0x7f831c10fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f83080a2d80 0x7f83080a5290 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f831c1079d0 0x7f831c10f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 --2- 172.21.15.78:0/3254879824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f831c101aa0 0x7f831c10f190 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 -- 172.21.15.78:0/3254879824 >> 172.21.15.78:0/3254879824 conn(0x7f831c0770b0 msgr2=0x7f831c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 -- 172.21.15.78:0/3254879824 shutdown_connections 2024-05-21T15:20:38.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.706+0000 7f8323984640 1 -- 172.21.15.78:0/3254879824 wait complete. 2024-05-21T15:20:38.725 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:20:38.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 -- 172.21.15.78:0/3801639253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 msgr2=0x7f0650101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 --2- 172.21.15.78:0/3801639253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 0x7f0650101eb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0638009900 tx=0x7f063802a410 comp rx=0 tx=0).stop 2024-05-21T15:20:38.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 -- 172.21.15.78:0/3801639253 shutdown_connections 2024-05-21T15:20:38.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 --2- 172.21.15.78:0/3801639253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065010c330 0x7f065010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 --2- 172.21.15.78:0/3801639253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 0x7f0650101eb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 --2- 172.21.15.78:0/3801639253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06501079d0 0x7f0650101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 -- 172.21.15.78:0/3801639253 >> 172.21.15.78:0/3801639253 conn(0x7f06500770b0 msgr2=0x7f06500774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 -- 172.21.15.78:0/3801639253 shutdown_connections 2024-05-21T15:20:38.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.836+0000 7f06565e3640 1 -- 172.21.15.78:0/3801639253 wait complete. 2024-05-21T15:20:38.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f06565e3640 1 Processor -- start 2024-05-21T15:20:38.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f06565e3640 1 -- start start 2024-05-21T15:20:38.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f06565e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 0x7f065019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f06565e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06501079d0 0x7f065019fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f06565e3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f065010c330 0x7f0650199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f06565e3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0650112e30 con 0x7f0650101aa0 2024-05-21T15:20:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f06565e3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0650112cb0 con 0x7f06501079d0 2024-05-21T15:20:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f06565e3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0650112fb0 con 0x7f065010c330 2024-05-21T15:20:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f064ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 0x7f065019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f064ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 0x7f065019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52584/0 (socket says 172.21.15.78:52584) 2024-05-21T15:20:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f064ffff640 1 -- 172.21.15.78:0/736829959 learned_addr learned my addr 172.21.15.78:0/736829959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.837+0000 7f0647fff640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06501079d0 0x7f065019fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f0654b80640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f065010c330 0x7f0650199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064ffff640 1 -- 172.21.15.78:0/736829959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f065010c330 msgr2=0x7f0650199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064ffff640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f065010c330 0x7f0650199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064ffff640 1 -- 172.21.15.78:0/736829959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06501079d0 msgr2=0x7f065019fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064ffff640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06501079d0 0x7f065019fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064ffff640 1 -- 172.21.15.78:0/736829959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0650199cb0 con 0x7f0650101aa0 2024-05-21T15:20:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064ffff640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 0x7f065019f4f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f064000e9f0 tx=0x7f064000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064dffb640 1 -- 172.21.15.78:0/736829959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064000ccc0 con 0x7f0650101aa0 2024-05-21T15:20:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064dffb640 1 -- 172.21.15.78:0/736829959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0640005500 con 0x7f0650101aa0 2024-05-21T15:20:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f06565e3640 1 -- 172.21.15.78:0/736829959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0650199fa0 con 0x7f0650101aa0 2024-05-21T15:20:38.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f064dffb640 1 -- 172.21.15.78:0/736829959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640010690 con 0x7f0650101aa0 2024-05-21T15:20:38.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.838+0000 7f06565e3640 1 -- 172.21.15.78:0/736829959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065019a4c0 con 0x7f0650101aa0 2024-05-21T15:20:38.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.840+0000 7f064dffb640 1 -- 172.21.15.78:0/736829959 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f06400046c0 con 0x7f0650101aa0 2024-05-21T15:20:38.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f065d563740 1 Processor -- start 2024-05-21T15:20:38.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f065d563740 1 -- start start 2024-05-21T15:20:38.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f065d563740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc316390 0x560fbc6b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f065d563740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f065d563740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f065d563740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560fbc61ac40 con 0x560fbc2f56b0 2024-05-21T15:20:38.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f065d563740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560fbc5c2580 con 0x560fbc2ed4d0 2024-05-21T15:20:38.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f065d563740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560fbc616c00 con 0x560fbc316390 2024-05-21T15:20:38.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f064dffb640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f063c0a2d80 0x7f063c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f064ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f0647fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f0654b80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc316390 0x560fbc6b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f0647fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52596/0 (socket says 172.21.15.78:52596) 2024-05-21T15:20:38.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f0647fff640 1 -- 172.21.15.78:0/302648966 learned_addr learned my addr 172.21.15.78:0/302648966 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.841+0000 7f0654b80640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f063c0a2d80 0x7f063c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f0647fff640 1 -- 172.21.15.78:0/302648966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc316390 msgr2=0x560fbc6b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f0647fff640 1 --2- 172.21.15.78:0/302648966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc316390 0x560fbc6b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f0647fff640 1 -- 172.21.15.78:0/302648966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 msgr2=0x560fbc6b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f0647fff640 1 --2- 172.21.15.78:0/302648966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f0647fff640 1 -- 172.21.15.78:0/302648966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbc32e3f0 con 0x560fbc2f56b0 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f064ffff640 1 --2- 172.21.15.78:0/302648966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f064dffb640 1 -- 172.21.15.78:0/736829959 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f06400cf830 con 0x7f0650101aa0 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f0654b80640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f063c0a2d80 0x7f063c0a5290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0648011fd0 tx=0x7f06480118f0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f0647fff640 1 --2- 172.21.15.78:0/302648966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6b57a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0638009900 tx=0x7f0638005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46edee2cc1a8edb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f06467fc640 1 -- 172.21.15.78:0/302648966 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638035040 con 0x560fbc2f56b0 2024-05-21T15:20:38.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f06467fc640 1 -- 172.21.15.78:0/302648966 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0638005520 con 0x560fbc2f56b0 2024-05-21T15:20:38.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.842+0000 7f06467fc640 1 -- 172.21.15.78:0/302648966 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06380035f0 con 0x560fbc2f56b0 2024-05-21T15:20:38.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 -- 172.21.15.78:0/302648966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 msgr2=0x560fbc6b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 --2- 172.21.15.78:0/302648966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6b57a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0638009900 tx=0x7f0638005fd0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 -- 172.21.15.78:0/302648966 shutdown_connections 2024-05-21T15:20:38.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 --2- 172.21.15.78:0/302648966 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 --2- 172.21.15.78:0/302648966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6b57a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 --2- 172.21.15.78:0/302648966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc316390 0x560fbc6b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 -- 172.21.15.78:0/302648966 >> 172.21.15.78:0/302648966 conn(0x560fbc30a4e0 msgr2=0x560fbc6b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 -- 172.21.15.78:0/302648966 shutdown_connections 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 -- 172.21.15.78:0/302648966 wait complete. 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.843+0000 7f065d563740 1 -- start start 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f065d563740 1 Processor -- start 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f065d563740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f065d563740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2f56b0 0x560fbc6c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f065d563740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc316390 0x560fbc6bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f065d563740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560fbc6c1fb0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f065d563740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560fbc5d5c40 con 0x560fbc316390 2024-05-21T15:20:38.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f065d563740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560fbc6c2130 con 0x560fbc2f56b0 2024-05-21T15:20:38.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f065d563740 10 client.0 _ll_register_callbacks cb 0x560fbc32dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f064ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f064ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52610/0 (socket says 172.21.15.78:52610) 2024-05-21T15:20:38.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f064ffff640 1 -- 172.21.15.78:0/4150638626 learned_addr learned my addr 172.21.15.78:0/4150638626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.844+0000 7f0654b80640 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc316390 0x560fbc6bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f064ffff640 1 -- 172.21.15.78:0/4150638626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2f56b0 msgr2=0x560fbc6c13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f0647fff640 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2f56b0 0x560fbc6c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f064ffff640 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2f56b0 0x560fbc6c13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f064ffff640 1 -- 172.21.15.78:0/4150638626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc316390 msgr2=0x560fbc6bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f064ffff640 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc316390 0x560fbc6bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f064ffff640 1 -- 172.21.15.78:0/4150638626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbc3f87d0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f0647fff640 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2f56b0 0x560fbc6c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:38.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f064ffff640 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c0eb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f06400986c0 tx=0x7f064009b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e39a2acbae7edaa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f062f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f062f7fe640 1 -- 172.21.15.78:0/4150638626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064009c3f0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f062f7fe640 1 -- 172.21.15.78:0/4150638626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f064009bf50 con 0x560fbc2ed4d0 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 1 -- 172.21.15.78:0/4150638626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0638005520 con 0x560fbc2ed4d0 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f062f7fe640 1 -- 172.21.15.78:0/4150638626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f064009bcf0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 10 client.16953 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 1 -- 172.21.15.78:0/4150638626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560fbc6ce7d0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f062f7fe640 1 -- 172.21.15.78:0/4150638626 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0638005520 con 0x560fbc2ed4d0 2024-05-21T15:20:38.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f062f7fe640 1 -- 172.21.15.78:0/4150638626 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f06400cfce0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 10 client.16953 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:38.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 20 client.16953 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560fbc2d4f60 0x560fbc6d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 4 client.16953 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:38.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.845+0000 7f065d563740 1 -- 172.21.15.78:0/4150638626 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fbc628370 con 0x560fbc2d4f60 2024-05-21T15:20:38.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.846+0000 7f0647fff640 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560fbc2d4f60 0x560fbc6d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.846+0000 7f062f7fe640 10 client.16953 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:38.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.846+0000 7f0647fff640 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560fbc2d4f60 0x560fbc6d1cf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c6a465bf8e7a01 server_cookie=47fba65f14e204e5 in_seq=0 out_seq=0 2024-05-21T15:20:38.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f062f7fe640 1 -- 172.21.15.78:0/4150638626 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560fbc628370 con 0x560fbc2d4f60 2024-05-21T15:20:38.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f062f7fe640 10 client.16953 handle_command_reply: tid=0 2024-05-21T15:20:38.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 client.16953 shutdown 2024-05-21T15:20:38.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 -- 172.21.15.78:0/4150638626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 msgr2=0x560fbc6c0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c0eb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f06400986c0 tx=0x7f064009b830 comp rx=0 tx=0).stop 2024-05-21T15:20:38.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 -- 172.21.15.78:0/4150638626 shutdown_connections 2024-05-21T15:20:38.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560fbc2d4f60 0x560fbc6d1cf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc316390 0x560fbc6bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2f56b0 0x560fbc6c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f062f7fe640 0 client.16953 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:38.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 --2- 172.21.15.78:0/4150638626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c0eb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 -- 172.21.15.78:0/4150638626 >> 172.21.15.78:0/4150638626 conn(0x560fbc30a4e0 msgr2=0x560fbc6b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 -- 172.21.15.78:0/4150638626 shutdown_connections 2024-05-21T15:20:38.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 1 -- 172.21.15.78:0/4150638626 wait complete. 2024-05-21T15:20:38.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.847+0000 7f065d563740 20 client.16953 trim_cache size 0 max 16384 2024-05-21T15:20:38.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f065d563740 1 Processor -- start 2024-05-21T15:20:38.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f065d563740 1 -- start start 2024-05-21T15:20:38.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f065d563740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 0x560fbc6bef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f065d563740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f065d563740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f065d563740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06501ac7f0 con 0x560fbc2f56b0 2024-05-21T15:20:38.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f065d563740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06501ae4b0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f065d563740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f06501acec0 con 0x560fbc2d4f60 2024-05-21T15:20:38.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f0654b80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f0654b80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6d7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52624/0 (socket says 172.21.15.78:52624) 2024-05-21T15:20:38.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f0654b80640 1 -- 172.21.15.78:0/2200403592 learned_addr learned my addr 172.21.15.78:0/2200403592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f0647fff640 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.861+0000 7f064ffff640 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 0x560fbc6bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0654b80640 1 -- 172.21.15.78:0/2200403592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 msgr2=0x560fbc6bef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0654b80640 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 0x560fbc6bef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0654b80640 1 -- 172.21.15.78:0/2200403592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 msgr2=0x560fbc6d74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0654b80640 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6d74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0654b80640 1 -- 172.21.15.78:0/2200403592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbc6bd810 con 0x560fbc2f56b0 2024-05-21T15:20:38.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0654b80640 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6d7a30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0648007dd0 tx=0x7f0648004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836b4dcee5d77b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0645ffb640 1 -- 172.21.15.78:0/2200403592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064801a040 con 0x560fbc2f56b0 2024-05-21T15:20:38.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0645ffb640 1 -- 172.21.15.78:0/2200403592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06480045b0 con 0x560fbc2f56b0 2024-05-21T15:20:38.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.862+0000 7f0645ffb640 1 -- 172.21.15.78:0/2200403592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648010740 con 0x560fbc2f56b0 2024-05-21T15:20:38.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 -- 172.21.15.78:0/2200403592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 msgr2=0x560fbc6d7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6d7a30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0648007dd0 tx=0x7f0648004ab0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 -- 172.21.15.78:0/2200403592 shutdown_connections 2024-05-21T15:20:38.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2ed4d0 0x560fbc6d74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2f56b0 0x560fbc6d7a30 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 --2- 172.21.15.78:0/2200403592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 0x560fbc6bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 -- 172.21.15.78:0/2200403592 >> 172.21.15.78:0/2200403592 conn(0x560fbc30a4e0 msgr2=0x560fbc6be800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 -- 172.21.15.78:0/2200403592 shutdown_connections 2024-05-21T15:20:38.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 -- 172.21.15.78:0/2200403592 wait complete. 2024-05-21T15:20:38.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 -- start start 2024-05-21T15:20:38.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.863+0000 7f065d563740 1 Processor -- start 2024-05-21T15:20:38.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f065d563740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 0x560fbc6c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f065d563740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f065d563740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2f56b0 0x560fbc6c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f065d563740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06501add20 con 0x560fbc2ed4d0 2024-05-21T15:20:38.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f065d563740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06501ad650 con 0x560fbc2f56b0 2024-05-21T15:20:38.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f065d563740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f06501aa7d0 con 0x560fbc2d4f60 2024-05-21T15:20:38.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f065d563740 10 client.0 _ll_register_callbacks cb 0x7f064000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:38.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52628/0 (socket says 172.21.15.78:52628) 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 -- 172.21.15.78:0/3775681288 learned_addr learned my addr 172.21.15.78:0/3775681288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0654b80640 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2f56b0 0x560fbc6c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f064ffff640 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 0x560fbc6c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 -- 172.21.15.78:0/3775681288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 msgr2=0x560fbc6c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 0x560fbc6c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 -- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2f56b0 msgr2=0x560fbc6c2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2f56b0 0x560fbc6c2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 -- 172.21.15.78:0/3775681288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbc6c33e0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0654b80640 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2f56b0 0x560fbc6c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:20:38.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f0647fff640 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c2960 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f063802aa00 tx=0x7f06380048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dfce9687696006 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:38.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f062f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:38.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f062f7fe640 1 -- 172.21.15.78:0/3775681288 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638035040 con 0x560fbc2ed4d0 2024-05-21T15:20:38.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f062f7fe640 1 -- 172.21.15.78:0/3775681288 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f0638050580 con 0x560fbc2ed4d0 2024-05-21T15:20:38.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f065d563740 1 -- 172.21.15.78:0/3775681288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06480045b0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.864+0000 7f062f7fe640 1 -- 172.21.15.78:0/3775681288 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f063805f050 con 0x560fbc2ed4d0 2024-05-21T15:20:38.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f065d563740 10 client.16959 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:38.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f065d563740 1 -- 172.21.15.78:0/3775681288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560fbc739ca0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f062f7fe640 1 -- 172.21.15.78:0/3775681288 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06480045b0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f062f7fe640 1 -- 172.21.15.78:0/3775681288 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f06380035f0 con 0x560fbc2ed4d0 2024-05-21T15:20:38.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f065d563740 10 client.16959 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:38.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f065d563740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:38.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f065d563740 20 client.16959 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:38.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f065d563740 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560fbc316390 0x560fbc73c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f065d563740 4 client.16959 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f065d563740 1 -- 172.21.15.78:0/3775681288 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fbc6226c0 con 0x560fbc316390 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f064ffff640 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560fbc316390 0x560fbc73c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f062f7fe640 10 client.16959 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.865+0000 7f064ffff640 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560fbc316390 0x560fbc73c010 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423e107ce861c408 server_cookie=74a47a2ae2111152 in_seq=0 out_seq=0 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f062f7fe640 1 -- 172.21.15.78:0/3775681288 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fbc6226c0 con 0x560fbc316390 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f062f7fe640 10 client.16959 handle_command_reply: tid=0 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 client.16959 shutdown 2024-05-21T15:20:38.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 -- 172.21.15.78:0/3775681288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 msgr2=0x560fbc6c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c2960 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f063802aa00 tx=0x7f06380048d0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 -- 172.21.15.78:0/3775681288 shutdown_connections 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560fbc316390 0x560fbc73c010 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560fbc2f56b0 0x560fbc6c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fbc2ed4d0 0x560fbc6c2960 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f062f7fe640 0 client.16959 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 --2- 172.21.15.78:0/3775681288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560fbc2d4f60 0x560fbc6c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 -- 172.21.15.78:0/3775681288 >> 172.21.15.78:0/3775681288 conn(0x560fbc30a4e0 msgr2=0x560fbc6be800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 -- 172.21.15.78:0/3775681288 shutdown_connections 2024-05-21T15:20:38.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 1 -- 172.21.15.78:0/3775681288 wait complete. 2024-05-21T15:20:38.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f065d563740 20 client.16959 trim_cache size 0 max 16384 2024-05-21T15:20:38.881 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:20:38.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f06565e3640 1 -- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f063c0a2d80 msgr2=0x7f063c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f06565e3640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f063c0a2d80 0x7f063c0a5290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0648011fd0 tx=0x7f06480118f0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f06565e3640 1 -- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 msgr2=0x7f065019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:38.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.866+0000 7f06565e3640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 0x7f065019f4f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f064000e9f0 tx=0x7f064000eeb0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.867+0000 7f06565e3640 1 -- 172.21.15.78:0/736829959 shutdown_connections 2024-05-21T15:20:38.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.867+0000 7f06565e3640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06501079d0 0x7f065019fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.867+0000 7f06565e3640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f063c0a2d80 0x7f063c0a5290 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.867+0000 7f06565e3640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f065010c330 0x7f0650199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.867+0000 7f06565e3640 1 --2- 172.21.15.78:0/736829959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0650101aa0 0x7f065019f4f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:38.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.867+0000 7f06565e3640 1 -- 172.21.15.78:0/736829959 >> 172.21.15.78:0/736829959 conn(0x7f06500770b0 msgr2=0x7f0650103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:38.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.867+0000 7f06565e3640 1 -- 172.21.15.78:0/736829959 shutdown_connections 2024-05-21T15:20:38.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:38.867+0000 7f06565e3640 1 -- 172.21.15.78:0/736829959 wait complete. 2024-05-21T15:20:39.885 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:20:39.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:20:39.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:20:40.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 -- 172.21.15.78:0/1540448994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9030107a10 msgr2=0x7f90301015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 --2- 172.21.15.78:0/1540448994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9030107a10 0x7f90301015a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f902c009900 tx=0x7f902c02a410 comp rx=0 tx=0).stop 2024-05-21T15:20:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 -- 172.21.15.78:0/1540448994 shutdown_connections 2024-05-21T15:20:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 --2- 172.21.15.78:0/1540448994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9030101ae0 0x7f9030101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 --2- 172.21.15.78:0/1540448994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f903010c370 0x7f903010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 --2- 172.21.15.78:0/1540448994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9030107a10 0x7f90301015a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 -- 172.21.15.78:0/1540448994 >> 172.21.15.78:0/1540448994 conn(0x7f90300770c0 msgr2=0x7f9030077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 -- 172.21.15.78:0/1540448994 shutdown_connections 2024-05-21T15:20:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.017+0000 7f9036f45640 1 -- 172.21.15.78:0/1540448994 wait complete. 2024-05-21T15:20:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f9036f45640 1 Processor -- start 2024-05-21T15:20:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f9036f45640 1 -- start start 2024-05-21T15:20:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f9036f45640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9030101ae0 0x7f90301986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f9036f45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030107a10 0x7f9030198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f9036f45640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f903010c370 0x7f903019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f9036f45640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9030111d20 con 0x7f903010c370 2024-05-21T15:20:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f9036f45640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9030111ba0 con 0x7f9030101ae0 2024-05-21T15:20:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f9036f45640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9030111ea0 con 0x7f9030107a10 2024-05-21T15:20:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f90354e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f903010c370 0x7f903019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f90354e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f903010c370 0x7f903019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52646/0 (socket says 172.21.15.78:52646) 2024-05-21T15:20:40.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.018+0000 7f90354e2640 1 -- 172.21.15.78:0/805237665 learned_addr learned my addr 172.21.15.78:0/805237665 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f9034ce1640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9030101ae0 0x7f90301986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f9027fff640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030107a10 0x7f9030198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f90354e2640 1 -- 172.21.15.78:0/805237665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030107a10 msgr2=0x7f9030198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f90354e2640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030107a10 0x7f9030198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f90354e2640 1 -- 172.21.15.78:0/805237665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9030101ae0 msgr2=0x7f90301986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f90354e2640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9030101ae0 0x7f90301986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f90354e2640 1 -- 172.21.15.78:0/805237665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903019d500 con 0x7f903010c370 2024-05-21T15:20:40.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f90354e2640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f903010c370 0x7f903019cec0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f902800ebb0 tx=0x7f9028007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f9025ffb640 1 -- 172.21.15.78:0/805237665 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028007ca0 con 0x7f903010c370 2024-05-21T15:20:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f9025ffb640 1 -- 172.21.15.78:0/805237665 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9028007e40 con 0x7f903010c370 2024-05-21T15:20:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f9036f45640 1 -- 172.21.15.78:0/805237665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903019d790 con 0x7f903010c370 2024-05-21T15:20:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.019+0000 7f9025ffb640 1 -- 172.21.15.78:0/805237665 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90280058a0 con 0x7f903010c370 2024-05-21T15:20:40.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.020+0000 7f9036f45640 1 -- 172.21.15.78:0/805237665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903006a260 con 0x7f903010c370 2024-05-21T15:20:40.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.022+0000 7f9025ffb640 1 -- 172.21.15.78:0/805237665 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9028004a40 con 0x7f903010c370 2024-05-21T15:20:40.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.022+0000 7f903ddf4740 1 Processor -- start 2024-05-21T15:20:40.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.022+0000 7f903ddf4740 1 -- start start 2024-05-21T15:20:40.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.022+0000 7f9025ffb640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f90180a2d80 0x7f90180a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.022+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.022+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f90180a2d80 0x7f90180a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f903ddf4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561288013990 con 0x561287ce6550 2024-05-21T15:20:40.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f903ddf4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561287fbb5b0 con 0x561287cee730 2024-05-21T15:20:40.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f903ddf4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56128800f950 con 0x561287d0f410 2024-05-21T15:20:40.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9025ffb640 1 -- 172.21.15.78:0/805237665 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f90280d0a50 con 0x7f903010c370 2024-05-21T15:20:40.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52650/0 (socket says 172.21.15.78:52650) 2024-05-21T15:20:40.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 -- 172.21.15.78:0/1664179309 learned_addr learned my addr 172.21.15.78:0/1664179309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9034ce1640 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f90354e2640 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 -- 172.21.15.78:0/1664179309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 msgr2=0x5612880ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 -- 172.21.15.78:0/1664179309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 msgr2=0x5612880af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 -- 172.21.15.78:0/1664179309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561287def7e0 con 0x561287ce6550 2024-05-21T15:20:40.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f90354e2640 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:40.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f90180a2d80 0x7f90180a5290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9020014920 tx=0x7f902000bb50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9027fff640 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880aeb60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9020011750 tx=0x7f902001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c313a9e57561df server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.023+0000 7f9016ffd640 1 -- 172.21.15.78:0/1664179309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020011e80 con 0x561287ce6550 2024-05-21T15:20:40.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f9016ffd640 1 -- 172.21.15.78:0/1664179309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9020014070 con 0x561287ce6550 2024-05-21T15:20:40.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f9016ffd640 1 -- 172.21.15.78:0/1664179309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020020b40 con 0x561287ce6550 2024-05-21T15:20:40.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f903ddf4740 1 -- 172.21.15.78:0/1664179309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 msgr2=0x5612880aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880aeb60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9020011750 tx=0x7f902001b000 comp rx=0 tx=0).stop 2024-05-21T15:20:40.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f903ddf4740 1 -- 172.21.15.78:0/1664179309 shutdown_connections 2024-05-21T15:20:40.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880aeb60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1664179309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.024+0000 7f903ddf4740 1 -- 172.21.15.78:0/1664179309 >> 172.21.15.78:0/1664179309 conn(0x561287d03560 msgr2=0x5612880aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.025+0000 7f903ddf4740 1 -- 172.21.15.78:0/1664179309 shutdown_connections 2024-05-21T15:20:40.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.025+0000 7f903ddf4740 1 -- 172.21.15.78:0/1664179309 wait complete. 2024-05-21T15:20:40.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.025+0000 7f903ddf4740 1 -- start start 2024-05-21T15:20:40.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.025+0000 7f903ddf4740 1 Processor -- start 2024-05-21T15:20:40.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880ba660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f903ddf4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5612880bb910 con 0x561287ce6550 2024-05-21T15:20:40.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f903ddf4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561287fceb40 con 0x561287cee730 2024-05-21T15:20:40.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f903ddf4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5612880bba90 con 0x561287d0f410 2024-05-21T15:20:40.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f903ddf4740 10 client.0 _ll_register_callbacks cb 0x561287fc30f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:40.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880ba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880ba660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52652/0 (socket says 172.21.15.78:52652) 2024-05-21T15:20:40.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 -- 172.21.15.78:0/3030527936 learned_addr learned my addr 172.21.15.78:0/3030527936 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f90354e2640 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9027fff640 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 -- 172.21.15.78:0/3030527936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 msgr2=0x5612880b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 -- 172.21.15.78:0/3030527936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 msgr2=0x5612880babc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880babc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 -- 172.21.15.78:0/3030527936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561287e4ff70 con 0x561287ce6550 2024-05-21T15:20:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f9034ce1640 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880ba660 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f902c009900 tx=0x7f902c0318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f02cf8d0706dfdf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f900ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f900ffff640 1 -- 172.21.15.78:0/3030527936 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c035040 con 0x561287ce6550 2024-05-21T15:20:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f900ffff640 1 -- 172.21.15.78:0/3030527936 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f902c004040 con 0x561287ce6550 2024-05-21T15:20:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.026+0000 7f903ddf4740 1 -- 172.21.15.78:0/3030527936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9020014070 con 0x561287ce6550 2024-05-21T15:20:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f900ffff640 1 -- 172.21.15.78:0/3030527936 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f902c004240 con 0x561287ce6550 2024-05-21T15:20:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f903ddf4740 10 client.16971 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f903ddf4740 1 -- 172.21.15.78:0/3030527936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561287ce7170 con 0x561287ce6550 2024-05-21T15:20:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f900ffff640 1 -- 172.21.15.78:0/3030527936 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9020014070 con 0x561287ce6550 2024-05-21T15:20:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f900ffff640 1 -- 172.21.15.78:0/3030527936 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f902c004a80 con 0x561287ce6550 2024-05-21T15:20:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f903ddf4740 10 client.16971 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f903ddf4740 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f903ddf4740 20 client.16971 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f903ddf4740 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561287cce1f0 0x5612880cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f903ddf4740 4 client.16971 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f903ddf4740 1 -- 172.21.15.78:0/3030527936 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612880210c0 con 0x561287cce1f0 2024-05-21T15:20:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.027+0000 7f9027fff640 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561287cce1f0 0x5612880cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.028+0000 7f9027fff640 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561287cce1f0 0x5612880cb250 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cd25b1bb970376 server_cookie=a02b5ed8fe14e590 in_seq=0 out_seq=0 2024-05-21T15:20:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.028+0000 7f900ffff640 10 client.16971 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f900ffff640 1 -- 172.21.15.78:0/3030527936 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612880210c0 con 0x561287cce1f0 2024-05-21T15:20:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f900ffff640 10 client.16971 handle_command_reply: tid=0 2024-05-21T15:20:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 client.16971 shutdown 2024-05-21T15:20:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 -- 172.21.15.78:0/3030527936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 msgr2=0x5612880ba660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880ba660 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f902c009900 tx=0x7f902c0318b0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 -- 172.21.15.78:0/3030527936 shutdown_connections 2024-05-21T15:20:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561287cce1f0 0x5612880cb250 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287cee730 0x5612880babc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 --2- 172.21.15.78:0/3030527936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287ce6550 0x5612880ba660 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f900ffff640 0 client.16971 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:40.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 -- 172.21.15.78:0/3030527936 >> 172.21.15.78:0/3030527936 conn(0x561287d03560 msgr2=0x5612880ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 -- 172.21.15.78:0/3030527936 shutdown_connections 2024-05-21T15:20:40.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 1 -- 172.21.15.78:0/3030527936 wait complete. 2024-05-21T15:20:40.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.029+0000 7f903ddf4740 20 client.16971 trim_cache size 0 max 16384 2024-05-21T15:20:40.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.047+0000 7f903ddf4740 1 Processor -- start 2024-05-21T15:20:40.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.047+0000 7f903ddf4740 1 -- start start 2024-05-21T15:20:40.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.047+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 0x5612880aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.047+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287cee730 0x5612880c0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.047+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.047+0000 7f903ddf4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90301aca40 con 0x561287cee730 2024-05-21T15:20:40.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.047+0000 7f903ddf4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f90301ae700 con 0x561287ce6550 2024-05-21T15:20:40.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.047+0000 7f903ddf4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90301ad110 con 0x561287d0f410 2024-05-21T15:20:40.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287cee730 0x5612880c0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287cee730 0x5612880c0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52668/0 (socket says 172.21.15.78:52668) 2024-05-21T15:20:40.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 -- 172.21.15.78:0/2605094763 learned_addr learned my addr 172.21.15.78:0/2605094763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f90354e2640 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9034ce1640 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 0x5612880aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 -- 172.21.15.78:0/2605094763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 msgr2=0x5612880c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 -- 172.21.15.78:0/2605094763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 msgr2=0x5612880aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 0x5612880aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 -- 172.21.15.78:0/2605094763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612880b7b20 con 0x561287cee730 2024-05-21T15:20:40.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f9027fff640 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287cee730 0x5612880c0d70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f902000bf50 tx=0x7f902000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d64738f262c9841 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f90167fc640 1 -- 172.21.15.78:0/2605094763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020038070 con 0x561287cee730 2024-05-21T15:20:40.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f90167fc640 1 -- 172.21.15.78:0/2605094763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902000fc50 con 0x561287cee730 2024-05-21T15:20:40.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.048+0000 7f90167fc640 1 -- 172.21.15.78:0/2605094763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90200143c0 con 0x561287cee730 2024-05-21T15:20:40.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 -- 172.21.15.78:0/2605094763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287cee730 msgr2=0x5612880c0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287cee730 0x5612880c0d70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f902000bf50 tx=0x7f902000bec0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 -- 172.21.15.78:0/2605094763 shutdown_connections 2024-05-21T15:20:40.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287d0f410 0x5612880c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287cee730 0x5612880c0d70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 --2- 172.21.15.78:0/2605094763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 0x5612880aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 -- 172.21.15.78:0/2605094763 >> 172.21.15.78:0/2605094763 conn(0x561287d03560 msgr2=0x5612880ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 -- 172.21.15.78:0/2605094763 shutdown_connections 2024-05-21T15:20:40.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 -- 172.21.15.78:0/2605094763 wait complete. 2024-05-21T15:20:40.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 -- start start 2024-05-21T15:20:40.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.049+0000 7f903ddf4740 1 Processor -- start 2024-05-21T15:20:40.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 0x5612880c0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287cee730 0x5612880c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f903ddf4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287d0f410 0x56128812a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f903ddf4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90301adf70 con 0x561287d0f410 2024-05-21T15:20:40.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f903ddf4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f90301ad8a0 con 0x561287ce6550 2024-05-21T15:20:40.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f903ddf4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90301aaa20 con 0x561287cee730 2024-05-21T15:20:40.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f903ddf4740 10 client.0 _ll_register_callbacks cb 0x7f902c0314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:40.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f90354e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287d0f410 0x56128812a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f90354e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287d0f410 0x56128812a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52670/0 (socket says 172.21.15.78:52670) 2024-05-21T15:20:40.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f90354e2640 1 -- 172.21.15.78:0/1027340630 learned_addr learned my addr 172.21.15.78:0/1027340630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f9027fff640 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287cee730 0x5612880c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f9034ce1640 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 0x5612880c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f90354e2640 1 -- 172.21.15.78:0/1027340630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287cee730 msgr2=0x5612880c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f90354e2640 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287cee730 0x5612880c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f90354e2640 1 -- 172.21.15.78:0/1027340630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 msgr2=0x5612880c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f90354e2640 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 0x5612880c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.050+0000 7f90354e2640 1 -- 172.21.15.78:0/1027340630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128812ae20 con 0x561287d0f410 2024-05-21T15:20:40.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f90354e2640 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287d0f410 0x56128812a860 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9028099370 tx=0x7f902809c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4b300eb157674a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f900ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:40.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f900ffff640 1 -- 172.21.15.78:0/1027340630 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90280bbd50 con 0x561287d0f410 2024-05-21T15:20:40.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f900ffff640 1 -- 172.21.15.78:0/1027340630 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f902809c360 con 0x561287d0f410 2024-05-21T15:20:40.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 1 -- 172.21.15.78:0/1027340630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f902000fc50 con 0x561287d0f410 2024-05-21T15:20:40.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f900ffff640 1 -- 172.21.15.78:0/1027340630 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f90280a5b80 con 0x561287d0f410 2024-05-21T15:20:40.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 10 client.16977 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:40.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 1 -- 172.21.15.78:0/1027340630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5612881333f0 con 0x561287d0f410 2024-05-21T15:20:40.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f900ffff640 1 -- 172.21.15.78:0/1027340630 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902000fc50 con 0x561287d0f410 2024-05-21T15:20:40.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f900ffff640 1 -- 172.21.15.78:0/1027340630 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f902809bdd0 con 0x561287d0f410 2024-05-21T15:20:40.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 10 client.16977 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:40.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:40.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 20 client.16977 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:40.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561287ca3630 0x561288135c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 4 client.16977 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:20:40.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.051+0000 7f903ddf4740 1 -- 172.21.15.78:0/1027340630 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56128801b410 con 0x561287ca3630 2024-05-21T15:20:40.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.052+0000 7f9034ce1640 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561287ca3630 0x561288135c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.052+0000 7f900ffff640 10 client.16977 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:40.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.052+0000 7f9034ce1640 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561287ca3630 0x561288135c30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04c42361bf01b59 server_cookie=21256c593b438206 in_seq=0 out_seq=0 2024-05-21T15:20:40.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f900ffff640 1 -- 172.21.15.78:0/1027340630 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56128801b410 con 0x561287ca3630 2024-05-21T15:20:40.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f900ffff640 10 client.16977 handle_command_reply: tid=0 2024-05-21T15:20:40.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 client.16977 shutdown 2024-05-21T15:20:40.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 -- 172.21.15.78:0/1027340630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287d0f410 msgr2=0x56128812a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287d0f410 0x56128812a860 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9028099370 tx=0x7f902809c9a0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 -- 172.21.15.78:0/1027340630 shutdown_connections 2024-05-21T15:20:40.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561287ca3630 0x561288135c30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561287d0f410 0x56128812a860 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561287cee730 0x5612880c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 --2- 172.21.15.78:0/1027340630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561287ce6550 0x5612880c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 -- 172.21.15.78:0/1027340630 >> 172.21.15.78:0/1027340630 conn(0x561287d03560 msgr2=0x5612880b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f900ffff640 0 client.16977 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:40.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.053+0000 7f903ddf4740 1 -- 172.21.15.78:0/1027340630 shutdown_connections 2024-05-21T15:20:40.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f903ddf4740 1 -- 172.21.15.78:0/1027340630 wait complete. 2024-05-21T15:20:40.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f903ddf4740 20 client.16977 trim_cache size 0 max 16384 2024-05-21T15:20:40.067 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:20:40.067 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:20:40.068 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c9738c2b-92a7-4cb8-a3eb-33f123225731", 2024-05-21T15:20:40.068 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:20:40.068 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:20:40.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 -- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f90180a2d80 msgr2=0x7f90180a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f90180a2d80 0x7f90180a5290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9020014920 tx=0x7f902000bb50 comp rx=0 tx=0).stop 2024-05-21T15:20:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 -- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f903010c370 msgr2=0x7f903019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f903010c370 0x7f903019cec0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f902800ebb0 tx=0x7f9028007590 comp rx=0 tx=0).stop 2024-05-21T15:20:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 -- 172.21.15.78:0/805237665 shutdown_connections 2024-05-21T15:20:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f90180a2d80 0x7f90180a5290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f903010c370 0x7f903019cec0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9030107a10 0x7f9030198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 --2- 172.21.15.78:0/805237665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9030101ae0 0x7f90301986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 -- 172.21.15.78:0/805237665 >> 172.21.15.78:0/805237665 conn(0x7f90300770c0 msgr2=0x7f90301030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 -- 172.21.15.78:0/805237665 shutdown_connections 2024-05-21T15:20:40.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.054+0000 7f9036f45640 1 -- 172.21.15.78:0/805237665 wait complete. 2024-05-21T15:20:40.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:20:40.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:20:40.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 -- 172.21.15.78:0/3414005732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14101b20 msgr2=0x7fdf14101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/3414005732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14101b20 0x7fdf14101f30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009900 tx=0x7fdf0002a410 comp rx=0 tx=0).stop 2024-05-21T15:20:40.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 -- 172.21.15.78:0/3414005732 shutdown_connections 2024-05-21T15:20:40.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/3414005732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1410c360 0x7fdf1410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/3414005732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14101b20 0x7fdf14101f30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/3414005732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf14107a00 0x7fdf141015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 -- 172.21.15.78:0/3414005732 >> 172.21.15.78:0/3414005732 conn(0x7fdf140770c0 msgr2=0x7fdf14077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 -- 172.21.15.78:0/3414005732 shutdown_connections 2024-05-21T15:20:40.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.185+0000 7fdf1ac21640 1 -- 172.21.15.78:0/3414005732 wait complete. 2024-05-21T15:20:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf1ac21640 1 Processor -- start 2024-05-21T15:20:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf1ac21640 1 -- start start 2024-05-21T15:20:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf1ac21640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf14101b20 0x7fdf14199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf1ac21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14107a00 0x7fdf14199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf1ac21640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1410c360 0x7fdf1419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf1ac21640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14111d10 con 0x7fdf14107a00 2024-05-21T15:20:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf1ac21640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14111b90 con 0x7fdf14101b20 2024-05-21T15:20:40.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf1ac21640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14111e90 con 0x7fdf1410c360 2024-05-21T15:20:40.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14107a00 0x7fdf14199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf13fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14107a00 0x7fdf14199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52690/0 (socket says 172.21.15.78:52690) 2024-05-21T15:20:40.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.186+0000 7fdf13fff640 1 -- 172.21.15.78:0/1796873218 learned_addr learned my addr 172.21.15.78:0/1796873218 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf189bd640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf14101b20 0x7fdf14199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf191be640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1410c360 0x7fdf1419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf13fff640 1 -- 172.21.15.78:0/1796873218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1410c360 msgr2=0x7fdf1419e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf13fff640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1410c360 0x7fdf1419e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf13fff640 1 -- 172.21.15.78:0/1796873218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf14101b20 msgr2=0x7fdf14199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf13fff640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf14101b20 0x7fdf14199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf13fff640 1 -- 172.21.15.78:0/1796873218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1419e660 con 0x7fdf14107a00 2024-05-21T15:20:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf191be640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1410c360 0x7fdf1419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf189bd640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf14101b20 0x7fdf14199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:40.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf13fff640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14107a00 0x7fdf14199d60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdf00005940 tx=0x7fdf000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf11ffb640 1 -- 172.21.15.78:0/1796873218 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0003f030 con 0x7fdf14107a00 2024-05-21T15:20:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf11ffb640 1 -- 172.21.15.78:0/1796873218 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf00003ce0 con 0x7fdf14107a00 2024-05-21T15:20:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf1ac21640 1 -- 172.21.15.78:0/1796873218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1419e8f0 con 0x7fdf14107a00 2024-05-21T15:20:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf11ffb640 1 -- 172.21.15.78:0/1796873218 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00035070 con 0x7fdf14107a00 2024-05-21T15:20:40.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.187+0000 7fdf1ac21640 1 -- 172.21.15.78:0/1796873218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1419ede0 con 0x7fdf14107a00 2024-05-21T15:20:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf11ffb640 1 -- 172.21.15.78:0/1796873218 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdf00042030 con 0x7fdf14107a00 2024-05-21T15:20:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf21b9d740 1 Processor -- start 2024-05-21T15:20:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf21b9d740 1 -- start start 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a2303390 0x55e8a26a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22da4d0 0x55e8a26a2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8a2607d00 con 0x55e8a2303390 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e8a25af4c0 con 0x55e8a22da4d0 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e8a2603cc0 con 0x55e8a22e26b0 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf191be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a2303390 0x55e8a26a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.190+0000 7fdf191be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a2303390 0x55e8a26a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52706/0 (socket says 172.21.15.78:52706) 2024-05-21T15:20:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf191be640 1 -- 172.21.15.78:0/3500241608 learned_addr learned my addr 172.21.15.78:0/3500241608 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf11ffb640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdf040a2d00 0x7fdf040a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf189bd640 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22da4d0 0x55e8a26a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf13fff640 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf191be640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdf040a2d00 0x7fdf040a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf11ffb640 1 -- 172.21.15.78:0/1796873218 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fdf000efad0 con 0x7fdf14107a00 2024-05-21T15:20:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf191be640 1 -- 172.21.15.78:0/3500241608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 msgr2=0x55e8a26a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf191be640 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26a27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf191be640 1 -- 172.21.15.78:0/3500241608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22da4d0 msgr2=0x55e8a26a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf191be640 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22da4d0 0x55e8a26a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf191be640 1 -- 172.21.15.78:0/3500241608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a23e57d0 con 0x55e8a2303390 2024-05-21T15:20:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdf191be640 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a2303390 0x55e8a26a1d30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c013950 tx=0x7fdf0c013e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202deb02cc88da87 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdefeffd640 1 -- 172.21.15.78:0/3500241608 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c013420 con 0x55e8a2303390 2024-05-21T15:20:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdefeffd640 1 -- 172.21.15.78:0/3500241608 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c011c40 con 0x55e8a2303390 2024-05-21T15:20:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.191+0000 7fdefeffd640 1 -- 172.21.15.78:0/3500241608 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c005260 con 0x55e8a2303390 2024-05-21T15:20:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3500241608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a2303390 msgr2=0x55e8a26a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf191be640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdf040a2d00 0x7fdf040a5210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c020000 tx=0x7fdf0c01f000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a2303390 0x55e8a26a1d30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c013950 tx=0x7fdf0c013e10 comp rx=0 tx=0).stop 2024-05-21T15:20:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3500241608 shutdown_connections 2024-05-21T15:20:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22da4d0 0x55e8a26a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26a27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3500241608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a2303390 0x55e8a26a1d30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3500241608 >> 172.21.15.78:0/3500241608 conn(0x55e8a22f74e0 msgr2=0x55e8a269e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3500241608 shutdown_connections 2024-05-21T15:20:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.192+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3500241608 wait complete. 2024-05-21T15:20:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 1 -- start start 2024-05-21T15:20:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 1 Processor -- start 2024-05-21T15:20:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26ade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8a26aefc0 con 0x55e8a22da4d0 2024-05-21T15:20:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e8a25c2b90 con 0x55e8a2303390 2024-05-21T15:20:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e8a26af140 con 0x55e8a22e26b0 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf21b9d740 10 client.0 _ll_register_callbacks cb 0x55e8a25feda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf189bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf189bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52722/0 (socket says 172.21.15.78:52722) 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf189bd640 1 -- 172.21.15.78:0/3658784888 learned_addr learned my addr 172.21.15.78:0/3658784888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.193+0000 7fdf191be640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf13fff640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf189bd640 1 -- 172.21.15.78:0/3658784888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 msgr2=0x55e8a26ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf189bd640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf189bd640 1 -- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 msgr2=0x55e8a26ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf189bd640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf189bd640 1 -- 172.21.15.78:0/3658784888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a231b3f0 con 0x55e8a22da4d0 2024-05-21T15:20:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf191be640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf13fff640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:20:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf189bd640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26ade60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800bf00 tx=0x7fdf08012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c923b28fe8b4ac6f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdef3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdef3fff640 1 -- 172.21.15.78:0/3658784888 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080035f0 con 0x55e8a22da4d0 2024-05-21T15:20:40.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdef3fff640 1 -- 172.21.15.78:0/3658784888 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fdf08003790 con 0x55e8a22da4d0 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3658784888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0c011c40 con 0x55e8a22da4d0 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdef3fff640 1 -- 172.21.15.78:0/3658784888 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdf08003990 con 0x55e8a22da4d0 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf21b9d740 10 client.16989 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3658784888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e8a22db0f0 con 0x55e8a22da4d0 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdef3fff640 1 -- 172.21.15.78:0/3658784888 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c011c40 con 0x55e8a22da4d0 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdef3fff640 1 -- 172.21.15.78:0/3658784888 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdf08019070 con 0x55e8a22da4d0 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf21b9d740 10 client.16989 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf21b9d740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:40.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.194+0000 7fdf21b9d740 20 client.16989 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.195+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8a22c1f60 0x55e8a26bed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.195+0000 7fdf21b9d740 4 client.16989 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:20:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.195+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3658784888 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8a2615430 con 0x55e8a22c1f60 2024-05-21T15:20:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.195+0000 7fdf13fff640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8a22c1f60 0x55e8a26bed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.195+0000 7fdef3fff640 10 client.16989 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.195+0000 7fdf13fff640 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8a22c1f60 0x55e8a26bed50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302a478d8ed6cc1b server_cookie=8ae6f1cc547fe4a3 in_seq=0 out_seq=0 2024-05-21T15:20:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdef3fff640 1 -- 172.21.15.78:0/3658784888 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e8a2615430 con 0x55e8a22c1f60 2024-05-21T15:20:40.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdef3fff640 10 client.16989 handle_command_reply: tid=0 2024-05-21T15:20:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 client.16989 shutdown 2024-05-21T15:20:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3658784888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 msgr2=0x55e8a26ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26ade60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800bf00 tx=0x7fdf08012820 comp rx=0 tx=0).stop 2024-05-21T15:20:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3658784888 shutdown_connections 2024-05-21T15:20:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8a22c1f60 0x55e8a26bed50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdef3fff640 0 client.16989 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3658784888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26ade60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3658784888 >> 172.21.15.78:0/3658784888 conn(0x55e8a22f74e0 msgr2=0x55e8a26a3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3658784888 shutdown_connections 2024-05-21T15:20:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3658784888 wait complete. 2024-05-21T15:20:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.196+0000 7fdf21b9d740 20 client.16989 trim_cache size 0 max 16384 2024-05-21T15:20:40.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf21b9d740 1 Processor -- start 2024-05-21T15:20:40.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf21b9d740 1 -- start start 2024-05-21T15:20:40.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22e26b0 0x55e8a26c4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a2303390 0x55e8a26c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141aec70 con 0x55e8a22da4d0 2024-05-21T15:20:40.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b0930 con 0x55e8a22e26b0 2024-05-21T15:20:40.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141af340 con 0x55e8a2303390 2024-05-21T15:20:40.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf189bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf189bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52726/0 (socket says 172.21.15.78:52726) 2024-05-21T15:20:40.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.214+0000 7fdf189bd640 1 -- 172.21.15.78:0/2587102255 learned_addr learned my addr 172.21.15.78:0/2587102255 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdf13fff640 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22e26b0 0x55e8a26c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdf191be640 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a2303390 0x55e8a26c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdf189bd640 1 -- 172.21.15.78:0/2587102255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a2303390 msgr2=0x55e8a26c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdf189bd640 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a2303390 0x55e8a26c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdf189bd640 1 -- 172.21.15.78:0/2587102255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22e26b0 msgr2=0x55e8a26c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdf189bd640 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22e26b0 0x55e8a26c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdf189bd640 1 -- 172.21.15.78:0/2587102255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a268f990 con 0x55e8a22da4d0 2024-05-21T15:20:40.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdf189bd640 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26a4200 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdf08004400 tx=0x7fdf08004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717efaecc91d8c2c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdefe7fc640 1 -- 172.21.15.78:0/2587102255 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080037f0 con 0x55e8a22da4d0 2024-05-21T15:20:40.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdefe7fc640 1 -- 172.21.15.78:0/2587102255 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf08005040 con 0x55e8a22da4d0 2024-05-21T15:20:40.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.215+0000 7fdefe7fc640 1 -- 172.21.15.78:0/2587102255 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080109d0 con 0x55e8a22da4d0 2024-05-21T15:20:40.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 -- 172.21.15.78:0/2587102255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 msgr2=0x55e8a26a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26a4200 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdf08004400 tx=0x7fdf08004430 comp rx=0 tx=0).stop 2024-05-21T15:20:40.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 -- 172.21.15.78:0/2587102255 shutdown_connections 2024-05-21T15:20:40.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a22e26b0 0x55e8a26c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a2303390 0x55e8a26c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/2587102255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26a4200 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 -- 172.21.15.78:0/2587102255 >> 172.21.15.78:0/2587102255 conn(0x55e8a22f74e0 msgr2=0x55e8a26ae150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 -- 172.21.15.78:0/2587102255 shutdown_connections 2024-05-21T15:20:40.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 -- 172.21.15.78:0/2587102255 wait complete. 2024-05-21T15:20:40.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 -- start start 2024-05-21T15:20:40.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.216+0000 7fdf21b9d740 1 Processor -- start 2024-05-21T15:20:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26b8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf21b9d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141b01a0 con 0x55e8a22da4d0 2024-05-21T15:20:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141afad0 con 0x55e8a2303390 2024-05-21T15:20:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf21b9d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141acc50 con 0x55e8a22e26b0 2024-05-21T15:20:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf21b9d740 10 client.0 _ll_register_callbacks cb 0x7fdf0800b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:20:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26b8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52730/0 (socket says 172.21.15.78:52730) 2024-05-21T15:20:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 -- 172.21.15.78:0/3450345191 learned_addr learned my addr 172.21.15.78:0/3450345191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:20:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf13fff640 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf191be640 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 -- 172.21.15.78:0/3450345191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 msgr2=0x55e8a26b8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26b8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 -- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 msgr2=0x55e8a26b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 -- 172.21.15.78:0/3450345191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a26b2270 con 0x55e8a22da4d0 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf191be640 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdf189bd640 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26b8880 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdf08010e30 tx=0x7fdf0800bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395ad8f091468526 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.217+0000 7fdef3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdef3fff640 1 -- 172.21.15.78:0/3450345191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080053f0 con 0x55e8a22da4d0 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdef3fff640 1 -- 172.21.15.78:0/3450345191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fdf08005590 con 0x55e8a22da4d0 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3450345191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf08005040 con 0x55e8a22da4d0 2024-05-21T15:20:40.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdef3fff640 1 -- 172.21.15.78:0/3450345191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdf08003e80 con 0x55e8a22da4d0 2024-05-21T15:20:40.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 10 client.16995 fetch_fsmap learned FSMap version 32 2024-05-21T15:20:40.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3450345191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e8a26b6d60 con 0x55e8a22da4d0 2024-05-21T15:20:40.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdef3fff640 1 -- 172.21.15.78:0/3450345191 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf08005040 con 0x55e8a22da4d0 2024-05-21T15:20:40.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdef3fff640 1 -- 172.21.15.78:0/3450345191 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdf08019070 con 0x55e8a22da4d0 2024-05-21T15:20:40.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 10 client.16995 fetch_fsmap finished waiting for FSMap version 32 2024-05-21T15:20:40.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:20:40.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 20 client.16995 populate_metadata read hostname 'smithi078' 2024-05-21T15:20:40.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8a22973a0 0x55e8a27290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:20:40.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 4 client.16995 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:20:40.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3450345191 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8a260f780 con 0x55e8a22973a0 2024-05-21T15:20:40.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.218+0000 7fdf13fff640 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8a22973a0 0x55e8a27290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:20:40.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdef3fff640 10 client.16995 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:40.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf13fff640 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8a22973a0 0x55e8a27290d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f4deba68c54ac server_cookie=2cdf406b36801141 in_seq=0 out_seq=0 2024-05-21T15:20:40.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdef3fff640 1 -- 172.21.15.78:0/3450345191 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e8a260f780 con 0x55e8a22973a0 2024-05-21T15:20:40.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdef3fff640 10 client.16995 handle_command_reply: tid=0 2024-05-21T15:20:40.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 client.16995 shutdown 2024-05-21T15:20:40.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3450345191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 msgr2=0x55e8a26b8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26b8880 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdf08010e30 tx=0x7fdf0800bad0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3450345191 shutdown_connections 2024-05-21T15:20:40.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e8a22973a0 0x55e8a27290d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e8a2303390 0x55e8a26b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e8a22e26b0 0x55e8a26b8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 --2- 172.21.15.78:0/3450345191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8a22da4d0 0x55e8a26b8880 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdef3fff640 0 client.16995 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:20:40.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3450345191 >> 172.21.15.78:0/3450345191 conn(0x55e8a22f74e0 msgr2=0x55e8a269cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3450345191 shutdown_connections 2024-05-21T15:20:40.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 1 -- 172.21.15.78:0/3450345191 wait complete. 2024-05-21T15:20:40.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.219+0000 7fdf21b9d740 20 client.16995 trim_cache size 0 max 16384 2024-05-21T15:20:40.233 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:20:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:20:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:20:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: "c9738c2b-92a7-4cb8-a3eb-33f123225731": { 2024-05-21T15:20:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:20:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c9738c2b-92a7-4cb8-a3eb-33f123225731", 2024-05-21T15:20:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:20:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:20:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:20:40.235 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:20:40.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 -- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdf040a2d00 msgr2=0x7fdf040a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdf040a2d00 0x7fdf040a5210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c020000 tx=0x7fdf0c01f000 comp rx=0 tx=0).stop 2024-05-21T15:20:40.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 -- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14107a00 msgr2=0x7fdf14199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:20:40.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14107a00 0x7fdf14199d60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdf00005940 tx=0x7fdf000042f0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 -- 172.21.15.78:0/1796873218 shutdown_connections 2024-05-21T15:20:40.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdf040a2d00 0x7fdf040a5210 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1410c360 0x7fdf1419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf14107a00 0x7fdf14199d60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 --2- 172.21.15.78:0/1796873218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf14101b20 0x7fdf14199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:20:40.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 -- 172.21.15.78:0/1796873218 >> 172.21.15.78:0/1796873218 conn(0x7fdf140770c0 msgr2=0x7fdf14073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:20:40.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 -- 172.21.15.78:0/1796873218 shutdown_connections 2024-05-21T15:20:40.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:20:40.220+0000 7fdf1ac21640 1 -- 172.21.15.78:0/1796873218 wait complete. 2024-05-21T15:20:40.238 INFO:tasks.cephfs.filesystem:scrub status for tag:c9738c2b-92a7-4cb8-a3eb-33f123225731 - {'path': '/', 'tag': 'c9738c2b-92a7-4cb8-a3eb-33f123225731', 'options': 'recursive,force'} 2024-05-21T15:20:40.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:40 smithi078 ceph-mon[31997]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 1.77k op/s 2024-05-21T15:20:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:40 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:20:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:40 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:40.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:40 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:40 smithi084 ceph-mon[36334]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 1.77k op/s 2024-05-21T15:20:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:40 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:20:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:40 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:40.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:40 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:40 smithi172 ceph-mon[36402]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 7.3 MiB/s wr, 1.77k op/s 2024-05-21T15:20:40.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:40 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:20:40.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:40 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:40.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:40 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:41.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:41.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:20:41.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.599 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:41 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:41.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:41.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:20:41.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:41 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:41.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:41.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:41.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:41.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:20:41.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:41.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:41 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:20:42.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:42 smithi078 ceph-mon[31997]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 1.78k op/s 2024-05-21T15:20:42.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:42 smithi084 ceph-mon[36334]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 1.78k op/s 2024-05-21T15:20:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:42 smithi172 ceph-mon[36402]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 1.78k op/s 2024-05-21T15:20:44.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:44 smithi078 ceph-mon[31997]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-05-21T15:20:44.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:44 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:20:44.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:44 smithi084 ceph-mon[36334]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-05-21T15:20:44.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:44 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:20:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:44 smithi172 ceph-mon[36402]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 60 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-05-21T15:20:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:44 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:20:46.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:46 smithi078 ceph-mon[31997]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 50 KiB/s rd, 6.3 MiB/s wr, 1.56k op/s 2024-05-21T15:20:46.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:46 smithi084 ceph-mon[36334]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 50 KiB/s rd, 6.3 MiB/s wr, 1.56k op/s 2024-05-21T15:20:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:46 smithi172 ceph-mon[36402]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 50 KiB/s rd, 6.3 MiB/s wr, 1.56k op/s 2024-05-21T15:20:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:47 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:47.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:47 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:47 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:47.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:47 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:47 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:47.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:47 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:48.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:48 smithi078 ceph-mon[31997]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 2.08k op/s 2024-05-21T15:20:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:48 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:48 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:48 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:20:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:48 smithi084 ceph-mon[36334]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 2.08k op/s 2024-05-21T15:20:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:48 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:48 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:48.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:48 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:20:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:48 smithi172 ceph-mon[36402]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 2.08k op/s 2024-05-21T15:20:48.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:48 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:48.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:48 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:48.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:48 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:20:50.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:50 smithi084 ceph-mon[36334]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:50.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:50 smithi172 ceph-mon[36402]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:50.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:50 smithi078 ceph-mon[31997]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:52.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:52 smithi084 ceph-mon[36334]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:52.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:52 smithi172 ceph-mon[36402]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:52 smithi078 ceph-mon[31997]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:54.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:54 smithi084 ceph-mon[36334]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 59 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:54.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:54 smithi172 ceph-mon[36402]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 59 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:54.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:54 smithi078 ceph-mon[31997]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 59 KiB/s rd, 6.9 MiB/s wr, 1.70k op/s 2024-05-21T15:20:56.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:56 smithi084 ceph-mon[36334]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 771 op/s 2024-05-21T15:20:56.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:56 smithi172 ceph-mon[36402]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 771 op/s 2024-05-21T15:20:56.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:56 smithi078 ceph-mon[31997]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 771 op/s 2024-05-21T15:20:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:58 smithi084 ceph-mon[36334]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 3.4 MiB/s wr, 837 op/s 2024-05-21T15:20:58.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:20:58 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:20:58.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:58 smithi172 ceph-mon[36402]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 3.4 MiB/s wr, 837 op/s 2024-05-21T15:20:58.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:20:58 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:20:58.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:58 smithi078 ceph-mon[31997]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 3.4 MiB/s wr, 837 op/s 2024-05-21T15:20:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:20:58 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:00.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:00 smithi084 ceph-mon[36334]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 336 op/s 2024-05-21T15:21:00.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:00 smithi172 ceph-mon[36402]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 336 op/s 2024-05-21T15:21:00.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:00 smithi078 ceph-mon[31997]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 336 op/s 2024-05-21T15:21:02.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:02 smithi084 ceph-mon[36334]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 395 KiB/s wr, 101 op/s 2024-05-21T15:21:02.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:02 smithi172 ceph-mon[36402]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 395 KiB/s wr, 101 op/s 2024-05-21T15:21:02.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:02 smithi078 ceph-mon[31997]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 395 KiB/s wr, 101 op/s 2024-05-21T15:21:03.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:03 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:03.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:03 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:03.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:03 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:04.645 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:04 smithi084 ceph-mon[36334]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 308 op/s 2024-05-21T15:21:04.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:04 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:04.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:04 smithi172 ceph-mon[36402]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 308 op/s 2024-05-21T15:21:04.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:04 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:04.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:04 smithi078 ceph-mon[31997]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 308 op/s 2024-05-21T15:21:04.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:04 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:06.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:06 smithi084 ceph-mon[36334]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 1.2 MiB/s wr, 302 op/s 2024-05-21T15:21:06.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:06 smithi172 ceph-mon[36402]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 1.2 MiB/s wr, 302 op/s 2024-05-21T15:21:06.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:06 smithi078 ceph-mon[31997]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 1.2 MiB/s wr, 302 op/s 2024-05-21T15:21:08.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:08 smithi172 ceph-mon[36402]: pgmap v154: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 1.6 MiB/s wr, 407 op/s 2024-05-21T15:21:08.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:08 smithi172 ceph-mon[36402]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:08.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:08 smithi078 ceph-mon[31997]: pgmap v154: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 1.6 MiB/s wr, 407 op/s 2024-05-21T15:21:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:08 smithi078 ceph-mon[31997]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:08.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:08 smithi084 ceph-mon[36334]: pgmap v154: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 1.6 MiB/s wr, 407 op/s 2024-05-21T15:21:08.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:08 smithi084 ceph-mon[36334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T15:21:10.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:21:10.239 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:21:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.361+0000 7efd6c26c640 1 -- 172.21.15.78:0/1167487513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd64102210 msgr2=0x7efd64102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.361+0000 7efd6c26c640 1 --2- 172.21.15.78:0/1167487513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd64102210 0x7efd64102620 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7efd60009900 tx=0x7efd6002a410 comp rx=0 tx=0).stop 2024-05-21T15:21:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.361+0000 7efd6c26c640 1 -- 172.21.15.78:0/1167487513 shutdown_connections 2024-05-21T15:21:10.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.361+0000 7efd6c26c640 1 --2- 172.21.15.78:0/1167487513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd6410c350 0x7efd6410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.361+0000 7efd6c26c640 1 --2- 172.21.15.78:0/1167487513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd64102210 0x7efd64102620 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.362+0000 7efd6c26c640 1 --2- 172.21.15.78:0/1167487513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd641079f0 0x7efd64101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.362+0000 7efd6c26c640 1 -- 172.21.15.78:0/1167487513 >> 172.21.15.78:0/1167487513 conn(0x7efd64076830 msgr2=0x7efd64076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.362+0000 7efd6c26c640 1 -- 172.21.15.78:0/1167487513 shutdown_connections 2024-05-21T15:21:10.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.362+0000 7efd6c26c640 1 -- 172.21.15.78:0/1167487513 wait complete. 2024-05-21T15:21:10.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.362+0000 7efd6c26c640 1 Processor -- start 2024-05-21T15:21:10.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.362+0000 7efd6c26c640 1 -- start start 2024-05-21T15:21:10.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6c26c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64102210 0x7efd6419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6c26c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd641079f0 0x7efd6419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6c26c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd6410c350 0x7efd6419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6c26c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd64111d00 con 0x7efd6410c350 2024-05-21T15:21:10.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6c26c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efd64111b80 con 0x7efd641079f0 2024-05-21T15:21:10.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6c26c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd64111e80 con 0x7efd64102210 2024-05-21T15:21:10.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd6410c350 0x7efd6419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd6410c350 0x7efd6419b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54068/0 (socket says 172.21.15.78:54068) 2024-05-21T15:21:10.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 -- 172.21.15.78:0/4000740183 learned_addr learned my addr 172.21.15.78:0/4000740183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd69807640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd641079f0 0x7efd6419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a008640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64102210 0x7efd6419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 -- 172.21.15.78:0/4000740183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64102210 msgr2=0x7efd6419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64102210 0x7efd6419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 -- 172.21.15.78:0/4000740183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd641079f0 msgr2=0x7efd6419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd641079f0 0x7efd6419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 -- 172.21.15.78:0/4000740183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd641a58e0 con 0x7efd6410c350 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd69807640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd641079f0 0x7efd6419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a008640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64102210 0x7efd6419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6a809640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd6410c350 0x7efd6419b0e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00eaa0 tx=0x7efd5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd577fe640 1 -- 172.21.15.78:0/4000740183 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00cdb0 con 0x7efd6410c350 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd577fe640 1 -- 172.21.15.78:0/4000740183 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd5c005020 con 0x7efd6410c350 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6c26c640 1 -- 172.21.15.78:0/4000740183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd641a5bd0 con 0x7efd6410c350 2024-05-21T15:21:10.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd577fe640 1 -- 172.21.15.78:0/4000740183 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0116b0 con 0x7efd6410c350 2024-05-21T15:21:10.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.363+0000 7efd6c26c640 1 -- 172.21.15.78:0/4000740183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641a60f0 con 0x7efd6410c350 2024-05-21T15:21:10.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.366+0000 7efd577fe640 1 -- 172.21.15.78:0/4000740183 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efd5c005560 con 0x7efd6410c350 2024-05-21T15:21:10.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.366+0000 7efd731eb740 1 Processor -- start 2024-05-21T15:21:10.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd731eb740 1 -- start start 2024-05-21T15:21:10.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd731eb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e195621390 0x55e1959bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd731eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1955f84d0 0x55e1959c0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd731eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1956006b0 0x55e1959c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd731eb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e195925d00 con 0x55e195621390 2024-05-21T15:21:10.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd731eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e1958cd4c0 con 0x55e1955f84d0 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd731eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e195921cc0 con 0x55e1956006b0 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd577fe640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efd500a2d00 0x7efd500a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e195621390 0x55e1959bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e195621390 0x55e1959bfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54082/0 (socket says 172.21.15.78:54082) 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 -- 172.21.15.78:0/2386387879 learned_addr learned my addr 172.21.15.78:0/2386387879 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd6a809640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efd500a2d00 0x7efd500a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd6a008640 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1955f84d0 0x55e1959c0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1956006b0 0x55e1959c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 -- 172.21.15.78:0/2386387879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1956006b0 msgr2=0x55e1959c07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1956006b0 0x55e1959c07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 -- 172.21.15.78:0/2386387879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1955f84d0 msgr2=0x55e1959c0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1955f84d0 0x55e1959c0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd69807640 1 -- 172.21.15.78:0/2386387879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1957037d0 con 0x55e195621390 2024-05-21T15:21:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd577fe640 1 -- 172.21.15.78:0/4000740183 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7efd5c0cf740 con 0x7efd6410c350 2024-05-21T15:21:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.367+0000 7efd6a008640 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1955f84d0 0x55e1959c0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.368+0000 7efd69807640 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e195621390 0x55e1959bfd30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7efd60000c00 tx=0x7efd60009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65de785749e4fee2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.368+0000 7efd6a809640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efd500a2d00 0x7efd500a5210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ccee0 tx=0x7efd5c06dbc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.368+0000 7efd54bfe640 1 -- 172.21.15.78:0/2386387879 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60035050 con 0x55e195621390 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.368+0000 7efd54bfe640 1 -- 172.21.15.78:0/2386387879 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd60004030 con 0x55e195621390 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.368+0000 7efd54bfe640 1 -- 172.21.15.78:0/2386387879 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60005620 con 0x55e195621390 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 -- 172.21.15.78:0/2386387879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e195621390 msgr2=0x55e1959bfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e195621390 0x55e1959bfd30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7efd60000c00 tx=0x7efd60009560 comp rx=0 tx=0).stop 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 -- 172.21.15.78:0/2386387879 shutdown_connections 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1955f84d0 0x55e1959c0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1956006b0 0x55e1959c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 --2- 172.21.15.78:0/2386387879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e195621390 0x55e1959bfd30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 -- 172.21.15.78:0/2386387879 >> 172.21.15.78:0/2386387879 conn(0x55e1956154e0 msgr2=0x55e1959bc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 -- 172.21.15.78:0/2386387879 shutdown_connections 2024-05-21T15:21:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 -- 172.21.15.78:0/2386387879 wait complete. 2024-05-21T15:21:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 -- start start 2024-05-21T15:21:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.369+0000 7efd731eb740 1 Processor -- start 2024-05-21T15:21:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd731eb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955f84d0 0x55e1959cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd731eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd731eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e195621390 0x55e1959ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd731eb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1959ccfc0 con 0x55e1955f84d0 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd731eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e1958e0b90 con 0x55e1956006b0 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd731eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e1959cd140 con 0x55e195621390 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd731eb740 10 client.0 _ll_register_callbacks cb 0x55e19591cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955f84d0 0x55e1959cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955f84d0 0x55e1959cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54084/0 (socket says 172.21.15.78:54084) 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 -- 172.21.15.78:0/1991363746 learned_addr learned my addr 172.21.15.78:0/1991363746 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd6a008640 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e195621390 0x55e1959ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 -- 172.21.15.78:0/1991363746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e195621390 msgr2=0x55e1959ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e195621390 0x55e1959ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 -- 172.21.15.78:0/1991363746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 msgr2=0x55e1959cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 -- 172.21.15.78:0/1991363746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1956393f0 con 0x55e1955f84d0 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd69807640 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955f84d0 0x55e1959cbec0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7efd6002a930 tx=0x7efd60005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab15bc9b93da5e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd4dffb640 1 -- 172.21.15.78:0/1991363746 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60051060 con 0x55e1955f84d0 2024-05-21T15:21:10.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd4dffb640 1 -- 172.21.15.78:0/1991363746 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7efd600035f0 con 0x55e1955f84d0 2024-05-21T15:21:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.370+0000 7efd731eb740 1 -- 172.21.15.78:0/1991363746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd60004030 con 0x55e1955f84d0 2024-05-21T15:21:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd4dffb640 1 -- 172.21.15.78:0/1991363746 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efd60035040 con 0x55e1955f84d0 2024-05-21T15:21:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd731eb740 10 client.17007 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd731eb740 1 -- 172.21.15.78:0/1991363746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e1955f90f0 con 0x55e1955f84d0 2024-05-21T15:21:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd4dffb640 1 -- 172.21.15.78:0/1991363746 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd60004030 con 0x55e1955f84d0 2024-05-21T15:21:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd4dffb640 1 -- 172.21.15.78:0/1991363746 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7efd600548b0 con 0x55e1955f84d0 2024-05-21T15:21:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd731eb740 10 client.17007 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:10.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd731eb740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd731eb740 20 client.17007 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd731eb740 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e1955dff60 0x55e1959dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd731eb740 4 client.17007 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd731eb740 1 -- 172.21.15.78:0/1991363746 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e195933430 con 0x55e1955dff60 2024-05-21T15:21:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.371+0000 7efd6a008640 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e1955dff60 0x55e1959dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.372+0000 7efd4dffb640 10 client.17007 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.372+0000 7efd6a008640 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e1955dff60 0x55e1959dca30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5e8807bef236ac server_cookie=f7a1ff9bfaa4873d in_seq=0 out_seq=0 2024-05-21T15:21:10.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd4dffb640 1 -- 172.21.15.78:0/1991363746 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e195933430 con 0x55e1955dff60 2024-05-21T15:21:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd4dffb640 10 client.17007 handle_command_reply: tid=0 2024-05-21T15:21:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 client.17007 shutdown 2024-05-21T15:21:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 -- 172.21.15.78:0/1991363746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955f84d0 msgr2=0x55e1959cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955f84d0 0x55e1959cbec0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7efd6002a930 tx=0x7efd60005ed0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 -- 172.21.15.78:0/1991363746 shutdown_connections 2024-05-21T15:21:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e1955dff60 0x55e1959dca30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e195621390 0x55e1959ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd4dffb640 0 client.17007 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:10.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 --2- 172.21.15.78:0/1991363746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955f84d0 0x55e1959cbec0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 -- 172.21.15.78:0/1991363746 >> 172.21.15.78:0/1991363746 conn(0x55e1956154e0 msgr2=0x55e1959c1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 -- 172.21.15.78:0/1991363746 shutdown_connections 2024-05-21T15:21:10.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 1 -- 172.21.15.78:0/1991363746 wait complete. 2024-05-21T15:21:10.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.373+0000 7efd731eb740 20 client.17007 trim_cache size 0 max 16384 2024-05-21T15:21:10.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd731eb740 1 Processor -- start 2024-05-21T15:21:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd731eb740 1 -- start start 2024-05-21T15:21:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd731eb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e1959cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd731eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 0x55e1959dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd731eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd731eb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd641aea20 con 0x55e1955dff60 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd731eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efd641b0560 con 0x55e1956006b0 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd731eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd641af0f0 con 0x55e1955f84d0 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e1959cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e1959cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54098/0 (socket says 172.21.15.78:54098) 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 -- 172.21.15.78:0/1271696372 learned_addr learned my addr 172.21.15.78:0/1271696372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 -- 172.21.15.78:0/1271696372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 msgr2=0x55e1959dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 --2- 172.21.15.78:0/1271696372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 0x55e1959dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 -- 172.21.15.78:0/1271696372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 msgr2=0x55e1959dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 --2- 172.21.15.78:0/1271696372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 -- 172.21.15.78:0/1271696372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1959bd0d0 con 0x55e1955dff60 2024-05-21T15:21:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd69807640 1 --2- 172.21.15.78:0/1271696372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e1959cb8d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7efd60054cb0 tx=0x7efd60009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceafe0963ec711db server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.391+0000 7efd4ffff640 1 -- 172.21.15.78:0/1271696372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60051040 con 0x55e1955dff60 2024-05-21T15:21:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd4ffff640 1 -- 172.21.15.78:0/1271696372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd60031d80 con 0x55e1955dff60 2024-05-21T15:21:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd4ffff640 1 -- 172.21.15.78:0/1271696372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6004a690 con 0x55e1955dff60 2024-05-21T15:21:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 -- 172.21.15.78:0/1271696372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 msgr2=0x55e1959cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 --2- 172.21.15.78:0/1271696372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e1959cb8d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7efd60054cb0 tx=0x7efd60009520 comp rx=0 tx=0).stop 2024-05-21T15:21:10.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 -- 172.21.15.78:0/1271696372 shutdown_connections 2024-05-21T15:21:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 --2- 172.21.15.78:0/1271696372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 --2- 172.21.15.78:0/1271696372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 0x55e1959dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 --2- 172.21.15.78:0/1271696372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e1959cb8d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 -- 172.21.15.78:0/1271696372 >> 172.21.15.78:0/1271696372 conn(0x55e1956154e0 msgr2=0x55e1959c0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 -- 172.21.15.78:0/1271696372 shutdown_connections 2024-05-21T15:21:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.392+0000 7efd731eb740 1 -- 172.21.15.78:0/1271696372 wait complete. 2024-05-21T15:21:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 1 -- start start 2024-05-21T15:21:10.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 1 Processor -- start 2024-05-21T15:21:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e195a3c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 0x55e195a3c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd641afe90 con 0x55e1955dff60 2024-05-21T15:21:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efd641af7c0 con 0x55e1956006b0 2024-05-21T15:21:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd641aca00 con 0x55e1955f84d0 2024-05-21T15:21:10.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd731eb740 10 client.0 _ll_register_callbacks cb 0x7efd60032ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e195a3c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e195a3c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54104/0 (socket says 172.21.15.78:54104) 2024-05-21T15:21:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 -- 172.21.15.78:0/2359798768 learned_addr learned my addr 172.21.15.78:0/2359798768 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd6a008640 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 0x55e195a3c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 -- 172.21.15.78:0/2359798768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 msgr2=0x55e195a3c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 0x55e195a3c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 -- 172.21.15.78:0/2359798768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 msgr2=0x55e1959df090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 -- 172.21.15.78:0/2359798768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1959df680 con 0x55e1955dff60 2024-05-21T15:21:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd69807640 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e195a3c010 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7efd6002a930 tx=0x7efd60032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d4e7d3e22f36a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.393+0000 7efd4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd4dffb640 1 -- 172.21.15.78:0/2359798768 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60062030 con 0x55e1955dff60 2024-05-21T15:21:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd4dffb640 1 -- 172.21.15.78:0/2359798768 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7efd60044040 con 0x55e1955dff60 2024-05-21T15:21:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 1 -- 172.21.15.78:0/2359798768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd60031d80 con 0x55e1955dff60 2024-05-21T15:21:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd4dffb640 1 -- 172.21.15.78:0/2359798768 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efd60051040 con 0x55e1955dff60 2024-05-21T15:21:10.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 10 client.17013 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 1 -- 172.21.15.78:0/2359798768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e195a44750 con 0x55e1955dff60 2024-05-21T15:21:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd4dffb640 1 -- 172.21.15.78:0/2359798768 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd60031d80 con 0x55e1955dff60 2024-05-21T15:21:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd4dffb640 1 -- 172.21.15.78:0/2359798768 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7efd6005c070 con 0x55e1955dff60 2024-05-21T15:21:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 10 client.17013 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 20 client.17013 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e195621390 0x55e195a46e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 4 client.17013 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd731eb740 1 -- 172.21.15.78:0/2359798768 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e19592d780 con 0x55e195621390 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.394+0000 7efd6a008640 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e195621390 0x55e195a46e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd6a008640 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e195621390 0x55e195a46e00 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caefb70153f3c9f0 server_cookie=1f491dec75730d9b in_seq=0 out_seq=0 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd4dffb640 10 client.17013 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd4dffb640 1 -- 172.21.15.78:0/2359798768 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e19592d780 con 0x55e195621390 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd4dffb640 10 client.17013 handle_command_reply: tid=0 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 client.17013 shutdown 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 -- 172.21.15.78:0/2359798768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 msgr2=0x55e195a3c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e195a3c010 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7efd6002a930 tx=0x7efd60032d90 comp rx=0 tx=0).stop 2024-05-21T15:21:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 -- 172.21.15.78:0/2359798768 shutdown_connections 2024-05-21T15:21:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e195621390 0x55e195a46e00 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1956006b0 0x55e1959df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1955f84d0 0x55e195a3c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 --2- 172.21.15.78:0/2359798768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1955dff60 0x55e195a3c010 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd4dffb640 0 client.17013 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 -- 172.21.15.78:0/2359798768 >> 172.21.15.78:0/2359798768 conn(0x55e1956154e0 msgr2=0x55e1959c03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 -- 172.21.15.78:0/2359798768 shutdown_connections 2024-05-21T15:21:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 1 -- 172.21.15.78:0/2359798768 wait complete. 2024-05-21T15:21:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.395+0000 7efd731eb740 20 client.17013 trim_cache size 0 max 16384 2024-05-21T15:21:10.411 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:21:10.411 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:21:10.411 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:21:10.411 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:21:10.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 -- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efd500a2d00 msgr2=0x7efd500a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efd500a2d00 0x7efd500a5210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0ccee0 tx=0x7efd5c06dbc0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 -- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd6410c350 msgr2=0x7efd6419b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd6410c350 0x7efd6419b0e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00eaa0 tx=0x7efd5c00ef60 comp rx=0 tx=0).stop 2024-05-21T15:21:10.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 -- 172.21.15.78:0/4000740183 shutdown_connections 2024-05-21T15:21:10.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd641079f0 0x7efd6419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efd500a2d00 0x7efd500a5210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd6410c350 0x7efd6419b0e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 --2- 172.21.15.78:0/4000740183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd64102210 0x7efd6419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 -- 172.21.15.78:0/4000740183 >> 172.21.15.78:0/4000740183 conn(0x7efd64076830 msgr2=0x7efd641030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 -- 172.21.15.78:0/4000740183 shutdown_connections 2024-05-21T15:21:10.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.396+0000 7efd6c26c640 1 -- 172.21.15.78:0/4000740183 wait complete. 2024-05-21T15:21:10.414 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:21:10.415 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:21:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 -- 172.21.15.78:0/3611323567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08010c330 msgr2=0x7fa08010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 --2- 172.21.15.78:0/3611323567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08010c330 0x7fa08010e750 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa078009ca0 tx=0x7fa07802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 -- 172.21.15.78:0/3611323567 shutdown_connections 2024-05-21T15:21:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 --2- 172.21.15.78:0/3611323567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa080101aa0 0x7fa080101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 --2- 172.21.15.78:0/3611323567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa08010c330 0x7fa08010e750 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 --2- 172.21.15.78:0/3611323567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0801079d0 0x7fa080101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 -- 172.21.15.78:0/3611323567 >> 172.21.15.78:0/3611323567 conn(0x7fa0800770c0 msgr2=0x7fa080077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 -- 172.21.15.78:0/3611323567 shutdown_connections 2024-05-21T15:21:10.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.523+0000 7fa0886af640 1 -- 172.21.15.78:0/3611323567 wait complete. 2024-05-21T15:21:10.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa0886af640 1 Processor -- start 2024-05-21T15:21:10.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa0886af640 1 -- start start 2024-05-21T15:21:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa0886af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa080101aa0 0x7fa0801985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa0886af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0801079d0 0x7fa080198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa0886af640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08010c330 0x7fa08019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa0886af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa080111d00 con 0x7fa080101aa0 2024-05-21T15:21:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa0886af640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa080111b80 con 0x7fa0801079d0 2024-05-21T15:21:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa0886af640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa080111e80 con 0x7fa08010c330 2024-05-21T15:21:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa08644b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa080101aa0 0x7fa0801985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa08644b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa080101aa0 0x7fa0801985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54118/0 (socket says 172.21.15.78:54118) 2024-05-21T15:21:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.524+0000 7fa08644b640 1 -- 172.21.15.78:0/304416213 learned_addr learned my addr 172.21.15.78:0/304416213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa08644b640 1 -- 172.21.15.78:0/304416213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08010c330 msgr2=0x7fa08019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa08644b640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08010c330 0x7fa08019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa08644b640 1 -- 172.21.15.78:0/304416213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0801079d0 msgr2=0x7fa080198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa08644b640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0801079d0 0x7fa080198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa08644b640 1 -- 172.21.15.78:0/304416213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08019d440 con 0x7fa080101aa0 2024-05-21T15:21:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa08644b640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa080101aa0 0x7fa0801985e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa07400eb00 tx=0x7fa074009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa0737fe640 1 -- 172.21.15.78:0/304416213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07400bc30 con 0x7fa080101aa0 2024-05-21T15:21:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa0737fe640 1 -- 172.21.15.78:0/304416213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa07400bdd0 con 0x7fa080101aa0 2024-05-21T15:21:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08019d730 con 0x7fa080101aa0 2024-05-21T15:21:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa0737fe640 1 -- 172.21.15.78:0/304416213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074017410 con 0x7fa080101aa0 2024-05-21T15:21:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.525+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa080068e40 con 0x7fa080101aa0 2024-05-21T15:21:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.526+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa080102c80 con 0x7fa080101aa0 2024-05-21T15:21:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.527+0000 7fa0737fe640 1 -- 172.21.15.78:0/304416213 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa074005020 con 0x7fa080101aa0 2024-05-21T15:21:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.528+0000 7fa0737fe640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa06c0a2d80 0x7fa06c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.528+0000 7fa085c4a640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa06c0a2d80 0x7fa06c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.528+0000 7fa0737fe640 1 -- 172.21.15.78:0/304416213 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fa0740d0ea0 con 0x7fa080101aa0 2024-05-21T15:21:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.528+0000 7fa085c4a640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa06c0a2d80 0x7fa06c0a5290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005fd0 tx=0x7fa07c00b410 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.533+0000 7fa0737fe640 1 -- 172.21.15.78:0/304416213 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa07409da60 con 0x7fa080101aa0 2024-05-21T15:21:10.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:10 smithi078 ceph-mon[31997]: pgmap v155: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-05-21T15:21:10.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:10 smithi172 ceph-mon[36402]: pgmap v155: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-05-21T15:21:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.747+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa080105b00 con 0x7fa080101aa0 2024-05-21T15:21:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.747+0000 7fa0737fe640 1 -- 172.21.15.78:0/304416213 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7fa07409d360 con 0x7fa080101aa0 2024-05-21T15:21:10.752 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:21:10.752 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:21:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:21:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa06c0a2d80 msgr2=0x7fa06c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa06c0a2d80 0x7fa06c0a5290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005fd0 tx=0x7fa07c00b410 comp rx=0 tx=0).stop 2024-05-21T15:21:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa080101aa0 msgr2=0x7fa0801985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa080101aa0 0x7fa0801985e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa07400eb00 tx=0x7fa074009ee0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 shutdown_connections 2024-05-21T15:21:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0801079d0 0x7fa080198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa06c0a2d80 0x7fa06c0a5290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08010c330 0x7fa08019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 --2- 172.21.15.78:0/304416213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa080101aa0 0x7fa0801985e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 >> 172.21.15.78:0/304416213 conn(0x7fa0800770c0 msgr2=0x7fa080103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 shutdown_connections 2024-05-21T15:21:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.749+0000 7fa0886af640 1 -- 172.21.15.78:0/304416213 wait complete. 2024-05-21T15:21:10.765 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:21:10.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 -- 172.21.15.78:0/3585802746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54410d280 msgr2=0x7fe54410f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 --2- 172.21.15.78:0/3585802746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54410d280 0x7fe54410f6a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009ca0 tx=0x7fe53c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 -- 172.21.15.78:0/3585802746 shutdown_connections 2024-05-21T15:21:10.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 --2- 172.21.15.78:0/3585802746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5441029f0 0x7fe544102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 --2- 172.21.15.78:0/3585802746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe54410d280 0x7fe54410f6a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 --2- 172.21.15.78:0/3585802746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe544108920 0x7fe5441024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 -- 172.21.15.78:0/3585802746 >> 172.21.15.78:0/3585802746 conn(0x7fe5440fdf70 msgr2=0x7fe5441003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 -- 172.21.15.78:0/3585802746 shutdown_connections 2024-05-21T15:21:10.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.874+0000 7fe54939d640 1 -- 172.21.15.78:0/3585802746 wait complete. 2024-05-21T15:21:10.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe54939d640 1 Processor -- start 2024-05-21T15:21:10.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe54939d640 1 -- start start 2024-05-21T15:21:10.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe54939d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5441029f0 0x7fe5441995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe54939d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe544108920 0x7fe544199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe54939d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe54410d280 0x7fe54419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe54939d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe544112c30 con 0x7fe544108920 2024-05-21T15:21:10.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe54939d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe544112ab0 con 0x7fe5441029f0 2024-05-21T15:21:10.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe54939d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe544112db0 con 0x7fe54410d280 2024-05-21T15:21:10.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe544108920 0x7fe544199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe544108920 0x7fe544199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54130/0 (socket says 172.21.15.78:54130) 2024-05-21T15:21:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe53bfff640 1 -- 172.21.15.78:0/2045153761 learned_addr learned my addr 172.21.15.78:0/2045153761 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.875+0000 7fe548b9c640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe54410d280 0x7fe54419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe543fff640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5441029f0 0x7fe5441995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe53bfff640 1 -- 172.21.15.78:0/2045153761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe54410d280 msgr2=0x7fe54419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe53bfff640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe54410d280 0x7fe54419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe53bfff640 1 -- 172.21.15.78:0/2045153761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5441029f0 msgr2=0x7fe5441995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe53bfff640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5441029f0 0x7fe5441995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe53bfff640 1 -- 172.21.15.78:0/2045153761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54419e410 con 0x7fe544108920 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe543fff640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5441029f0 0x7fe5441995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe53bfff640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe544108920 0x7fe544199b10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c970 tx=0x7fe52c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe541ffb640 1 -- 172.21.15.78:0/2045153761 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c004220 con 0x7fe544108920 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe541ffb640 1 -- 172.21.15.78:0/2045153761 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52c0043c0 con 0x7fe544108920 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe54939d640 1 -- 172.21.15.78:0/2045153761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54419e700 con 0x7fe544108920 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe541ffb640 1 -- 172.21.15.78:0/2045153761 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00f450 con 0x7fe544108920 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.876+0000 7fe54939d640 1 -- 172.21.15.78:0/2045153761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5441a6080 con 0x7fe544108920 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.878+0000 7fe55157f740 1 Processor -- start 2024-05-21T15:21:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe55157f740 1 -- start start 2024-05-21T15:21:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe541ffb640 1 -- 172.21.15.78:0/2045153761 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe52c00f5f0 con 0x7fe544108920 2024-05-21T15:21:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe55157f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596d0390 0x55d459a6ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe55157f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 0x55d459a6f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe55157f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 0x55d459a6f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe55157f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d4599d4c40 con 0x55d4596d0390 2024-05-21T15:21:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe55157f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d45997c580 con 0x55d4596af6b0 2024-05-21T15:21:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe55157f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4599d0c00 con 0x55d4596a74d0 2024-05-21T15:21:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596d0390 0x55d459a6ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596d0390 0x55d459a6ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54136/0 (socket says 172.21.15.78:54136) 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 -- 172.21.15.78:0/724657562 learned_addr learned my addr 172.21.15.78:0/724657562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe543fff640 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 0x55d459a6f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe53bfff640 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 0x55d459a6f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 -- 172.21.15.78:0/724657562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 msgr2=0x55d459a6f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 0x55d459a6f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 -- 172.21.15.78:0/724657562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 msgr2=0x55d459a6f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 0x55d459a6f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 -- 172.21.15.78:0/724657562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4596e83f0 con 0x55d4596d0390 2024-05-21T15:21:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe548b9c640 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596d0390 0x55d459a6ef60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009ca0 tx=0x7fe53c02f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0e0c12b640ba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.879+0000 7fe541ffb640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe5300a2d70 0x7fe5300a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe53a7fc640 1 -- 172.21.15.78:0/724657562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c031db0 con 0x55d4596d0390 2024-05-21T15:21:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe543fff640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe5300a2d70 0x7fe5300a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe541ffb640 1 -- 172.21.15.78:0/2045153761 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe52c0cf510 con 0x7fe544108920 2024-05-21T15:21:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe53a7fc640 1 -- 172.21.15.78:0/724657562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe53c036040 con 0x55d4596d0390 2024-05-21T15:21:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe53a7fc640 1 -- 172.21.15.78:0/724657562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c004330 con 0x55d4596d0390 2024-05-21T15:21:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe55157f740 1 -- 172.21.15.78:0/724657562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596d0390 msgr2=0x55d459a6ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe55157f740 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596d0390 0x55d459a6ef60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009ca0 tx=0x7fe53c02f4f0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe543fff640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe5300a2d70 0x7fe5300a5280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55d459a70520 tx=0x7fe534009450 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe55157f740 1 -- 172.21.15.78:0/724657562 shutdown_connections 2024-05-21T15:21:10.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe55157f740 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 0x55d459a6f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe55157f740 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 0x55d459a6f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe55157f740 1 --2- 172.21.15.78:0/724657562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596d0390 0x55d459a6ef60 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.880+0000 7fe55157f740 1 -- 172.21.15.78:0/724657562 >> 172.21.15.78:0/724657562 conn(0x55d4596c44e0 msgr2=0x55d459a6b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.881+0000 7fe55157f740 1 -- 172.21.15.78:0/724657562 shutdown_connections 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.881+0000 7fe55157f740 1 -- 172.21.15.78:0/724657562 wait complete. 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.881+0000 7fe55157f740 1 -- start start 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.881+0000 7fe55157f740 1 Processor -- start 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.881+0000 7fe55157f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 0x55d459a795a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.881+0000 7fe55157f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a79ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.881+0000 7fe55157f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596d0390 0x55d459a7f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe55157f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d459a7c250 con 0x55d4596af6b0 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe55157f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d45998fc40 con 0x55d4596d0390 2024-05-21T15:21:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe55157f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d459a7c3d0 con 0x55d4596a74d0 2024-05-21T15:21:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe55157f740 10 client.0 _ll_register_callbacks cb 0x55d4596e7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a79ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a79ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54140/0 (socket says 172.21.15.78:54140) 2024-05-21T15:21:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 -- 172.21.15.78:0/177050019 learned_addr learned my addr 172.21.15.78:0/177050019 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe543fff640 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 0x55d459a795a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe548b9c640 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596d0390 0x55d459a7f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 -- 172.21.15.78:0/177050019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 msgr2=0x55d459a795a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 0x55d459a795a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 -- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596d0390 msgr2=0x55d459a7f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596d0390 0x55d459a7f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 -- 172.21.15.78:0/177050019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4597b27d0 con 0x55d4596af6b0 2024-05-21T15:21:10.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe548b9c640 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596d0390 0x55d459a7f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:10.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe53bfff640 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a79ae0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c940 tx=0x7fe52c09b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8369c6651d1b18f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe5237fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:10.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe5237fe640 1 -- 172.21.15.78:0/177050019 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c09fd80 con 0x55d4596af6b0 2024-05-21T15:21:10.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe5237fe640 1 -- 172.21.15.78:0/177050019 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe52c098040 con 0x55d4596af6b0 2024-05-21T15:21:10.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe55157f740 1 -- 172.21.15.78:0/177050019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe53c036040 con 0x55d4596af6b0 2024-05-21T15:21:10.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe5237fe640 1 -- 172.21.15.78:0/177050019 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe52c0a4640 con 0x55d4596af6b0 2024-05-21T15:21:10.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe55157f740 10 client.17031 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:10.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.882+0000 7fe55157f740 1 -- 172.21.15.78:0/177050019 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d459a889e0 con 0x55d4596af6b0 2024-05-21T15:21:10.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe5237fe640 1 -- 172.21.15.78:0/177050019 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe53c036040 con 0x55d4596af6b0 2024-05-21T15:21:10.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe5237fe640 1 -- 172.21.15.78:0/177050019 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe52c0860b0 con 0x55d4596af6b0 2024-05-21T15:21:10.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe55157f740 10 client.17031 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:10.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe55157f740 10 client.17031 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:21:10.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe55157f740 20 client.17031 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:10.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe55157f740 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55d45968ef60 0x55d459a8bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe55157f740 4 client.17031 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:10.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe55157f740 1 -- 172.21.15.78:0/177050019 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4599e2370 con 0x55d45968ef60 2024-05-21T15:21:10.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.883+0000 7fe543fff640 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55d45968ef60 0x55d459a8bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe5237fe640 10 client.17031 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:21:10.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe543fff640 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55d45968ef60 0x55d459a8bed0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=369b83715d3aa0d server_cookie=bcdeb4ab8ed70ca5 in_seq=0 out_seq=0 2024-05-21T15:21:10.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe5237fe640 1 -- 172.21.15.78:0/177050019 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d4599e2370 con 0x55d45968ef60 2024-05-21T15:21:10.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe5237fe640 10 client.17031 handle_command_reply: tid=0 2024-05-21T15:21:10.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe55157f740 1 client.17031 shutdown 2024-05-21T15:21:10.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe55157f740 1 -- 172.21.15.78:0/177050019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 msgr2=0x55d459a79ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe55157f740 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a79ae0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c940 tx=0x7fe52c09b5d0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe55157f740 1 -- 172.21.15.78:0/177050019 shutdown_connections 2024-05-21T15:21:10.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.884+0000 7fe55157f740 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55d45968ef60 0x55d459a8bed0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.885+0000 7fe55157f740 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596d0390 0x55d459a7f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.885+0000 7fe55157f740 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a79ae0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.885+0000 7fe5237fe640 0 client.17031 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:21:10.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.885+0000 7fe55157f740 1 --2- 172.21.15.78:0/177050019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596a74d0 0x55d459a795a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.885+0000 7fe55157f740 1 -- 172.21.15.78:0/177050019 >> 172.21.15.78:0/177050019 conn(0x55d4596c44e0 msgr2=0x55d459a6ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.885+0000 7fe55157f740 1 -- 172.21.15.78:0/177050019 shutdown_connections 2024-05-21T15:21:10.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.885+0000 7fe55157f740 1 -- 172.21.15.78:0/177050019 wait complete. 2024-05-21T15:21:10.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.885+0000 7fe55157f740 20 client.17031 trim_cache size 0 max 16384 2024-05-21T15:21:10.903 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:10 smithi084 ceph-mon[36334]: pgmap v155: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-05-21T15:21:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe55157f740 1 Processor -- start 2024-05-21T15:21:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe55157f740 1 -- start start 2024-05-21T15:21:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe55157f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596a74d0 0x55d459a7b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe55157f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a7b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe55157f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 0x55d459a91640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe55157f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5441ae7e0 con 0x55d4596af6b0 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe55157f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5441b0320 con 0x55d4596a74d0 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe55157f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5441aeeb0 con 0x55d4596d0390 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a7b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a7b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54144/0 (socket says 172.21.15.78:54144) 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.898+0000 7fe53bfff640 1 -- 172.21.15.78:0/3926163971 learned_addr learned my addr 172.21.15.78:0/3926163971 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe543fff640 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596a74d0 0x55d459a7b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe53bfff640 1 -- 172.21.15.78:0/3926163971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 msgr2=0x55d459a91640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe53bfff640 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 0x55d459a91640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe53bfff640 1 -- 172.21.15.78:0/3926163971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596a74d0 msgr2=0x55d459a7b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe53bfff640 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596a74d0 0x55d459a7b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe53bfff640 1 -- 172.21.15.78:0/3926163971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d459a70550 con 0x55d4596af6b0 2024-05-21T15:21:10.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe543fff640 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596a74d0 0x55d459a7b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:10.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe53bfff640 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a7b700 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe52c097e30 tx=0x7fe52c099ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0cdbdf243d08b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe539ffb640 1 -- 172.21.15.78:0/3926163971 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0860e0 con 0x55d4596af6b0 2024-05-21T15:21:10.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe539ffb640 1 -- 172.21.15.78:0/3926163971 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52c098040 con 0x55d4596af6b0 2024-05-21T15:21:10.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.899+0000 7fe539ffb640 1 -- 172.21.15.78:0/3926163971 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0cf2a0 con 0x55d4596af6b0 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 -- 172.21.15.78:0/3926163971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 msgr2=0x55d459a7b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a7b700 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe52c097e30 tx=0x7fe52c099ee0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 -- 172.21.15.78:0/3926163971 shutdown_connections 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 0x55d459a91640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596af6b0 0x55d459a7b700 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 --2- 172.21.15.78:0/3926163971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596a74d0 0x55d459a7b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 -- 172.21.15.78:0/3926163971 >> 172.21.15.78:0/3926163971 conn(0x55d4596c44e0 msgr2=0x55d459a7aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 -- 172.21.15.78:0/3926163971 shutdown_connections 2024-05-21T15:21:10.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 -- 172.21.15.78:0/3926163971 wait complete. 2024-05-21T15:21:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 -- start start 2024-05-21T15:21:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.900+0000 7fe55157f740 1 Processor -- start 2024-05-21T15:21:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe55157f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596a74d0 0x55d459a83d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe55157f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 0x55d459a842d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe55157f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 0x55d459a84810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe55157f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5441afc50 con 0x55d4596a74d0 2024-05-21T15:21:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe55157f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5441af580 con 0x55d4596af6b0 2024-05-21T15:21:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe55157f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5441ac7c0 con 0x55d4596d0390 2024-05-21T15:21:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe55157f740 10 client.0 _ll_register_callbacks cb 0x7fe52c003910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596a74d0 0x55d459a83d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596a74d0 0x55d459a83d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54152/0 (socket says 172.21.15.78:54152) 2024-05-21T15:21:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 -- 172.21.15.78:0/418166977 learned_addr learned my addr 172.21.15.78:0/418166977 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe53bfff640 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 0x55d459a842d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe548b9c640 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 0x55d459a84810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 -- 172.21.15.78:0/418166977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 msgr2=0x55d459a84810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 0x55d459a84810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 -- 172.21.15.78:0/418166977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 msgr2=0x55d459a842d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 0x55d459a842d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 -- 172.21.15.78:0/418166977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d459af3300 con 0x55d4596a74d0 2024-05-21T15:21:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe548b9c640 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 0x55d459a84810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe543fff640 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596a74d0 0x55d459a83d90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe534009ec0 tx=0x7fe5340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55837d6f8133af0b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe5237fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.901+0000 7fe5237fe640 1 -- 172.21.15.78:0/418166977 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534012b40 con 0x55d4596a74d0 2024-05-21T15:21:10.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe5237fe640 1 -- 172.21.15.78:0/418166977 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe534012ce0 con 0x55d4596a74d0 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 1 -- 172.21.15.78:0/418166977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe52c098040 con 0x55d4596a74d0 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe5237fe640 1 -- 172.21.15.78:0/418166977 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe534032470 con 0x55d4596a74d0 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 10 client.17037 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 1 -- 172.21.15.78:0/418166977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d459af45c0 con 0x55d4596a74d0 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe5237fe640 1 -- 172.21.15.78:0/418166977 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52c098040 con 0x55d4596a74d0 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe5237fe640 1 -- 172.21.15.78:0/418166977 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe53400f040 con 0x55d4596a74d0 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 10 client.17037 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 10 client.17037 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:21:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 20 client.17037 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55d4596643a0 0x55d459af6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 4 client.17037 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe55157f740 1 -- 172.21.15.78:0/418166977 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4599dc6c0 con 0x55d4596643a0 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.902+0000 7fe53bfff640 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55d4596643a0 0x55d459af6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe5237fe640 10 client.17037 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe53bfff640 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55d4596643a0 0x55d459af6d60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9b4156e5508891e server_cookie=f2b3761f7d560eb1 in_seq=0 out_seq=0 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe5237fe640 1 -- 172.21.15.78:0/418166977 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4599dc6c0 con 0x55d4596643a0 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe5237fe640 10 client.17037 handle_command_reply: tid=0 2024-05-21T15:21:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 client.17037 shutdown 2024-05-21T15:21:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 -- 172.21.15.78:0/418166977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596a74d0 msgr2=0x55d459a83d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596a74d0 0x55d459a83d90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe534009ec0 tx=0x7fe5340046c0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 -- 172.21.15.78:0/418166977 shutdown_connections 2024-05-21T15:21:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55d4596643a0 0x55d459af6d60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d4596af6b0 0x55d459a842d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4596d0390 0x55d459a84810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 --2- 172.21.15.78:0/418166977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4596a74d0 0x55d459a83d90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe5237fe640 0 client.17037 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:21:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 -- 172.21.15.78:0/418166977 >> 172.21.15.78:0/418166977 conn(0x55d4596c44e0 msgr2=0x55d459a7b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 -- 172.21.15.78:0/418166977 shutdown_connections 2024-05-21T15:21:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 1 -- 172.21.15.78:0/418166977 wait complete. 2024-05-21T15:21:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.903+0000 7fe55157f740 20 client.17037 trim_cache size 0 max 16384 2024-05-21T15:21:10.919 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 -- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe5300a2d70 msgr2=0x7fe5300a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe5300a2d70 0x7fe5300a5280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55d459a70520 tx=0x7fe534009450 comp rx=0 tx=0).stop 2024-05-21T15:21:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 -- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe544108920 msgr2=0x7fe544199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe544108920 0x7fe544199b10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c970 tx=0x7fe52c00ce30 comp rx=0 tx=0).stop 2024-05-21T15:21:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 -- 172.21.15.78:0/2045153761 shutdown_connections 2024-05-21T15:21:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe5300a2d70 0x7fe5300a5280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe54410d280 0x7fe54419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe544108920 0x7fe544199b10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 --2- 172.21.15.78:0/2045153761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5441029f0 0x7fe5441995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 -- 172.21.15.78:0/2045153761 >> 172.21.15.78:0/2045153761 conn(0x7fe5440fdf70 msgr2=0x7fe5441091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 -- 172.21.15.78:0/2045153761 shutdown_connections 2024-05-21T15:21:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:10.904+0000 7fe54939d640 1 -- 172.21.15.78:0/2045153761 wait complete. 2024-05-21T15:21:10.923 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:21:11.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.031+0000 7f7352faa640 1 -- 172.21.15.78:0/2456619831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c101b20 msgr2=0x7f734c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.031+0000 7f7352faa640 1 --2- 172.21.15.78:0/2456619831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c101b20 0x7f734c101f30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7338009900 tx=0x7f733802a410 comp rx=0 tx=0).stop 2024-05-21T15:21:11.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 -- 172.21.15.78:0/2456619831 shutdown_connections 2024-05-21T15:21:11.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 --2- 172.21.15.78:0/2456619831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f734c10c360 0x7f734c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 --2- 172.21.15.78:0/2456619831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c101b20 0x7f734c101f30 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 --2- 172.21.15.78:0/2456619831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f734c107a00 0x7f734c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 -- 172.21.15.78:0/2456619831 >> 172.21.15.78:0/2456619831 conn(0x7f734c0770c0 msgr2=0x7f734c077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 -- 172.21.15.78:0/2456619831 shutdown_connections 2024-05-21T15:21:11.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 -- 172.21.15.78:0/2456619831 wait complete. 2024-05-21T15:21:11.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 Processor -- start 2024-05-21T15:21:11.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 -- start start 2024-05-21T15:21:11.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.032+0000 7f7352faa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f734c101b20 0x7f734c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7352faa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f734c107a00 0x7f734c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7352faa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c10c360 0x7f734c19b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7352faa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f734c111d10 con 0x7f734c10c360 2024-05-21T15:21:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7352faa640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f734c111b90 con 0x7f734c107a00 2024-05-21T15:21:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7352faa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f734c111e90 con 0x7f734c101b20 2024-05-21T15:21:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c10c360 0x7f734c19b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c10c360 0x7f734c19b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54164/0 (socket says 172.21.15.78:54164) 2024-05-21T15:21:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 -- 172.21.15.78:0/1769049626 learned_addr learned my addr 172.21.15.78:0/1769049626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f734bfff640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f734c107a00 0x7f734c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7350d46640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f734c101b20 0x7f734c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 -- 172.21.15.78:0/1769049626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f734c101b20 msgr2=0x7f734c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f734c101b20 0x7f734c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 -- 172.21.15.78:0/1769049626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f734c107a00 msgr2=0x7f734c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f734c107a00 0x7f734c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 -- 172.21.15.78:0/1769049626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c1a5970 con 0x7f734c10c360 2024-05-21T15:21:11.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7350d46640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f734c101b20 0x7f734c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:11.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7351547640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c10c360 0x7f734c19b0d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f734400eb10 tx=0x7f734400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7349ffb640 1 -- 172.21.15.78:0/1769049626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344004b20 con 0x7f734c10c360 2024-05-21T15:21:11.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7352faa640 1 -- 172.21.15.78:0/1769049626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c1a5c60 con 0x7f734c10c360 2024-05-21T15:21:11.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7349ffb640 1 -- 172.21.15.78:0/1769049626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7344004cc0 con 0x7f734c10c360 2024-05-21T15:21:11.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.033+0000 7f7349ffb640 1 -- 172.21.15.78:0/1769049626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344010760 con 0x7f734c10c360 2024-05-21T15:21:11.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.034+0000 7f7352faa640 1 -- 172.21.15.78:0/1769049626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c1a6180 con 0x7f734c10c360 2024-05-21T15:21:11.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.036+0000 7f7349ffb640 1 -- 172.21.15.78:0/1769049626 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7344005020 con 0x7f734c10c360 2024-05-21T15:21:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.036+0000 7f7359e53740 1 Processor -- start 2024-05-21T15:21:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.036+0000 7f7359e53740 1 -- start start 2024-05-21T15:21:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.036+0000 7f7359e53740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 0x564a4a855d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.036+0000 7f7359e53740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a856260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.036+0000 7f7359e53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.036+0000 7f7359e53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a4a7bbc40 con 0x564a4a4966b0 2024-05-21T15:21:11.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7359e53740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564a4a763580 con 0x564a4a4b7390 2024-05-21T15:21:11.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7359e53740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a4a7b7c00 con 0x564a4a48e4d0 2024-05-21T15:21:11.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7349ffb640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f733c0a2d00 0x7f733c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54178/0 (socket says 172.21.15.78:54178) 2024-05-21T15:21:11.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 -- 172.21.15.78:0/1607176235 learned_addr learned my addr 172.21.15.78:0/1607176235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7351547640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f733c0a2d00 0x7f733c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7350d46640 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a856260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 0x564a4a855d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7349ffb640 1 -- 172.21.15.78:0/1769049626 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f73440cfba0 con 0x7f734c10c360 2024-05-21T15:21:11.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 -- 172.21.15.78:0/1607176235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 msgr2=0x564a4a856260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a856260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 -- 172.21.15.78:0/1607176235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 msgr2=0x564a4a855d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 0x564a4a855d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 -- 172.21.15.78:0/1607176235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a4a4cf3f0 con 0x564a4a4966b0 2024-05-21T15:21:11.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7350d46640 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a856260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:21:11.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7351547640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f733c0a2d00 0x7f733c0a5210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f734409bff0 tx=0x7f734406f0d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f734bfff640 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8567a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7338009900 tx=0x7f7338000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89f110ca3bb27b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.037+0000 7f7336ffd640 1 -- 172.21.15.78:0/1607176235 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338035070 con 0x564a4a4966b0 2024-05-21T15:21:11.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7336ffd640 1 -- 172.21.15.78:0/1607176235 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7338031860 con 0x564a4a4966b0 2024-05-21T15:21:11.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7336ffd640 1 -- 172.21.15.78:0/1607176235 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338003bb0 con 0x564a4a4966b0 2024-05-21T15:21:11.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 -- 172.21.15.78:0/1607176235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 msgr2=0x564a4a8567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8567a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7338009900 tx=0x7f7338000bc0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 -- 172.21.15.78:0/1607176235 shutdown_connections 2024-05-21T15:21:11.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8567a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a856260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 --2- 172.21.15.78:0/1607176235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 0x564a4a855d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 -- 172.21.15.78:0/1607176235 >> 172.21.15.78:0/1607176235 conn(0x564a4a4ab4e0 msgr2=0x564a4a8524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 -- 172.21.15.78:0/1607176235 shutdown_connections 2024-05-21T15:21:11.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.038+0000 7f7359e53740 1 -- 172.21.15.78:0/1607176235 wait complete. 2024-05-21T15:21:11.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 1 -- start start 2024-05-21T15:21:11.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 1 Processor -- start 2024-05-21T15:21:11.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a861d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 0x564a4a860240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564a4a862fb0 con 0x564a4a4966b0 2024-05-21T15:21:11.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564a4a776c40 con 0x564a4a4b7390 2024-05-21T15:21:11.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a4a863130 con 0x564a4a48e4d0 2024-05-21T15:21:11.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7359e53740 10 client.0 _ll_register_callbacks cb 0x564a4a4cebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:11.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7350d46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7350d46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54190/0 (socket says 172.21.15.78:54190) 2024-05-21T15:21:11.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.039+0000 7f7350d46640 1 -- 172.21.15.78:0/773448017 learned_addr learned my addr 172.21.15.78:0/773448017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f734bfff640 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a861d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f734bfff640 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 0x564a4a860240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7350d46640 1 -- 172.21.15.78:0/773448017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 msgr2=0x564a4a861d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7350d46640 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a861d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7350d46640 1 -- 172.21.15.78:0/773448017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 msgr2=0x564a4a860240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7350d46640 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 0x564a4a860240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7350d46640 1 -- 172.21.15.78:0/773448017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a4a5997d0 con 0x564a4a4966b0 2024-05-21T15:21:11.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7350d46640 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8622e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f734000bf00 tx=0x7f7340012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249f5ad4a18ba7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f732bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:11.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f732bfff640 1 -- 172.21.15.78:0/773448017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340010960 con 0x564a4a4966b0 2024-05-21T15:21:11.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f732bfff640 1 -- 172.21.15.78:0/773448017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7340010b00 con 0x564a4a4966b0 2024-05-21T15:21:11.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7359e53740 1 -- 172.21.15.78:0/773448017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7338031860 con 0x564a4a4966b0 2024-05-21T15:21:11.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f732bfff640 1 -- 172.21.15.78:0/773448017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7340010d00 con 0x564a4a4966b0 2024-05-21T15:21:11.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7359e53740 10 client.17049 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:11.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f7359e53740 1 -- 172.21.15.78:0/773448017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564a4a86f8b0 con 0x564a4a4966b0 2024-05-21T15:21:11.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.040+0000 7f732bfff640 1 -- 172.21.15.78:0/773448017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7338031860 con 0x564a4a4966b0 2024-05-21T15:21:11.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f732bfff640 1 -- 172.21.15.78:0/773448017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7340019070 con 0x564a4a4966b0 2024-05-21T15:21:11.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f7359e53740 10 client.17049 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:11.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f7359e53740 10 client.17049 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:21:11.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f7359e53740 20 client.17049 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:11.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f7359e53740 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564a4a475f60 0x564a4a872e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f7359e53740 4 client.17049 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:11.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f7359e53740 1 -- 172.21.15.78:0/773448017 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a4a7c9370 con 0x564a4a475f60 2024-05-21T15:21:11.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f734bfff640 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564a4a475f60 0x564a4a872e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f732bfff640 10 client.17049 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:21:11.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.041+0000 7f734bfff640 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564a4a475f60 0x564a4a872e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f63eb77d4623331 server_cookie=88b723043bdd82ef in_seq=0 out_seq=0 2024-05-21T15:21:11.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f732bfff640 1 -- 172.21.15.78:0/773448017 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a4a7c9370 con 0x564a4a475f60 2024-05-21T15:21:11.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f732bfff640 10 client.17049 handle_command_reply: tid=0 2024-05-21T15:21:11.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 client.17049 shutdown 2024-05-21T15:21:11.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 -- 172.21.15.78:0/773448017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 msgr2=0x564a4a8622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8622e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f734000bf00 tx=0x7f7340012820 comp rx=0 tx=0).stop 2024-05-21T15:21:11.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 -- 172.21.15.78:0/773448017 shutdown_connections 2024-05-21T15:21:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564a4a475f60 0x564a4a872e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a4b7390 0x564a4a860240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a8622e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 --2- 172.21.15.78:0/773448017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a48e4d0 0x564a4a861d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f732bfff640 0 client.17049 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:21:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 -- 172.21.15.78:0/773448017 >> 172.21.15.78:0/773448017 conn(0x564a4a4ab4e0 msgr2=0x564a4a857a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 -- 172.21.15.78:0/773448017 shutdown_connections 2024-05-21T15:21:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 1 -- 172.21.15.78:0/773448017 wait complete. 2024-05-21T15:21:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.042+0000 7f7359e53740 20 client.17049 trim_cache size 0 max 16384 2024-05-21T15:21:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f7359e53740 1 Processor -- start 2024-05-21T15:21:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f7359e53740 1 -- start start 2024-05-21T15:21:11.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f7359e53740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a475f60 0x564a4a85f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f7359e53740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 0x564a4a8601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f7359e53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a878630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f7359e53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1aeb30 con 0x564a4a4966b0 2024-05-21T15:21:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f7359e53740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1b0670 con 0x564a4a48e4d0 2024-05-21T15:21:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f7359e53740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1af200 con 0x564a4a475f60 2024-05-21T15:21:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f734bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a878630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f734bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a878630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54200/0 (socket says 172.21.15.78:54200) 2024-05-21T15:21:11.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f734bfff640 1 -- 172.21.15.78:0/565916389 learned_addr learned my addr 172.21.15.78:0/565916389 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f734bfff640 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a475f60 0x564a4a85f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.056+0000 7f734bfff640 1 -- 172.21.15.78:0/565916389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a475f60 msgr2=0x564a4a85f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f734bfff640 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a475f60 0x564a4a85f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f734bfff640 1 -- 172.21.15.78:0/565916389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 msgr2=0x564a4a8601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f7350d46640 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 0x564a4a8601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f734bfff640 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 0x564a4a8601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f734bfff640 1 -- 172.21.15.78:0/565916389 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a4a8733c0 con 0x564a4a4966b0 2024-05-21T15:21:11.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f734bfff640 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a878630 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7338005f70 tx=0x7f7338005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd91b474db8e215 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f73367fc640 1 -- 172.21.15.78:0/565916389 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338035040 con 0x564a4a4966b0 2024-05-21T15:21:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f73367fc640 1 -- 172.21.15.78:0/565916389 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73380035f0 con 0x564a4a4966b0 2024-05-21T15:21:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.057+0000 7f73367fc640 1 -- 172.21.15.78:0/565916389 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338053430 con 0x564a4a4966b0 2024-05-21T15:21:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 -- 172.21.15.78:0/565916389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 msgr2=0x564a4a878630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a878630 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7338005f70 tx=0x7f7338005780 comp rx=0 tx=0).stop 2024-05-21T15:21:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 -- 172.21.15.78:0/565916389 shutdown_connections 2024-05-21T15:21:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 0x564a4a8601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a4966b0 0x564a4a878630 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 --2- 172.21.15.78:0/565916389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a475f60 0x564a4a85f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 -- 172.21.15.78:0/565916389 >> 172.21.15.78:0/565916389 conn(0x564a4a4ab4e0 msgr2=0x564a4a85fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 -- 172.21.15.78:0/565916389 shutdown_connections 2024-05-21T15:21:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 -- 172.21.15.78:0/565916389 wait complete. 2024-05-21T15:21:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 -- start start 2024-05-21T15:21:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.058+0000 7f7359e53740 1 Processor -- start 2024-05-21T15:21:11.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7359e53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a475f60 0x564a4a868a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7359e53740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 0x564a4a865db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7359e53740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a4966b0 0x564a4a866370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7359e53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1affa0 con 0x564a4a475f60 2024-05-21T15:21:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7359e53740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1af8d0 con 0x564a4a48e4d0 2024-05-21T15:21:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7359e53740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1acb10 con 0x564a4a4966b0 2024-05-21T15:21:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7359e53740 10 client.0 _ll_register_callbacks cb 0x7f734000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a475f60 0x564a4a868a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a475f60 0x564a4a868a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54212/0 (socket says 172.21.15.78:54212) 2024-05-21T15:21:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 -- 172.21.15.78:0/1337722488 learned_addr learned my addr 172.21.15.78:0/1337722488 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7350d46640 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 0x564a4a865db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a4966b0 0x564a4a866370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 -- 172.21.15.78:0/1337722488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a4966b0 msgr2=0x564a4a866370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a4966b0 0x564a4a866370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 -- 172.21.15.78:0/1337722488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 msgr2=0x564a4a865db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 0x564a4a865db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 -- 172.21.15.78:0/1337722488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a4a866b00 con 0x564a4a475f60 2024-05-21T15:21:11.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f734bfff640 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a475f60 0x564a4a868a40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7338033000 tx=0x7f733804d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458f88628e4921ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f732bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:11.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f732bfff640 1 -- 172.21.15.78:0/1337722488 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338035040 con 0x564a4a475f60 2024-05-21T15:21:11.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f732bfff640 1 -- 172.21.15.78:0/1337722488 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f73380039a0 con 0x564a4a475f60 2024-05-21T15:21:11.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.059+0000 7f7359e53740 1 -- 172.21.15.78:0/1337722488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73380035f0 con 0x564a4a475f60 2024-05-21T15:21:11.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f732bfff640 1 -- 172.21.15.78:0/1337722488 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7338004490 con 0x564a4a475f60 2024-05-21T15:21:11.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7359e53740 10 client.17055 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:11.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7359e53740 1 -- 172.21.15.78:0/1337722488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564a4a8db1b0 con 0x564a4a475f60 2024-05-21T15:21:11.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f732bfff640 1 -- 172.21.15.78:0/1337722488 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73380035f0 con 0x564a4a475f60 2024-05-21T15:21:11.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f732bfff640 1 -- 172.21.15.78:0/1337722488 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f73380530b0 con 0x564a4a475f60 2024-05-21T15:21:11.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7359e53740 10 client.17055 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:11.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7359e53740 10 client.17055 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:21:11.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7359e53740 20 client.17055 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:11.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7359e53740 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564a4a4b7390 0x564a4a8dd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7359e53740 4 client.17055 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:11.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7359e53740 1 -- 172.21.15.78:0/1337722488 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a4a7c36c0 con 0x564a4a4b7390 2024-05-21T15:21:11.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.060+0000 7f7350d46640 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564a4a4b7390 0x564a4a8dd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f732bfff640 10 client.17055 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:21:11.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7350d46640 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564a4a4b7390 0x564a4a8dd390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2de97b4dbfce21c server_cookie=306fc412b1a4d9d4 in_seq=0 out_seq=0 2024-05-21T15:21:11.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f732bfff640 1 -- 172.21.15.78:0/1337722488 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a4a7c36c0 con 0x564a4a4b7390 2024-05-21T15:21:11.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f732bfff640 10 client.17055 handle_command_reply: tid=0 2024-05-21T15:21:11.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 client.17055 shutdown 2024-05-21T15:21:11.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 -- 172.21.15.78:0/1337722488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a475f60 msgr2=0x564a4a868a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a475f60 0x564a4a868a40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7338033000 tx=0x7f733804d040 comp rx=0 tx=0).stop 2024-05-21T15:21:11.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 -- 172.21.15.78:0/1337722488 shutdown_connections 2024-05-21T15:21:11.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564a4a4b7390 0x564a4a8dd390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564a4a48e4d0 0x564a4a865db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a4a4966b0 0x564a4a866370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 --2- 172.21.15.78:0/1337722488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564a4a475f60 0x564a4a868a40 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f732bfff640 0 client.17055 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:21:11.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 -- 172.21.15.78:0/1337722488 >> 172.21.15.78:0/1337722488 conn(0x564a4a4ab4e0 msgr2=0x564a4a861700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 -- 172.21.15.78:0/1337722488 shutdown_connections 2024-05-21T15:21:11.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 1 -- 172.21.15.78:0/1337722488 wait complete. 2024-05-21T15:21:11.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.061+0000 7f7359e53740 20 client.17055 trim_cache size 0 max 16384 2024-05-21T15:21:11.080 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:11.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 -- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f733c0a2d00 msgr2=0x7f733c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f733c0a2d00 0x7f733c0a5210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f734409bff0 tx=0x7f734406f0d0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 -- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c10c360 msgr2=0x7f734c19b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c10c360 0x7f734c19b0d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f734400eb10 tx=0x7f734400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 -- 172.21.15.78:0/1769049626 shutdown_connections 2024-05-21T15:21:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f734c107a00 0x7f734c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f733c0a2d00 0x7f733c0a5210 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f734c10c360 0x7f734c19b0d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 --2- 172.21.15.78:0/1769049626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f734c101b20 0x7f734c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 -- 172.21.15.78:0/1769049626 >> 172.21.15.78:0/1769049626 conn(0x7f734c0770c0 msgr2=0x7f734c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 -- 172.21.15.78:0/1769049626 shutdown_connections 2024-05-21T15:21:11.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.062+0000 7f7352faa640 1 -- 172.21.15.78:0/1769049626 wait complete. 2024-05-21T15:21:11.083 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:21:11.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/1753878584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810c330 msgr2=0x7f8f7810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/1753878584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810c330 0x7f8f7810e750 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009ca0 tx=0x7f8f7002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/1753878584 shutdown_connections 2024-05-21T15:21:11.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/1753878584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f7810c330 0x7f8f7810e750 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/1753878584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f78101aa0 0x7f8f78101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/1753878584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f781079d0 0x7f8f78101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/1753878584 >> 172.21.15.78:0/1753878584 conn(0x7f8f780770b0 msgr2=0x7f8f780774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/1753878584 shutdown_connections 2024-05-21T15:21:11.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.192+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/1753878584 wait complete. 2024-05-21T15:21:11.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7f1b5640 1 Processor -- start 2024-05-21T15:21:11.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7f1b5640 1 -- start start 2024-05-21T15:21:11.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7f1b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f78101aa0 0x7f8f781986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7f1b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f781079d0 0x7f8f78198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7f1b5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f7810c330 0x7f8f7819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7f1b5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78111d00 con 0x7f8f781079d0 2024-05-21T15:21:11.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7f1b5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78111b80 con 0x7f8f7810c330 2024-05-21T15:21:11.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7f1b5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78111e80 con 0x7f8f78101aa0 2024-05-21T15:21:11.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f781079d0 0x7f8f78198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f781079d0 0x7f8f78198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54238/0 (socket says 172.21.15.78:54238) 2024-05-21T15:21:11.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f77fff640 1 -- 172.21.15.78:0/44359963 learned_addr learned my addr 172.21.15.78:0/44359963 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f78101aa0 0x7f8f781986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7d752640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f7810c330 0x7f8f7819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f77fff640 1 -- 172.21.15.78:0/44359963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f78101aa0 msgr2=0x7f8f781986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f77fff640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f78101aa0 0x7f8f781986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f77fff640 1 -- 172.21.15.78:0/44359963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f7810c330 msgr2=0x7f8f7819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f77fff640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f7810c330 0x7f8f7819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f77fff640 1 -- 172.21.15.78:0/44359963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7819d540 con 0x7f8f781079d0 2024-05-21T15:21:11.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.193+0000 7f8f7d752640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f7810c330 0x7f8f7819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:11.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.194+0000 7f8f77fff640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f781079d0 0x7f8f78198c40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8f64005d00 tx=0x7f8f64009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.194+0000 7f8f75ffb640 1 -- 172.21.15.78:0/44359963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6401d070 con 0x7f8f781079d0 2024-05-21T15:21:11.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.194+0000 7f8f75ffb640 1 -- 172.21.15.78:0/44359963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f64007c80 con 0x7f8f781079d0 2024-05-21T15:21:11.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.194+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/44359963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7819d830 con 0x7f8f781079d0 2024-05-21T15:21:11.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.194+0000 7f8f75ffb640 1 -- 172.21.15.78:0/44359963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64016320 con 0x7f8f781079d0 2024-05-21T15:21:11.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.194+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/44359963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7819dd30 con 0x7f8f781079d0 2024-05-21T15:21:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.196+0000 7f8f75ffb640 1 -- 172.21.15.78:0/44359963 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8f64020070 con 0x7f8f781079d0 2024-05-21T15:21:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f8605e740 1 Processor -- start 2024-05-21T15:21:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f8605e740 1 -- start start 2024-05-21T15:21:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 0x557864f29bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f2a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f2a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f8605e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557864e8fc40 con 0x557864b624d0 2024-05-21T15:21:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f8605e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557864e37580 con 0x557864b6a6b0 2024-05-21T15:21:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f8605e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557864e8bc00 con 0x557864b8b390 2024-05-21T15:21:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f75ffb640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8f680a2d00 0x7f8f680a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7cf51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f2a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7cf51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f2a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54246/0 (socket says 172.21.15.78:54246) 2024-05-21T15:21:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7cf51640 1 -- 172.21.15.78:0/3624135060 learned_addr learned my addr 172.21.15.78:0/3624135060 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7d752640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8f680a2d00 0x7f8f680a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7d752640 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 0x557864f29bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f75ffb640 1 -- 172.21.15.78:0/44359963 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8f640d09e0 con 0x7f8f781079d0 2024-05-21T15:21:11.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f77fff640 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f2a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7cf51640 1 -- 172.21.15.78:0/3624135060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 msgr2=0x557864f29bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 0x557864f29bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7cf51640 1 -- 172.21.15.78:0/3624135060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 msgr2=0x557864f2a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f2a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.197+0000 7f8f7cf51640 1 -- 172.21.15.78:0/3624135060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557864ba33f0 con 0x557864b624d0 2024-05-21T15:21:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.198+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f2a0f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009c10 tx=0x7f8f6c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c453ed0fb2a703f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.198+0000 7f8f7d752640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8f680a2d00 0x7f8f680a5210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x557864f26ec0 tx=0x7f8f700093a0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.198+0000 7f8f62ffd640 1 -- 172.21.15.78:0/3624135060 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c004230 con 0x557864b624d0 2024-05-21T15:21:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.198+0000 7f8f62ffd640 1 -- 172.21.15.78:0/3624135060 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f6c0043d0 con 0x557864b624d0 2024-05-21T15:21:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.198+0000 7f8f62ffd640 1 -- 172.21.15.78:0/3624135060 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c004df0 con 0x557864b624d0 2024-05-21T15:21:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 -- 172.21.15.78:0/3624135060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 msgr2=0x557864f2a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f2a0f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009c10 tx=0x7f8f6c011810 comp rx=0 tx=0).stop 2024-05-21T15:21:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 -- 172.21.15.78:0/3624135060 shutdown_connections 2024-05-21T15:21:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f2a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f2a0f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 --2- 172.21.15.78:0/3624135060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 0x557864f29bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 -- 172.21.15.78:0/3624135060 >> 172.21.15.78:0/3624135060 conn(0x557864b7f4e0 msgr2=0x557864f264d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 -- 172.21.15.78:0/3624135060 shutdown_connections 2024-05-21T15:21:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 -- 172.21.15.78:0/3624135060 wait complete. 2024-05-21T15:21:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 -- start start 2024-05-21T15:21:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.199+0000 7f8f8605e740 1 Processor -- start 2024-05-21T15:21:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b624d0 0x557864f38720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b6a6b0 0x557864f38c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 0x557864f44930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f8605e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557864f3b480 con 0x557864b6a6b0 2024-05-21T15:21:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f8605e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557864e4ac40 con 0x557864b624d0 2024-05-21T15:21:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f8605e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557864f3b600 con 0x557864b8b390 2024-05-21T15:21:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f8605e740 10 client.0 _ll_register_callbacks cb 0x557864ba2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b6a6b0 0x557864f38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b6a6b0 0x557864f38c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54256/0 (socket says 172.21.15.78:54256) 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 -- 172.21.15.78:0/859806981 learned_addr learned my addr 172.21.15.78:0/859806981 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b624d0 0x557864f38720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f7d752640 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 0x557864f44930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 -- 172.21.15.78:0/859806981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 msgr2=0x557864f44930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 0x557864f44930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 -- 172.21.15.78:0/859806981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b624d0 msgr2=0x557864f38720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b624d0 0x557864f38720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 -- 172.21.15.78:0/859806981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557864c6d7d0 con 0x557864b6a6b0 2024-05-21T15:21:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f77fff640 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b6a6b0 0x557864f38c60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8f6409ce30 tx=0x7f8f6409c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89c47d5d5f31702 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f57fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f57fff640 1 -- 172.21.15.78:0/859806981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640a1250 con 0x557864b6a6b0 2024-05-21T15:21:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f57fff640 1 -- 172.21.15.78:0/859806981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8f64018420 con 0x557864b6a6b0 2024-05-21T15:21:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.200+0000 7f8f8605e740 1 -- 172.21.15.78:0/859806981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f6c0043d0 con 0x557864b6a6b0 2024-05-21T15:21:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f57fff640 1 -- 172.21.15.78:0/859806981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8f640a5b10 con 0x557864b6a6b0 2024-05-21T15:21:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f8605e740 10 client.17067 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f8605e740 1 -- 172.21.15.78:0/859806981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557864f47b50 con 0x557864b6a6b0 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f57fff640 1 -- 172.21.15.78:0/859806981 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f640b9030 con 0x557864b6a6b0 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f57fff640 1 -- 172.21.15.78:0/859806981 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8f640ba030 con 0x557864b6a6b0 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f8605e740 10 client.17067 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f8605e740 10 client.17067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f8605e740 20 client.17067 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f8605e740 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x557864b49f60 0x557864f4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f8605e740 4 client.17067 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f8605e740 1 -- 172.21.15.78:0/859806981 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557864e9d370 con 0x557864b49f60 2024-05-21T15:21:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.201+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x557864b49f60 0x557864f4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.202+0000 7f8f57fff640 10 client.17067 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:21:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.202+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x557864b49f60 0x557864f4b0a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e46fc05fb7c2368f server_cookie=a8dd340301914c45 in_seq=0 out_seq=0 2024-05-21T15:21:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.202+0000 7f8f57fff640 1 -- 172.21.15.78:0/859806981 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557864e9d370 con 0x557864b49f60 2024-05-21T15:21:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.202+0000 7f8f57fff640 10 client.17067 handle_command_reply: tid=0 2024-05-21T15:21:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.202+0000 7f8f8605e740 1 client.17067 shutdown 2024-05-21T15:21:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 -- 172.21.15.78:0/859806981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b6a6b0 msgr2=0x557864f38c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b6a6b0 0x557864f38c60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8f6409ce30 tx=0x7f8f6409c8c0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 -- 172.21.15.78:0/859806981 shutdown_connections 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x557864b49f60 0x557864f4b0a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b8b390 0x557864f44930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f57fff640 0 client.17067 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b6a6b0 0x557864f38c60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 --2- 172.21.15.78:0/859806981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b624d0 0x557864f38720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 -- 172.21.15.78:0/859806981 >> 172.21.15.78:0/859806981 conn(0x557864b7f4e0 msgr2=0x557864f2ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 -- 172.21.15.78:0/859806981 shutdown_connections 2024-05-21T15:21:11.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 1 -- 172.21.15.78:0/859806981 wait complete. 2024-05-21T15:21:11.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.203+0000 7f8f8605e740 20 client.17067 trim_cache size 0 max 16384 2024-05-21T15:21:11.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.216+0000 7f8f8605e740 1 Processor -- start 2024-05-21T15:21:11.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.216+0000 7f8f8605e740 1 -- start start 2024-05-21T15:21:11.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b49f60 0x557864f2bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f44510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f50830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f8605e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781aca80 con 0x557864b624d0 2024-05-21T15:21:11.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f8605e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781ae5c0 con 0x557864b6a6b0 2024-05-21T15:21:11.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f8605e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781ad150 con 0x557864b49f60 2024-05-21T15:21:11.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f44510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54270/0 (socket says 172.21.15.78:54270) 2024-05-21T15:21:11.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 -- 172.21.15.78:0/2894787760 learned_addr learned my addr 172.21.15.78:0/2894787760 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b49f60 0x557864f2bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f7d752640 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f50830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 -- 172.21.15.78:0/2894787760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b49f60 msgr2=0x557864f2bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b49f60 0x557864f2bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 -- 172.21.15.78:0/2894787760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 msgr2=0x557864f50830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f50830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 -- 172.21.15.78:0/2894787760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557864f36a60 con 0x557864b624d0 2024-05-21T15:21:11.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b49f60 0x557864f2bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:11.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f7d752640 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f50830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:11.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f77fff640 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f44510 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8f6409cba0 tx=0x7f8f6409c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dd8e74bbf1c866 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f627fc640 1 -- 172.21.15.78:0/2894787760 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6409c620 con 0x557864b624d0 2024-05-21T15:21:11.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f627fc640 1 -- 172.21.15.78:0/2894787760 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f6c0043d0 con 0x557864b624d0 2024-05-21T15:21:11.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.217+0000 7f8f627fc640 1 -- 172.21.15.78:0/2894787760 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64099140 con 0x557864b624d0 2024-05-21T15:21:11.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 -- 172.21.15.78:0/2894787760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 msgr2=0x557864f44510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f44510 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8f6409cba0 tx=0x7f8f6409c930 comp rx=0 tx=0).stop 2024-05-21T15:21:11.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 -- 172.21.15.78:0/2894787760 shutdown_connections 2024-05-21T15:21:11.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b6a6b0 0x557864f50830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f44510 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 --2- 172.21.15.78:0/2894787760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b49f60 0x557864f2bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 -- 172.21.15.78:0/2894787760 >> 172.21.15.78:0/2894787760 conn(0x557864b7f4e0 msgr2=0x557864f2b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 -- 172.21.15.78:0/2894787760 shutdown_connections 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.218+0000 7f8f8605e740 1 -- 172.21.15.78:0/2894787760 wait complete. 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 1 -- start start 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 1 Processor -- start 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b49f60 0x557864f40aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f3be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b6a6b0 0x557864f3c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781adef0 con 0x557864b624d0 2024-05-21T15:21:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781ad820 con 0x557864b49f60 2024-05-21T15:21:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781aaa60 con 0x557864b6a6b0 2024-05-21T15:21:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f8605e740 10 client.0 _ll_register_callbacks cb 0x7f8f64005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f3be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f3be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54272/0 (socket says 172.21.15.78:54272) 2024-05-21T15:21:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f77fff640 1 -- 172.21.15.78:0/1995703647 learned_addr learned my addr 172.21.15.78:0/1995703647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b49f60 0x557864f40aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f7d752640 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b6a6b0 0x557864f3c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f77fff640 1 -- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b6a6b0 msgr2=0x557864f3c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f77fff640 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b6a6b0 0x557864f3c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f77fff640 1 -- 172.21.15.78:0/1995703647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b49f60 msgr2=0x557864f40aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f77fff640 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b49f60 0x557864f40aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.219+0000 7f8f77fff640 1 -- 172.21.15.78:0/1995703647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557864f3c8d0 con 0x557864b624d0 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f7d752640 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b6a6b0 0x557864f3c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f77fff640 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f3be50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8f64005c40 tx=0x7f8f64018890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdf9a5f3aa6594 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f57fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f57fff640 1 -- 172.21.15.78:0/1995703647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64099140 con 0x557864b624d0 2024-05-21T15:21:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f57fff640 1 -- 172.21.15.78:0/1995703647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8f64098db0 con 0x557864b624d0 2024-05-21T15:21:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 1 -- 172.21.15.78:0/1995703647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f6c0043d0 con 0x557864b624d0 2024-05-21T15:21:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f57fff640 1 -- 172.21.15.78:0/1995703647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8f640861a0 con 0x557864b624d0 2024-05-21T15:21:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 10 client.17073 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 1 -- 172.21.15.78:0/1995703647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557864fb2e40 con 0x557864b624d0 2024-05-21T15:21:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f57fff640 1 -- 172.21.15.78:0/1995703647 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f6c0043d0 con 0x557864b624d0 2024-05-21T15:21:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f57fff640 1 -- 172.21.15.78:0/1995703647 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8f640a23e0 con 0x557864b624d0 2024-05-21T15:21:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 10 client.17073 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 10 client.17073 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:21:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 20 client.17073 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x557864b8b390 0x557864fb5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 4 client.17073 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.220+0000 7f8f8605e740 1 -- 172.21.15.78:0/1995703647 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557864e976c0 con 0x557864b8b390 2024-05-21T15:21:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x557864b8b390 0x557864fb5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f57fff640 10 client.17073 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:21:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f7cf51640 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x557864b8b390 0x557864fb5460 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2186feadc5590c57 server_cookie=930f0538b205ce38 in_seq=0 out_seq=0 2024-05-21T15:21:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f57fff640 1 -- 172.21.15.78:0/1995703647 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557864e976c0 con 0x557864b8b390 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f57fff640 10 client.17073 handle_command_reply: tid=0 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f8605e740 1 client.17073 shutdown 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f8605e740 1 -- 172.21.15.78:0/1995703647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 msgr2=0x557864f3be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f8605e740 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f3be50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8f64005c40 tx=0x7f8f64018890 comp rx=0 tx=0).stop 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f8605e740 1 -- 172.21.15.78:0/1995703647 shutdown_connections 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f8605e740 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x557864b8b390 0x557864fb5460 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f8605e740 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557864b6a6b0 0x557864f3c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f57fff640 0 client.17073 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:21:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f8605e740 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557864b624d0 0x557864f3be50 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.221+0000 7f8f8605e740 1 --2- 172.21.15.78:0/1995703647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557864b49f60 0x557864f40aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f8605e740 1 -- 172.21.15.78:0/1995703647 >> 172.21.15.78:0/1995703647 conn(0x557864b7f4e0 msgr2=0x557864f29ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f8605e740 1 -- 172.21.15.78:0/1995703647 shutdown_connections 2024-05-21T15:21:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f8605e740 1 -- 172.21.15.78:0/1995703647 wait complete. 2024-05-21T15:21:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f8605e740 20 client.17073 trim_cache size 0 max 16384 2024-05-21T15:21:11.292 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8f680a2d00 msgr2=0x7f8f680a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8f680a2d00 0x7f8f680a5210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x557864f26ec0 tx=0x7f8f700093a0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f781079d0 msgr2=0x7f8f78198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f781079d0 0x7f8f78198c40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8f64005d00 tx=0x7f8f64009ce0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/44359963 shutdown_connections 2024-05-21T15:21:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f7810c330 0x7f8f7819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8f680a2d00 0x7f8f680a5210 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f781079d0 0x7f8f78198c40 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 --2- 172.21.15.78:0/44359963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f78101aa0 0x7f8f781986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/44359963 >> 172.21.15.78:0/44359963 conn(0x7f8f780770b0 msgr2=0x7f8f78103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/44359963 shutdown_connections 2024-05-21T15:21:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.222+0000 7f8f7f1b5640 1 -- 172.21.15.78:0/44359963 wait complete. 2024-05-21T15:21:11.296 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:21:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.406+0000 7f052cc22640 1 -- 172.21.15.78:0/880805302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 msgr2=0x7f052810ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.406+0000 7f052cc22640 1 --2- 172.21.15.78:0/880805302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 0x7f052810ca70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f051c009900 tx=0x7f051c02a410 comp rx=0 tx=0).stop 2024-05-21T15:21:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.406+0000 7f052cc22640 1 -- 172.21.15.78:0/880805302 shutdown_connections 2024-05-21T15:21:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.406+0000 7f052cc22640 1 --2- 172.21.15.78:0/880805302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f052810d040 0x7f052810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.406+0000 7f052cc22640 1 --2- 172.21.15.78:0/880805302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 0x7f052810ca70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.406+0000 7f052cc22640 1 --2- 172.21.15.78:0/880805302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0528103bf0 0x7f0528104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.406+0000 7f052cc22640 1 -- 172.21.15.78:0/880805302 >> 172.21.15.78:0/880805302 conn(0x7f05280fd210 msgr2=0x7f05280ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.407+0000 7f052cc22640 1 -- 172.21.15.78:0/880805302 shutdown_connections 2024-05-21T15:21:11.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.407+0000 7f052cc22640 1 -- 172.21.15.78:0/880805302 wait complete. 2024-05-21T15:21:11.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.407+0000 7f052cc22640 1 Processor -- start 2024-05-21T15:21:11.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.407+0000 7f052cc22640 1 -- start start 2024-05-21T15:21:11.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f052cc22640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0528103bf0 0x7f0528199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f052cc22640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 0x7f0528199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f052cc22640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052810d040 0x7f052819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f052cc22640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0528112e40 con 0x7f052810a650 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f052cc22640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0528112cc0 con 0x7f0528103bf0 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f052cc22640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0528112fc0 con 0x7f052810d040 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 0x7f0528199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 0x7f0528199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54292/0 (socket says 172.21.15.78:54292) 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 -- 172.21.15.78:0/2546774651 learned_addr learned my addr 172.21.15.78:0/2546774651 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f052659c640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0528103bf0 0x7f0528199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0526d9d640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052810d040 0x7f052819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 -- 172.21.15.78:0/2546774651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052810d040 msgr2=0x7f052819dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052810d040 0x7f052819dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 -- 172.21.15.78:0/2546774651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0528103bf0 msgr2=0x7f0528199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0528103bf0 0x7f0528199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 -- 172.21.15.78:0/2546774651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052819e5e0 con 0x7f052810a650 2024-05-21T15:21:11.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f0525d9b640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 0x7f0528199ce0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f051c004770 tx=0x7f051c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.408+0000 7f05177fe640 1 -- 172.21.15.78:0/2546774651 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c03f070 con 0x7f052810a650 2024-05-21T15:21:11.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.409+0000 7f052cc22640 1 -- 172.21.15.78:0/2546774651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052819e870 con 0x7f052810a650 2024-05-21T15:21:11.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.409+0000 7f05177fe640 1 -- 172.21.15.78:0/2546774651 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c031ca0 con 0x7f052810a650 2024-05-21T15:21:11.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.409+0000 7f05177fe640 1 -- 172.21.15.78:0/2546774651 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c035030 con 0x7f052810a650 2024-05-21T15:21:11.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.409+0000 7f052cc22640 1 -- 172.21.15.78:0/2546774651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052819ed60 con 0x7f052810a650 2024-05-21T15:21:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.411+0000 7f05177fe640 1 -- 172.21.15.78:0/2546774651 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f051c042070 con 0x7f052810a650 2024-05-21T15:21:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.411+0000 7f0533ba1740 1 Processor -- start 2024-05-21T15:21:11.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0533ba1740 1 -- start start 2024-05-21T15:21:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f05177fe640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f05100a2d00 0x7f05100a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0526d9d640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f05100a2d00 0x7f05100a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f05177fe640 1 -- 172.21.15.78:0/2546774651 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f051c0efb60 con 0x7f052810a650 2024-05-21T15:21:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b30390 0x55ec10eced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10ecf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0533ba1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ec10e34c40 con 0x55ec10b074d0 2024-05-21T15:21:11.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0533ba1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ec10ddc580 con 0x55ec10b30390 2024-05-21T15:21:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0533ba1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec10e30c00 con 0x55ec10b0f6b0 2024-05-21T15:21:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54308/0 (socket says 172.21.15.78:54308) 2024-05-21T15:21:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 -- 172.21.15.78:0/2864465583 learned_addr learned my addr 172.21.15.78:0/2864465583 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f052659c640 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b30390 0x55ec10eced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0526d9d640 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10ecf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 -- 172.21.15.78:0/2864465583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 msgr2=0x55ec10ecf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10ecf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 -- 172.21.15.78:0/2864465583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b30390 msgr2=0x55ec10eced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b30390 0x55ec10eced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 -- 172.21.15.78:0/2864465583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec10b483f0 con 0x55ec10b074d0 2024-05-21T15:21:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0526d9d640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f05100a2d00 0x7f05100a5210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0520005fd0 tx=0x7f0520011810 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.412+0000 7f0525d9b640 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf260 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f051c0b9540 tx=0x7f051c03b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b59953eff4627fb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.413+0000 7f0514ff9640 1 -- 172.21.15.78:0/2864465583 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c049040 con 0x55ec10b074d0 2024-05-21T15:21:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.413+0000 7f0514ff9640 1 -- 172.21.15.78:0/2864465583 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c0dc0d0 con 0x55ec10b074d0 2024-05-21T15:21:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.413+0000 7f0514ff9640 1 -- 172.21.15.78:0/2864465583 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c0bc790 con 0x55ec10b074d0 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.413+0000 7f0533ba1740 1 -- 172.21.15.78:0/2864465583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 msgr2=0x55ec10ecf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.413+0000 7f0533ba1740 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf260 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f051c0b9540 tx=0x7f051c03b6d0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.413+0000 7f0533ba1740 1 -- 172.21.15.78:0/2864465583 shutdown_connections 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.413+0000 7f0533ba1740 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10ecf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf260 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 --2- 172.21.15.78:0/2864465583 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b30390 0x55ec10eced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 -- 172.21.15.78:0/2864465583 >> 172.21.15.78:0/2864465583 conn(0x55ec10b244e0 msgr2=0x55ec10ecb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 -- 172.21.15.78:0/2864465583 shutdown_connections 2024-05-21T15:21:11.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 -- 172.21.15.78:0/2864465583 wait complete. 2024-05-21T15:21:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 -- start start 2024-05-21T15:21:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 Processor -- start 2024-05-21T15:21:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 0x55ec10edae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10edb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b30390 0x55ec10ed92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ec10edbe30 con 0x55ec10b30390 2024-05-21T15:21:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ec10defc40 con 0x55ec10b074d0 2024-05-21T15:21:11.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec10edbfb0 con 0x55ec10b0f6b0 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0533ba1740 10 client.0 _ll_register_callbacks cb 0x55ec10b47be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.414+0000 7f0526d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b30390 0x55ec10ed92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0526d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b30390 0x55ec10ed92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54314/0 (socket says 172.21.15.78:54314) 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0526d9d640 1 -- 172.21.15.78:0/388316976 learned_addr learned my addr 172.21.15.78:0/388316976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f052659c640 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 0x55ec10edae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0525d9b640 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10edb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0526d9d640 1 -- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 msgr2=0x55ec10edb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0526d9d640 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10edb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0526d9d640 1 -- 172.21.15.78:0/388316976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 msgr2=0x55ec10edae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0526d9d640 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 0x55ec10edae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0526d9d640 1 -- 172.21.15.78:0/388316976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec10c127d0 con 0x55ec10b30390 2024-05-21T15:21:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0525d9b640 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10edb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0526d9d640 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b30390 0x55ec10ed92b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0520004160 tx=0x7f0520004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215a17a3be242545 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f050dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f050dffb640 1 -- 172.21.15.78:0/388316976 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520017040 con 0x55ec10b30390 2024-05-21T15:21:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f050dffb640 1 -- 172.21.15.78:0/388316976 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f052001e800 con 0x55ec10b30390 2024-05-21T15:21:11.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0533ba1740 1 -- 172.21.15.78:0/388316976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f051c0dc0d0 con 0x55ec10b30390 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f050dffb640 1 -- 172.21.15.78:0/388316976 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f052001ea00 con 0x55ec10b30390 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0533ba1740 10 client.17085 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f0533ba1740 1 -- 172.21.15.78:0/388316976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ec10ee8560 con 0x55ec10b30390 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.415+0000 7f050dffb640 1 -- 172.21.15.78:0/388316976 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c0dc0d0 con 0x55ec10b30390 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f050dffb640 1 -- 172.21.15.78:0/388316976 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0520025020 con 0x55ec10b30390 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f0533ba1740 10 client.17085 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f0533ba1740 10 client.17085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f0533ba1740 20 client.17085 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:11.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f0533ba1740 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ec10aeef60 0x55ec10eeba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f0533ba1740 4 client.17085 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f0533ba1740 1 -- 172.21.15.78:0/388316976 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec10e42370 con 0x55ec10aeef60 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f052659c640 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ec10aeef60 0x55ec10eeba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f050dffb640 10 client.17085 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.416+0000 7f052659c640 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ec10aeef60 0x55ec10eeba90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a04de72633cff49b server_cookie=9c6035dfef639eb2 in_seq=0 out_seq=0 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f050dffb640 1 -- 172.21.15.78:0/388316976 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec10e42370 con 0x55ec10aeef60 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f050dffb640 10 client.17085 handle_command_reply: tid=0 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 client.17085 shutdown 2024-05-21T15:21:11.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 -- 172.21.15.78:0/388316976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b30390 msgr2=0x55ec10ed92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b30390 0x55ec10ed92b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0520004160 tx=0x7f0520004ab0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 -- 172.21.15.78:0/388316976 shutdown_connections 2024-05-21T15:21:11.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ec10aeef60 0x55ec10eeba90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b30390 0x55ec10ed92b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f050dffb640 0 client.17085 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:21:11.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b0f6b0 0x55ec10edb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 --2- 172.21.15.78:0/388316976 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 0x55ec10edae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 -- 172.21.15.78:0/388316976 >> 172.21.15.78:0/388316976 conn(0x55ec10b244e0 msgr2=0x55ec10ed0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 -- 172.21.15.78:0/388316976 shutdown_connections 2024-05-21T15:21:11.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.417+0000 7f0533ba1740 1 -- 172.21.15.78:0/388316976 wait complete. 2024-05-21T15:21:11.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.418+0000 7f0533ba1740 20 client.17085 trim_cache size 0 max 16384 2024-05-21T15:21:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0533ba1740 1 Processor -- start 2024-05-21T15:21:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0533ba1740 1 -- start start 2024-05-21T15:21:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b0f6b0 0x55ec10ef1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b30390 0x55ec10ef1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0533ba1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05281aec60 con 0x55ec10b074d0 2024-05-21T15:21:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0533ba1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05281b07a0 con 0x55ec10b0f6b0 2024-05-21T15:21:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0533ba1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05281af330 con 0x55ec10b30390 2024-05-21T15:21:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f052659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f052659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54320/0 (socket says 172.21.15.78:54320) 2024-05-21T15:21:11.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f052659c640 1 -- 172.21.15.78:0/543464431 learned_addr learned my addr 172.21.15.78:0/543464431 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.431+0000 7f0525d9b640 1 --2- 172.21.15.78:0/543464431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b0f6b0 0x55ec10ef1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f052659c640 1 -- 172.21.15.78:0/543464431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b30390 msgr2=0x55ec10ef1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f052659c640 1 --2- 172.21.15.78:0/543464431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b30390 0x55ec10ef1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f052659c640 1 -- 172.21.15.78:0/543464431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b0f6b0 msgr2=0x55ec10ef1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f052659c640 1 --2- 172.21.15.78:0/543464431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b0f6b0 0x55ec10ef1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f052659c640 1 -- 172.21.15.78:0/543464431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec10ecc070 con 0x55ec10b074d0 2024-05-21T15:21:11.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f052659c640 1 --2- 172.21.15.78:0/543464431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf4e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0518005940 tx=0x7f051800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cbfd041771f455 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f050ffff640 1 -- 172.21.15.78:0/543464431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518004220 con 0x55ec10b074d0 2024-05-21T15:21:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f050ffff640 1 -- 172.21.15.78:0/543464431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05180043c0 con 0x55ec10b074d0 2024-05-21T15:21:11.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.432+0000 7f050ffff640 1 -- 172.21.15.78:0/543464431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518020890 con 0x55ec10b074d0 2024-05-21T15:21:11.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- 172.21.15.78:0/543464431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 msgr2=0x55ec10ecf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 --2- 172.21.15.78:0/543464431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf4e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0518005940 tx=0x7f051800b410 comp rx=0 tx=0).stop 2024-05-21T15:21:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- 172.21.15.78:0/543464431 shutdown_connections 2024-05-21T15:21:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 --2- 172.21.15.78:0/543464431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b0f6b0 0x55ec10ef1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 --2- 172.21.15.78:0/543464431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b30390 0x55ec10ef1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 --2- 172.21.15.78:0/543464431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b074d0 0x55ec10ecf4e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- 172.21.15.78:0/543464431 >> 172.21.15.78:0/543464431 conn(0x55ec10b244e0 msgr2=0x55ec10ed0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- 172.21.15.78:0/543464431 shutdown_connections 2024-05-21T15:21:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- 172.21.15.78:0/543464431 wait complete. 2024-05-21T15:21:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- start start 2024-05-21T15:21:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 Processor -- start 2024-05-21T15:21:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 0x55ec10f4c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b0f6b0 0x55ec10f4c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b30390 0x55ec10ede610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05281b00d0 con 0x55ec10b0f6b0 2024-05-21T15:21:11.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05281afa00 con 0x55ec10b074d0 2024-05-21T15:21:11.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05281acc40 con 0x55ec10b30390 2024-05-21T15:21:11.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0533ba1740 10 client.0 _ll_register_callbacks cb 0x7f0520003a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:11.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.433+0000 7f0525d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b0f6b0 0x55ec10f4c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0525d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b0f6b0 0x55ec10f4c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54334/0 (socket says 172.21.15.78:54334) 2024-05-21T15:21:11.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0525d9b640 1 -- 172.21.15.78:0/1426844438 learned_addr learned my addr 172.21.15.78:0/1426844438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f052659c640 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 0x55ec10f4c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0525d9b640 1 -- 172.21.15.78:0/1426844438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b30390 msgr2=0x55ec10ede610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0525d9b640 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b30390 0x55ec10ede610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0525d9b640 1 -- 172.21.15.78:0/1426844438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 msgr2=0x55ec10f4c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0525d9b640 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 0x55ec10f4c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0525d9b640 1 -- 172.21.15.78:0/1426844438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec10edec20 con 0x55ec10b0f6b0 2024-05-21T15:21:11.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0525d9b640 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b0f6b0 0x55ec10f4c750 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f051c0c4ec0 tx=0x7f051c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd0208414d002ab server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f050dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:11.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f050dffb640 1 -- 172.21.15.78:0/1426844438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c049070 con 0x55ec10b0f6b0 2024-05-21T15:21:11.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f050dffb640 1 -- 172.21.15.78:0/1426844438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f051c0b9750 con 0x55ec10b0f6b0 2024-05-21T15:21:11.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0533ba1740 1 -- 172.21.15.78:0/1426844438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05180043c0 con 0x55ec10b0f6b0 2024-05-21T15:21:11.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f050dffb640 1 -- 172.21.15.78:0/1426844438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f051c0dc190 con 0x55ec10b0f6b0 2024-05-21T15:21:11.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0533ba1740 10 client.17091 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:11.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f0533ba1740 1 -- 172.21.15.78:0/1426844438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ec10f538b0 con 0x55ec10b0f6b0 2024-05-21T15:21:11.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.434+0000 7f050dffb640 1 -- 172.21.15.78:0/1426844438 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05180043c0 con 0x55ec10b0f6b0 2024-05-21T15:21:11.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f050dffb640 1 -- 172.21.15.78:0/1426844438 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f051c0b8e30 con 0x55ec10b0f6b0 2024-05-21T15:21:11.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f0533ba1740 10 client.17091 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:11.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f0533ba1740 10 client.17091 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:21:11.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f0533ba1740 20 client.17091 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:11.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f0533ba1740 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ec10ac43a0 0x55ec10f55f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f0533ba1740 4 client.17091 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:11.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f0533ba1740 1 -- 172.21.15.78:0/1426844438 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec10e3c6c0 con 0x55ec10ac43a0 2024-05-21T15:21:11.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f052659c640 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ec10ac43a0 0x55ec10f55f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f050dffb640 10 client.17091 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:21:11.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f052659c640 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ec10ac43a0 0x55ec10f55f00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ba75b28220777d2 server_cookie=bd8b94f4b7953d70 in_seq=0 out_seq=0 2024-05-21T15:21:11.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f050dffb640 1 -- 172.21.15.78:0/1426844438 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec10e3c6c0 con 0x55ec10ac43a0 2024-05-21T15:21:11.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.435+0000 7f050dffb640 10 client.17091 handle_command_reply: tid=0 2024-05-21T15:21:11.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 client.17091 shutdown 2024-05-21T15:21:11.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 -- 172.21.15.78:0/1426844438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b0f6b0 msgr2=0x55ec10f4c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b0f6b0 0x55ec10f4c750 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f051c0c4ec0 tx=0x7f051c009560 comp rx=0 tx=0).stop 2024-05-21T15:21:11.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 -- 172.21.15.78:0/1426844438 shutdown_connections 2024-05-21T15:21:11.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ec10ac43a0 0x55ec10f55f00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec10b30390 0x55ec10ede610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f050dffb640 0 client.17091 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:21:11.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec10b0f6b0 0x55ec10f4c750 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 --2- 172.21.15.78:0/1426844438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ec10b074d0 0x55ec10f4c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 -- 172.21.15.78:0/1426844438 >> 172.21.15.78:0/1426844438 conn(0x55ec10b244e0 msgr2=0x55ec10ecf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 -- 172.21.15.78:0/1426844438 shutdown_connections 2024-05-21T15:21:11.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 1 -- 172.21.15.78:0/1426844438 wait complete. 2024-05-21T15:21:11.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f0533ba1740 20 client.17091 trim_cache size 0 max 16384 2024-05-21T15:21:11.459 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 -- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f05100a2d00 msgr2=0x7f05100a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f05100a2d00 0x7f05100a5210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0520005fd0 tx=0x7f0520011810 comp rx=0 tx=0).stop 2024-05-21T15:21:11.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 -- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 msgr2=0x7f0528199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 0x7f0528199ce0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f051c004770 tx=0x7f051c003c10 comp rx=0 tx=0).stop 2024-05-21T15:21:11.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 -- 172.21.15.78:0/2546774651 shutdown_connections 2024-05-21T15:21:11.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f05100a2d00 0x7f05100a5210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052810d040 0x7f052819dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052810a650 0x7f0528199ce0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 --2- 172.21.15.78:0/2546774651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0528103bf0 0x7f0528199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 -- 172.21.15.78:0/2546774651 >> 172.21.15.78:0/2546774651 conn(0x7f05280fd210 msgr2=0x7f052810d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 -- 172.21.15.78:0/2546774651 shutdown_connections 2024-05-21T15:21:11.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.436+0000 7f052cc22640 1 -- 172.21.15.78:0/2546774651 wait complete. 2024-05-21T15:21:11.463 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:21:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 -- 172.21.15.78:0/3902180728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff40103bd0 msgr2=0x7eff40103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 --2- 172.21.15.78:0/3902180728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff40103bd0 0x7eff40103fe0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff3002a480 comp rx=0 tx=0).stop 2024-05-21T15:21:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 -- 172.21.15.78:0/3902180728 shutdown_connections 2024-05-21T15:21:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 --2- 172.21.15.78:0/3902180728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff4010d020 0x7eff4010f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 --2- 172.21.15.78:0/3902180728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff4010a630 0x7eff4010ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 --2- 172.21.15.78:0/3902180728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff40103bd0 0x7eff40103fe0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 -- 172.21.15.78:0/3902180728 >> 172.21.15.78:0/3902180728 conn(0x7eff40077130 msgr2=0x7eff40077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 -- 172.21.15.78:0/3902180728 shutdown_connections 2024-05-21T15:21:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 -- 172.21.15.78:0/3902180728 wait complete. 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 Processor -- start 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.573+0000 7eff464e9640 1 -- start start 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff464e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff40103bd0 0x7eff401997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff464e9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff4010a630 0x7eff40199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff464e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff4010d020 0x7eff4019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff464e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff40112dd0 con 0x7eff4010d020 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff464e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7eff40112c50 con 0x7eff40103bd0 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff464e9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff40112f50 con 0x7eff4010a630 2024-05-21T15:21:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff4010d020 0x7eff4019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff4010d020 0x7eff4019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54342/0 (socket says 172.21.15.78:54342) 2024-05-21T15:21:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 -- 172.21.15.78:0/3457871010 learned_addr learned my addr 172.21.15.78:0/3457871010 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff3ffff640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff40103bd0 0x7eff401997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff37fff640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff4010a630 0x7eff40199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 -- 172.21.15.78:0/3457871010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff4010a630 msgr2=0x7eff40199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff4010a630 0x7eff40199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 -- 172.21.15.78:0/3457871010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff40103bd0 msgr2=0x7eff401997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff40103bd0 0x7eff401997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 -- 172.21.15.78:0/3457871010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4019e610 con 0x7eff4010d020 2024-05-21T15:21:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff37fff640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff4010a630 0x7eff40199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff44a86640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff4010d020 0x7eff4019dfd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7eff3800eb10 tx=0x7eff3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff3dffb640 1 -- 172.21.15.78:0/3457871010 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38004b20 con 0x7eff4010d020 2024-05-21T15:21:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff464e9640 1 -- 172.21.15.78:0/3457871010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4019e900 con 0x7eff4010d020 2024-05-21T15:21:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff3dffb640 1 -- 172.21.15.78:0/3457871010 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff38004cc0 con 0x7eff4010d020 2024-05-21T15:21:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff3dffb640 1 -- 172.21.15.78:0/3457871010 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38010760 con 0x7eff4010d020 2024-05-21T15:21:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.574+0000 7eff464e9640 1 -- 172.21.15.78:0/3457871010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4019ee00 con 0x7eff4010d020 2024-05-21T15:21:11.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff4d392740 1 Processor -- start 2024-05-21T15:21:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff3dffb640 1 -- 172.21.15.78:0/3457871010 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7eff38010900 con 0x7eff4010d020 2024-05-21T15:21:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff4d392740 1 -- start start 2024-05-21T15:21:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff4d392740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 0x560293eccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff4d392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ecd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff4d392740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 0x560293ecd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff4d392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560293e32c40 con 0x560293b054d0 2024-05-21T15:21:11.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff4d392740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560293dda580 con 0x560293b0d6b0 2024-05-21T15:21:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.577+0000 7eff4d392740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560293e2ec00 con 0x560293b2e390 2024-05-21T15:21:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ecd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ecd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54352/0 (socket says 172.21.15.78:54352) 2024-05-21T15:21:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 -- 172.21.15.78:0/3075620484 learned_addr learned my addr 172.21.15.78:0/3075620484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff37fff640 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 0x560293eccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3dffb640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7eff2c0a2d00 0x7eff2c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff37fff640 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 0x560293ecd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 -- 172.21.15.78:0/3075620484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 msgr2=0x560293eccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 0x560293eccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 -- 172.21.15.78:0/3075620484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 msgr2=0x560293ecd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 0x560293ecd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 -- 172.21.15.78:0/3075620484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560293b463f0 con 0x560293b054d0 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff44a86640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7eff2c0a2d00 0x7eff2c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3dffb640 1 -- 172.21.15.78:0/3457871010 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7eff380cfab0 con 0x7eff4010d020 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff3ffff640 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ecd260 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff30005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c1ef337f793280 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff367fc640 1 -- 172.21.15.78:0/3075620484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30040070 con 0x560293b054d0 2024-05-21T15:21:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff367fc640 1 -- 172.21.15.78:0/3075620484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30031970 con 0x560293b054d0 2024-05-21T15:21:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.578+0000 7eff367fc640 1 -- 172.21.15.78:0/3075620484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30035040 con 0x560293b054d0 2024-05-21T15:21:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 -- 172.21.15.78:0/3075620484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 msgr2=0x560293ecd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ecd260 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff30005e80 comp rx=0 tx=0).stop 2024-05-21T15:21:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff44a86640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7eff2c0a2d00 0x7eff2c0a5210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff3809bc70 tx=0x7eff3806df30 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 -- 172.21.15.78:0/3075620484 shutdown_connections 2024-05-21T15:21:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 0x560293ecd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ecd260 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 --2- 172.21.15.78:0/3075620484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 0x560293eccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 -- 172.21.15.78:0/3075620484 >> 172.21.15.78:0/3075620484 conn(0x560293b224e0 msgr2=0x560293ec94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 -- 172.21.15.78:0/3075620484 shutdown_connections 2024-05-21T15:21:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.579+0000 7eff4d392740 1 -- 172.21.15.78:0/3075620484 wait complete. 2024-05-21T15:21:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 1 -- start start 2024-05-21T15:21:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 1 Processor -- start 2024-05-21T15:21:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ed8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 0x560293ed9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 0x560293ed72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560293ed9fb0 con 0x560293b054d0 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560293dedc40 con 0x560293b0d6b0 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560293eda130 con 0x560293b2e390 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff4d392740 10 client.0 _ll_register_callbacks cb 0x560293b45be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ed8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ed8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54364/0 (socket says 172.21.15.78:54364) 2024-05-21T15:21:11.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.580+0000 7eff37fff640 1 -- 172.21.15.78:0/3115581077 learned_addr learned my addr 172.21.15.78:0/3115581077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff37fff640 1 -- 172.21.15.78:0/3115581077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 msgr2=0x560293ed72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff37fff640 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 0x560293ed72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff37fff640 1 -- 172.21.15.78:0/3115581077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 msgr2=0x560293ed9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff37fff640 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 0x560293ed9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff37fff640 1 -- 172.21.15.78:0/3115581077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560293c107d0 con 0x560293b054d0 2024-05-21T15:21:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff37fff640 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ed8e10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7eff2800bfa0 tx=0x7eff28013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1567ba324477cec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff1f7fe640 1 -- 172.21.15.78:0/3115581077 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28010d30 con 0x560293b054d0 2024-05-21T15:21:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff1f7fe640 1 -- 172.21.15.78:0/3115581077 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7eff2801b410 con 0x560293b054d0 2024-05-21T15:21:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 1 -- 172.21.15.78:0/3115581077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff30031970 con 0x560293b054d0 2024-05-21T15:21:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff1f7fe640 1 -- 172.21.15.78:0/3115581077 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7eff2801b610 con 0x560293b054d0 2024-05-21T15:21:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 10 client.17103 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 1 -- 172.21.15.78:0/3115581077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560293ee6810 con 0x560293b054d0 2024-05-21T15:21:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff1f7fe640 1 -- 172.21.15.78:0/3115581077 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff30031970 con 0x560293b054d0 2024-05-21T15:21:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff1f7fe640 1 -- 172.21.15.78:0/3115581077 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7eff2800f040 con 0x560293b054d0 2024-05-21T15:21:11.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 10 client.17103 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:11.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:11.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 20 client.17103 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:11.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560293aecf60 0x560293ee9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 4 client.17103 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:11.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.581+0000 7eff4d392740 1 -- 172.21.15.78:0/3115581077 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560293e40370 con 0x560293aecf60 2024-05-21T15:21:11.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.582+0000 7eff3ffff640 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560293aecf60 0x560293ee9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.582+0000 7eff1f7fe640 10 client.17103 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:11.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.582+0000 7eff3ffff640 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560293aecf60 0x560293ee9db0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44981e51aba70e7 server_cookie=8124551aec0f792b in_seq=0 out_seq=0 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff1f7fe640 1 -- 172.21.15.78:0/3115581077 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560293e40370 con 0x560293aecf60 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff1f7fe640 10 client.17103 handle_command_reply: tid=0 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 client.17103 shutdown 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 -- 172.21.15.78:0/3115581077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 msgr2=0x560293ed8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ed8e10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7eff2800bfa0 tx=0x7eff28013ce0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 -- 172.21.15.78:0/3115581077 shutdown_connections 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560293aecf60 0x560293ee9db0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b0d6b0 0x560293ed9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b2e390 0x560293ed72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff1f7fe640 0 client.17103 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 --2- 172.21.15.78:0/3115581077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b054d0 0x560293ed8e10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 -- 172.21.15.78:0/3115581077 >> 172.21.15.78:0/3115581077 conn(0x560293b224e0 msgr2=0x560293ecea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 -- 172.21.15.78:0/3115581077 shutdown_connections 2024-05-21T15:21:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 1 -- 172.21.15.78:0/3115581077 wait complete. 2024-05-21T15:21:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.583+0000 7eff4d392740 20 client.17103 trim_cache size 0 max 16384 2024-05-21T15:21:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff4d392740 1 Processor -- start 2024-05-21T15:21:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff4d392740 1 -- start start 2024-05-21T15:21:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff4d392740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 0x560293ed7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff4d392740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 0x560293ec8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff4d392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293ec9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff4d392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff401aecd0 con 0x560293b2e390 2024-05-21T15:21:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff4d392740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7eff401b0810 con 0x560293b054d0 2024-05-21T15:21:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff4d392740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff401af3a0 con 0x560293b0d6b0 2024-05-21T15:21:11.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293ec9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293ec9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54368/0 (socket says 172.21.15.78:54368) 2024-05-21T15:21:11.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 -- 172.21.15.78:0/476948325 learned_addr learned my addr 172.21.15.78:0/476948325 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 -- 172.21.15.78:0/476948325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 msgr2=0x560293ec8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 --2- 172.21.15.78:0/476948325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 0x560293ec8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 -- 172.21.15.78:0/476948325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 msgr2=0x560293ed7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 --2- 172.21.15.78:0/476948325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 0x560293ed7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 -- 172.21.15.78:0/476948325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560293eca0a0 con 0x560293b2e390 2024-05-21T15:21:11.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.597+0000 7eff37fff640 1 --2- 172.21.15.78:0/476948325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293ec9090 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7eff28005f30 tx=0x7eff28013f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e122f0c975542e2b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff35ffb640 1 -- 172.21.15.78:0/476948325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff280113e0 con 0x560293b2e390 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff35ffb640 1 -- 172.21.15.78:0/476948325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff28030070 con 0x560293b2e390 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff35ffb640 1 -- 172.21.15.78:0/476948325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28005360 con 0x560293b2e390 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 -- 172.21.15.78:0/476948325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 msgr2=0x560293ec9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 --2- 172.21.15.78:0/476948325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293ec9090 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7eff28005f30 tx=0x7eff28013f80 comp rx=0 tx=0).stop 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 -- 172.21.15.78:0/476948325 shutdown_connections 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 --2- 172.21.15.78:0/476948325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293ec9090 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 --2- 172.21.15.78:0/476948325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 0x560293ec8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 --2- 172.21.15.78:0/476948325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 0x560293ed7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 -- 172.21.15.78:0/476948325 >> 172.21.15.78:0/476948325 conn(0x560293b224e0 msgr2=0x560293ed6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 -- 172.21.15.78:0/476948325 shutdown_connections 2024-05-21T15:21:11.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.598+0000 7eff4d392740 1 -- 172.21.15.78:0/476948325 wait complete. 2024-05-21T15:21:11.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 1 -- start start 2024-05-21T15:21:11.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 1 Processor -- start 2024-05-21T15:21:11.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 0x560293edf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 0x560293edf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293edc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff401b0140 con 0x560293b2e390 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7eff401afa70 con 0x560293b054d0 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff401accb0 con 0x560293b0d6b0 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff4d392740 10 client.0 _ll_register_callbacks cb 0x7eff2800b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293edc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff37fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293edc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54372/0 (socket says 172.21.15.78:54372) 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.599+0000 7eff37fff640 1 -- 172.21.15.78:0/56695938 learned_addr learned my addr 172.21.15.78:0/56695938 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff37fff640 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 0x560293edf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff3ffff640 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 0x560293edf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff37fff640 1 -- 172.21.15.78:0/56695938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 msgr2=0x560293edf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff37fff640 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 0x560293edf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff37fff640 1 -- 172.21.15.78:0/56695938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 msgr2=0x560293edf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:11.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff37fff640 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 0x560293edf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff37fff640 1 -- 172.21.15.78:0/56695938 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560293edcd70 con 0x560293b2e390 2024-05-21T15:21:11.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff3ffff640 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 0x560293edf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:11.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff37fff640 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293edc680 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7eff2802d040 tx=0x7eff28011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb1033d55d73c96 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:11.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:11.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff1f7fe640 1 -- 172.21.15.78:0/56695938 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2803b030 con 0x560293b2e390 2024-05-21T15:21:11.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff1f7fe640 1 -- 172.21.15.78:0/56695938 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7eff28011c50 con 0x560293b2e390 2024-05-21T15:21:11.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff4d392740 1 -- 172.21.15.78:0/56695938 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff28030070 con 0x560293b2e390 2024-05-21T15:21:11.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff1f7fe640 1 -- 172.21.15.78:0/56695938 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7eff280053b0 con 0x560293b2e390 2024-05-21T15:21:11.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff4d392740 10 client.17109 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:11.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff4d392740 1 -- 172.21.15.78:0/56695938 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560293f51a90 con 0x560293b2e390 2024-05-21T15:21:11.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff1f7fe640 1 -- 172.21.15.78:0/56695938 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff28030070 con 0x560293b2e390 2024-05-21T15:21:11.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.600+0000 7eff1f7fe640 1 -- 172.21.15.78:0/56695938 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7eff2801b9b0 con 0x560293b2e390 2024-05-21T15:21:11.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff4d392740 10 client.17109 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:11.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff4d392740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:11.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff4d392740 20 client.17109 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:11.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff4d392740 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560293ac23a0 0x560293f53fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:11.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff4d392740 4 client.17109 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:11.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff4d392740 1 -- 172.21.15.78:0/56695938 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560293e3a6c0 con 0x560293ac23a0 2024-05-21T15:21:11.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff3ffff640 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560293ac23a0 0x560293f53fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:11.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff3ffff640 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560293ac23a0 0x560293f53fe0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f498adead53a9cd server_cookie=db02de93c6bd83ec in_seq=0 out_seq=0 2024-05-21T15:21:11.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff1f7fe640 10 client.17109 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:11.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff1f7fe640 1 -- 172.21.15.78:0/56695938 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560293e3a6c0 con 0x560293ac23a0 2024-05-21T15:21:11.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff1f7fe640 10 client.17109 handle_command_reply: tid=0 2024-05-21T15:21:11.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.601+0000 7eff4d392740 1 client.17109 shutdown 2024-05-21T15:21:11.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 -- 172.21.15.78:0/56695938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 msgr2=0x560293edc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293edc680 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7eff2802d040 tx=0x7eff28011ef0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 -- 172.21.15.78:0/56695938 shutdown_connections 2024-05-21T15:21:11.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x560293ac23a0 0x560293f53fe0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560293b2e390 0x560293edc680 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560293b0d6b0 0x560293edf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 --2- 172.21.15.78:0/56695938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560293b054d0 0x560293edf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 -- 172.21.15.78:0/56695938 >> 172.21.15.78:0/56695938 conn(0x560293b224e0 msgr2=0x560293ed8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff1f7fe640 0 client.17109 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:11.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 -- 172.21.15.78:0/56695938 shutdown_connections 2024-05-21T15:21:11.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 1 -- 172.21.15.78:0/56695938 wait complete. 2024-05-21T15:21:11.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff4d392740 20 client.17109 trim_cache size 0 max 16384 2024-05-21T15:21:11.617 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 -- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7eff2c0a2d00 msgr2=0x7eff2c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7eff2c0a2d00 0x7eff2c0a5210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff3809bc70 tx=0x7eff3806df30 comp rx=0 tx=0).stop 2024-05-21T15:21:11.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 -- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff4010d020 msgr2=0x7eff4019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:11.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff4010d020 0x7eff4019dfd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7eff3800eb10 tx=0x7eff3800c2d0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 -- 172.21.15.78:0/3457871010 shutdown_connections 2024-05-21T15:21:11.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7eff2c0a2d00 0x7eff2c0a5210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff4010d020 0x7eff4019dfd0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff4010a630 0x7eff40199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 --2- 172.21.15.78:0/3457871010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff40103bd0 0x7eff401997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:11.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 -- 172.21.15.78:0/3457871010 >> 172.21.15.78:0/3457871010 conn(0x7eff40077130 msgr2=0x7eff40100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:11.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 -- 172.21.15.78:0/3457871010 shutdown_connections 2024-05-21T15:21:11.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:11.602+0000 7eff464e9640 1 -- 172.21.15.78:0/3457871010 wait complete. 2024-05-21T15:21:11.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:11 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/304416213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:21:11.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:11 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/304416213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:21:11.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:11 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/304416213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:21:12.621 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:21:12.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:21:12.621 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:21:12.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:12 smithi172 ceph-mon[36402]: pgmap v156: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 322 op/s 2024-05-21T15:21:12.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 -- 172.21.15.78:0/2159104849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 msgr2=0x7fd170102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 --2- 172.21.15.78:0/2159104849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 0x7fd170102780 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd160009900 tx=0x7fd16002a410 comp rx=0 tx=0).stop 2024-05-21T15:21:12.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 -- 172.21.15.78:0/2159104849 shutdown_connections 2024-05-21T15:21:12.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 --2- 172.21.15.78:0/2159104849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170106230 0x7fd170106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 --2- 172.21.15.78:0/2159104849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd170106cb0 0x7fd17010e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 --2- 172.21.15.78:0/2159104849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 0x7fd170102780 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 -- 172.21.15.78:0/2159104849 >> 172.21.15.78:0/2159104849 conn(0x7fd1700768d0 msgr2=0x7fd170076d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 -- 172.21.15.78:0/2159104849 shutdown_connections 2024-05-21T15:21:12.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.743+0000 7fd175ca6640 1 -- 172.21.15.78:0/2159104849 wait complete. 2024-05-21T15:21:12.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.744+0000 7fd175ca6640 1 Processor -- start 2024-05-21T15:21:12.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.744+0000 7fd175ca6640 1 -- start start 2024-05-21T15:21:12.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd175ca6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 0x7fd170198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd175ca6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170106230 0x7fd170198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd175ca6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd170106cb0 0x7fd17019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd175ca6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd170111d00 con 0x7fd170102370 2024-05-21T15:21:12.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd175ca6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd170111b80 con 0x7fd170106230 2024-05-21T15:21:12.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd175ca6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd170111e80 con 0x7fd170106cb0 2024-05-21T15:21:12.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 0x7fd170198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 0x7fd170198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54380/0 (socket says 172.21.15.78:54380) 2024-05-21T15:21:12.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 -- 172.21.15.78:0/3227730320 learned_addr learned my addr 172.21.15.78:0/3227730320 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16effd640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170106230 0x7fd170198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd170106cb0 0x7fd17019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 -- 172.21.15.78:0/3227730320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd170106cb0 msgr2=0x7fd17019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd170106cb0 0x7fd17019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 -- 172.21.15.78:0/3227730320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170106230 msgr2=0x7fd170198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170106230 0x7fd170198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 -- 172.21.15.78:0/3227730320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17019d460 con 0x7fd170102370 2024-05-21T15:21:12.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd170106cb0 0x7fd17019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:12.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 0x7fd170198680 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd1600042f0 tx=0x7fd160004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16cff9640 1 -- 172.21.15.78:0/3227730320 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16003f030 con 0x7fd170102370 2024-05-21T15:21:12.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd175ca6640 1 -- 172.21.15.78:0/3227730320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17019d6f0 con 0x7fd170102370 2024-05-21T15:21:12.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16cff9640 1 -- 172.21.15.78:0/3227730320 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160003ce0 con 0x7fd170102370 2024-05-21T15:21:12.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.745+0000 7fd16cff9640 1 -- 172.21.15.78:0/3227730320 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160035070 con 0x7fd170102370 2024-05-21T15:21:12.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.746+0000 7fd175ca6640 1 -- 172.21.15.78:0/3227730320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17019dc30 con 0x7fd170102370 2024-05-21T15:21:12.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.748+0000 7fd16cff9640 1 -- 172.21.15.78:0/3227730320 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd160042030 con 0x7fd170102370 2024-05-21T15:21:12.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.748+0000 7fd17cc1f740 1 Processor -- start 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.748+0000 7fd17cc1f740 1 -- start start 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.748+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c216830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.748+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be4c550 0x55760c216d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.748+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c2172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55760c179990 con 0x55760be75410 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55760c1215b0 con 0x55760be4c550 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55760c175950 con 0x55760be54730 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c216830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c216830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54392/0 (socket says 172.21.15.78:54392) 2024-05-21T15:21:12.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 -- 172.21.15.78:0/3932050360 learned_addr learned my addr 172.21.15.78:0/3932050360 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be4c550 0x55760c216d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16effd640 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c2172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16cff9640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd1580a2d00 0x7fd1580a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 -- 172.21.15.78:0/3932050360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 msgr2=0x55760c2172b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c2172b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 -- 172.21.15.78:0/3932050360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be4c550 msgr2=0x55760c216d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be4c550 0x55760c216d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 -- 172.21.15.78:0/3932050360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55760bf557e0 con 0x55760be75410 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be4c550 0x55760c216d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd1580a2d00 0x7fd1580a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16cff9640 1 -- 172.21.15.78:0/3227730320 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd1600efb90 con 0x7fd170102370 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c216830 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd16800d340 tx=0x7fd16800d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2318124776859bc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.749+0000 7fd155ffb640 1 -- 172.21.15.78:0/3932050360 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16801a920 con 0x55760be75410 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd1580a2d00 0x7fd1580a5210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd16801e000 tx=0x7fd16801f000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd155ffb640 1 -- 172.21.15.78:0/3932050360 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd168019410 con 0x55760be75410 2024-05-21T15:21:12.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd155ffb640 1 -- 172.21.15.78:0/3932050360 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16800e610 con 0x55760be75410 2024-05-21T15:21:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3932050360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 msgr2=0x55760c216830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c216830 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd16800d340 tx=0x7fd16800d800 comp rx=0 tx=0).stop 2024-05-21T15:21:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3932050360 shutdown_connections 2024-05-21T15:21:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be4c550 0x55760c216d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c2172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/3932050360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c216830 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3932050360 >> 172.21.15.78:0/3932050360 conn(0x55760be69560 msgr2=0x55760c212fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3932050360 shutdown_connections 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.750+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3932050360 wait complete. 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.751+0000 7fd17cc1f740 1 -- start start 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.751+0000 7fd17cc1f740 1 Processor -- start 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.751+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c2228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.751+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c222e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.751+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c220d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.751+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55760c223b80 con 0x55760be4c550 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.751+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55760c134b40 con 0x55760be75410 2024-05-21T15:21:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.751+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55760c223d00 con 0x55760be54730 2024-05-21T15:21:12.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c2228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd17cc1f740 10 client.0 _ll_register_callbacks cb 0x55760c1290f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:12.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c2228e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54408/0 (socket says 172.21.15.78:54408) 2024-05-21T15:21:12.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 -- 172.21.15.78:0/236338771 learned_addr learned my addr 172.21.15.78:0/236338771 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16effd640 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c222e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16ffff640 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c220d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 -- 172.21.15.78:0/236338771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 msgr2=0x55760c222e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c222e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 -- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 msgr2=0x55760c220d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c220d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 -- 172.21.15.78:0/236338771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55760bfb5f70 con 0x55760be4c550 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16ffff640 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c220d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c2228e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd160006010 tx=0x7fd16002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c052b4b8ea65fc98 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd14effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd14effd640 1 -- 172.21.15.78:0/236338771 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160049040 con 0x55760be4c550 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd14effd640 1 -- 172.21.15.78:0/236338771 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd16003b460 con 0x55760be4c550 2024-05-21T15:21:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd17cc1f740 1 -- 172.21.15.78:0/236338771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd168019410 con 0x55760be4c550 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd14effd640 1 -- 172.21.15.78:0/236338771 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd1600f4040 con 0x55760be4c550 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd17cc1f740 10 client.17121 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd17cc1f740 1 -- 172.21.15.78:0/236338771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55760be4d170 con 0x55760be4c550 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.752+0000 7fd14effd640 1 -- 172.21.15.78:0/236338771 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd168019410 con 0x55760be4c550 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.753+0000 7fd14effd640 1 -- 172.21.15.78:0/236338771 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd1600ed4a0 con 0x55760be4c550 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.753+0000 7fd17cc1f740 10 client.17121 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.753+0000 7fd17cc1f740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.753+0000 7fd17cc1f740 20 client.17121 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.753+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55760be341f0 0x55760c233410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.753+0000 7fd17cc1f740 4 client.17121 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:12.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.753+0000 7fd17cc1f740 1 -- 172.21.15.78:0/236338771 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55760c1870c0 con 0x55760be341f0 2024-05-21T15:21:12.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.753+0000 7fd16effd640 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55760be341f0 0x55760c233410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.754+0000 7fd16effd640 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55760be341f0 0x55760c233410 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6869378e078903 server_cookie=77d8e87aa3abe7bc in_seq=0 out_seq=0 2024-05-21T15:21:12.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.754+0000 7fd14effd640 10 client.17121 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:12.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd14effd640 1 -- 172.21.15.78:0/236338771 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55760c1870c0 con 0x55760be341f0 2024-05-21T15:21:12.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd14effd640 10 client.17121 handle_command_reply: tid=0 2024-05-21T15:21:12.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 client.17121 shutdown 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 -- 172.21.15.78:0/236338771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 msgr2=0x55760c2228e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c2228e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd160006010 tx=0x7fd16002a410 comp rx=0 tx=0).stop 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 -- 172.21.15.78:0/236338771 shutdown_connections 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55760be341f0 0x55760c233410 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c220d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c222e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/236338771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c2228e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd14effd640 0 client.17121 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:12.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 -- 172.21.15.78:0/236338771 >> 172.21.15.78:0/236338771 conn(0x55760be69560 msgr2=0x55760c216600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 -- 172.21.15.78:0/236338771 shutdown_connections 2024-05-21T15:21:12.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 1 -- 172.21.15.78:0/236338771 wait complete. 2024-05-21T15:21:12.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.755+0000 7fd17cc1f740 20 client.17121 trim_cache size 0 max 16384 2024-05-21T15:21:12.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.773+0000 7fd17cc1f740 1 Processor -- start 2024-05-21T15:21:12.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.773+0000 7fd17cc1f740 1 -- start start 2024-05-21T15:21:12.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.773+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be4c550 0x55760c238bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.773+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be54730 0x55760c239110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.773+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c2205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.773+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ac9c0 con 0x55760be75410 2024-05-21T15:21:12.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.773+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ae500 con 0x55760be54730 2024-05-21T15:21:12.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.773+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ad090 con 0x55760be4c550 2024-05-21T15:21:12.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c2205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c2205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54414/0 (socket says 172.21.15.78:54414) 2024-05-21T15:21:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 -- 172.21.15.78:0/3597037792 learned_addr learned my addr 172.21.15.78:0/3597037792 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16effd640 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be54730 0x55760c239110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be4c550 0x55760c238bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 -- 172.21.15.78:0/3597037792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be4c550 msgr2=0x55760c238bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be4c550 0x55760c238bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 -- 172.21.15.78:0/3597037792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be54730 msgr2=0x55760c239110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be54730 0x55760c239110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 -- 172.21.15.78:0/3597037792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55760c21f2b0 con 0x55760be75410 2024-05-21T15:21:12.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16effd640 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be54730 0x55760c239110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:12.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd16ffff640 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c2205f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd16800dd00 tx=0x7fd1680084b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2879e4461699d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd1557fa640 1 -- 172.21.15.78:0/3597037792 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168008580 con 0x55760be75410 2024-05-21T15:21:12.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd1557fa640 1 -- 172.21.15.78:0/3597037792 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16801ae00 con 0x55760be75410 2024-05-21T15:21:12.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.774+0000 7fd1557fa640 1 -- 172.21.15.78:0/3597037792 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16800ea80 con 0x55760be75410 2024-05-21T15:21:12.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3597037792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 msgr2=0x55760c2205f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c2205f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd16800dd00 tx=0x7fd1680084b0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3597037792 shutdown_connections 2024-05-21T15:21:12.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be54730 0x55760c239110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be75410 0x55760c2205f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/3597037792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be4c550 0x55760c238bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3597037792 >> 172.21.15.78:0/3597037792 conn(0x55760be69560 msgr2=0x55760c2221f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3597037792 shutdown_connections 2024-05-21T15:21:12.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 -- 172.21.15.78:0/3597037792 wait complete. 2024-05-21T15:21:12.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.775+0000 7fd17cc1f740 1 -- start start 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd17cc1f740 1 Processor -- start 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c226b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c2270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd17cc1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c227600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ade30 con 0x55760be4c550 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ad760 con 0x55760be75410 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd17cc1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701aa9a0 con 0x55760be54730 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd17cc1f740 10 client.0 _ll_register_callbacks cb 0x7fd160005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:12.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c226b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c226b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54428/0 (socket says 172.21.15.78:54428) 2024-05-21T15:21:12.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16f7fe640 1 -- 172.21.15.78:0/1689496616 learned_addr learned my addr 172.21.15.78:0/1689496616 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16ffff640 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c227600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16effd640 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c2270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16f7fe640 1 -- 172.21.15.78:0/1689496616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 msgr2=0x55760c2270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c2270c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16f7fe640 1 -- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 msgr2=0x55760c227600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c227600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16f7fe640 1 -- 172.21.15.78:0/1689496616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55760c29a790 con 0x55760be4c550 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.776+0000 7fd16ffff640 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c227600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd16f7fe640 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c226b80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd16003d040 tx=0x7fd1600bc5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f13f72b1342d1bc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd14effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd14effd640 1 -- 172.21.15.78:0/1689496616 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160049040 con 0x55760be4c550 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd14effd640 1 -- 172.21.15.78:0/1689496616 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd160031dd0 con 0x55760be4c550 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 1 -- 172.21.15.78:0/1689496616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16801ae00 con 0x55760be4c550 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd14effd640 1 -- 172.21.15.78:0/1689496616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd1600b8290 con 0x55760be4c550 2024-05-21T15:21:12.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 10 client.17127 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 1 -- 172.21.15.78:0/1689496616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55760c29b960 con 0x55760be4c550 2024-05-21T15:21:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd14effd640 1 -- 172.21.15.78:0/1689496616 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16801ae00 con 0x55760be4c550 2024-05-21T15:21:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd14effd640 1 -- 172.21.15.78:0/1689496616 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd1600d8030 con 0x55760be4c550 2024-05-21T15:21:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 10 client.17127 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 20 client.17127 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55760be09630 0x55760c29df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 4 client.17127 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:21:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.777+0000 7fd17cc1f740 1 -- 172.21.15.78:0/1689496616 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55760c181410 con 0x55760be09630 2024-05-21T15:21:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.778+0000 7fd16effd640 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55760be09630 0x55760c29df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.778+0000 7fd14effd640 10 client.17127 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.778+0000 7fd16effd640 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55760be09630 0x55760c29df90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f557818eb2a3614 server_cookie=80370e94bc62837c in_seq=0 out_seq=0 2024-05-21T15:21:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd14effd640 1 -- 172.21.15.78:0/1689496616 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55760c181410 con 0x55760be09630 2024-05-21T15:21:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd14effd640 10 client.17127 handle_command_reply: tid=0 2024-05-21T15:21:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 client.17127 shutdown 2024-05-21T15:21:12.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 -- 172.21.15.78:0/1689496616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 msgr2=0x55760c226b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c226b80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd16003d040 tx=0x7fd1600bc5c0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 -- 172.21.15.78:0/1689496616 shutdown_connections 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55760be09630 0x55760c29df90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55760be75410 0x55760c227600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55760be54730 0x55760c2270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 --2- 172.21.15.78:0/1689496616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55760be4c550 0x55760c226b80 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd14effd640 0 client.17127 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.779+0000 7fd17cc1f740 1 -- 172.21.15.78:0/1689496616 >> 172.21.15.78:0/1689496616 conn(0x55760be69560 msgr2=0x55760c2128f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd17cc1f740 1 -- 172.21.15.78:0/1689496616 shutdown_connections 2024-05-21T15:21:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd17cc1f740 1 -- 172.21.15.78:0/1689496616 wait complete. 2024-05-21T15:21:12.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd17cc1f740 20 client.17127 trim_cache size 0 max 16384 2024-05-21T15:21:12.793 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:21:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:21:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "94ffe5d9-24fe-4e3d-8062-98040e8fe38f", 2024-05-21T15:21:12.793 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:21:12.794 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:21:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 -- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd1580a2d00 msgr2=0x7fd1580a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd1580a2d00 0x7fd1580a5210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd16801e000 tx=0x7fd16801f000 comp rx=0 tx=0).stop 2024-05-21T15:21:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 -- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 msgr2=0x7fd170198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 0x7fd170198680 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd1600042f0 tx=0x7fd160004320 comp rx=0 tx=0).stop 2024-05-21T15:21:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 -- 172.21.15.78:0/3227730320 shutdown_connections 2024-05-21T15:21:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd170106230 0x7fd170198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd1580a2d00 0x7fd1580a5210 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd170106cb0 0x7fd17019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 --2- 172.21.15.78:0/3227730320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd170102370 0x7fd170198680 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 -- 172.21.15.78:0/3227730320 >> 172.21.15.78:0/3227730320 conn(0x7fd1700768d0 msgr2=0x7fd1700fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.780+0000 7fd175ca6640 1 -- 172.21.15.78:0/3227730320 shutdown_connections 2024-05-21T15:21:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.781+0000 7fd175ca6640 1 -- 172.21.15.78:0/3227730320 wait complete. 2024-05-21T15:21:12.796 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:12 smithi078 ceph-mon[31997]: pgmap v156: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 322 op/s 2024-05-21T15:21:12.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:21:12.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:21:12.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:12 smithi084 ceph-mon[36334]: pgmap v156: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 322 op/s 2024-05-21T15:21:12.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 -- 172.21.15.78:0/3178302207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 msgr2=0x7f3c4c101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 --2- 172.21.15.78:0/3178302207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 0x7f3c4c101590 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009900 tx=0x7f3c4802a410 comp rx=0 tx=0).stop 2024-05-21T15:21:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 -- 172.21.15.78:0/3178302207 shutdown_connections 2024-05-21T15:21:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 --2- 172.21.15.78:0/3178302207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c4c10c360 0x7f3c4c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 --2- 172.21.15.78:0/3178302207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c4c101ad0 0x7f3c4c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 --2- 172.21.15.78:0/3178302207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 0x7f3c4c101590 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 -- 172.21.15.78:0/3178302207 >> 172.21.15.78:0/3178302207 conn(0x7f3c4c077100 msgr2=0x7f3c4c077540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 -- 172.21.15.78:0/3178302207 shutdown_connections 2024-05-21T15:21:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.908+0000 7f3c52b96640 1 -- 172.21.15.78:0/3178302207 wait complete. 2024-05-21T15:21:12.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c52b96640 1 Processor -- start 2024-05-21T15:21:12.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c52b96640 1 -- start start 2024-05-21T15:21:12.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c52b96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c4c101ad0 0x7f3c4c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c52b96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 0x7f3c4c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c52b96640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c4c10c360 0x7f3c4c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c52b96640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c111d60 con 0x7f3c4c107a00 2024-05-21T15:21:12.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c52b96640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c111be0 con 0x7f3c4c10c360 2024-05-21T15:21:12.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c52b96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c111ee0 con 0x7f3c4c101ad0 2024-05-21T15:21:12.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.909+0000 7f3c43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 0x7f3c4c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 0x7f3c4c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54456/0 (socket says 172.21.15.78:54456) 2024-05-21T15:21:12.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c43fff640 1 -- 172.21.15.78:0/1807669384 learned_addr learned my addr 172.21.15.78:0/1807669384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c50932640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c4c101ad0 0x7f3c4c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c43fff640 1 -- 172.21.15.78:0/1807669384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c4c101ad0 msgr2=0x7f3c4c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c4c101ad0 0x7f3c4c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c43fff640 1 -- 172.21.15.78:0/1807669384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c4c10c360 msgr2=0x7f3c4c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c51133640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c4c10c360 0x7f3c4c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c4c10c360 0x7f3c4c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c43fff640 1 -- 172.21.15.78:0/1807669384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c19d540 con 0x7f3c4c107a00 2024-05-21T15:21:12.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c51133640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c4c10c360 0x7f3c4c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:12.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 0x7f3c4c198c40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005d00 tx=0x7f3c3c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c41ffb640 1 -- 172.21.15.78:0/1807669384 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c017070 con 0x7f3c4c107a00 2024-05-21T15:21:12.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c41ffb640 1 -- 172.21.15.78:0/1807669384 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c009c30 con 0x7f3c4c107a00 2024-05-21T15:21:12.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c52b96640 1 -- 172.21.15.78:0/1807669384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c19d830 con 0x7f3c4c107a00 2024-05-21T15:21:12.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c41ffb640 1 -- 172.21.15.78:0/1807669384 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c00f620 con 0x7f3c4c107a00 2024-05-21T15:21:12.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.910+0000 7f3c52b96640 1 -- 172.21.15.78:0/1807669384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c19dd30 con 0x7f3c4c107a00 2024-05-21T15:21:12.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c41ffb640 1 -- 172.21.15.78:0/1807669384 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3c3c01a070 con 0x7f3c4c107a00 2024-05-21T15:21:12.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c59a3f740 1 Processor -- start 2024-05-21T15:21:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c59a3f740 1 -- start start 2024-05-21T15:21:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x5598794e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c41ffb640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c340a2d00 0x7f3c340a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55987944ad00 con 0x5598791256b0 2024-05-21T15:21:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5598793f24c0 con 0x559879146390 2024-05-21T15:21:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559879446cc0 con 0x55987911d4d0 2024-05-21T15:21:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.913+0000 7f3c51133640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c340a2d00 0x7f3c340a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c41ffb640 1 -- 172.21.15.78:0/1807669384 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3c3c0cefe0 con 0x7f3c4c107a00 2024-05-21T15:21:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54458/0 (socket says 172.21.15.78:54458) 2024-05-21T15:21:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 -- 172.21.15.78:0/2092778977 learned_addr learned my addr 172.21.15.78:0/2092778977 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c50932640 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x5598794e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c43fff640 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 -- 172.21.15.78:0/2092778977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 msgr2=0x5598794e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794e7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 -- 172.21.15.78:0/2092778977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 msgr2=0x5598794e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x5598794e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 -- 172.21.15.78:0/2092778977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598792287d0 con 0x5598791256b0 2024-05-21T15:21:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c340a2d00 0x7f3c340a5210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005fd0 tx=0x7f3c4400ad50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c51133640 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e79c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400fc10 tx=0x7f3c44019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77b183c06ff8555 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c32ffd640 1 -- 172.21.15.78:0/2092778977 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44022070 con 0x5598791256b0 2024-05-21T15:21:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c32ffd640 1 -- 172.21.15.78:0/2092778977 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4400dde0 con 0x5598791256b0 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.914+0000 7f3c32ffd640 1 -- 172.21.15.78:0/2092778977 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44010630 con 0x5598791256b0 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 -- 172.21.15.78:0/2092778977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 msgr2=0x5598794e79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e79c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400fc10 tx=0x7f3c44019000 comp rx=0 tx=0).stop 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 -- 172.21.15.78:0/2092778977 shutdown_connections 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e79c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/2092778977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x5598794e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 -- 172.21.15.78:0/2092778977 >> 172.21.15.78:0/2092778977 conn(0x55987913a4e0 msgr2=0x5598794e36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 -- 172.21.15.78:0/2092778977 shutdown_connections 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.915+0000 7f3c59a3f740 1 -- 172.21.15.78:0/2092778977 wait complete. 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 1 -- start start 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 1 Processor -- start 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598794f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598791256b0 0x5598794f3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x5598794f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5598794f4230 con 0x55987911d4d0 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559879405b90 con 0x559879146390 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5598794f43b0 con 0x5598791256b0 2024-05-21T15:21:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c59a3f740 10 client.0 _ll_register_callbacks cb 0x559879441da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c50932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598794f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c50932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598794f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54462/0 (socket says 172.21.15.78:54462) 2024-05-21T15:21:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c50932640 1 -- 172.21.15.78:0/3624665476 learned_addr learned my addr 172.21.15.78:0/3624665476 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c43fff640 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598791256b0 0x5598794f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c50932640 1 -- 172.21.15.78:0/3624665476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598791256b0 msgr2=0x5598794f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c50932640 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598791256b0 0x5598794f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c50932640 1 -- 172.21.15.78:0/3624665476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 msgr2=0x5598794f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.916+0000 7f3c50932640 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x5598794f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c50932640 1 -- 172.21.15.78:0/3624665476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987915e3f0 con 0x55987911d4d0 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c50932640 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598794f3040 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c102a00 tx=0x7f3c48005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66f62f5dbd9977f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c2bfff640 1 -- 172.21.15.78:0/3624665476 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48035040 con 0x55987911d4d0 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c2bfff640 1 -- 172.21.15.78:0/3624665476 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3c480048c0 con 0x55987911d4d0 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c59a3f740 1 -- 172.21.15.78:0/3624665476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c4400dde0 con 0x55987911d4d0 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c2bfff640 1 -- 172.21.15.78:0/3624665476 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3c48005650 con 0x55987911d4d0 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c59a3f740 10 client.17139 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c59a3f740 1 -- 172.21.15.78:0/3624665476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55987911e0f0 con 0x55987911d4d0 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c2bfff640 1 -- 172.21.15.78:0/3624665476 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4400dde0 con 0x55987911d4d0 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c2bfff640 1 -- 172.21.15.78:0/3624665476 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3c480315b0 con 0x55987911d4d0 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c59a3f740 10 client.17139 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c59a3f740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c59a3f740 20 client.17139 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.917+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559879104f60 0x559879503e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.918+0000 7f3c59a3f740 4 client.17139 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.918+0000 7f3c59a3f740 1 -- 172.21.15.78:0/3624665476 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559879458430 con 0x559879104f60 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.918+0000 7f3c43fff640 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559879104f60 0x559879503e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.918+0000 7f3c2bfff640 10 client.17139 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:12.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.918+0000 7f3c43fff640 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559879104f60 0x559879503e60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23150d3de9dcf38f server_cookie=fc02c59d43f08caa in_seq=0 out_seq=0 2024-05-21T15:21:12.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c2bfff640 1 -- 172.21.15.78:0/3624665476 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559879458430 con 0x559879104f60 2024-05-21T15:21:12.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c2bfff640 10 client.17139 handle_command_reply: tid=0 2024-05-21T15:21:12.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 client.17139 shutdown 2024-05-21T15:21:12.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 -- 172.21.15.78:0/3624665476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 msgr2=0x5598794f3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598794f3040 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c102a00 tx=0x7f3c48005250 comp rx=0 tx=0).stop 2024-05-21T15:21:12.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 -- 172.21.15.78:0/3624665476 shutdown_connections 2024-05-21T15:21:12.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559879104f60 0x559879503e60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x5598794f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c2bfff640 0 client.17139 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598791256b0 0x5598794f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/3624665476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598794f3040 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 -- 172.21.15.78:0/3624665476 >> 172.21.15.78:0/3624665476 conn(0x55987913a4e0 msgr2=0x5598794e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 -- 172.21.15.78:0/3624665476 shutdown_connections 2024-05-21T15:21:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 1 -- 172.21.15.78:0/3624665476 wait complete. 2024-05-21T15:21:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.919+0000 7f3c59a3f740 20 client.17139 trim_cache size 0 max 16384 2024-05-21T15:21:12.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c59a3f740 1 Processor -- start 2024-05-21T15:21:12.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c59a3f740 1 -- start start 2024-05-21T15:21:12.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x559879509620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1aca40 con 0x5598791256b0 2024-05-21T15:21:12.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1ae580 con 0x559879146390 2024-05-21T15:21:12.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1ad110 con 0x55987911d4d0 2024-05-21T15:21:12.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c43fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54468/0 (socket says 172.21.15.78:54468) 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c43fff640 1 -- 172.21.15.78:0/1139315107 learned_addr learned my addr 172.21.15.78:0/1139315107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c50932640 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c51133640 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x559879509620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c43fff640 1 -- 172.21.15.78:0/1139315107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 msgr2=0x5598794f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c43fff640 1 -- 172.21.15.78:0/1139315107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 msgr2=0x559879509620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x559879509620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c43fff640 1 -- 172.21.15.78:0/1139315107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598794e3c30 con 0x5598791256b0 2024-05-21T15:21:12.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c50932640 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:12.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.937+0000 7f3c51133640 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x559879509620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:12.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e36f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0cc710 tx=0x7f3c3c010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42df0daa78797dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c327fc640 1 -- 172.21.15.78:0/1139315107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c09bb80 con 0x5598791256b0 2024-05-21T15:21:12.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c327fc640 1 -- 172.21.15.78:0/1139315107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c09b480 con 0x5598791256b0 2024-05-21T15:21:12.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c327fc640 1 -- 172.21.15.78:0/1139315107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c099780 con 0x5598791256b0 2024-05-21T15:21:12.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1139315107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 msgr2=0x5598794e36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e36f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0cc710 tx=0x7f3c3c010c70 comp rx=0 tx=0).stop 2024-05-21T15:21:12.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1139315107 shutdown_connections 2024-05-21T15:21:12.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559879146390 0x559879509620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598791256b0 0x5598794e36f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.938+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1139315107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987911d4d0 0x5598794f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1139315107 >> 172.21.15.78:0/1139315107 conn(0x55987913a4e0 msgr2=0x5598794e2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1139315107 shutdown_connections 2024-05-21T15:21:12.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1139315107 wait complete. 2024-05-21T15:21:12.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 -- start start 2024-05-21T15:21:12.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 Processor -- start 2024-05-21T15:21:12.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598795644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5598791256b0 0x559879564a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559879146390 0x5598794f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1adeb0 con 0x55987911d4d0 2024-05-21T15:21:12.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1ad7e0 con 0x5598791256b0 2024-05-21T15:21:12.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c1aaa20 con 0x559879146390 2024-05-21T15:21:12.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.939+0000 7f3c59a3f740 10 client.0 _ll_register_callbacks cb 0x7f3c48004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598795644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598795644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54470/0 (socket says 172.21.15.78:54470) 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 -- 172.21.15.78:0/1992712895 learned_addr learned my addr 172.21.15.78:0/1992712895 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5598791256b0 0x559879564a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c51133640 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559879146390 0x5598794f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 -- 172.21.15.78:0/1992712895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559879146390 msgr2=0x5598794f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559879146390 0x5598794f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 -- 172.21.15.78:0/1992712895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5598791256b0 msgr2=0x559879564a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:12.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5598791256b0 0x559879564a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 -- 172.21.15.78:0/1992712895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598794f50a0 con 0x55987911d4d0 2024-05-21T15:21:12.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c50932640 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598795644f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3c48031a00 tx=0x7f3c48005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0a2bf09b70e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:12.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:12.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c2bfff640 1 -- 172.21.15.78:0/1992712895 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48035050 con 0x55987911d4d0 2024-05-21T15:21:12.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c2bfff640 1 -- 172.21.15.78:0/1992712895 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3c48003a00 con 0x55987911d4d0 2024-05-21T15:21:12.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1992712895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c3c09b480 con 0x55987911d4d0 2024-05-21T15:21:12.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c2bfff640 1 -- 172.21.15.78:0/1992712895 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3c48003c00 con 0x55987911d4d0 2024-05-21T15:21:12.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c59a3f740 10 client.17145 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:12.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1992712895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55987956ba40 con 0x55987911d4d0 2024-05-21T15:21:12.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.940+0000 7f3c2bfff640 1 -- 172.21.15.78:0/1992712895 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3c09b480 con 0x55987911d4d0 2024-05-21T15:21:12.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c2bfff640 1 -- 172.21.15.78:0/1992712895 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3c48054020 con 0x55987911d4d0 2024-05-21T15:21:12.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c59a3f740 10 client.17145 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:12.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c59a3f740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:12.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c59a3f740 20 client.17145 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:12.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5598790da3a0 0x55987956e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:12.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c59a3f740 4 client.17145 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:21:12.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1992712895 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559879452780 con 0x5598790da3a0 2024-05-21T15:21:12.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5598790da3a0 0x55987956e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:12.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c2bfff640 10 client.17145 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:12.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.941+0000 7f3c43fff640 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5598790da3a0 0x55987956e170 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d0dd9fb7ca00eb server_cookie=e113bfa52aca8db4 in_seq=0 out_seq=0 2024-05-21T15:21:12.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c2bfff640 1 -- 172.21.15.78:0/1992712895 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559879452780 con 0x5598790da3a0 2024-05-21T15:21:12.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c2bfff640 10 client.17145 handle_command_reply: tid=0 2024-05-21T15:21:12.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 client.17145 shutdown 2024-05-21T15:21:12.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1992712895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 msgr2=0x5598795644f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598795644f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3c48031a00 tx=0x7f3c48005990 comp rx=0 tx=0).stop 2024-05-21T15:21:12.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1992712895 shutdown_connections 2024-05-21T15:21:12.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5598790da3a0 0x55987956e170 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5598791256b0 0x559879564a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559879146390 0x5598794f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 --2- 172.21.15.78:0/1992712895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987911d4d0 0x5598795644f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c2bfff640 0 client.17145 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:12.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1992712895 >> 172.21.15.78:0/1992712895 conn(0x55987913a4e0 msgr2=0x5598794f0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1992712895 shutdown_connections 2024-05-21T15:21:12.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 1 -- 172.21.15.78:0/1992712895 wait complete. 2024-05-21T15:21:12.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c59a3f740 20 client.17145 trim_cache size 0 max 16384 2024-05-21T15:21:12.957 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:21:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:21:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:21:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "94ffe5d9-24fe-4e3d-8062-98040e8fe38f": { 2024-05-21T15:21:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:21:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "94ffe5d9-24fe-4e3d-8062-98040e8fe38f", 2024-05-21T15:21:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:21:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:21:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:21:12.958 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:21:12.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c52b96640 1 -- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c340a2d00 msgr2=0x7f3c340a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c52b96640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c340a2d00 0x7f3c340a5210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005fd0 tx=0x7f3c4400ad50 comp rx=0 tx=0).stop 2024-05-21T15:21:12.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c52b96640 1 -- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 msgr2=0x7f3c4c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:12.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c52b96640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 0x7f3c4c198c40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005d00 tx=0x7f3c3c00dde0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c52b96640 1 -- 172.21.15.78:0/1807669384 shutdown_connections 2024-05-21T15:21:12.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c52b96640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c4c10c360 0x7f3c4c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c52b96640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3c340a2d00 0x7f3c340a5210 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.942+0000 7f3c52b96640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c4c107a00 0x7f3c4c198c40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.943+0000 7f3c52b96640 1 --2- 172.21.15.78:0/1807669384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c4c101ad0 0x7f3c4c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:12.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.943+0000 7f3c52b96640 1 -- 172.21.15.78:0/1807669384 >> 172.21.15.78:0/1807669384 conn(0x7f3c4c077100 msgr2=0x7f3c4c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:12.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.943+0000 7f3c52b96640 1 -- 172.21.15.78:0/1807669384 shutdown_connections 2024-05-21T15:21:12.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:12.943+0000 7f3c52b96640 1 -- 172.21.15.78:0/1807669384 wait complete. 2024-05-21T15:21:12.961 INFO:tasks.cephfs.filesystem:scrub status for tag:94ffe5d9-24fe-4e3d-8062-98040e8fe38f - {'path': '/', 'tag': '94ffe5d9-24fe-4e3d-8062-98040e8fe38f', 'options': 'recursive,force'} 2024-05-21T15:21:13.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:21:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:13.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:13.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.685 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:13 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:21:13.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:13.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:13.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:13.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:13.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:13.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:13.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:13 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:21:13.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:13.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:13.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:13.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:13.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:13.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:13.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:13.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:13 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:14.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:14 smithi172 ceph-mon[36402]: pgmap v157: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 321 op/s 2024-05-21T15:21:14.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:14 smithi078 ceph-mon[31997]: pgmap v157: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 321 op/s 2024-05-21T15:21:14.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:14 smithi084 ceph-mon[36334]: pgmap v157: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 321 op/s 2024-05-21T15:21:16.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:16 smithi172 ceph-mon[36402]: pgmap v158: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 11 KiB/s rd, 420 KiB/s wr, 110 op/s 2024-05-21T15:21:16.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:16 smithi078 ceph-mon[31997]: pgmap v158: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 11 KiB/s rd, 420 KiB/s wr, 110 op/s 2024-05-21T15:21:16.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:16 smithi084 ceph-mon[36334]: pgmap v158: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 11 KiB/s rd, 420 KiB/s wr, 110 op/s 2024-05-21T15:21:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:18 smithi172 ceph-mon[36402]: pgmap v159: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 422 KiB/s wr, 139 op/s 2024-05-21T15:21:18.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:18 smithi078 ceph-mon[31997]: pgmap v159: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 422 KiB/s wr, 139 op/s 2024-05-21T15:21:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:18 smithi084 ceph-mon[36334]: pgmap v159: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 422 KiB/s wr, 139 op/s 2024-05-21T15:21:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:20 smithi172 ceph-mon[36402]: pgmap v160: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 41 KiB/s rd, 4.3 KiB/s wr, 44 op/s 2024-05-21T15:21:20.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:20 smithi078 ceph-mon[31997]: pgmap v160: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 41 KiB/s rd, 4.3 KiB/s wr, 44 op/s 2024-05-21T15:21:20.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:20 smithi084 ceph-mon[36334]: pgmap v160: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 41 KiB/s rd, 4.3 KiB/s wr, 44 op/s 2024-05-21T15:21:21.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:21 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:21:21.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:21 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:21:21.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:21 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:21:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:22 smithi078 ceph-mon[31997]: pgmap v161: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 KiB/s wr, 44 op/s 2024-05-21T15:21:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:21:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:21:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:21:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:22 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:22 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:22 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:22 smithi084 ceph-mon[36334]: pgmap v161: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 KiB/s wr, 44 op/s 2024-05-21T15:21:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:21:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:21:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:21:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:22 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:22 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:22 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:22 smithi172 ceph-mon[36402]: pgmap v161: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 KiB/s wr, 44 op/s 2024-05-21T15:21:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:21:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:21:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:21:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:22 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:22 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:22 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:23.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:23 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:23.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:23 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:23.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:23 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:23.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:23 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:23.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:23 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:23.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:23 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:24.835 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:24 smithi172 ceph-mon[36402]: pgmap v162: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 KiB/s wr, 49 op/s 2024-05-21T15:21:24.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:24 smithi078 ceph-mon[31997]: pgmap v162: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 KiB/s wr, 49 op/s 2024-05-21T15:21:24.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:24 smithi084 ceph-mon[36334]: pgmap v162: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 65 KiB/s rd, 26 KiB/s wr, 49 op/s 2024-05-21T15:21:26.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:26 smithi078 ceph-mon[31997]: pgmap v163: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 61 KiB/s rd, 24 KiB/s wr, 44 op/s 2024-05-21T15:21:26.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:26 smithi084 ceph-mon[36334]: pgmap v163: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 61 KiB/s rd, 24 KiB/s wr, 44 op/s 2024-05-21T15:21:26.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:26 smithi172 ceph-mon[36402]: pgmap v163: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 61 KiB/s rd, 24 KiB/s wr, 44 op/s 2024-05-21T15:21:28.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:28 smithi078 ceph-mon[31997]: pgmap v164: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 67 KiB/s rd, 28 KiB/s wr, 49 op/s 2024-05-21T15:21:28.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:28 smithi084 ceph-mon[36334]: pgmap v164: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 67 KiB/s rd, 28 KiB/s wr, 49 op/s 2024-05-21T15:21:28.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:28 smithi172 ceph-mon[36402]: pgmap v164: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 67 KiB/s rd, 28 KiB/s wr, 49 op/s 2024-05-21T15:21:30.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:30 smithi078 ceph-mon[31997]: pgmap v165: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 27 KiB/s wr, 20 op/s 2024-05-21T15:21:30.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:30 smithi084 ceph-mon[36334]: pgmap v165: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 27 KiB/s wr, 20 op/s 2024-05-21T15:21:30.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:30 smithi172 ceph-mon[36402]: pgmap v165: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 27 KiB/s wr, 20 op/s 2024-05-21T15:21:32.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:32 smithi078 ceph-mon[31997]: pgmap v166: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-21T15:21:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:32 smithi084 ceph-mon[36334]: pgmap v166: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-21T15:21:32.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:32 smithi172 ceph-mon[36402]: pgmap v166: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-21T15:21:34.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:34 smithi078 ceph-mon[31997]: pgmap v167: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-05-21T15:21:34.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:34 smithi084 ceph-mon[36334]: pgmap v167: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-05-21T15:21:34.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:34 smithi172 ceph-mon[36402]: pgmap v167: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-05-21T15:21:36.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:36 smithi078 ceph-mon[31997]: pgmap v168: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-21T15:21:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:36 smithi084 ceph-mon[36334]: pgmap v168: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-21T15:21:36.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:36 smithi172 ceph-mon[36402]: pgmap v168: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-05-21T15:21:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:38 smithi078 ceph-mon[31997]: pgmap v169: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-05-21T15:21:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:38 smithi084 ceph-mon[36334]: pgmap v169: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-05-21T15:21:38.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:38 smithi172 ceph-mon[36402]: pgmap v169: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 KiB/s wr, 14 op/s 2024-05-21T15:21:40.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:40 smithi078 ceph-mon[31997]: pgmap v170: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-05-21T15:21:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:40 smithi084 ceph-mon[36334]: pgmap v170: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-05-21T15:21:40.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:40 smithi172 ceph-mon[36402]: pgmap v170: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-05-21T15:21:42.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:42 smithi078 ceph-mon[31997]: pgmap v171: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-21T15:21:42.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:42 smithi084 ceph-mon[36334]: pgmap v171: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-21T15:21:42.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:42 smithi172 ceph-mon[36402]: pgmap v171: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-05-21T15:21:42.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:21:42.963 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:21:43.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 -- 172.21.15.78:0/2025031183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10a630 msgr2=0x7f473c10ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 --2- 172.21.15.78:0/2025031183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10a630 0x7f473c10ca50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4730009900 tx=0x7f473002a410 comp rx=0 tx=0).stop 2024-05-21T15:21:43.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 -- 172.21.15.78:0/2025031183 shutdown_connections 2024-05-21T15:21:43.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 --2- 172.21.15.78:0/2025031183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f473c10d020 0x7f473c10f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 --2- 172.21.15.78:0/2025031183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10a630 0x7f473c10ca50 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 --2- 172.21.15.78:0/2025031183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c103bd0 0x7f473c103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 -- 172.21.15.78:0/2025031183 >> 172.21.15.78:0/2025031183 conn(0x7f473c077100 msgr2=0x7f473c077540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 -- 172.21.15.78:0/2025031183 shutdown_connections 2024-05-21T15:21:43.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.085+0000 7f474272f640 1 -- 172.21.15.78:0/2025031183 wait complete. 2024-05-21T15:21:43.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.086+0000 7f474272f640 1 Processor -- start 2024-05-21T15:21:43.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.086+0000 7f474272f640 1 -- start start 2024-05-21T15:21:43.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.086+0000 7f474272f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f473c103bd0 0x7f473c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.086+0000 7f474272f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c10a630 0x7f473c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f474272f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10d020 0x7f473c19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f474272f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f473c112dd0 con 0x7f473c10d020 2024-05-21T15:21:43.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f474272f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f473c112c50 con 0x7f473c103bd0 2024-05-21T15:21:43.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f474272f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f473c112f50 con 0x7f473c10a630 2024-05-21T15:21:43.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10d020 0x7f473c19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10d020 0x7f473c19e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35336/0 (socket says 172.21.15.78:35336) 2024-05-21T15:21:43.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 -- 172.21.15.78:0/2473094702 learned_addr learned my addr 172.21.15.78:0/2473094702 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f473b7fe640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c10a630 0x7f473c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f473bfff640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f473c103bd0 0x7f473c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 -- 172.21.15.78:0/2473094702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c10a630 msgr2=0x7f473c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c10a630 0x7f473c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 -- 172.21.15.78:0/2473094702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f473c103bd0 msgr2=0x7f473c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f473c103bd0 0x7f473c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 -- 172.21.15.78:0/2473094702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c19e650 con 0x7f473c10d020 2024-05-21T15:21:43.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f4740ccc640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10d020 0x7f473c19e010 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f473400cad0 tx=0x7f473400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f47397fa640 1 -- 172.21.15.78:0/2473094702 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473400ae20 con 0x7f473c10d020 2024-05-21T15:21:43.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f47397fa640 1 -- 172.21.15.78:0/2473094702 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4734005020 con 0x7f473c10d020 2024-05-21T15:21:43.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f474272f640 1 -- 172.21.15.78:0/2473094702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c19e940 con 0x7f473c10d020 2024-05-21T15:21:43.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f47397fa640 1 -- 172.21.15.78:0/2473094702 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473400f6f0 con 0x7f473c10d020 2024-05-21T15:21:43.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.087+0000 7f474272f640 1 -- 172.21.15.78:0/2473094702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c19ee40 con 0x7f473c10d020 2024-05-21T15:21:43.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47397fa640 1 -- 172.21.15.78:0/2473094702 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f47340051d0 con 0x7f473c10d020 2024-05-21T15:21:43.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47495d1740 1 Processor -- start 2024-05-21T15:21:43.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47495d1740 1 -- start start 2024-05-21T15:21:43.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47495d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d906390 0x55cc7dca4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47495d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dca5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47495d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 0x55cc7dca5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47495d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dc0ad00 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47495d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dbb24c0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47495d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dc06cc0 con 0x55cc7d906390 2024-05-21T15:21:43.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f473bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dca5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f473bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dca5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35348/0 (socket says 172.21.15.78:35348) 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f473bfff640 1 -- 172.21.15.78:0/2704336933 learned_addr learned my addr 172.21.15.78:0/2704336933 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.090+0000 7f47397fa640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f47240a2d00 0x7f47240a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f473b7fe640 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d906390 0x55cc7dca4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f473b7fe640 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 0x55cc7dca5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f4740ccc640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f47240a2d00 0x7f47240a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f473bfff640 1 -- 172.21.15.78:0/2704336933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d906390 msgr2=0x55cc7dca4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f473bfff640 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d906390 0x55cc7dca4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f473bfff640 1 -- 172.21.15.78:0/2704336933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 msgr2=0x55cc7dca5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f473bfff640 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 0x55cc7dca5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f473bfff640 1 -- 172.21.15.78:0/2704336933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7d9e87d0 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f47397fa640 1 -- 172.21.15.78:0/2473094702 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f47340cf590 con 0x7f473c10d020 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f473bfff640 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dca5120 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f472c00cbe0 tx=0x7f472c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ba1bb997fd50b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f47227fc640 1 -- 172.21.15.78:0/2704336933 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c010630 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f4740ccc640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f47240a2d00 0x7f47240a5210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f473409b750 tx=0x7f47340665d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f47227fc640 1 -- 172.21.15.78:0/2704336933 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472c010cb0 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.091+0000 7f47227fc640 1 -- 172.21.15.78:0/2704336933 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c01ac20 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 -- 172.21.15.78:0/2704336933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 msgr2=0x55cc7dca5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dca5120 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f472c00cbe0 tx=0x7f472c00e720 comp rx=0 tx=0).stop 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 -- 172.21.15.78:0/2704336933 shutdown_connections 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 0x55cc7dca5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dca5120 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 --2- 172.21.15.78:0/2704336933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d906390 0x55cc7dca4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 -- 172.21.15.78:0/2704336933 >> 172.21.15.78:0/2704336933 conn(0x55cc7d8fa4e0 msgr2=0x55cc7dca14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 -- 172.21.15.78:0/2704336933 shutdown_connections 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 -- 172.21.15.78:0/2704336933 wait complete. 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.092+0000 7f47495d1740 1 -- start start 2024-05-21T15:21:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f47495d1740 1 Processor -- start 2024-05-21T15:21:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f47495d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f47495d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f47495d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d906390 0x55cc7dcb3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f47495d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dcb6490 con 0x55cc7d8e56b0 2024-05-21T15:21:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f47495d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dbc5b90 con 0x55cc7d906390 2024-05-21T15:21:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f47495d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cc7dcb6610 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f47495d1740 10 client.0 _ll_register_callbacks cb 0x55cc7dc01da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35352/0 (socket says 172.21.15.78:35352) 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473bfff640 1 -- 172.21.15.78:0/535395923 learned_addr learned my addr 172.21.15.78:0/535395923 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473b7fe640 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d906390 0x55cc7dcb3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473b7fe640 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473bfff640 1 -- 172.21.15.78:0/535395923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 msgr2=0x55cc7dcb51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473bfff640 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473bfff640 1 -- 172.21.15.78:0/535395923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d906390 msgr2=0x55cc7dcb3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473bfff640 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d906390 0x55cc7dcb3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.093+0000 7f473bfff640 1 -- 172.21.15.78:0/535395923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7d91e3f0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f473bfff640 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb56e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f472c00ec20 tx=0x7f472c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809dbe3465efe680 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f471b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:43.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f471b7fe640 1 -- 172.21.15.78:0/535395923 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c01a9e0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f471b7fe640 1 -- 172.21.15.78:0/535395923 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f472c01ab80 con 0x55cc7d8e56b0 2024-05-21T15:21:43.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 1 -- 172.21.15.78:0/535395923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f472c010cb0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f471b7fe640 1 -- 172.21.15.78:0/535395923 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f472c01adc0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 10 client.17157 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:43.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 1 -- 172.21.15.78:0/535395923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cc7d8de0f0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f471b7fe640 1 -- 172.21.15.78:0/535395923 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472c010cb0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f471b7fe640 1 -- 172.21.15.78:0/535395923 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f472c0046c0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 10 client.17157 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:43.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:43.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 20 client.17157 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:43.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cc7d8c4f60 0x55cc7dcc5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 4 client.17157 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:43.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.094+0000 7f47495d1740 1 -- 172.21.15.78:0/535395923 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc7dc18430 con 0x55cc7d8c4f60 2024-05-21T15:21:43.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.095+0000 7f473b7fe640 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cc7d8c4f60 0x55cc7dcc5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.095+0000 7f471b7fe640 10 client.17157 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:43.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.095+0000 7f473b7fe640 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cc7d8c4f60 0x55cc7dcc5f20 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29cdadd9556af9e7 server_cookie=ab774f6e9e253e83 in_seq=0 out_seq=0 2024-05-21T15:21:43.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f471b7fe640 1 -- 172.21.15.78:0/535395923 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cc7dc18430 con 0x55cc7d8c4f60 2024-05-21T15:21:43.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f471b7fe640 10 client.17157 handle_command_reply: tid=0 2024-05-21T15:21:43.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 client.17157 shutdown 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 -- 172.21.15.78:0/535395923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 msgr2=0x55cc7dcb56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb56e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f472c00ec20 tx=0x7f472c00c920 comp rx=0 tx=0).stop 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 -- 172.21.15.78:0/535395923 shutdown_connections 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cc7d8c4f60 0x55cc7dcc5f20 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d906390 0x55cc7dcb3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f471b7fe640 0 client.17157 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb56e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 --2- 172.21.15.78:0/535395923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 -- 172.21.15.78:0/535395923 >> 172.21.15.78:0/535395923 conn(0x55cc7d8fa4e0 msgr2=0x55cc7dca69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 -- 172.21.15.78:0/535395923 shutdown_connections 2024-05-21T15:21:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 1 -- 172.21.15.78:0/535395923 wait complete. 2024-05-21T15:21:43.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.096+0000 7f47495d1740 20 client.17157 trim_cache size 0 max 16384 2024-05-21T15:21:43.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f47495d1740 1 Processor -- start 2024-05-21T15:21:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f47495d1740 1 -- start start 2024-05-21T15:21:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f47495d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f47495d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f47495d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f47495d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1acad0 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f47495d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1ae610 con 0x55cc7d8e56b0 2024-05-21T15:21:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f47495d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1ad1a0 con 0x55cc7d8c4f60 2024-05-21T15:21:43.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35364/0 (socket says 172.21.15.78:35364) 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 -- 172.21.15.78:0/262627780 learned_addr learned my addr 172.21.15.78:0/262627780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473b7fe640 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 -- 172.21.15.78:0/262627780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8c4f60 msgr2=0x55cc7dcae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 -- 172.21.15.78:0/262627780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 msgr2=0x55cc7dcb2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 -- 172.21.15.78:0/262627780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7dcb65f0 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473b7fe640 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f473bfff640 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb3590 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f472c004c80 tx=0x7f472c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c284e9d23e26519 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.114+0000 7f4721ffb640 1 -- 172.21.15.78:0/262627780 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c004660 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f4721ffb640 1 -- 172.21.15.78:0/262627780 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472c004800 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f4721ffb640 1 -- 172.21.15.78:0/262627780 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c0052e0 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 -- 172.21.15.78:0/262627780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 msgr2=0x55cc7dcb3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb3590 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f472c004c80 tx=0x7f472c00ed50 comp rx=0 tx=0).stop 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 -- 172.21.15.78:0/262627780 shutdown_connections 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb3590 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 --2- 172.21.15.78:0/262627780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 -- 172.21.15.78:0/262627780 >> 172.21.15.78:0/262627780 conn(0x55cc7d8fa4e0 msgr2=0x55cc7dcb1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 -- 172.21.15.78:0/262627780 shutdown_connections 2024-05-21T15:21:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.115+0000 7f47495d1740 1 -- 172.21.15.78:0/262627780 wait complete. 2024-05-21T15:21:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 1 -- start start 2024-05-21T15:21:43.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 1 Processor -- start 2024-05-21T15:21:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcbbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1adf40 con 0x55cc7d8e56b0 2024-05-21T15:21:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1ad870 con 0x55cc7d8c4f60 2024-05-21T15:21:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1aaab0 con 0x55cc7d8dd4d0 2024-05-21T15:21:43.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f47495d1740 10 client.0 _ll_register_callbacks cb 0x7f472c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f473b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f473b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35380/0 (socket says 172.21.15.78:35380) 2024-05-21T15:21:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f473b7fe640 1 -- 172.21.15.78:0/1221190160 learned_addr learned my addr 172.21.15.78:0/1221190160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f473b7fe640 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcbbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.116+0000 7f473bfff640 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f473b7fe640 1 -- 172.21.15.78:0/1221190160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 msgr2=0x55cc7dcb6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f473b7fe640 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f473b7fe640 1 -- 172.21.15.78:0/1221190160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8c4f60 msgr2=0x55cc7dcbbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f473b7fe640 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcbbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f473b7fe640 1 -- 172.21.15.78:0/1221190160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7dcb7a40 con 0x55cc7d8e56b0 2024-05-21T15:21:43.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f473b7fe640 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb74d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4730005940 tx=0x7f47300092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feb16f326b5bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f471b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f471b7fe640 1 -- 172.21.15.78:0/1221190160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730050030 con 0x55cc7d8e56b0 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f471b7fe640 1 -- 172.21.15.78:0/1221190160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f47300043e0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 1 -- 172.21.15.78:0/1221190160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f472c004800 con 0x55cc7d8e56b0 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f471b7fe640 1 -- 172.21.15.78:0/1221190160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f473005a050 con 0x55cc7d8e56b0 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 10 client.17163 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 1 -- 172.21.15.78:0/1221190160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cc7dd2de00 con 0x55cc7d8e56b0 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f471b7fe640 1 -- 172.21.15.78:0/1221190160 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472c004800 con 0x55cc7d8e56b0 2024-05-21T15:21:43.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f471b7fe640 1 -- 172.21.15.78:0/1221190160 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f47300316d0 con 0x55cc7d8e56b0 2024-05-21T15:21:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 10 client.17163 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 20 client.17163 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cc7d906390 0x55cc7dd30530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 4 client.17163 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:21:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.117+0000 7f47495d1740 1 -- 172.21.15.78:0/1221190160 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc7dc12780 con 0x55cc7d906390 2024-05-21T15:21:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.118+0000 7f473bfff640 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cc7d906390 0x55cc7dd30530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.118+0000 7f471b7fe640 10 client.17163 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.118+0000 7f473bfff640 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cc7d906390 0x55cc7dd30530 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3031fc350282bb0e server_cookie=bdec4dbe5219c492 in_seq=0 out_seq=0 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.118+0000 7f471b7fe640 1 -- 172.21.15.78:0/1221190160 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cc7dc12780 con 0x55cc7d906390 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.118+0000 7f471b7fe640 10 client.17163 handle_command_reply: tid=0 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.118+0000 7f47495d1740 1 client.17163 shutdown 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.118+0000 7f47495d1740 1 -- 172.21.15.78:0/1221190160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 msgr2=0x55cc7dcb74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.118+0000 7f47495d1740 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb74d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4730005940 tx=0x7f47300092d0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 1 -- 172.21.15.78:0/1221190160 shutdown_connections 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cc7d906390 0x55cc7dd30530 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc7d8e56b0 0x55cc7dcb74d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc7d8dd4d0 0x55cc7dcb6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 1 --2- 172.21.15.78:0/1221190160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc7d8c4f60 0x55cc7dcbbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f471b7fe640 0 client.17163 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 1 -- 172.21.15.78:0/1221190160 >> 172.21.15.78:0/1221190160 conn(0x55cc7d8fa4e0 msgr2=0x55cc7dca4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 1 -- 172.21.15.78:0/1221190160 shutdown_connections 2024-05-21T15:21:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 1 -- 172.21.15.78:0/1221190160 wait complete. 2024-05-21T15:21:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f47495d1740 20 client.17163 trim_cache size 0 max 16384 2024-05-21T15:21:43.133 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:21:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:21:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:21:43.133 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:21:43.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 -- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f47240a2d00 msgr2=0x7f47240a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f47240a2d00 0x7f47240a5210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f473409b750 tx=0x7f47340665d0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 -- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10d020 msgr2=0x7f473c19e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10d020 0x7f473c19e010 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f473400cad0 tx=0x7f473400cf90 comp rx=0 tx=0).stop 2024-05-21T15:21:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 -- 172.21.15.78:0/2473094702 shutdown_connections 2024-05-21T15:21:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f47240a2d00 0x7f47240a5210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f473c10d020 0x7f473c19e010 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f473c10a630 0x7f473c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 --2- 172.21.15.78:0/2473094702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f473c103bd0 0x7f473c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 -- 172.21.15.78:0/2473094702 >> 172.21.15.78:0/2473094702 conn(0x7f473c077100 msgr2=0x7f473c1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 -- 172.21.15.78:0/2473094702 shutdown_connections 2024-05-21T15:21:43.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.119+0000 7f474272f640 1 -- 172.21.15.78:0/2473094702 wait complete. 2024-05-21T15:21:43.137 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:21:43.137 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:21:43.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 -- 172.21.15.78:0/2230360213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 msgr2=0x7f9ed81015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 --2- 172.21.15.78:0/2230360213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 0x7f9ed81015d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009900 tx=0x7f9ec802a410 comp rx=0 tx=0).stop 2024-05-21T15:21:43.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 -- 172.21.15.78:0/2230360213 shutdown_connections 2024-05-21T15:21:43.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 --2- 172.21.15.78:0/2230360213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed8101b10 0x7f9ed8101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 --2- 172.21.15.78:0/2230360213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed810c350 0x7f9ed810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 --2- 172.21.15.78:0/2230360213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 0x7f9ed81015d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 -- 172.21.15.78:0/2230360213 >> 172.21.15.78:0/2230360213 conn(0x7f9ed8076ff0 msgr2=0x7f9ed8077430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 -- 172.21.15.78:0/2230360213 shutdown_connections 2024-05-21T15:21:43.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 -- 172.21.15.78:0/2230360213 wait complete. 2024-05-21T15:21:43.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.245+0000 7f9edde80640 1 Processor -- start 2024-05-21T15:21:43.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9edde80640 1 -- start start 2024-05-21T15:21:43.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9edde80640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed8101b10 0x7f9ed81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9edde80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 0x7f9ed8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9edde80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed810c350 0x7f9ed819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9edde80640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8111d00 con 0x7f9ed81079f0 2024-05-21T15:21:43.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9edde80640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8111b80 con 0x7f9ed8101b10 2024-05-21T15:21:43.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9edde80640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8111e80 con 0x7f9ed810c350 2024-05-21T15:21:43.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 0x7f9ed8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 0x7f9ed8198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35398/0 (socket says 172.21.15.78:35398) 2024-05-21T15:21:43.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed6ffd640 1 -- 172.21.15.78:0/1715398600 learned_addr learned my addr 172.21.15.78:0/1715398600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed77fe640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed8101b10 0x7f9ed81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed7fff640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed810c350 0x7f9ed819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed6ffd640 1 -- 172.21.15.78:0/1715398600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed810c350 msgr2=0x7f9ed819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed6ffd640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed810c350 0x7f9ed819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed6ffd640 1 -- 172.21.15.78:0/1715398600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed8101b10 msgr2=0x7f9ed81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed6ffd640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed8101b10 0x7f9ed81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed6ffd640 1 -- 172.21.15.78:0/1715398600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed819d540 con 0x7f9ed81079f0 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed77fe640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed8101b10 0x7f9ed81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.246+0000 7f9ed7fff640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed810c350 0x7f9ed819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:43.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.247+0000 7f9ed6ffd640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 0x7f9ed8198c40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00d740 tx=0x7f9ecc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.247+0000 7f9ed4ff9640 1 -- 172.21.15.78:0/1715398600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc017070 con 0x7f9ed81079f0 2024-05-21T15:21:43.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.247+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed819d830 con 0x7f9ed81079f0 2024-05-21T15:21:43.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.247+0000 7f9ed4ff9640 1 -- 172.21.15.78:0/1715398600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ecc00de50 con 0x7f9ed81079f0 2024-05-21T15:21:43.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.247+0000 7f9ed4ff9640 1 -- 172.21.15.78:0/1715398600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc003e00 con 0x7f9ed81079f0 2024-05-21T15:21:43.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.247+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed81a51b0 con 0x7f9ed81079f0 2024-05-21T15:21:43.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.248+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed8069140 con 0x7f9ed81079f0 2024-05-21T15:21:43.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.249+0000 7f9ed4ff9640 1 -- 172.21.15.78:0/1715398600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9ecc003fa0 con 0x7f9ed81079f0 2024-05-21T15:21:43.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.250+0000 7f9ed4ff9640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9ec00a2d00 0x7f9ec00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.250+0000 7f9ed77fe640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9ec00a2d00 0x7f9ec00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.250+0000 7f9ed4ff9640 1 -- 172.21.15.78:0/1715398600 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9ecc0cf350 con 0x7f9ed81079f0 2024-05-21T15:21:43.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.251+0000 7f9ed77fe640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9ec00a2d00 0x7f9ec00a5210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8005a20 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.254+0000 7f9ed4ff9640 1 -- 172.21.15.78:0/1715398600 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ecc09bf10 con 0x7f9ed81079f0 2024-05-21T15:21:43.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.469+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ed8199a30 con 0x7f9ed81079f0 2024-05-21T15:21:43.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.470+0000 7f9ed4ff9640 1 -- 172.21.15.78:0/1715398600 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7f9ecc09b810 con 0x7f9ed81079f0 2024-05-21T15:21:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:21:43.473 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:21:43.473 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:21:43.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9ec00a2d00 msgr2=0x7f9ec00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9ec00a2d00 0x7f9ec00a5210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8005a20 comp rx=0 tx=0).stop 2024-05-21T15:21:43.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 msgr2=0x7f9ed8198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 0x7f9ed8198c40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00d740 tx=0x7f9ecc00dc00 comp rx=0 tx=0).stop 2024-05-21T15:21:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 shutdown_connections 2024-05-21T15:21:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9ec00a2d00 0x7f9ec00a5210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed810c350 0x7f9ed819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ed81079f0 0x7f9ed8198c40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 --2- 172.21.15.78:0/1715398600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed8101b10 0x7f9ed81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 >> 172.21.15.78:0/1715398600 conn(0x7f9ed8076ff0 msgr2=0x7f9ed8103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 shutdown_connections 2024-05-21T15:21:43.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.472+0000 7f9edde80640 1 -- 172.21.15.78:0/1715398600 wait complete. 2024-05-21T15:21:43.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:21:43.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3364859096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108ac0 msgr2=0x7f8e74102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3364859096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108ac0 0x7f8e74102660 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009900 tx=0x7f8e6402a410 comp rx=0 tx=0).stop 2024-05-21T15:21:43.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3364859096 shutdown_connections 2024-05-21T15:21:43.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3364859096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e74108f60 0x7f8e74102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3364859096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e74103210 0x7f8e7410f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3364859096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108ac0 0x7f8e74102660 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3364859096 >> 172.21.15.78:0/3364859096 conn(0x7f8e740fe1c0 msgr2=0x7f8e74100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3364859096 shutdown_connections 2024-05-21T15:21:43.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.597+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3364859096 wait complete. 2024-05-21T15:21:43.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.598+0000 7f8e7a7b5640 1 Processor -- start 2024-05-21T15:21:43.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.598+0000 7f8e7a7b5640 1 -- start start 2024-05-21T15:21:43.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e7a7b5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e74103210 0x7f8e741997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e7a7b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e74108ac0 0x7f8e74199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e7a7b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108f60 0x7f8e7419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e7a7b5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74112e00 con 0x7f8e74108f60 2024-05-21T15:21:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e7a7b5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74112c80 con 0x7f8e74103210 2024-05-21T15:21:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e7a7b5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74112f80 con 0x7f8e74108ac0 2024-05-21T15:21:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108f60 0x7f8e7419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108f60 0x7f8e7419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35414/0 (socket says 172.21.15.78:35414) 2024-05-21T15:21:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 -- 172.21.15.78:0/3855793426 learned_addr learned my addr 172.21.15.78:0/3855793426 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e737fe640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e74108ac0 0x7f8e74199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e73fff640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e74103210 0x7f8e741997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 -- 172.21.15.78:0/3855793426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e74108ac0 msgr2=0x7f8e74199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e74108ac0 0x7f8e74199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 -- 172.21.15.78:0/3855793426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e74103210 msgr2=0x7f8e741997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e74103210 0x7f8e741997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 -- 172.21.15.78:0/3855793426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7419e620 con 0x7f8e74108f60 2024-05-21T15:21:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e78d52640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108f60 0x7f8e7419dfe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00eaa0 tx=0x7f8e6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e717fa640 1 -- 172.21.15.78:0/3855793426 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c00cdb0 con 0x7f8e74108f60 2024-05-21T15:21:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e717fa640 1 -- 172.21.15.78:0/3855793426 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c005020 con 0x7f8e74108f60 2024-05-21T15:21:43.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3855793426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7419e910 con 0x7f8e74108f60 2024-05-21T15:21:43.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e717fa640 1 -- 172.21.15.78:0/3855793426 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c0116b0 con 0x7f8e74108f60 2024-05-21T15:21:43.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.599+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3855793426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e741a6290 con 0x7f8e74108f60 2024-05-21T15:21:43.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e717fa640 1 -- 172.21.15.78:0/3855793426 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8e6c005560 con 0x7f8e74108f60 2024-05-21T15:21:43.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e81664740 1 Processor -- start 2024-05-21T15:21:43.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e81664740 1 -- start start 2024-05-21T15:21:43.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e81664740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a88f390 0x55c77ac2dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e81664740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a8664d0 0x55c77ac2e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e81664740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e81664740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c77ab93c40 con 0x55c77a88f390 2024-05-21T15:21:43.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e81664740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c77ab3b580 con 0x55c77a8664d0 2024-05-21T15:21:43.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.602+0000 7f8e81664740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c77ab8fc00 con 0x55c77a86e6b0 2024-05-21T15:21:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a88f390 0x55c77ac2dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a88f390 0x55c77ac2dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35416/0 (socket says 172.21.15.78:35416) 2024-05-21T15:21:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 -- 172.21.15.78:0/1712806422 learned_addr learned my addr 172.21.15.78:0/1712806422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e717fa640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e5c0a2d00 0x7f8e5c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e73fff640 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a8664d0 0x55c77ac2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e78d52640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e5c0a2d00 0x7f8e5c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 -- 172.21.15.78:0/1712806422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 msgr2=0x55c77ac2e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 -- 172.21.15.78:0/1712806422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a8664d0 msgr2=0x55c77ac2e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a8664d0 0x55c77ac2e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 -- 172.21.15.78:0/1712806422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77a8a73f0 con 0x55c77a88f390 2024-05-21T15:21:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e717fa640 1 -- 172.21.15.78:0/3855793426 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8e6c0cf720 con 0x7f8e74108f60 2024-05-21T15:21:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a88f390 0x55c77ac2dd20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800add0 tx=0x7f8e6800a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e6bf80fcb4c5a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e78d52640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e5c0a2d00 0x7f8e5c0a5210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c09b8e0 tx=0x7f8e6c0c6770 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.603+0000 7f8e5a7fc640 1 -- 172.21.15.78:0/1712806422 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68004030 con 0x55c77a88f390 2024-05-21T15:21:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e5a7fc640 1 -- 172.21.15.78:0/1712806422 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e680041d0 con 0x55c77a88f390 2024-05-21T15:21:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e5a7fc640 1 -- 172.21.15.78:0/1712806422 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800f6b0 con 0x55c77a88f390 2024-05-21T15:21:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 -- 172.21.15.78:0/1712806422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a88f390 msgr2=0x55c77ac2dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a88f390 0x55c77ac2dd20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800add0 tx=0x7f8e6800a680 comp rx=0 tx=0).stop 2024-05-21T15:21:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 -- 172.21.15.78:0/1712806422 shutdown_connections 2024-05-21T15:21:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a8664d0 0x55c77ac2e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 --2- 172.21.15.78:0/1712806422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a88f390 0x55c77ac2dd20 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 -- 172.21.15.78:0/1712806422 >> 172.21.15.78:0/1712806422 conn(0x55c77a8834e0 msgr2=0x55c77ac2a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 -- 172.21.15.78:0/1712806422 shutdown_connections 2024-05-21T15:21:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.604+0000 7f8e81664740 1 -- 172.21.15.78:0/1712806422 wait complete. 2024-05-21T15:21:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 1 -- start start 2024-05-21T15:21:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 1 Processor -- start 2024-05-21T15:21:43.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a86e6b0 0x55c77ac388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a88f390 0x55c77ac3e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c77ac3afb0 con 0x55c77a8664d0 2024-05-21T15:21:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c77ab4ec40 con 0x55c77a86e6b0 2024-05-21T15:21:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c77ac3b130 con 0x55c77a88f390 2024-05-21T15:21:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e81664740 10 client.0 _ll_register_callbacks cb 0x55c77a8a6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.605+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac383b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35418/0 (socket says 172.21.15.78:35418) 2024-05-21T15:21:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e737fe640 1 -- 172.21.15.78:0/1018248011 learned_addr learned my addr 172.21.15.78:0/1018248011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e73fff640 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a86e6b0 0x55c77ac388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e737fe640 1 -- 172.21.15.78:0/1018248011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a88f390 msgr2=0x55c77ac3e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a88f390 0x55c77ac3e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e737fe640 1 -- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a86e6b0 msgr2=0x55c77ac388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a86e6b0 0x55c77ac388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e737fe640 1 -- 172.21.15.78:0/1018248011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77a9717d0 con 0x55c77a8664d0 2024-05-21T15:21:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e73fff640 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a86e6b0 0x55c77ac388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac383b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800a770 tx=0x7f8e6800a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53055269b912b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e537fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e537fe640 1 -- 172.21.15.78:0/1018248011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68003a60 con 0x55c77a8664d0 2024-05-21T15:21:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e537fe640 1 -- 172.21.15.78:0/1018248011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8e68003ed0 con 0x55c77a8664d0 2024-05-21T15:21:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e81664740 1 -- 172.21.15.78:0/1018248011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e680041d0 con 0x55c77a8664d0 2024-05-21T15:21:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e537fe640 1 -- 172.21.15.78:0/1018248011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8e6800f450 con 0x55c77a8664d0 2024-05-21T15:21:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e81664740 10 client.17181 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:43.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.606+0000 7f8e81664740 1 -- 172.21.15.78:0/1018248011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c77ac476e0 con 0x55c77a8664d0 2024-05-21T15:21:43.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e537fe640 1 -- 172.21.15.78:0/1018248011 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e680041d0 con 0x55c77a8664d0 2024-05-21T15:21:43.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e537fe640 1 -- 172.21.15.78:0/1018248011 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8e68012570 con 0x55c77a8664d0 2024-05-21T15:21:43.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e81664740 10 client.17181 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:43.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e81664740 10 client.17181 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:21:43.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e81664740 20 client.17181 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:43.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e81664740 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55c77a84df60 0x55c77ac4aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e81664740 4 client.17181 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:43.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e81664740 1 -- 172.21.15.78:0/1018248011 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c77aba1370 con 0x55c77a84df60 2024-05-21T15:21:43.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e73fff640 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55c77a84df60 0x55c77ac4aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e537fe640 10 client.17181 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:21:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.607+0000 7f8e73fff640 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55c77a84df60 0x55c77ac4aaa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1b10f3e77fb654d5 server_cookie=5955de46c391f7c2 in_seq=0 out_seq=0 2024-05-21T15:21:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e537fe640 1 -- 172.21.15.78:0/1018248011 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c77aba1370 con 0x55c77a84df60 2024-05-21T15:21:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e537fe640 10 client.17181 handle_command_reply: tid=0 2024-05-21T15:21:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e81664740 1 client.17181 shutdown 2024-05-21T15:21:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e81664740 1 -- 172.21.15.78:0/1018248011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 msgr2=0x55c77ac383b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e81664740 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac383b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800a770 tx=0x7f8e6800a7a0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e81664740 1 -- 172.21.15.78:0/1018248011 shutdown_connections 2024-05-21T15:21:43.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e81664740 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55c77a84df60 0x55c77ac4aaa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e81664740 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a86e6b0 0x55c77ac388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e81664740 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a88f390 0x55c77ac3e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e537fe640 0 client.17181 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:21:43.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.608+0000 7f8e81664740 1 --2- 172.21.15.78:0/1018248011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac383b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.609+0000 7f8e81664740 1 -- 172.21.15.78:0/1018248011 >> 172.21.15.78:0/1018248011 conn(0x55c77a8834e0 msgr2=0x55c77ac2fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.609+0000 7f8e81664740 1 -- 172.21.15.78:0/1018248011 shutdown_connections 2024-05-21T15:21:43.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.609+0000 7f8e81664740 1 -- 172.21.15.78:0/1018248011 wait complete. 2024-05-21T15:21:43.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.609+0000 7f8e81664740 20 client.17181 trim_cache size 0 max 16384 2024-05-21T15:21:43.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.622+0000 7f8e81664740 1 Processor -- start 2024-05-21T15:21:43.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.622+0000 7f8e81664740 1 -- start start 2024-05-21T15:21:43.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.622+0000 7f8e81664740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac3a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e81664740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e81664740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 0x55c77ac2e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e81664740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741aed00 con 0x55c77a8664d0 2024-05-21T15:21:43.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e81664740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741b0840 con 0x55c77a88f390 2024-05-21T15:21:43.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e81664740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741af3d0 con 0x55c77a86e6b0 2024-05-21T15:21:43.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac3a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac3a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35422/0 (socket says 172.21.15.78:35422) 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 -- 172.21.15.78:0/1391741476 learned_addr learned my addr 172.21.15.78:0/1391741476 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e73fff640 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 -- 172.21.15.78:0/1391741476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 msgr2=0x55c77ac2e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 -- 172.21.15.78:0/1391741476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 msgr2=0x55c77ac2e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 0x55c77ac2e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 -- 172.21.15.78:0/1391741476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77ac2ae70 con 0x55c77a8664d0 2024-05-21T15:21:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e73fff640 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:43.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e737fe640 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac3a2f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8e68011cc0 tx=0x7f8e68005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3566a738f77286 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e59ffb640 1 -- 172.21.15.78:0/1391741476 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68038070 con 0x55c77a8664d0 2024-05-21T15:21:43.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e59ffb640 1 -- 172.21.15.78:0/1391741476 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e68023070 con 0x55c77a8664d0 2024-05-21T15:21:43.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.623+0000 7f8e59ffb640 1 -- 172.21.15.78:0/1391741476 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68012b80 con 0x55c77a8664d0 2024-05-21T15:21:43.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 -- 172.21.15.78:0/1391741476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 msgr2=0x55c77ac3a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac3a2f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8e68011cc0 tx=0x7f8e68005690 comp rx=0 tx=0).stop 2024-05-21T15:21:43.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 -- 172.21.15.78:0/1391741476 shutdown_connections 2024-05-21T15:21:43.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 0x55c77ac2e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a86e6b0 0x55c77ac2e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 --2- 172.21.15.78:0/1391741476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a8664d0 0x55c77ac3a2f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 -- 172.21.15.78:0/1391741476 >> 172.21.15.78:0/1391741476 conn(0x55c77a8834e0 msgr2=0x55c77ac2dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 -- 172.21.15.78:0/1391741476 shutdown_connections 2024-05-21T15:21:43.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 -- 172.21.15.78:0/1391741476 wait complete. 2024-05-21T15:21:43.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.624+0000 7f8e81664740 1 -- start start 2024-05-21T15:21:43.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e81664740 1 Processor -- start 2024-05-21T15:21:43.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e81664740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a8664d0 0x55c77ac41730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e81664740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a86e6b0 0x55c77ac41c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e81664740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 0x55c77acb0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e81664740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741b0170 con 0x55c77a86e6b0 2024-05-21T15:21:43.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e81664740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741afaa0 con 0x55c77a88f390 2024-05-21T15:21:43.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e81664740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741acce0 con 0x55c77a8664d0 2024-05-21T15:21:43.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e81664740 10 client.0 _ll_register_callbacks cb 0x7f8e64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:43.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a86e6b0 0x55c77ac41c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a86e6b0 0x55c77ac41c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35428/0 (socket says 172.21.15.78:35428) 2024-05-21T15:21:43.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e73fff640 1 -- 172.21.15.78:0/2780820258 learned_addr learned my addr 172.21.15.78:0/2780820258 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e737fe640 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a8664d0 0x55c77ac41730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e737fe640 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 0x55c77acb0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e73fff640 1 -- 172.21.15.78:0/2780820258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a8664d0 msgr2=0x55c77ac41730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e73fff640 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a8664d0 0x55c77ac41730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e73fff640 1 -- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 msgr2=0x55c77acb0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e73fff640 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 0x55c77acb0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e73fff640 1 -- 172.21.15.78:0/2780820258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77ac42510 con 0x55c77a86e6b0 2024-05-21T15:21:43.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.625+0000 7f8e737fe640 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 0x55c77acb0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:43.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e73fff640 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a86e6b0 0x55c77ac41c70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005ab0 tx=0x7f8e64004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44cfa7a62709194 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e537fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:43.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e537fe640 1 -- 172.21.15.78:0/2780820258 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64035040 con 0x55c77a86e6b0 2024-05-21T15:21:43.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e537fe640 1 -- 172.21.15.78:0/2780820258 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8e64005240 con 0x55c77a86e6b0 2024-05-21T15:21:43.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 1 -- 172.21.15.78:0/2780820258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e68023070 con 0x55c77a86e6b0 2024-05-21T15:21:43.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e537fe640 1 -- 172.21.15.78:0/2780820258 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8e64051050 con 0x55c77a86e6b0 2024-05-21T15:21:43.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 10 client.17187 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 1 -- 172.21.15.78:0/2780820258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c77acb2120 con 0x55c77a86e6b0 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e537fe640 1 -- 172.21.15.78:0/2780820258 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e68023070 con 0x55c77a86e6b0 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e537fe640 1 -- 172.21.15.78:0/2780820258 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8e6404c030 con 0x55c77a86e6b0 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 10 client.17187 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 10 client.17187 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 20 client.17187 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55c77a8233a0 0x55c77acb4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 4 client.17187 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:43.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.626+0000 7f8e81664740 1 -- 172.21.15.78:0/2780820258 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c77ab9b6c0 con 0x55c77a8233a0 2024-05-21T15:21:43.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e737fe640 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55c77a8233a0 0x55c77acb4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e537fe640 10 client.17187 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:21:43.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e737fe640 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55c77a8233a0 0x55c77acb4960 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51a9a4546a22184a server_cookie=1f12611ae3014454 in_seq=0 out_seq=0 2024-05-21T15:21:43.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e537fe640 1 -- 172.21.15.78:0/2780820258 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c77ab9b6c0 con 0x55c77a8233a0 2024-05-21T15:21:43.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e537fe640 10 client.17187 handle_command_reply: tid=0 2024-05-21T15:21:43.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 client.17187 shutdown 2024-05-21T15:21:43.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 -- 172.21.15.78:0/2780820258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a86e6b0 msgr2=0x55c77ac41c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a86e6b0 0x55c77ac41c70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005ab0 tx=0x7f8e64004250 comp rx=0 tx=0).stop 2024-05-21T15:21:43.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 -- 172.21.15.78:0/2780820258 shutdown_connections 2024-05-21T15:21:43.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55c77a8233a0 0x55c77acb4960 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c77a88f390 0x55c77acb0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e537fe640 0 client.17187 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:21:43.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c77a86e6b0 0x55c77ac41c70 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 --2- 172.21.15.78:0/2780820258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c77a8664d0 0x55c77ac41730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.627+0000 7f8e81664740 1 -- 172.21.15.78:0/2780820258 >> 172.21.15.78:0/2780820258 conn(0x55c77a8834e0 msgr2=0x55c77ac2e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e81664740 1 -- 172.21.15.78:0/2780820258 shutdown_connections 2024-05-21T15:21:43.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e81664740 1 -- 172.21.15.78:0/2780820258 wait complete. 2024-05-21T15:21:43.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e81664740 20 client.17187 trim_cache size 0 max 16384 2024-05-21T15:21:43.644 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:43.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e5c0a2d00 msgr2=0x7f8e5c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e5c0a2d00 0x7f8e5c0a5210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c09b8e0 tx=0x7f8e6c0c6770 comp rx=0 tx=0).stop 2024-05-21T15:21:43.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108f60 msgr2=0x7f8e7419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108f60 0x7f8e7419dfe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00eaa0 tx=0x7f8e6c00ef60 comp rx=0 tx=0).stop 2024-05-21T15:21:43.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3855793426 shutdown_connections 2024-05-21T15:21:43.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8e5c0a2d00 0x7f8e5c0a5210 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e74108f60 0x7f8e7419dfe0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e74108ac0 0x7f8e74199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 --2- 172.21.15.78:0/3855793426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8e74103210 0x7f8e741997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3855793426 >> 172.21.15.78:0/3855793426 conn(0x7f8e740fe1c0 msgr2=0x7f8e740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3855793426 shutdown_connections 2024-05-21T15:21:43.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.628+0000 7f8e7a7b5640 1 -- 172.21.15.78:0/3855793426 wait complete. 2024-05-21T15:21:43.647 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:21:43.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 -- 172.21.15.78:0/3908264124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c10c350 msgr2=0x7fcb8c10e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 --2- 172.21.15.78:0/3908264124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c10c350 0x7fcb8c10e770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009ca0 tx=0x7fcb8402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 -- 172.21.15.78:0/3908264124 shutdown_connections 2024-05-21T15:21:43.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 --2- 172.21.15.78:0/3908264124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c10c350 0x7fcb8c10e770 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 --2- 172.21.15.78:0/3908264124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8c102210 0x7fcb8c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 --2- 172.21.15.78:0/3908264124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb8c1079f0 0x7fcb8c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 -- 172.21.15.78:0/3908264124 >> 172.21.15.78:0/3908264124 conn(0x7fcb8c076830 msgr2=0x7fcb8c076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 -- 172.21.15.78:0/3908264124 shutdown_connections 2024-05-21T15:21:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.757+0000 7fcb90e32640 1 -- 172.21.15.78:0/3908264124 wait complete. 2024-05-21T15:21:43.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.758+0000 7fcb90e32640 1 Processor -- start 2024-05-21T15:21:43.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.758+0000 7fcb90e32640 1 -- start start 2024-05-21T15:21:43.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.758+0000 7fcb90e32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb8c102210 0x7fcb8c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.758+0000 7fcb90e32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c1079f0 0x7fcb8c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.758+0000 7fcb90e32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8c10c350 0x7fcb8c19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.758+0000 7fcb90e32640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c111d20 con 0x7fcb8c1079f0 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.758+0000 7fcb90e32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c111ba0 con 0x7fcb8c102210 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.758+0000 7fcb90e32640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c111ea0 con 0x7fcb8c10c350 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c1079f0 0x7fcb8c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c1079f0 0x7fcb8c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35448/0 (socket says 172.21.15.78:35448) 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 -- 172.21.15.78:0/2206248940 learned_addr learned my addr 172.21.15.78:0/2206248940 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb8a59c640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb8c102210 0x7fcb8c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb8ad9d640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8c10c350 0x7fcb8c19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 -- 172.21.15.78:0/2206248940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8c10c350 msgr2=0x7fcb8c19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8c10c350 0x7fcb8c19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 -- 172.21.15.78:0/2206248940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb8c102210 msgr2=0x7fcb8c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb8c102210 0x7fcb8c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 -- 172.21.15.78:0/2206248940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c1a5920 con 0x7fcb8c1079f0 2024-05-21T15:21:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb83fff640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c1079f0 0x7fcb8c19ab20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400e9f0 tx=0x7fcb7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb837fe640 1 -- 172.21.15.78:0/2206248940 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400ccc0 con 0x7fcb8c1079f0 2024-05-21T15:21:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb837fe640 1 -- 172.21.15.78:0/2206248940 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb740035f0 con 0x7fcb8c1079f0 2024-05-21T15:21:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb90e32640 1 -- 172.21.15.78:0/2206248940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c1a5c10 con 0x7fcb8c1079f0 2024-05-21T15:21:43.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb837fe640 1 -- 172.21.15.78:0/2206248940 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7400ce60 con 0x7fcb8c1079f0 2024-05-21T15:21:43.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.759+0000 7fcb90e32640 1 -- 172.21.15.78:0/2206248940 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c1a6110 con 0x7fcb8c1079f0 2024-05-21T15:21:43.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb837fe640 1 -- 172.21.15.78:0/2206248940 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcb740037a0 con 0x7fcb8c1079f0 2024-05-21T15:21:43.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb97dbb740 1 Processor -- start 2024-05-21T15:21:43.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb97dbb740 1 -- start start 2024-05-21T15:21:43.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbee390 0x564bacf8cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 0x564bacf8d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbcd6b0 0x564bacf8d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564bacef2c40 con 0x564bacbee390 2024-05-21T15:21:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564bace9a580 con 0x564bacbc54d0 2024-05-21T15:21:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564baceeec00 con 0x564bacbcd6b0 2024-05-21T15:21:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb8ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbee390 0x564bacf8cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb8ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbee390 0x564bacf8cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35452/0 (socket says 172.21.15.78:35452) 2024-05-21T15:21:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.762+0000 7fcb8ad9d640 1 -- 172.21.15.78:0/1906513234 learned_addr learned my addr 172.21.15.78:0/1906513234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb83fff640 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbcd6b0 0x564bacf8d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb837fe640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcb780a2d00 0x7fcb780a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8a59c640 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 0x564bacf8d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8ad9d640 1 -- 172.21.15.78:0/1906513234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbcd6b0 msgr2=0x564bacf8d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8ad9d640 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbcd6b0 0x564bacf8d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8ad9d640 1 -- 172.21.15.78:0/1906513234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 msgr2=0x564bacf8d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8ad9d640 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 0x564bacf8d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8ad9d640 1 -- 172.21.15.78:0/1906513234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bacc063f0 con 0x564bacbee390 2024-05-21T15:21:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8a59c640 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 0x564bacf8d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:43.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb83fff640 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbcd6b0 0x564bacf8d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8ad9d640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcb780a2d00 0x7fcb780a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb837fe640 1 -- 172.21.15.78:0/2206248940 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcb740cfa50 con 0x7fcb8c1079f0 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb8ad9d640 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbee390 0x564bacf8cd20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcb8402ade0 tx=0x7fcb8402f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7df091628219bbe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb80ff9640 1 -- 172.21.15.78:0/1906513234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84041040 con 0x564bacbee390 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb80ff9640 1 -- 172.21.15.78:0/1906513234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb84036070 con 0x564bacbee390 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.763+0000 7fcb80ff9640 1 -- 172.21.15.78:0/1906513234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8402f870 con 0x564bacbee390 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 -- 172.21.15.78:0/1906513234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbee390 msgr2=0x564bacf8cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbee390 0x564bacf8cd20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcb8402ade0 tx=0x7fcb8402f3e0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb8ad9d640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcb780a2d00 0x7fcb780a5210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb84048000 tx=0x7fcb8404d000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 -- 172.21.15.78:0/1906513234 shutdown_connections 2024-05-21T15:21:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 0x564bacf8d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbcd6b0 0x564bacf8d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/1906513234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbee390 0x564bacf8cd20 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 -- 172.21.15.78:0/1906513234 >> 172.21.15.78:0/1906513234 conn(0x564bacbe24e0 msgr2=0x564bacf894d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 -- 172.21.15.78:0/1906513234 shutdown_connections 2024-05-21T15:21:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.764+0000 7fcb97dbb740 1 -- 172.21.15.78:0/1906513234 wait complete. 2024-05-21T15:21:43.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 1 -- start start 2024-05-21T15:21:43.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 1 Processor -- start 2024-05-21T15:21:43.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbc54d0 0x564bacf973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbcd6b0 0x564bacf97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbee390 0x564bacf9d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564bacf9a010 con 0x564bacbc54d0 2024-05-21T15:21:43.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564baceadc40 con 0x564bacbcd6b0 2024-05-21T15:21:43.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564bacf9a190 con 0x564bacbee390 2024-05-21T15:21:43.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb97dbb740 10 client.0 _ll_register_callbacks cb 0x564bacc05be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:43.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb8a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbc54d0 0x564bacf973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb8a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbc54d0 0x564bacf973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35460/0 (socket says 172.21.15.78:35460) 2024-05-21T15:21:43.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb8a59c640 1 -- 172.21.15.78:0/4154208627 learned_addr learned my addr 172.21.15.78:0/4154208627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb8a59c640 1 -- 172.21.15.78:0/4154208627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbee390 msgr2=0x564bacf9d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb8a59c640 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbee390 0x564bacf9d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb8a59c640 1 -- 172.21.15.78:0/4154208627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbcd6b0 msgr2=0x564bacf97910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb8a59c640 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbcd6b0 0x564bacf97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.765+0000 7fcb8a59c640 1 -- 172.21.15.78:0/4154208627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564baccd07d0 con 0x564bacbc54d0 2024-05-21T15:21:43.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb8a59c640 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbc54d0 0x564bacf973d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c06a230 tx=0x7fcb7c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919581ce96a69211 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:43.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb71ffb640 1 -- 172.21.15.78:0/4154208627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c012860 con 0x564bacbc54d0 2024-05-21T15:21:43.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb71ffb640 1 -- 172.21.15.78:0/4154208627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcb7c012a00 con 0x564bacbc54d0 2024-05-21T15:21:43.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 1 -- 172.21.15.78:0/4154208627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb84036070 con 0x564bacbc54d0 2024-05-21T15:21:43.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb71ffb640 1 -- 172.21.15.78:0/4154208627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcb7c012c00 con 0x564bacbc54d0 2024-05-21T15:21:43.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 10 client.17199 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:43.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 1 -- 172.21.15.78:0/4154208627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564bacfa67d0 con 0x564bacbc54d0 2024-05-21T15:21:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb71ffb640 1 -- 172.21.15.78:0/4154208627 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb84036070 con 0x564bacbc54d0 2024-05-21T15:21:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb71ffb640 1 -- 172.21.15.78:0/4154208627 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fcb7c0254a0 con 0x564bacbc54d0 2024-05-21T15:21:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 10 client.17199 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 10 client.17199 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:21:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 20 client.17199 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564bacbacf60 0x564bacfa9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 4 client.17199 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.766+0000 7fcb97dbb740 1 -- 172.21.15.78:0/4154208627 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bacf00370 con 0x564bacbacf60 2024-05-21T15:21:43.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.767+0000 7fcb83fff640 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564bacbacf60 0x564bacfa9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.767+0000 7fcb71ffb640 10 client.17199 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:21:43.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.767+0000 7fcb83fff640 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564bacbacf60 0x564bacfa9d90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=33afe39a7250927a server_cookie=450422c690feee45 in_seq=0 out_seq=0 2024-05-21T15:21:43.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb71ffb640 1 -- 172.21.15.78:0/4154208627 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564bacf00370 con 0x564bacbacf60 2024-05-21T15:21:43.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb71ffb640 10 client.17199 handle_command_reply: tid=0 2024-05-21T15:21:43.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 client.17199 shutdown 2024-05-21T15:21:43.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 -- 172.21.15.78:0/4154208627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbc54d0 msgr2=0x564bacf973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbc54d0 0x564bacf973d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c06a230 tx=0x7fcb7c0117b0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 -- 172.21.15.78:0/4154208627 shutdown_connections 2024-05-21T15:21:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564bacbacf60 0x564bacfa9d90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbcd6b0 0x564bacf97910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbee390 0x564bacf9d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/4154208627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbc54d0 0x564bacf973d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb71ffb640 0 client.17199 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:21:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 -- 172.21.15.78:0/4154208627 >> 172.21.15.78:0/4154208627 conn(0x564bacbe24e0 msgr2=0x564bacf8ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 -- 172.21.15.78:0/4154208627 shutdown_connections 2024-05-21T15:21:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 1 -- 172.21.15.78:0/4154208627 wait complete. 2024-05-21T15:21:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.768+0000 7fcb97dbb740 20 client.17199 trim_cache size 0 max 16384 2024-05-21T15:21:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb97dbb740 1 Processor -- start 2024-05-21T15:21:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb97dbb740 1 -- start start 2024-05-21T15:21:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbc54d0 0x564bacf91120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbee390 0x564bacfaf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1aead0 con 0x564bacbcd6b0 2024-05-21T15:21:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1b0610 con 0x564bacbee390 2024-05-21T15:21:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1af1a0 con 0x564bacbc54d0 2024-05-21T15:21:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35472/0 (socket says 172.21.15.78:35472) 2024-05-21T15:21:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.782+0000 7fcb83fff640 1 -- 172.21.15.78:0/5277972 learned_addr learned my addr 172.21.15.78:0/5277972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb83fff640 1 -- 172.21.15.78:0/5277972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbc54d0 msgr2=0x564bacf91120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb83fff640 1 --2- 172.21.15.78:0/5277972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbc54d0 0x564bacf91120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb83fff640 1 -- 172.21.15.78:0/5277972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbee390 msgr2=0x564bacfaf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb83fff640 1 --2- 172.21.15.78:0/5277972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbee390 0x564bacfaf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb83fff640 1 -- 172.21.15.78:0/5277972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bacf95710 con 0x564bacbcd6b0 2024-05-21T15:21:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb83fff640 1 --2- 172.21.15.78:0/5277972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9d3f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcb7409bc10 tx=0x7fcb740d1f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b4756ab6402691 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb73fff640 1 -- 172.21.15.78:0/5277972 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7409c610 con 0x564bacbcd6b0 2024-05-21T15:21:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb73fff640 1 -- 172.21.15.78:0/5277972 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb740d6070 con 0x564bacbcd6b0 2024-05-21T15:21:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb73fff640 1 -- 172.21.15.78:0/5277972 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb740cd3c0 con 0x564bacbcd6b0 2024-05-21T15:21:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb97dbb740 1 -- 172.21.15.78:0/5277972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 msgr2=0x564bacf9d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.783+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/5277972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9d3f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcb7409bc10 tx=0x7fcb740d1f60 comp rx=0 tx=0).stop 2024-05-21T15:21:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 -- 172.21.15.78:0/5277972 shutdown_connections 2024-05-21T15:21:43.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/5277972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbee390 0x564bacfaf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/5277972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9d3f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/5277972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbc54d0 0x564bacf91120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 -- 172.21.15.78:0/5277972 >> 172.21.15.78:0/5277972 conn(0x564bacbe24e0 msgr2=0x564bacf90a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 -- 172.21.15.78:0/5277972 shutdown_connections 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 -- 172.21.15.78:0/5277972 wait complete. 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 -- start start 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 Processor -- start 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 0x564bacfa3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbee390 0x564bacf9f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1aff40 con 0x564bacbcd6b0 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1af870 con 0x564bacbc54d0 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1acab0 con 0x564bacbee390 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb97dbb740 10 client.0 _ll_register_callbacks cb 0x7fcb7c00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35482/0 (socket says 172.21.15.78:35482) 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.784+0000 7fcb83fff640 1 -- 172.21.15.78:0/2649793834 learned_addr learned my addr 172.21.15.78:0/2649793834 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb83fff640 1 -- 172.21.15.78:0/2649793834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbee390 msgr2=0x564bacf9f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb83fff640 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbee390 0x564bacf9f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb83fff640 1 -- 172.21.15.78:0/2649793834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 msgr2=0x564bacfa3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb83fff640 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 0x564bacfa3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb83fff640 1 -- 172.21.15.78:0/2649793834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bacf9fe80 con 0x564bacbcd6b0 2024-05-21T15:21:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb83fff640 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9f1d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcb740cd740 tx=0x7fcb740a02c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e606a117e27fb25 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb71ffb640 1 -- 172.21.15.78:0/2649793834 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7409c610 con 0x564bacbcd6b0 2024-05-21T15:21:43.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb71ffb640 1 -- 172.21.15.78:0/2649793834 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fcb740cd3c0 con 0x564bacbcd6b0 2024-05-21T15:21:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 1 -- 172.21.15.78:0/2649793834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb740d6070 con 0x564bacbcd6b0 2024-05-21T15:21:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb71ffb640 1 -- 172.21.15.78:0/2649793834 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcb7409bf10 con 0x564bacbcd6b0 2024-05-21T15:21:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 10 client.17205 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 1 -- 172.21.15.78:0/2649793834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564bad012050 con 0x564bacbcd6b0 2024-05-21T15:21:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb71ffb640 1 -- 172.21.15.78:0/2649793834 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb740d6070 con 0x564bacbcd6b0 2024-05-21T15:21:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb71ffb640 1 -- 172.21.15.78:0/2649793834 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fcb7409b360 con 0x564bacbcd6b0 2024-05-21T15:21:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 10 client.17205 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:43.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 10 client.17205 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 20 client.17205 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564bacb823a0 0x564bad0147e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 4 client.17205 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.785+0000 7fcb97dbb740 1 -- 172.21.15.78:0/2649793834 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bacefa6c0 con 0x564bacb823a0 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.786+0000 7fcb8a59c640 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564bacb823a0 0x564bad0147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.786+0000 7fcb71ffb640 10 client.17205 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.786+0000 7fcb8a59c640 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564bacb823a0 0x564bad0147e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96132cb2b16cea3a server_cookie=4959d5252c0add88 in_seq=0 out_seq=0 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.786+0000 7fcb71ffb640 1 -- 172.21.15.78:0/2649793834 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bacefa6c0 con 0x564bacb823a0 2024-05-21T15:21:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.786+0000 7fcb71ffb640 10 client.17205 handle_command_reply: tid=0 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.786+0000 7fcb97dbb740 1 client.17205 shutdown 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 -- 172.21.15.78:0/2649793834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 msgr2=0x564bacf9f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9f1d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcb740cd740 tx=0x7fcb740a02c0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 -- 172.21.15.78:0/2649793834 shutdown_connections 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x564bacb823a0 0x564bad0147e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564bacbee390 0x564bacf9f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb71ffb640 0 client.17205 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564bacbcd6b0 0x564bacf9f1d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 --2- 172.21.15.78:0/2649793834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564bacbc54d0 0x564bacfa3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 -- 172.21.15.78:0/2649793834 >> 172.21.15.78:0/2649793834 conn(0x564bacbe24e0 msgr2=0x564bacf87b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 -- 172.21.15.78:0/2649793834 shutdown_connections 2024-05-21T15:21:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 1 -- 172.21.15.78:0/2649793834 wait complete. 2024-05-21T15:21:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb97dbb740 20 client.17205 trim_cache size 0 max 16384 2024-05-21T15:21:43.801 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 -- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcb780a2d00 msgr2=0x7fcb780a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcb780a2d00 0x7fcb780a5210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb84048000 tx=0x7fcb8404d000 comp rx=0 tx=0).stop 2024-05-21T15:21:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 -- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c1079f0 msgr2=0x7fcb8c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c1079f0 0x7fcb8c19ab20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400e9f0 tx=0x7fcb7400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 -- 172.21.15.78:0/2206248940 shutdown_connections 2024-05-21T15:21:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcb780a2d00 0x7fcb780a5210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8c10c350 0x7fcb8c19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb8c1079f0 0x7fcb8c19ab20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 --2- 172.21.15.78:0/2206248940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb8c102210 0x7fcb8c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 -- 172.21.15.78:0/2206248940 >> 172.21.15.78:0/2206248940 conn(0x7fcb8c076830 msgr2=0x7fcb8c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 -- 172.21.15.78:0/2206248940 shutdown_connections 2024-05-21T15:21:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.787+0000 7fcb90e32640 1 -- 172.21.15.78:0/2206248940 wait complete. 2024-05-21T15:21:43.804 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:43 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1715398600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:21:43.805 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:21:43.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:43 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1715398600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:21:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 -- 172.21.15.78:0/1301131042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c1079d0 msgr2=0x7f9d4c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1301131042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c1079d0 0x7f9d4c101560 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009ca0 tx=0x7f9d4802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 -- 172.21.15.78:0/1301131042 shutdown_connections 2024-05-21T15:21:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1301131042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d4c10c330 0x7f9d4c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1301131042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d4c101aa0 0x7f9d4c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1301131042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c1079d0 0x7f9d4c101560 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 -- 172.21.15.78:0/1301131042 >> 172.21.15.78:0/1301131042 conn(0x7f9d4c0770b0 msgr2=0x7f9d4c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 -- 172.21.15.78:0/1301131042 shutdown_connections 2024-05-21T15:21:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.915+0000 7f9d5420d640 1 -- 172.21.15.78:0/1301131042 wait complete. 2024-05-21T15:21:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d5420d640 1 Processor -- start 2024-05-21T15:21:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d5420d640 1 -- start start 2024-05-21T15:21:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d5420d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d4c101aa0 0x7f9d4c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d5420d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d4c1079d0 0x7f9d4c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d5420d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c10c330 0x7f9d4c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d5420d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c111ce0 con 0x7f9d4c10c330 2024-05-21T15:21:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d5420d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c111b60 con 0x7f9d4c1079d0 2024-05-21T15:21:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d5420d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c111e60 con 0x7f9d4c101aa0 2024-05-21T15:21:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d527aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c10c330 0x7f9d4c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d527aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c10c330 0x7f9d4c19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35504/0 (socket says 172.21.15.78:35504) 2024-05-21T15:21:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d527aa640 1 -- 172.21.15.78:0/1352688784 learned_addr learned my addr 172.21.15.78:0/1352688784 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d51fa9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d4c101aa0 0x7f9d4c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d517a8640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d4c1079d0 0x7f9d4c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d527aa640 1 -- 172.21.15.78:0/1352688784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d4c101aa0 msgr2=0x7f9d4c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d527aa640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d4c101aa0 0x7f9d4c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d527aa640 1 -- 172.21.15.78:0/1352688784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d4c1079d0 msgr2=0x7f9d4c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d527aa640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d4c1079d0 0x7f9d4c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d527aa640 1 -- 172.21.15.78:0/1352688784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c19d580 con 0x7f9d4c10c330 2024-05-21T15:21:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.916+0000 7f9d517a8640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d4c1079d0 0x7f9d4c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.917+0000 7f9d527aa640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c10c330 0x7f9d4c19cf40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400e970 tx=0x7f9d4400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.917+0000 7f9d42ffd640 1 -- 172.21.15.78:0/1352688784 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400ccc0 con 0x7f9d4c10c330 2024-05-21T15:21:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.917+0000 7f9d42ffd640 1 -- 172.21.15.78:0/1352688784 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d4400ce60 con 0x7f9d4c10c330 2024-05-21T15:21:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.917+0000 7f9d42ffd640 1 -- 172.21.15.78:0/1352688784 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44010490 con 0x7f9d4c10c330 2024-05-21T15:21:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.917+0000 7f9d5420d640 1 -- 172.21.15.78:0/1352688784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4c19d870 con 0x7f9d4c10c330 2024-05-21T15:21:43.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.917+0000 7f9d5420d640 1 -- 172.21.15.78:0/1352688784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4c1a51f0 con 0x7f9d4c10c330 2024-05-21T15:21:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.919+0000 7f9d42ffd640 1 -- 172.21.15.78:0/1352688784 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9d44005020 con 0x7f9d4c10c330 2024-05-21T15:21:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d5b191740 1 Processor -- start 2024-05-21T15:21:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d5b191740 1 -- start start 2024-05-21T15:21:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 0x563877ae5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877ae6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877ae67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d5b191740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563877a4bc40 con 0x5638777266b0 2024-05-21T15:21:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d5b191740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5638779f3580 con 0x56387771e4d0 2024-05-21T15:21:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d5b191740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563877a47c00 con 0x563877747390 2024-05-21T15:21:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d517a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877ae67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d517a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877ae67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35518/0 (socket says 172.21.15.78:35518) 2024-05-21T15:21:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d517a8640 1 -- 172.21.15.78:0/789023176 learned_addr learned my addr 172.21.15.78:0/789023176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d42ffd640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9d380a2d80 0x7f9d380a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.920+0000 7f9d517a8640 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 0x563877ae5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877ae6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d527aa640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9d380a2d80 0x7f9d380a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d517a8640 1 -- 172.21.15.78:0/789023176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 msgr2=0x563877ae5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d517a8640 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 0x563877ae5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d517a8640 1 -- 172.21.15.78:0/789023176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 msgr2=0x563877ae6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d517a8640 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877ae6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d517a8640 1 -- 172.21.15.78:0/789023176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387775f3f0 con 0x5638777266b0 2024-05-21T15:21:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d42ffd640 1 -- 172.21.15.78:0/1352688784 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9d440cf960 con 0x7f9d4c10c330 2024-05-21T15:21:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d517a8640 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877ae67a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c013820 tx=0x7f9d3c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85e5bccb4438ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d527aa640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9d380a2d80 0x7f9d380a5290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d4409bac0 tx=0x7f9d4406dd80 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d37fff640 1 -- 172.21.15.78:0/789023176 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c013e60 con 0x5638777266b0 2024-05-21T15:21:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d37fff640 1 -- 172.21.15.78:0/789023176 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d3c00edd0 con 0x5638777266b0 2024-05-21T15:21:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.921+0000 7f9d37fff640 1 -- 172.21.15.78:0/789023176 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c01ed50 con 0x5638777266b0 2024-05-21T15:21:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 -- 172.21.15.78:0/789023176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 msgr2=0x563877ae67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877ae67a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c013820 tx=0x7f9d3c00b3e0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 -- 172.21.15.78:0/789023176 shutdown_connections 2024-05-21T15:21:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877ae6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877ae67a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 --2- 172.21.15.78:0/789023176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 0x563877ae5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 -- 172.21.15.78:0/789023176 >> 172.21.15.78:0/789023176 conn(0x56387773b4e0 msgr2=0x563877ae24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 -- 172.21.15.78:0/789023176 shutdown_connections 2024-05-21T15:21:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.922+0000 7f9d5b191740 1 -- 172.21.15.78:0/789023176 wait complete. 2024-05-21T15:21:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 1 -- start start 2024-05-21T15:21:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 1 Processor -- start 2024-05-21T15:21:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56387771e4d0 0x563877af7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877af7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563877747390 0x563877af33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563877af2fb0 con 0x56387771e4d0 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563877a06c40 con 0x563877747390 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563877af3130 con 0x5638777266b0 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d5b191740 10 client.0 _ll_register_callbacks cb 0x56387775ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d517a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56387771e4d0 0x563877af7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d517a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56387771e4d0 0x563877af7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35534/0 (socket says 172.21.15.78:35534) 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d517a8640 1 -- 172.21.15.78:0/1260994833 learned_addr learned my addr 172.21.15.78:0/1260994833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d517a8640 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563877747390 0x563877af33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.923+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877af7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d517a8640 1 -- 172.21.15.78:0/1260994833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 msgr2=0x563877af7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d517a8640 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877af7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d517a8640 1 -- 172.21.15.78:0/1260994833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563877747390 msgr2=0x563877af33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d517a8640 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563877747390 0x563877af33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d517a8640 1 -- 172.21.15.78:0/1260994833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638778297d0 con 0x56387771e4d0 2024-05-21T15:21:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d517a8640 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56387771e4d0 0x563877af7810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c005940 tx=0x7f9d3c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa46e57dfff08c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d357fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d357fa640 1 -- 172.21.15.78:0/1260994833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c013e60 con 0x56387771e4d0 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d357fa640 1 -- 172.21.15.78:0/1260994833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9d3c01ebd0 con 0x56387771e4d0 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d5b191740 1 -- 172.21.15.78:0/1260994833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d3c00edd0 con 0x56387771e4d0 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d357fa640 1 -- 172.21.15.78:0/1260994833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9d3c01edd0 con 0x56387771e4d0 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d5b191740 10 client.17217 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d5b191740 1 -- 172.21.15.78:0/1260994833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563877b019c0 con 0x56387771e4d0 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d357fa640 1 -- 172.21.15.78:0/1260994833 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d3c00edd0 con 0x56387771e4d0 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d357fa640 1 -- 172.21.15.78:0/1260994833 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9d3c00f040 con 0x56387771e4d0 2024-05-21T15:21:43.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d5b191740 10 client.17217 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:43.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d5b191740 10 client.17217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:21:43.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.924+0000 7f9d5b191740 20 client.17217 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:43.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.925+0000 7f9d5b191740 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x563877705f60 0x563877b04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.925+0000 7f9d5b191740 4 client.17217 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:43.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.925+0000 7f9d5b191740 1 -- 172.21.15.78:0/1260994833 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563877a59370 con 0x563877705f60 2024-05-21T15:21:43.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.925+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x563877705f60 0x563877b04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.925+0000 7f9d357fa640 10 client.17217 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:21:43.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.925+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x563877705f60 0x563877b04d20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=463fb4bce7e8444a server_cookie=2c4209d79620cf7d in_seq=0 out_seq=0 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d357fa640 1 -- 172.21.15.78:0/1260994833 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563877a59370 con 0x563877705f60 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d357fa640 10 client.17217 handle_command_reply: tid=0 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 client.17217 shutdown 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 -- 172.21.15.78:0/1260994833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56387771e4d0 msgr2=0x563877af7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56387771e4d0 0x563877af7810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c005940 tx=0x7f9d3c001e50 comp rx=0 tx=0).stop 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 -- 172.21.15.78:0/1260994833 shutdown_connections 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x563877705f60 0x563877b04d20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563877747390 0x563877af33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877af7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 --2- 172.21.15.78:0/1260994833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56387771e4d0 0x563877af7810 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d357fa640 0 client.17217 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:21:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 -- 172.21.15.78:0/1260994833 >> 172.21.15.78:0/1260994833 conn(0x56387773b4e0 msgr2=0x563877ae7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 -- 172.21.15.78:0/1260994833 shutdown_connections 2024-05-21T15:21:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 1 -- 172.21.15.78:0/1260994833 wait complete. 2024-05-21T15:21:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.926+0000 7f9d5b191740 20 client.17217 trim_cache size 0 max 16384 2024-05-21T15:21:43.941 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:43 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1715398600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:21:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d5b191740 1 Processor -- start 2024-05-21T15:21:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d5b191740 1 -- start start 2024-05-21T15:21:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877ae80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877aebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563877747390 0x563877aec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d5b191740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c1acb00 con 0x563877747390 2024-05-21T15:21:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d5b191740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c1ae640 con 0x56387771e4d0 2024-05-21T15:21:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d5b191740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c1ad1d0 con 0x5638777266b0 2024-05-21T15:21:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563877747390 0x563877aec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563877747390 0x563877aec260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35538/0 (socket says 172.21.15.78:35538) 2024-05-21T15:21:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 -- 172.21.15.78:0/4090083385 learned_addr learned my addr 172.21.15.78:0/4090083385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877aebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 -- 172.21.15.78:0/4090083385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 msgr2=0x563877aebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877aebd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 -- 172.21.15.78:0/4090083385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 msgr2=0x563877ae80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877ae80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 -- 172.21.15.78:0/4090083385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563877aff490 con 0x563877747390 2024-05-21T15:21:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877aebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d517a8640 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563877747390 0x563877aec260 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c005600 tx=0x7f9d3c013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4b9b4ea855b822 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.940+0000 7f9d377fe640 1 -- 172.21.15.78:0/4090083385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c022070 con 0x563877747390 2024-05-21T15:21:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d377fe640 1 -- 172.21.15.78:0/4090083385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d3c01e430 con 0x563877747390 2024-05-21T15:21:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d377fe640 1 -- 172.21.15.78:0/4090083385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c013e60 con 0x563877747390 2024-05-21T15:21:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 -- 172.21.15.78:0/4090083385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563877747390 msgr2=0x563877aec260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563877747390 0x563877aec260 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c005600 tx=0x7f9d3c013720 comp rx=0 tx=0).stop 2024-05-21T15:21:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 -- 172.21.15.78:0/4090083385 shutdown_connections 2024-05-21T15:21:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563877747390 0x563877aec260 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5638777266b0 0x563877aebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 --2- 172.21.15.78:0/4090083385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877ae80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 -- 172.21.15.78:0/4090083385 >> 172.21.15.78:0/4090083385 conn(0x56387773b4e0 msgr2=0x563877ae79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 -- 172.21.15.78:0/4090083385 shutdown_connections 2024-05-21T15:21:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.941+0000 7f9d5b191740 1 -- 172.21.15.78:0/4090083385 wait complete. 2024-05-21T15:21:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 1 -- start start 2024-05-21T15:21:43.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 1 Processor -- start 2024-05-21T15:21:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877af9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877af99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 0x563877af9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c1adf70 con 0x5638777266b0 2024-05-21T15:21:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c1ad8a0 con 0x56387771e4d0 2024-05-21T15:21:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c1aaae0 con 0x563877747390 2024-05-21T15:21:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d5b191740 10 client.0 _ll_register_callbacks cb 0x7f9d3c00be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d51fa9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877af99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d51fa9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877af99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35540/0 (socket says 172.21.15.78:35540) 2024-05-21T15:21:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d51fa9640 1 -- 172.21.15.78:0/3328010885 learned_addr learned my addr 172.21.15.78:0/3328010885 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d517a8640 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877af9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d517a8640 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 0x563877af9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d51fa9640 1 -- 172.21.15.78:0/3328010885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 msgr2=0x563877af9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.942+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 0x563877af9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d51fa9640 1 -- 172.21.15.78:0/3328010885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 msgr2=0x563877af9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877af9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d51fa9640 1 -- 172.21.15.78:0/3328010885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563877b6af70 con 0x5638777266b0 2024-05-21T15:21:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d51fa9640 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877af99b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9d48005e30 tx=0x7f9d48009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18b8dc9fbe8c1d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d357fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d357fa640 1 -- 172.21.15.78:0/3328010885 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48031650 con 0x5638777266b0 2024-05-21T15:21:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d357fa640 1 -- 172.21.15.78:0/3328010885 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9d480317f0 con 0x5638777266b0 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 1 -- 172.21.15.78:0/3328010885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d3c01e430 con 0x5638777266b0 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d357fa640 1 -- 172.21.15.78:0/3328010885 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9d4802f610 con 0x5638777266b0 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 10 client.17223 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 1 -- 172.21.15.78:0/3328010885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563877b6c190 con 0x5638777266b0 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d357fa640 1 -- 172.21.15.78:0/3328010885 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d3c01e430 con 0x5638777266b0 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d357fa640 1 -- 172.21.15.78:0/3328010885 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9d48044040 con 0x5638777266b0 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 10 client.17223 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 10 client.17223 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:21:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 20 client.17223 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5638776db3a0 0x563877b6e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 4 client.17223 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.943+0000 7f9d5b191740 1 -- 172.21.15.78:0/3328010885 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563877a536c0 con 0x5638776db3a0 2024-05-21T15:21:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.944+0000 7f9d517a8640 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5638776db3a0 0x563877b6e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.944+0000 7f9d357fa640 10 client.17223 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:21:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.944+0000 7f9d517a8640 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5638776db3a0 0x563877b6e980 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99a7da3ecf35bc5e server_cookie=fdcc580d6e2432df in_seq=0 out_seq=0 2024-05-21T15:21:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.944+0000 7f9d357fa640 1 -- 172.21.15.78:0/3328010885 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563877a536c0 con 0x5638776db3a0 2024-05-21T15:21:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.944+0000 7f9d357fa640 10 client.17223 handle_command_reply: tid=0 2024-05-21T15:21:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.944+0000 7f9d5b191740 1 client.17223 shutdown 2024-05-21T15:21:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 -- 172.21.15.78:0/3328010885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 msgr2=0x563877af99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877af99b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9d48005e30 tx=0x7f9d48009990 comp rx=0 tx=0).stop 2024-05-21T15:21:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 -- 172.21.15.78:0/3328010885 shutdown_connections 2024-05-21T15:21:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5638776db3a0 0x563877b6e980 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563877747390 0x563877af9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638777266b0 0x563877af99b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 --2- 172.21.15.78:0/3328010885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56387771e4d0 0x563877af9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d357fa640 0 client.17223 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:21:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 -- 172.21.15.78:0/3328010885 >> 172.21.15.78:0/3328010885 conn(0x56387773b4e0 msgr2=0x563877af0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 -- 172.21.15.78:0/3328010885 shutdown_connections 2024-05-21T15:21:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 1 -- 172.21.15.78:0/3328010885 wait complete. 2024-05-21T15:21:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5b191740 20 client.17223 trim_cache size 0 max 16384 2024-05-21T15:21:43.958 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 -- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9d380a2d80 msgr2=0x7f9d380a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9d380a2d80 0x7f9d380a5290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d4409bac0 tx=0x7f9d4406dd80 comp rx=0 tx=0).stop 2024-05-21T15:21:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 -- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c10c330 msgr2=0x7f9d4c19cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c10c330 0x7f9d4c19cf40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400e970 tx=0x7f9d4400ed70 comp rx=0 tx=0).stop 2024-05-21T15:21:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 -- 172.21.15.78:0/1352688784 shutdown_connections 2024-05-21T15:21:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d4c1079d0 0x7f9d4c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f9d380a2d80 0x7f9d380a5290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d4c10c330 0x7f9d4c19cf40 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 --2- 172.21.15.78:0/1352688784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d4c101aa0 0x7f9d4c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 -- 172.21.15.78:0/1352688784 >> 172.21.15.78:0/1352688784 conn(0x7f9d4c0770b0 msgr2=0x7f9d4c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 -- 172.21.15.78:0/1352688784 shutdown_connections 2024-05-21T15:21:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:43.945+0000 7f9d5420d640 1 -- 172.21.15.78:0/1352688784 wait complete. 2024-05-21T15:21:43.961 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:21:44.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 -- 172.21.15.78:0/2636348661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 msgr2=0x7f4a201017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2636348661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 0x7f4a201017e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009900 tx=0x7f4a1802a410 comp rx=0 tx=0).stop 2024-05-21T15:21:44.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 -- 172.21.15.78:0/2636348661 shutdown_connections 2024-05-21T15:21:44.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2636348661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a20106420 0x7f4a2010dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2636348661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a20105960 0x7f4a20105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2636348661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 0x7f4a201017e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 -- 172.21.15.78:0/2636348661 >> 172.21.15.78:0/2636348661 conn(0x7f4a200768d0 msgr2=0x7f4a20076d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 -- 172.21.15.78:0/2636348661 shutdown_connections 2024-05-21T15:21:44.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.071+0000 7f4a26a79640 1 -- 172.21.15.78:0/2636348661 wait complete. 2024-05-21T15:21:44.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a26a79640 1 Processor -- start 2024-05-21T15:21:44.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a26a79640 1 -- start start 2024-05-21T15:21:44.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a26a79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 0x7f4a20198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a26a79640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a20105960 0x7f4a20198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a26a79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a20106420 0x7f4a2019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a26a79640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201114a0 con 0x7f4a201013d0 2024-05-21T15:21:44.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a26a79640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20111320 con 0x7f4a20105960 2024-05-21T15:21:44.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a26a79640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20111620 con 0x7f4a20106420 2024-05-21T15:21:44.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a24815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 0x7f4a20198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a24815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 0x7f4a20198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35550/0 (socket says 172.21.15.78:35550) 2024-05-21T15:21:44.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.072+0000 7f4a24815640 1 -- 172.21.15.78:0/2986995438 learned_addr learned my addr 172.21.15.78:0/2986995438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a20105960 0x7f4a20198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a25016640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a20106420 0x7f4a2019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a24815640 1 -- 172.21.15.78:0/2986995438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a20106420 msgr2=0x7f4a2019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a24815640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a20106420 0x7f4a2019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a24815640 1 -- 172.21.15.78:0/2986995438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a20105960 msgr2=0x7f4a20198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a24815640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a20105960 0x7f4a20198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a24815640 1 -- 172.21.15.78:0/2986995438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2019d590 con 0x7f4a201013d0 2024-05-21T15:21:44.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a25016640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a20106420 0x7f4a2019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:44.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a24815640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 0x7f4a20198730 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006010 tx=0x7f4a180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a1dffb640 1 -- 172.21.15.78:0/2986995438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1803f030 con 0x7f4a201013d0 2024-05-21T15:21:44.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a26a79640 1 -- 172.21.15.78:0/2986995438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2019d820 con 0x7f4a201013d0 2024-05-21T15:21:44.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a1dffb640 1 -- 172.21.15.78:0/2986995438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a18003ce0 con 0x7f4a201013d0 2024-05-21T15:21:44.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a1dffb640 1 -- 172.21.15.78:0/2986995438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18035070 con 0x7f4a201013d0 2024-05-21T15:21:44.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.073+0000 7f4a26a79640 1 -- 172.21.15.78:0/2986995438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2019dd60 con 0x7f4a201013d0 2024-05-21T15:21:44.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a2d9f5740 1 Processor -- start 2024-05-21T15:21:44.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a1dffb640 1 -- 172.21.15.78:0/2986995438 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4a18042030 con 0x7f4a201013d0 2024-05-21T15:21:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a2d9f5740 1 -- start start 2024-05-21T15:21:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb63b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce9eac9c40 con 0x55ce9e79c4d0 2024-05-21T15:21:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ce9ea71580 con 0x55ce9e7c5390 2024-05-21T15:21:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ce9eac5c00 con 0x55ce9e7a46b0 2024-05-21T15:21:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a24815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a24815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35554/0 (socket says 172.21.15.78:35554) 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a24815640 1 -- 172.21.15.78:0/383158047 learned_addr learned my addr 172.21.15.78:0/383158047 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a25016640 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb63b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a24815640 1 -- 172.21.15.78:0/383158047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 msgr2=0x55ce9eb645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a24815640 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a24815640 1 -- 172.21.15.78:0/383158047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 msgr2=0x55ce9eb63b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a24815640 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb63b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.076+0000 7f4a24815640 1 -- 172.21.15.78:0/383158047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce9e7dd3f0 con 0x55ce9e79c4d0 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a25016640 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb63b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a1dffb640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4a0c0a2d00 0x7f4a0c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a25016640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4a0c0a2d00 0x7f4a0c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a24815640 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb640b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4a180ed1c0 tx=0x7f4a180f3e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71656dbb2e7a259d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a1dffb640 1 -- 172.21.15.78:0/2986995438 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f4a180efb60 con 0x7f4a201013d0 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a16ffd640 1 -- 172.21.15.78:0/383158047 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18049070 con 0x55ce9e79c4d0 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a16ffd640 1 -- 172.21.15.78:0/383158047 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a1803b580 con 0x55ce9e79c4d0 2024-05-21T15:21:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.077+0000 7f4a16ffd640 1 -- 172.21.15.78:0/383158047 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a180b82c0 con 0x55ce9e79c4d0 2024-05-21T15:21:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/383158047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 msgr2=0x55ce9eb640b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb640b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4a180ed1c0 tx=0x7f4a180f3e30 comp rx=0 tx=0).stop 2024-05-21T15:21:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a25016640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4a0c0a2d00 0x7f4a0c0a5210 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ce9eb61730 tx=0x7f4a0800c6a0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/383158047 shutdown_connections 2024-05-21T15:21:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb640b0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/383158047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb63b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/383158047 >> 172.21.15.78:0/383158047 conn(0x55ce9e7b94e0 msgr2=0x55ce9eb604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/383158047 shutdown_connections 2024-05-21T15:21:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/383158047 wait complete. 2024-05-21T15:21:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 -- start start 2024-05-21T15:21:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.078+0000 7f4a2d9f5740 1 Processor -- start 2024-05-21T15:21:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb70430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb70970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb7c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce9eb73070 con 0x55ce9e79c4d0 2024-05-21T15:21:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ce9ea84c40 con 0x55ce9e7c5390 2024-05-21T15:21:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ce9eb731f0 con 0x55ce9e7a46b0 2024-05-21T15:21:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a2d9f5740 10 client.0 _ll_register_callbacks cb 0x55ce9e7dcbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb70430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb70430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35562/0 (socket says 172.21.15.78:35562) 2024-05-21T15:21:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 -- 172.21.15.78:0/4292822963 learned_addr learned my addr 172.21.15.78:0/4292822963 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb70970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a25016640 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb7c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 -- 172.21.15.78:0/4292822963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 msgr2=0x55ce9eb70970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb70970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 -- 172.21.15.78:0/4292822963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 msgr2=0x55ce9eb7c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb7c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 -- 172.21.15.78:0/4292822963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce9e8a77d0 con 0x55ce9e79c4d0 2024-05-21T15:21:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f4a24815640 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb70430 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4a180bbd20 tx=0x7f4a1803b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf65d99a7204ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.079+0000 7f49fffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f49fffff640 1 -- 172.21.15.78:0/4292822963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18049070 con 0x55ce9e79c4d0 2024-05-21T15:21:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f49fffff640 1 -- 172.21.15.78:0/4292822963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f4a180b8750 con 0x55ce9e79c4d0 2024-05-21T15:21:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/4292822963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ce9ead7370 con 0x55ce9e79c4d0 2024-05-21T15:21:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f49fffff640 1 -- 172.21.15.78:0/4292822963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4a1803bcb0 con 0x55ce9e79c4d0 2024-05-21T15:21:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 10 client.17235 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/4292822963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ce9eb73070 con 0x55ce9e79c4d0 2024-05-21T15:21:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f49fffff640 1 -- 172.21.15.78:0/4292822963 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ce9ead7370 con 0x55ce9e79c4d0 2024-05-21T15:21:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f49fffff640 1 -- 172.21.15.78:0/4292822963 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f4a180d8070 con 0x55ce9e79c4d0 2024-05-21T15:21:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 10 client.17235 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 10 client.17235 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:21:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 20 client.17235 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ce9e783f60 0x55ce9eb828c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 4 client.17235 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/4292822963 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce9ead16c0 con 0x55ce9e783f60 2024-05-21T15:21:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.080+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ce9e783f60 0x55ce9eb828c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.081+0000 7f49fffff640 10 client.17235 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:21:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.081+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ce9e783f60 0x55ce9eb828c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57a1f557b55e53ed server_cookie=20a0404fef3bea3e in_seq=0 out_seq=0 2024-05-21T15:21:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f49fffff640 1 -- 172.21.15.78:0/4292822963 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce9ead16c0 con 0x55ce9e783f60 2024-05-21T15:21:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f49fffff640 10 client.17235 handle_command_reply: tid=0 2024-05-21T15:21:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 client.17235 shutdown 2024-05-21T15:21:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/4292822963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 msgr2=0x55ce9eb70430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb70430 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4a180bbd20 tx=0x7f4a1803b4d0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/4292822963 shutdown_connections 2024-05-21T15:21:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ce9e783f60 0x55ce9eb828c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7c5390 0x55ce9eb7c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb70970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f49fffff640 0 client.17235 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:21:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/4292822963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb70430 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/4292822963 >> 172.21.15.78:0/4292822963 conn(0x55ce9e7b94e0 msgr2=0x55ce9eb644e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/4292822963 shutdown_connections 2024-05-21T15:21:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/4292822963 wait complete. 2024-05-21T15:21:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.082+0000 7f4a2d9f5740 20 client.17235 trim_cache size 0 max 16384 2024-05-21T15:21:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a2d9f5740 1 Processor -- start 2024-05-21T15:21:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a2d9f5740 1 -- start start 2024-05-21T15:21:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb6f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb7c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e7c5390 0x55ce9eb69d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201acb30 con 0x55ce9e7c5390 2024-05-21T15:21:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201ae670 con 0x55ce9e7a46b0 2024-05-21T15:21:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201ad200 con 0x55ce9e79c4d0 2024-05-21T15:21:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e7c5390 0x55ce9eb69d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e7c5390 0x55ce9eb69d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35574/0 (socket says 172.21.15.78:35574) 2024-05-21T15:21:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 -- 172.21.15.78:0/2183106834 learned_addr learned my addr 172.21.15.78:0/2183106834 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a24815640 1 --2- 172.21.15.78:0/2183106834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb6f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 -- 172.21.15.78:0/2183106834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e79c4d0 msgr2=0x55ce9eb6f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 --2- 172.21.15.78:0/2183106834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb6f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 -- 172.21.15.78:0/2183106834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 msgr2=0x55ce9eb7c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 --2- 172.21.15.78:0/2183106834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb7c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 -- 172.21.15.78:0/2183106834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce9eb65250 con 0x55ce9e7c5390 2024-05-21T15:21:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.096+0000 7f4a25016640 1 --2- 172.21.15.78:0/2183106834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e7c5390 0x55ce9eb69d60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800cfc0 tx=0x7f4a08013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c2595125e3449b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a167fc640 1 -- 172.21.15.78:0/2183106834 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08020630 con 0x55ce9e7c5390 2024-05-21T15:21:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a167fc640 1 -- 172.21.15.78:0/2183106834 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a08020cb0 con 0x55ce9e7c5390 2024-05-21T15:21:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a167fc640 1 -- 172.21.15.78:0/2183106834 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08028e30 con 0x55ce9e7c5390 2024-05-21T15:21:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2183106834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e7c5390 msgr2=0x55ce9eb69d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2183106834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e7c5390 0x55ce9eb69d60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800cfc0 tx=0x7f4a08013000 comp rx=0 tx=0).stop 2024-05-21T15:21:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2183106834 shutdown_connections 2024-05-21T15:21:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2183106834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb7c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2183106834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e7c5390 0x55ce9eb69d60 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2183106834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb6f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2183106834 >> 172.21.15.78:0/2183106834 conn(0x55ce9e7b94e0 msgr2=0x55ce9eb721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2183106834 shutdown_connections 2024-05-21T15:21:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.097+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2183106834 wait complete. 2024-05-21T15:21:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 1 -- start start 2024-05-21T15:21:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 1 Processor -- start 2024-05-21T15:21:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb5eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7c5390 0x55ce9eb5f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201adfa0 con 0x55ce9e79c4d0 2024-05-21T15:21:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201ad8d0 con 0x55ce9e7a46b0 2024-05-21T15:21:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a201aab10 con 0x55ce9e7c5390 2024-05-21T15:21:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a2d9f5740 10 client.0 _ll_register_callbacks cb 0x7f4a180f3bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a24815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb5eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a24815640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb5eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35586/0 (socket says 172.21.15.78:35586) 2024-05-21T15:21:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.098+0000 7f4a24815640 1 -- 172.21.15.78:0/2569327694 learned_addr learned my addr 172.21.15.78:0/2569327694 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a25016640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7c5390 0x55ce9eb5f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a24815640 1 -- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7c5390 msgr2=0x55ce9eb5f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a24815640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7c5390 0x55ce9eb5f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a24815640 1 -- 172.21.15.78:0/2569327694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 msgr2=0x55ce9eb5f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a24815640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb5f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a24815640 1 -- 172.21.15.78:0/2569327694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce9eb77440 con 0x55ce9e79c4d0 2024-05-21T15:21:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a25016640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7c5390 0x55ce9eb5f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb5f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a24815640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb5eb40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4a180b89f0 tx=0x7f4a180efe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3058ae315b01a08 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f49fffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f49fffff640 1 -- 172.21.15.78:0/2569327694 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18049070 con 0x55ce9e79c4d0 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f49fffff640 1 -- 172.21.15.78:0/2569327694 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f4a180c00d0 con 0x55ce9e79c4d0 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2569327694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a08020cb0 con 0x55ce9e79c4d0 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f49fffff640 1 -- 172.21.15.78:0/2569327694 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4a180bb980 con 0x55ce9e79c4d0 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a2d9f5740 10 client.17241 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2569327694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ce9eb77440 con 0x55ce9e79c4d0 2024-05-21T15:21:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f49fffff640 1 -- 172.21.15.78:0/2569327694 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a08020cb0 con 0x55ce9e79c4d0 2024-05-21T15:21:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.099+0000 7f49fffff640 1 -- 172.21.15.78:0/2569327694 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f4a180d8040 con 0x55ce9e79c4d0 2024-05-21T15:21:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f4a2d9f5740 10 client.17241 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f4a2d9f5740 10 client.17241 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:21:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f4a2d9f5740 20 client.17241 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ce9e7593a0 0x55ce9eb788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f4a2d9f5740 4 client.17241 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2569327694 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce9ea4e820 con 0x55ce9e7593a0 2024-05-21T15:21:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ce9e7593a0 0x55ce9eb788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f49fffff640 10 client.17241 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f4a1ffff640 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ce9e7593a0 0x55ce9eb788d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31a0558de4211b40 server_cookie=553f1ba0ef62bd94 in_seq=0 out_seq=0 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f49fffff640 1 -- 172.21.15.78:0/2569327694 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce9ea4e820 con 0x55ce9e7593a0 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.100+0000 7f49fffff640 10 client.17241 handle_command_reply: tid=0 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 client.17241 shutdown 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2569327694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 msgr2=0x55ce9eb5eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb5eb40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4a180b89f0 tx=0x7f4a180efe60 comp rx=0 tx=0).stop 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2569327694 shutdown_connections 2024-05-21T15:21:44.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ce9e7593a0 0x55ce9eb788d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ce9e7a46b0 0x55ce9eb5f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f49fffff640 0 client.17241 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:21:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce9e7c5390 0x55ce9eb5f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 --2- 172.21.15.78:0/2569327694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce9e79c4d0 0x55ce9eb5eb40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2569327694 >> 172.21.15.78:0/2569327694 conn(0x55ce9e7b94e0 msgr2=0x55ce9eb63ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2569327694 shutdown_connections 2024-05-21T15:21:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 1 -- 172.21.15.78:0/2569327694 wait complete. 2024-05-21T15:21:44.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a2d9f5740 20 client.17241 trim_cache size 0 max 16384 2024-05-21T15:21:44.115 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a26a79640 1 -- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4a0c0a2d00 msgr2=0x7f4a0c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4a0c0a2d00 0x7f4a0c0a5210 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ce9eb61730 tx=0x7f4a0800c6a0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a26a79640 1 -- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 msgr2=0x7f4a20198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 0x7f4a20198730 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006010 tx=0x7f4a180042f0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a26a79640 1 -- 172.21.15.78:0/2986995438 shutdown_connections 2024-05-21T15:21:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a20105960 0x7f4a20198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4a0c0a2d00 0x7f4a0c0a5210 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.101+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a20106420 0x7f4a2019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.102+0000 7f4a26a79640 1 --2- 172.21.15.78:0/2986995438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a201013d0 0x7f4a20198730 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.102+0000 7f4a26a79640 1 -- 172.21.15.78:0/2986995438 >> 172.21.15.78:0/2986995438 conn(0x7f4a200768d0 msgr2=0x7f4a201028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.102+0000 7f4a26a79640 1 -- 172.21.15.78:0/2986995438 shutdown_connections 2024-05-21T15:21:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.102+0000 7f4a26a79640 1 -- 172.21.15.78:0/2986995438 wait complete. 2024-05-21T15:21:44.118 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:21:44.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 -- 172.21.15.78:0/3941681289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868108ac0 msgr2=0x7f9868102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 --2- 172.21.15.78:0/3941681289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868108ac0 0x7f9868102660 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f985c009900 tx=0x7f985c02a410 comp rx=0 tx=0).stop 2024-05-21T15:21:44.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 -- 172.21.15.78:0/3941681289 shutdown_connections 2024-05-21T15:21:44.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 --2- 172.21.15.78:0/3941681289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9868108f60 0x7f9868102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 --2- 172.21.15.78:0/3941681289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9868103210 0x7f986810f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 --2- 172.21.15.78:0/3941681289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868108ac0 0x7f9868102660 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 -- 172.21.15.78:0/3941681289 >> 172.21.15.78:0/3941681289 conn(0x7f98680fe1c0 msgr2=0x7f9868100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 -- 172.21.15.78:0/3941681289 shutdown_connections 2024-05-21T15:21:44.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.228+0000 7f986eea6640 1 -- 172.21.15.78:0/3941681289 wait complete. 2024-05-21T15:21:44.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986eea6640 1 Processor -- start 2024-05-21T15:21:44.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986eea6640 1 -- start start 2024-05-21T15:21:44.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986eea6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868103210 0x7f98681997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986eea6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9868108ac0 0x7f9868199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986eea6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9868108f60 0x7f986819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986eea6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112e00 con 0x7f9868103210 2024-05-21T15:21:44.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986eea6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112c80 con 0x7f9868108ac0 2024-05-21T15:21:44.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986eea6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112f80 con 0x7f9868108f60 2024-05-21T15:21:44.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986cc42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868103210 0x7f98681997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986cc42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868103210 0x7f98681997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35604/0 (socket says 172.21.15.78:35604) 2024-05-21T15:21:44.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.229+0000 7f986cc42640 1 -- 172.21.15.78:0/1376594740 learned_addr learned my addr 172.21.15.78:0/1376594740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f9867fff640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9868108ac0 0x7f9868199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986d443640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9868108f60 0x7f986819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986cc42640 1 -- 172.21.15.78:0/1376594740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9868108f60 msgr2=0x7f986819dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986cc42640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9868108f60 0x7f986819dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986cc42640 1 -- 172.21.15.78:0/1376594740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9868108ac0 msgr2=0x7f9868199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986cc42640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9868108ac0 0x7f9868199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986cc42640 1 -- 172.21.15.78:0/1376594740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986819e600 con 0x7f9868103210 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986cc42640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868103210 0x7f98681997a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f985c005940 tx=0x7f985c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f9865ffb640 1 -- 172.21.15.78:0/1376594740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c03f030 con 0x7f9868103210 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f9865ffb640 1 -- 172.21.15.78:0/1376594740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985c003ce0 con 0x7f9868103210 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986eea6640 1 -- 172.21.15.78:0/1376594740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986819e890 con 0x7f9868103210 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f9865ffb640 1 -- 172.21.15.78:0/1376594740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c035070 con 0x7f9868103210 2024-05-21T15:21:44.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.230+0000 7f986eea6640 1 -- 172.21.15.78:0/1376594740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986819ed80 con 0x7f9868103210 2024-05-21T15:21:44.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9865ffb640 1 -- 172.21.15.78:0/1376594740 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f985c042030 con 0x7f9868103210 2024-05-21T15:21:44.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9875e1f740 1 Processor -- start 2024-05-21T15:21:44.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9875e1f740 1 -- start start 2024-05-21T15:21:44.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9865ffb640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f98580a2d00 0x7f98580a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 0x55b5433b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff14d0 0x55b5433b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff96b0 0x55b5433b97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9875e1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b54331ec40 con 0x55b542ff14d0 2024-05-21T15:21:44.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9875e1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b5432c6580 con 0x55b542ff96b0 2024-05-21T15:21:44.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9875e1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b54331ac00 con 0x55b54301a390 2024-05-21T15:21:44.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f986d443640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f98580a2d00 0x7f98580a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9865ffb640 1 -- 172.21.15.78:0/1376594740 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f985c0efae0 con 0x7f9868103210 2024-05-21T15:21:44.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.233+0000 7f9867fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff14d0 0x55b5433b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9867fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff14d0 0x55b5433b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35614/0 (socket says 172.21.15.78:35614) 2024-05-21T15:21:44.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9867fff640 1 -- 172.21.15.78:0/685216999 learned_addr learned my addr 172.21.15.78:0/685216999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f986cc42640 1 --2- 172.21.15.78:0/685216999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 0x55b5433b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9867fff640 1 -- 172.21.15.78:0/685216999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 msgr2=0x55b5433b8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9867fff640 1 --2- 172.21.15.78:0/685216999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 0x55b5433b8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9867fff640 1 -- 172.21.15.78:0/685216999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff96b0 msgr2=0x55b5433b97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9867fff640 1 --2- 172.21.15.78:0/685216999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff96b0 0x55b5433b97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9867fff640 1 -- 172.21.15.78:0/685216999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5430323f0 con 0x55b542ff14d0 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f986d443640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f98580a2d00 0x7f98580a5210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9860005fd0 tx=0x7f98600098c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9867fff640 1 --2- 172.21.15.78:0/685216999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff14d0 0x55b5433b9260 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f985400bbb0 tx=0x7f98540107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b687d5c645cfd31 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9852ffd640 1 -- 172.21.15.78:0/685216999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854004030 con 0x55b542ff14d0 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9852ffd640 1 -- 172.21.15.78:0/685216999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9854005500 con 0x55b542ff14d0 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.234+0000 7f9852ffd640 1 -- 172.21.15.78:0/685216999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854004c20 con 0x55b542ff14d0 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 -- 172.21.15.78:0/685216999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff14d0 msgr2=0x55b5433b9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 --2- 172.21.15.78:0/685216999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff14d0 0x55b5433b9260 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f985400bbb0 tx=0x7f98540107a0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 -- 172.21.15.78:0/685216999 shutdown_connections 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 --2- 172.21.15.78:0/685216999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff96b0 0x55b5433b97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 --2- 172.21.15.78:0/685216999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff14d0 0x55b5433b9260 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 --2- 172.21.15.78:0/685216999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 0x55b5433b8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 -- 172.21.15.78:0/685216999 >> 172.21.15.78:0/685216999 conn(0x55b54300e4e0 msgr2=0x55b5433b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 -- 172.21.15.78:0/685216999 shutdown_connections 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 -- 172.21.15.78:0/685216999 wait complete. 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 -- start start 2024-05-21T15:21:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.235+0000 7f9875e1f740 1 Processor -- start 2024-05-21T15:21:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff14d0 0x55b5433c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 0x55b5433c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9875e1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b5433c5fb0 con 0x55b542ff96b0 2024-05-21T15:21:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9875e1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b5432d9c40 con 0x55b542ff14d0 2024-05-21T15:21:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9875e1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b5433c6130 con 0x55b54301a390 2024-05-21T15:21:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9875e1f740 10 client.0 _ll_register_callbacks cb 0x55b543031be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9867fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9867fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433c52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35622/0 (socket says 172.21.15.78:35622) 2024-05-21T15:21:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9867fff640 1 -- 172.21.15.78:0/3847642408 learned_addr learned my addr 172.21.15.78:0/3847642408 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f986cc42640 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff14d0 0x55b5433c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f986d443640 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 0x55b5433c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9867fff640 1 -- 172.21.15.78:0/3847642408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 msgr2=0x55b5433c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9867fff640 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 0x55b5433c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9867fff640 1 -- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff14d0 msgr2=0x55b5433c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9867fff640 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff14d0 0x55b5433c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f9867fff640 1 -- 172.21.15.78:0/3847642408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5430fc7d0 con 0x55b542ff96b0 2024-05-21T15:21:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.236+0000 7f986cc42640 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff14d0 0x55b5433c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9867fff640 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433c52c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9854010ca0 tx=0x7f985400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ace0ec4515d907 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9847fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9847fff640 1 -- 172.21.15.78:0/3847642408 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985402b060 con 0x55b542ff96b0 2024-05-21T15:21:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9847fff640 1 -- 172.21.15.78:0/3847642408 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9854011650 con 0x55b542ff96b0 2024-05-21T15:21:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 1 -- 172.21.15.78:0/3847642408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9854005500 con 0x55b542ff96b0 2024-05-21T15:21:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9847fff640 1 -- 172.21.15.78:0/3847642408 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9854011850 con 0x55b542ff96b0 2024-05-21T15:21:44.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 10 client.17253 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:44.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 1 -- 172.21.15.78:0/3847642408 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b5433d27c0 con 0x55b542ff96b0 2024-05-21T15:21:44.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9847fff640 1 -- 172.21.15.78:0/3847642408 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9854005500 con 0x55b542ff96b0 2024-05-21T15:21:44.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9847fff640 1 -- 172.21.15.78:0/3847642408 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f985401bc20 con 0x55b542ff96b0 2024-05-21T15:21:44.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 10 client.17253 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:44.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:44.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 20 client.17253 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:44.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b542fd8f60 0x55b5433d5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 4 client.17253 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:44.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.237+0000 7f9875e1f740 1 -- 172.21.15.78:0/3847642408 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b54332c370 con 0x55b542fd8f60 2024-05-21T15:21:44.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.238+0000 7f986cc42640 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b542fd8f60 0x55b5433d5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.238+0000 7f986cc42640 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b542fd8f60 0x55b5433d5b20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90fa026e57859fb2 server_cookie=5717da7d61b4c0b6 in_seq=0 out_seq=0 2024-05-21T15:21:44.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.238+0000 7f9847fff640 10 client.17253 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:44.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9847fff640 1 -- 172.21.15.78:0/3847642408 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b54332c370 con 0x55b542fd8f60 2024-05-21T15:21:44.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9847fff640 10 client.17253 handle_command_reply: tid=0 2024-05-21T15:21:44.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 client.17253 shutdown 2024-05-21T15:21:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 -- 172.21.15.78:0/3847642408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 msgr2=0x55b5433c52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433c52c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9854010ca0 tx=0x7f985400bed0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 -- 172.21.15.78:0/3847642408 shutdown_connections 2024-05-21T15:21:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b542fd8f60 0x55b5433d5b20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b54301a390 0x55b5433c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9847fff640 0 client.17253 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433c52c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 --2- 172.21.15.78:0/3847642408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542ff14d0 0x55b5433c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 -- 172.21.15.78:0/3847642408 >> 172.21.15.78:0/3847642408 conn(0x55b54300e4e0 msgr2=0x55b5433baa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 -- 172.21.15.78:0/3847642408 shutdown_connections 2024-05-21T15:21:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 1 -- 172.21.15.78:0/3847642408 wait complete. 2024-05-21T15:21:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.239+0000 7f9875e1f740 20 client.17253 trim_cache size 0 max 16384 2024-05-21T15:21:44.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9875e1f740 1 Processor -- start 2024-05-21T15:21:44.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9875e1f740 1 -- start start 2024-05-21T15:21:44.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 0x55b5433bc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 0x55b5433bcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9875e1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98681aec90 con 0x55b542ff96b0 2024-05-21T15:21:44.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9875e1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f98681b07d0 con 0x55b542fd8f60 2024-05-21T15:21:44.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9875e1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98681af360 con 0x55b542ff14d0 2024-05-21T15:21:44.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35628/0 (socket says 172.21.15.78:35628) 2024-05-21T15:21:44.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 -- 172.21.15.78:0/2407968114 learned_addr learned my addr 172.21.15.78:0/2407968114 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986cc42640 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 0x55b5433bc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 -- 172.21.15.78:0/2407968114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 msgr2=0x55b5433bcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f9867fff640 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 0x55b5433bcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 0x55b5433bcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 -- 172.21.15.78:0/2407968114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 msgr2=0x55b5433bc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 0x55b5433bc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 -- 172.21.15.78:0/2407968114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5433b0ba0 con 0x55b542ff96b0 2024-05-21T15:21:44.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.253+0000 7f986d443640 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433b4e90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f98600137d0 tx=0x7f9860015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b903d669f1d7039c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f98527fc640 1 -- 172.21.15.78:0/2407968114 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986000fae0 con 0x55b542ff96b0 2024-05-21T15:21:44.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f98527fc640 1 -- 172.21.15.78:0/2407968114 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9860017070 con 0x55b542ff96b0 2024-05-21T15:21:44.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f98527fc640 1 -- 172.21.15.78:0/2407968114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860020390 con 0x55b542ff96b0 2024-05-21T15:21:44.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 -- 172.21.15.78:0/2407968114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 msgr2=0x55b5433b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433b4e90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f98600137d0 tx=0x7f9860015000 comp rx=0 tx=0).stop 2024-05-21T15:21:44.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 -- 172.21.15.78:0/2407968114 shutdown_connections 2024-05-21T15:21:44.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433b4e90 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 0x55b5433bcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 --2- 172.21.15.78:0/2407968114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 0x55b5433bc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 -- 172.21.15.78:0/2407968114 >> 172.21.15.78:0/2407968114 conn(0x55b54300e4e0 msgr2=0x55b5433c1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 -- 172.21.15.78:0/2407968114 shutdown_connections 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.254+0000 7f9875e1f740 1 -- 172.21.15.78:0/2407968114 wait complete. 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 1 -- start start 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 1 Processor -- start 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 0x55b5433d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 0x55b5433d61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98681b0100 con 0x55b542ff96b0 2024-05-21T15:21:44.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f98681afa30 con 0x55b542fd8f60 2024-05-21T15:21:44.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98681acc70 con 0x55b542ff14d0 2024-05-21T15:21:44.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f9875e1f740 10 client.0 _ll_register_callbacks cb 0x7f985400ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:44.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.255+0000 7f986d443640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986d443640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433d66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35634/0 (socket says 172.21.15.78:35634) 2024-05-21T15:21:44.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986d443640 1 -- 172.21.15.78:0/601846611 learned_addr learned my addr 172.21.15.78:0/601846611 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:44.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f9867fff640 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 0x55b5433d61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986cc42640 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 0x55b5433d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986d443640 1 -- 172.21.15.78:0/601846611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 msgr2=0x55b5433d61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986d443640 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 0x55b5433d61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986d443640 1 -- 172.21.15.78:0/601846611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 msgr2=0x55b5433d8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986d443640 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 0x55b5433d8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986d443640 1 -- 172.21.15.78:0/601846611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5433d6c20 con 0x55b542ff96b0 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f986d443640 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433d66e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9860009590 tx=0x7f986000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46aff13f775089f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f9847fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:44.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f9847fff640 1 -- 172.21.15.78:0/601846611 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860038060 con 0x55b542ff96b0 2024-05-21T15:21:44.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f9847fff640 1 -- 172.21.15.78:0/601846611 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9860021430 con 0x55b542ff96b0 2024-05-21T15:21:44.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f9875e1f740 1 -- 172.21.15.78:0/601846611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9860017070 con 0x55b542ff96b0 2024-05-21T15:21:44.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f9847fff640 1 -- 172.21.15.78:0/601846611 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9860021630 con 0x55b542ff96b0 2024-05-21T15:21:44.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f9875e1f740 10 client.17259 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:44.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.256+0000 7f9875e1f740 1 -- 172.21.15.78:0/601846611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b54343d890 con 0x55b542ff96b0 2024-05-21T15:21:44.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9847fff640 1 -- 172.21.15.78:0/601846611 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9860017070 con 0x55b542ff96b0 2024-05-21T15:21:44.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9847fff640 1 -- 172.21.15.78:0/601846611 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f986000f040 con 0x55b542ff96b0 2024-05-21T15:21:44.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9875e1f740 10 client.17259 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:44.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9875e1f740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:44.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9875e1f740 20 client.17259 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:44.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9875e1f740 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b54301a390 0x55b54343ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:44.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9875e1f740 4 client.17259 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:21:44.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9875e1f740 1 -- 172.21.15.78:0/601846611 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5433266c0 con 0x55b54301a390 2024-05-21T15:21:44.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f986cc42640 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b54301a390 0x55b54343ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:44.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9847fff640 10 client.17259 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f986cc42640 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b54301a390 0x55b54343ff70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd171193f6cdb4a server_cookie=5cf93da6ea99ad3d in_seq=0 out_seq=0 2024-05-21T15:21:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9847fff640 1 -- 172.21.15.78:0/601846611 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5433266c0 con 0x55b54301a390 2024-05-21T15:21:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.257+0000 7f9847fff640 10 client.17259 handle_command_reply: tid=0 2024-05-21T15:21:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 client.17259 shutdown 2024-05-21T15:21:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 -- 172.21.15.78:0/601846611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 msgr2=0x55b5433d66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433d66e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9860009590 tx=0x7f986000ff90 comp rx=0 tx=0).stop 2024-05-21T15:21:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 -- 172.21.15.78:0/601846611 shutdown_connections 2024-05-21T15:21:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b54301a390 0x55b54343ff70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b542ff96b0 0x55b5433d66e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b542ff14d0 0x55b5433d61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9847fff640 0 client.17259 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 --2- 172.21.15.78:0/601846611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b542fd8f60 0x55b5433d8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 -- 172.21.15.78:0/601846611 >> 172.21.15.78:0/601846611 conn(0x55b54300e4e0 msgr2=0x55b5433ba790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 -- 172.21.15.78:0/601846611 shutdown_connections 2024-05-21T15:21:44.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 1 -- 172.21.15.78:0/601846611 wait complete. 2024-05-21T15:21:44.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f9875e1f740 20 client.17259 trim_cache size 0 max 16384 2024-05-21T15:21:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:21:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 -- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f98580a2d00 msgr2=0x7f98580a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f98580a2d00 0x7f98580a5210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9860005fd0 tx=0x7f98600098c0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 -- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868103210 msgr2=0x7f98681997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868103210 0x7f98681997a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f985c005940 tx=0x7f985c0042f0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 -- 172.21.15.78:0/1376594740 shutdown_connections 2024-05-21T15:21:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9868108ac0 0x7f9868199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f98580a2d00 0x7f98580a5210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9868108f60 0x7f986819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.258+0000 7f986eea6640 1 --2- 172.21.15.78:0/1376594740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9868103210 0x7f98681997a0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.259+0000 7f986eea6640 1 -- 172.21.15.78:0/1376594740 >> 172.21.15.78:0/1376594740 conn(0x7f98680fe1c0 msgr2=0x7f98680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.259+0000 7f986eea6640 1 -- 172.21.15.78:0/1376594740 shutdown_connections 2024-05-21T15:21:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:44.259+0000 7f986eea6640 1 -- 172.21.15.78:0/1376594740 wait complete. 2024-05-21T15:21:44.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:44 smithi078 ceph-mon[31997]: pgmap v172: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-05-21T15:21:44.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:44 smithi084 ceph-mon[36334]: pgmap v172: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-05-21T15:21:44.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:44 smithi172 ceph-mon[36402]: pgmap v172: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-05-21T15:21:45.278 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:21:45.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:21:45.279 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:21:45.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 -- 172.21.15.78:0/6172568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10108b50 msgr2=0x7f6e10100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 --2- 172.21.15.78:0/6172568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10108b50 0x7f6e10100ac0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009900 tx=0x7f6e0002a410 comp rx=0 tx=0).stop 2024-05-21T15:21:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 -- 172.21.15.78:0/6172568 shutdown_connections 2024-05-21T15:21:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 --2- 172.21.15.78:0/6172568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e1010d500 0x7f6e1010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 --2- 172.21.15.78:0/6172568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e10101000 0x7f6e10101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 --2- 172.21.15.78:0/6172568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10108b50 0x7f6e10100ac0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 -- 172.21.15.78:0/6172568 >> 172.21.15.78:0/6172568 conn(0x7f6e10076830 msgr2=0x7f6e10076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 -- 172.21.15.78:0/6172568 shutdown_connections 2024-05-21T15:21:45.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.402+0000 7f6e16492640 1 -- 172.21.15.78:0/6172568 wait complete. 2024-05-21T15:21:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e16492640 1 Processor -- start 2024-05-21T15:21:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e16492640 1 -- start start 2024-05-21T15:21:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e16492640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10101000 0x7f6e1019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e16492640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e10108b50 0x7f6e1019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e16492640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e1010d500 0x7f6e101996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e16492640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10112ec0 con 0x7f6e10101000 2024-05-21T15:21:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e16492640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10112d40 con 0x7f6e10108b50 2024-05-21T15:21:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e16492640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10113040 con 0x7f6e1010d500 2024-05-21T15:21:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10101000 0x7f6e1019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10101000 0x7f6e1019f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35652/0 (socket says 172.21.15.78:35652) 2024-05-21T15:21:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.403+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3551801046 learned_addr learned my addr 172.21.15.78:0/3551801046 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e1010d500 0x7f6e101996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e07fff640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e10108b50 0x7f6e1019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3551801046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e1010d500 msgr2=0x7f6e101996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e1010d500 0x7f6e101996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3551801046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e10108b50 msgr2=0x7f6e1019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e10108b50 0x7f6e1019faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3551801046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e10199e10 con 0x7f6e10101000 2024-05-21T15:21:45.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e07fff640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e10108b50 0x7f6e1019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:21:45.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10101000 0x7f6e1019f560 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6e0002a930 tx=0x7f6e00003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0dffb640 1 -- 172.21.15.78:0/3551801046 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0003f070 con 0x7f6e10101000 2024-05-21T15:21:45.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0dffb640 1 -- 172.21.15.78:0/3551801046 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0003b460 con 0x7f6e10101000 2024-05-21T15:21:45.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e16492640 1 -- 172.21.15.78:0/3551801046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1019a040 con 0x7f6e10101000 2024-05-21T15:21:45.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e0dffb640 1 -- 172.21.15.78:0/3551801046 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00035030 con 0x7f6e10101000 2024-05-21T15:21:45.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.404+0000 7f6e16492640 1 -- 172.21.15.78:0/3551801046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e101a6300 con 0x7f6e10101000 2024-05-21T15:21:45.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e0dffb640 1 -- 172.21.15.78:0/3551801046 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6e00042070 con 0x7f6e10101000 2024-05-21T15:21:45.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e1d418740 1 Processor -- start 2024-05-21T15:21:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e1d418740 1 -- start start 2024-05-21T15:21:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf4e410 0x555edc2ed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf25550 0x555edc2edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf2d730 0x555edc2ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e1d418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555edc252990 con 0x555edbf2d730 2024-05-21T15:21:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e1d418740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555edc1fa5b0 con 0x555edbf4e410 2024-05-21T15:21:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e1d418740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555edc24e950 con 0x555edbf25550 2024-05-21T15:21:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf2d730 0x555edc2ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf2d730 0x555edc2ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35664/0 (socket says 172.21.15.78:35664) 2024-05-21T15:21:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.407+0000 7f6e07fff640 1 -- 172.21.15.78:0/186667747 learned_addr learned my addr 172.21.15.78:0/186667747 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf25550 0x555edc2edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf4e410 0x555edc2ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e0dffb640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6dfc0a2d80 0x7f6dfc0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e07fff640 1 -- 172.21.15.78:0/186667747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf25550 msgr2=0x555edc2edb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e07fff640 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf25550 0x555edc2edb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e07fff640 1 -- 172.21.15.78:0/186667747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf4e410 msgr2=0x555edc2ed620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e07fff640 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf4e410 0x555edc2ed620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e07fff640 1 -- 172.21.15.78:0/186667747 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555edc02e7e0 con 0x555edbf2d730 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf4e410 0x555edc2ed620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6dfc0a2d80 0x7f6dfc0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e0dffb640 1 -- 172.21.15.78:0/3551801046 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f6e000efe10 con 0x7f6e10101000 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e07fff640 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf2d730 0x555edc2ee0a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6df800a970 tx=0x7f6df800ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dd23d043438c25 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e067fc640 1 -- 172.21.15.78:0/186667747 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8015070 con 0x555edbf2d730 2024-05-21T15:21:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e067fc640 1 -- 172.21.15.78:0/186667747 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df800cc00 con 0x555edbf2d730 2024-05-21T15:21:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.408+0000 7f6e067fc640 1 -- 172.21.15.78:0/186667747 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df80105b0 con 0x555edbf2d730 2024-05-21T15:21:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 -- 172.21.15.78:0/186667747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf2d730 msgr2=0x555edc2ee0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6dfc0a2d80 0x7f6dfc0a5290 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x555edc2ea790 tx=0x7f6e08004a40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf2d730 0x555edc2ee0a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6df800a970 tx=0x7f6df800ae30 comp rx=0 tx=0).stop 2024-05-21T15:21:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 -- 172.21.15.78:0/186667747 shutdown_connections 2024-05-21T15:21:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf2d730 0x555edc2ee0a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf25550 0x555edc2edb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 --2- 172.21.15.78:0/186667747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf4e410 0x555edc2ed620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 -- 172.21.15.78:0/186667747 >> 172.21.15.78:0/186667747 conn(0x555edbf42560 msgr2=0x555edc2e9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 -- 172.21.15.78:0/186667747 shutdown_connections 2024-05-21T15:21:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 -- 172.21.15.78:0/186667747 wait complete. 2024-05-21T15:21:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.409+0000 7f6e1d418740 1 -- start start 2024-05-21T15:21:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e1d418740 1 Processor -- start 2024-05-21T15:21:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc2f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 0x555edc2fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e1d418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555edc2fa970 con 0x555edbf25550 2024-05-21T15:21:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e1d418740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555edc20db40 con 0x555edbf2d730 2024-05-21T15:21:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e1d418740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555edc2faaf0 con 0x555edbf4e410 2024-05-21T15:21:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e1d418740 10 client.0 _ll_register_callbacks cb 0x555edc2020f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2f7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35678/0 (socket says 172.21.15.78:35678) 2024-05-21T15:21:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3903286183 learned_addr learned my addr 172.21.15.78:0/3903286183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e07fff640 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc2f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3903286183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 msgr2=0x555edc2fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 0x555edc2fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3903286183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 msgr2=0x555edc2f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc2f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.410+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3903286183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555edc08ef70 con 0x555edbf25550 2024-05-21T15:21:45.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2f7ce0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6e00000c00 tx=0x7f6e00009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314fde6b1fe7c1cf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6def7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:45.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6def7fe640 1 -- 172.21.15.78:0/3903286183 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00049070 con 0x555edbf25550 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6def7fe640 1 -- 172.21.15.78:0/3903286183 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f6e0003b610 con 0x555edbf25550 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 1 -- 172.21.15.78:0/3903286183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df800cc00 con 0x555edbf25550 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6def7fe640 1 -- 172.21.15.78:0/3903286183 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e000bbe10 con 0x555edbf25550 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 10 client.17271 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 1 -- 172.21.15.78:0/3903286183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555edbf26170 con 0x555edbf25550 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6def7fe640 1 -- 172.21.15.78:0/3903286183 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df800cc00 con 0x555edbf25550 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6def7fe640 1 -- 172.21.15.78:0/3903286183 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6e0003b9a0 con 0x555edbf25550 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 10 client.17271 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:45.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:45.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 20 client.17271 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:45.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555edbf0d1f0 0x555edc30a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 4 client.17271 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:45.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.411+0000 7f6e1d418740 1 -- 172.21.15.78:0/3903286183 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555edc2600c0 con 0x555edbf0d1f0 2024-05-21T15:21:45.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.412+0000 7f6e07fff640 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555edbf0d1f0 0x555edc30a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.412+0000 7f6def7fe640 10 client.17271 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:45.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.412+0000 7f6e07fff640 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555edbf0d1f0 0x555edc30a180 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eedac2ca7cedb52d server_cookie=bec2285d7eca5463 in_seq=0 out_seq=0 2024-05-21T15:21:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6def7fe640 1 -- 172.21.15.78:0/3903286183 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555edc2600c0 con 0x555edbf0d1f0 2024-05-21T15:21:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6def7fe640 10 client.17271 handle_command_reply: tid=0 2024-05-21T15:21:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 client.17271 shutdown 2024-05-21T15:21:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 -- 172.21.15.78:0/3903286183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 msgr2=0x555edc2f7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2f7ce0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6e00000c00 tx=0x7f6e00009260 comp rx=0 tx=0).stop 2024-05-21T15:21:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 -- 172.21.15.78:0/3903286183 shutdown_connections 2024-05-21T15:21:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555edbf0d1f0 0x555edc30a180 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc2f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 0x555edc2fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6def7fe640 0 client.17271 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3903286183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2f7ce0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 -- 172.21.15.78:0/3903286183 >> 172.21.15.78:0/3903286183 conn(0x555edbf42560 msgr2=0x555edc2ef340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 -- 172.21.15.78:0/3903286183 shutdown_connections 2024-05-21T15:21:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 1 -- 172.21.15.78:0/3903286183 wait complete. 2024-05-21T15:21:45.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.413+0000 7f6e1d418740 20 client.17271 trim_cache size 0 max 16384 2024-05-21T15:21:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e1d418740 1 Processor -- start 2024-05-21T15:21:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e1d418740 1 -- start start 2024-05-21T15:21:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2fdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc30f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 0x555edc30fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e1d418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101aec50 con 0x555edbf25550 2024-05-21T15:21:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e1d418740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101b0790 con 0x555edbf2d730 2024-05-21T15:21:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e1d418740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101af320 con 0x555edbf4e410 2024-05-21T15:21:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2fdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2fdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35682/0 (socket says 172.21.15.78:35682) 2024-05-21T15:21:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.431+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3989750946 learned_addr learned my addr 172.21.15.78:0/3989750946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e07fff640 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc30f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3989750946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 msgr2=0x555edc30fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 0x555edc30fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 0x555edc30fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3989750946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 msgr2=0x555edc30f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc30f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e0ffff640 1 -- 172.21.15.78:0/3989750946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555edc2ee990 con 0x555edbf25550 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e07fff640 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc30f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2fdd60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6e000bc260 tx=0x7f6e00031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c26615546e741be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e05ffb640 1 -- 172.21.15.78:0/3989750946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00049070 con 0x555edbf25550 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e05ffb640 1 -- 172.21.15.78:0/3989750946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e000bc2d0 con 0x555edbf25550 2024-05-21T15:21:45.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.432+0000 7f6e05ffb640 1 -- 172.21.15.78:0/3989750946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000bbc30 con 0x555edbf25550 2024-05-21T15:21:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 -- 172.21.15.78:0/3989750946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 msgr2=0x555edc2fdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2fdd60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6e000bc260 tx=0x7f6e00031510 comp rx=0 tx=0).stop 2024-05-21T15:21:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 -- 172.21.15.78:0/3989750946 shutdown_connections 2024-05-21T15:21:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf2d730 0x555edc30f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf4e410 0x555edc30fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 --2- 172.21.15.78:0/3989750946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf25550 0x555edc2fdd60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 -- 172.21.15.78:0/3989750946 >> 172.21.15.78:0/3989750946 conn(0x555edbf42560 msgr2=0x555edc2ef940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 -- 172.21.15.78:0/3989750946 shutdown_connections 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 -- 172.21.15.78:0/3989750946 wait complete. 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 -- start start 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.433+0000 7f6e1d418740 1 Processor -- start 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf25550 0x555edc36a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf2d730 0x555edc36ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e1d418740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf4e410 0x555edc2fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e1d418740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101b00c0 con 0x555edbf4e410 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e1d418740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101af9f0 con 0x555edbf25550 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e1d418740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e101acc30 con 0x555edbf2d730 2024-05-21T15:21:45.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e1d418740 10 client.0 _ll_register_callbacks cb 0x7f6e00005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf4e410 0x555edc2fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf4e410 0x555edc2fefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35696/0 (socket says 172.21.15.78:35696) 2024-05-21T15:21:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 -- 172.21.15.78:0/2164335666 learned_addr learned my addr 172.21.15.78:0/2164335666 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e07fff640 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf2d730 0x555edc36ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf25550 0x555edc36a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 -- 172.21.15.78:0/2164335666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf2d730 msgr2=0x555edc36ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf2d730 0x555edc36ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 -- 172.21.15.78:0/2164335666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf25550 msgr2=0x555edc36a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf25550 0x555edc36a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 -- 172.21.15.78:0/2164335666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555edc2ff5f0 con 0x555edbf4e410 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6e14a2f640 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf4e410 0x555edc2fefb0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6e08005c00 tx=0x7f6e0801edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb56c8580fb17dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.434+0000 7f6def7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6def7fe640 1 -- 172.21.15.78:0/2164335666 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08012d60 con 0x555edbf4e410 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6def7fe640 1 -- 172.21.15.78:0/2164335666 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f6e0801b460 con 0x555edbf4e410 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 1 -- 172.21.15.78:0/2164335666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e000bc2d0 con 0x555edbf4e410 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6def7fe640 1 -- 172.21.15.78:0/2164335666 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e0801b660 con 0x555edbf4e410 2024-05-21T15:21:45.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 10 client.17277 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 1 -- 172.21.15.78:0/2164335666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555edc371fb0 con 0x555edbf4e410 2024-05-21T15:21:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6def7fe640 1 -- 172.21.15.78:0/2164335666 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e000bc2d0 con 0x555edbf4e410 2024-05-21T15:21:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6def7fe640 1 -- 172.21.15.78:0/2164335666 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6e08025020 con 0x555edbf4e410 2024-05-21T15:21:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 10 client.17277 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 20 client.17277 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555edbee2630 0x555edc374720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 4 client.17277 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:21:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e1d418740 1 -- 172.21.15.78:0/2164335666 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555edc25a410 con 0x555edbee2630 2024-05-21T15:21:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.435+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555edbee2630 0x555edc374720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.436+0000 7f6def7fe640 10 client.17277 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:45.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.436+0000 7f6e0ffff640 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555edbee2630 0x555edc374720 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785f7abba78b0b server_cookie=238b961cb42aa17f in_seq=0 out_seq=0 2024-05-21T15:21:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6def7fe640 1 -- 172.21.15.78:0/2164335666 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555edc25a410 con 0x555edbee2630 2024-05-21T15:21:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6def7fe640 10 client.17277 handle_command_reply: tid=0 2024-05-21T15:21:45.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 client.17277 shutdown 2024-05-21T15:21:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 -- 172.21.15.78:0/2164335666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf4e410 msgr2=0x555edc2fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf4e410 0x555edc2fefb0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6e08005c00 tx=0x7f6e0801edb0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 -- 172.21.15.78:0/2164335666 shutdown_connections 2024-05-21T15:21:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555edbee2630 0x555edc374720 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555edbf4e410 0x555edc2fefb0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555edbf2d730 0x555edc36ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 --2- 172.21.15.78:0/2164335666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555edbf25550 0x555edc36a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6def7fe640 0 client.17277 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 -- 172.21.15.78:0/2164335666 >> 172.21.15.78:0/2164335666 conn(0x555edbf42560 msgr2=0x555edc2ef940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 -- 172.21.15.78:0/2164335666 shutdown_connections 2024-05-21T15:21:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 1 -- 172.21.15.78:0/2164335666 wait complete. 2024-05-21T15:21:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.437+0000 7f6e1d418740 20 client.17277 trim_cache size 0 max 16384 2024-05-21T15:21:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:21:45.451 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:21:45.451 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "9240f5f8-3e44-4f3e-b9fa-019096e83413", 2024-05-21T15:21:45.451 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:21:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:21:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 -- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6dfc0a2d80 msgr2=0x7f6dfc0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6dfc0a2d80 0x7f6dfc0a5290 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x555edc2ea790 tx=0x7f6e08004a40 comp rx=0 tx=0).stop 2024-05-21T15:21:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 -- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10101000 msgr2=0x7f6e1019f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10101000 0x7f6e1019f560 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6e0002a930 tx=0x7f6e00003c00 comp rx=0 tx=0).stop 2024-05-21T15:21:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 -- 172.21.15.78:0/3551801046 shutdown_connections 2024-05-21T15:21:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e10108b50 0x7f6e1019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6dfc0a2d80 0x7f6dfc0a5290 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e1010d500 0x7f6e101996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 --2- 172.21.15.78:0/3551801046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e10101000 0x7f6e1019f560 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 -- 172.21.15.78:0/3551801046 >> 172.21.15.78:0/3551801046 conn(0x7f6e10076830 msgr2=0x7f6e101094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 -- 172.21.15.78:0/3551801046 shutdown_connections 2024-05-21T15:21:45.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.438+0000 7f6e16492640 1 -- 172.21.15.78:0/3551801046 wait complete. 2024-05-21T15:21:45.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:21:45.454 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:21:45.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 -- 172.21.15.78:0/3258833275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3810d070 msgr2=0x7f8d3810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 --2- 172.21.15.78:0/3258833275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3810d070 0x7f8d3810f550 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009ca0 tx=0x7f8d3002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 -- 172.21.15.78:0/3258833275 shutdown_connections 2024-05-21T15:21:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 --2- 172.21.15.78:0/3258833275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d3810a680 0x7f8d3810caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 --2- 172.21.15.78:0/3258833275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3810d070 0x7f8d3810f550 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 --2- 172.21.15.78:0/3258833275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d38103c20 0x7f8d38104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 -- 172.21.15.78:0/3258833275 >> 172.21.15.78:0/3258833275 conn(0x7f8d380770c0 msgr2=0x7f8d38077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 -- 172.21.15.78:0/3258833275 shutdown_connections 2024-05-21T15:21:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.567+0000 7f8d4014c640 1 -- 172.21.15.78:0/3258833275 wait complete. 2024-05-21T15:21:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d4014c640 1 Processor -- start 2024-05-21T15:21:45.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d4014c640 1 -- start start 2024-05-21T15:21:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d4014c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d38103c20 0x7f8d381997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d4014c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d3810a680 0x7f8d38199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d4014c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d3810d070 0x7f8d3819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d4014c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38112de0 con 0x7f8d38103c20 2024-05-21T15:21:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d4014c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38112c60 con 0x7f8d3810a680 2024-05-21T15:21:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d4014c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38112f60 con 0x7f8d3810d070 2024-05-21T15:21:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d38103c20 0x7f8d381997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d38103c20 0x7f8d381997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35712/0 (socket says 172.21.15.78:35712) 2024-05-21T15:21:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 -- 172.21.15.78:0/1008145563 learned_addr learned my addr 172.21.15.78:0/1008145563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d3810a680 0x7f8d38199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 -- 172.21.15.78:0/1008145563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d3810d070 msgr2=0x7f8d3819dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d3810d070 0x7f8d3819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 -- 172.21.15.78:0/1008145563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d3810a680 msgr2=0x7f8d38199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d3810a680 0x7f8d38199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 -- 172.21.15.78:0/1008145563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3819e600 con 0x7f8d38103c20 2024-05-21T15:21:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.568+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d38103c20 0x7f8d381997a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e9f0 tx=0x7f8d3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.569+0000 7f8d2effd640 1 -- 172.21.15.78:0/1008145563 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400cd80 con 0x7f8d38103c20 2024-05-21T15:21:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.569+0000 7f8d2effd640 1 -- 172.21.15.78:0/1008145563 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d34005500 con 0x7f8d38103c20 2024-05-21T15:21:45.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.569+0000 7f8d4014c640 1 -- 172.21.15.78:0/1008145563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3819e8f0 con 0x7f8d38103c20 2024-05-21T15:21:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.569+0000 7f8d2effd640 1 -- 172.21.15.78:0/1008145563 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34010690 con 0x7f8d38103c20 2024-05-21T15:21:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.569+0000 7f8d4014c640 1 -- 172.21.15.78:0/1008145563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3819edf0 con 0x7f8d38103c20 2024-05-21T15:21:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.571+0000 7f8d2effd640 1 -- 172.21.15.78:0/1008145563 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8d340046c0 con 0x7f8d38103c20 2024-05-21T15:21:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.571+0000 7f8d46ff6740 1 Processor -- start 2024-05-21T15:21:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d46ff6740 1 -- start start 2024-05-21T15:21:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 0x56206a051bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a052100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c926b0 0x56206a052640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562069fb7d00 con 0x562069c926b0 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562069f5f4c0 con 0x562069cb3390 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562069fb3cc0 con 0x562069c8a4d0 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3d6e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c926b0 0x56206a052640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d2effd640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8d240a2d00 0x7f8d240a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3d6e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c926b0 0x56206a052640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35722/0 (socket says 172.21.15.78:35722) 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3d6e7640 1 -- 172.21.15.78:0/1997927016 learned_addr learned my addr 172.21.15.78:0/1997927016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a052100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 0x56206a051bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8d240a2d00 0x7f8d240a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d2effd640 1 -- 172.21.15.78:0/1008145563 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8d340cfa10 con 0x7f8d38103c20 2024-05-21T15:21:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3d6e7640 1 -- 172.21.15.78:0/1997927016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 msgr2=0x56206a052100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a052100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3d6e7640 1 -- 172.21.15.78:0/1997927016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 msgr2=0x56206a051bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 0x56206a051bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.572+0000 7f8d3d6e7640 1 -- 172.21.15.78:0/1997927016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562069d957d0 con 0x562069c926b0 2024-05-21T15:21:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.573+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c926b0 0x56206a052640 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009b70 tx=0x7f8d28010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57641304f8aa6eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.573+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8d240a2d00 0x7f8d240a5210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d3002ade0 tx=0x7f8d300095c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.573+0000 7f8d23fff640 1 -- 172.21.15.78:0/1997927016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28009e60 con 0x562069c926b0 2024-05-21T15:21:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.573+0000 7f8d23fff640 1 -- 172.21.15.78:0/1997927016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d280035f0 con 0x562069c926b0 2024-05-21T15:21:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.573+0000 7f8d23fff640 1 -- 172.21.15.78:0/1997927016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d280126b0 con 0x562069c926b0 2024-05-21T15:21:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1997927016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c926b0 msgr2=0x56206a052640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c926b0 0x56206a052640 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009b70 tx=0x7f8d28010740 comp rx=0 tx=0).stop 2024-05-21T15:21:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1997927016 shutdown_connections 2024-05-21T15:21:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c926b0 0x56206a052640 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a052100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/1997927016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 0x56206a051bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1997927016 >> 172.21.15.78:0/1997927016 conn(0x562069ca74e0 msgr2=0x56206a04e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1997927016 shutdown_connections 2024-05-21T15:21:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1997927016 wait complete. 2024-05-21T15:21:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 -- start start 2024-05-21T15:21:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.574+0000 7f8d46ff6740 1 Processor -- start 2024-05-21T15:21:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c8a4d0 0x56206a062230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 0x56206a062770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a0606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56206a063490 con 0x562069cb3390 2024-05-21T15:21:45.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562069f72b90 con 0x562069c8a4d0 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56206a063610 con 0x562069c926b0 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d46ff6740 10 client.0 _ll_register_callbacks cb 0x562069faeda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a0606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a0606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35736/0 (socket says 172.21.15.78:35736) 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 -- 172.21.15.78:0/671650162 learned_addr learned my addr 172.21.15.78:0/671650162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c8a4d0 0x56206a062230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 0x56206a062770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 -- 172.21.15.78:0/671650162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 msgr2=0x56206a062770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 0x56206a062770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 -- 172.21.15.78:0/671650162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c8a4d0 msgr2=0x56206a062230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c8a4d0 0x56206a062230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 -- 172.21.15.78:0/671650162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562069ccb3f0 con 0x562069cb3390 2024-05-21T15:21:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a0606b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f8d30035000 tx=0x7f8d30005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a03dcb73f2b1dd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.575+0000 7f8d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d217fa640 1 -- 172.21.15.78:0/671650162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30050070 con 0x562069cb3390 2024-05-21T15:21:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d217fa640 1 -- 172.21.15.78:0/671650162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8d3002fa20 con 0x562069cb3390 2024-05-21T15:21:45.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 1 -- 172.21.15.78:0/671650162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d280035f0 con 0x562069cb3390 2024-05-21T15:21:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d217fa640 1 -- 172.21.15.78:0/671650162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8d3005c050 con 0x562069cb3390 2024-05-21T15:21:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 10 client.17289 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 1 -- 172.21.15.78:0/671650162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562069c8b0f0 con 0x562069cb3390 2024-05-21T15:21:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d217fa640 1 -- 172.21.15.78:0/671650162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d280035f0 con 0x562069cb3390 2024-05-21T15:21:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d217fa640 1 -- 172.21.15.78:0/671650162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8d3004b440 con 0x562069cb3390 2024-05-21T15:21:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 10 client.17289 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 20 client.17289 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562069c71f60 0x56206a073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 4 client.17289 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:21:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.576+0000 7f8d46ff6740 1 -- 172.21.15.78:0/671650162 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562069fc5430 con 0x562069c71f60 2024-05-21T15:21:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.577+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562069c71f60 0x56206a073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.577+0000 7f8d217fa640 10 client.17289 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.577+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562069c71f60 0x56206a073040 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19855c359ebca571 server_cookie=e4204320f3b106ad in_seq=0 out_seq=0 2024-05-21T15:21:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d217fa640 1 -- 172.21.15.78:0/671650162 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562069fc5430 con 0x562069c71f60 2024-05-21T15:21:45.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d217fa640 10 client.17289 handle_command_reply: tid=0 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 client.17289 shutdown 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 -- 172.21.15.78:0/671650162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 msgr2=0x56206a0606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a0606b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f8d30035000 tx=0x7f8d30005fd0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 -- 172.21.15.78:0/671650162 shutdown_connections 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562069c71f60 0x56206a073040 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a0606b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d217fa640 0 client.17289 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 0x56206a062770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/671650162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c8a4d0 0x56206a062230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 -- 172.21.15.78:0/671650162 >> 172.21.15.78:0/671650162 conn(0x562069ca74e0 msgr2=0x56206a053a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 -- 172.21.15.78:0/671650162 shutdown_connections 2024-05-21T15:21:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 1 -- 172.21.15.78:0/671650162 wait complete. 2024-05-21T15:21:45.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.578+0000 7f8d46ff6740 20 client.17289 trim_cache size 0 max 16384 2024-05-21T15:21:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d46ff6740 1 Processor -- start 2024-05-21T15:21:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d46ff6740 1 -- start start 2024-05-21T15:21:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a05f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c926b0 0x56206a0521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a052710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381aca00 con 0x562069cb3390 2024-05-21T15:21:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381ae540 con 0x562069c926b0 2024-05-21T15:21:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381ad0d0 con 0x562069c8a4d0 2024-05-21T15:21:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3e6e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a052710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3e6e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a052710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35744/0 (socket says 172.21.15.78:35744) 2024-05-21T15:21:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3e6e9640 1 -- 172.21.15.78:0/1034833193 learned_addr learned my addr 172.21.15.78:0/1034833193 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a05f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c926b0 0x56206a0521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3e6e9640 1 -- 172.21.15.78:0/1034833193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 msgr2=0x56206a05f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a05f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3e6e9640 1 -- 172.21.15.78:0/1034833193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c926b0 msgr2=0x56206a0521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c926b0 0x56206a0521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3e6e9640 1 -- 172.21.15.78:0/1034833193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56206a04f010 con 0x562069cb3390 2024-05-21T15:21:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.596+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a05f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:21:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.597+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a052710 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8d3004b810 tx=0x7f8d30031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c8a6ab034c771c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.597+0000 7f8d237fe640 1 -- 172.21.15.78:0/1034833193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30050040 con 0x562069cb3390 2024-05-21T15:21:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.597+0000 7f8d237fe640 1 -- 172.21.15.78:0/1034833193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3004be50 con 0x562069cb3390 2024-05-21T15:21:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.597+0000 7f8d237fe640 1 -- 172.21.15.78:0/1034833193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30045040 con 0x562069cb3390 2024-05-21T15:21:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1034833193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 msgr2=0x56206a052710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a052710 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8d3004b810 tx=0x7f8d30031b10 comp rx=0 tx=0).stop 2024-05-21T15:21:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1034833193 shutdown_connections 2024-05-21T15:21:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069c926b0 0x56206a0521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069cb3390 0x56206a052710 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/1034833193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c8a4d0 0x56206a05f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1034833193 >> 172.21.15.78:0/1034833193 conn(0x562069ca74e0 msgr2=0x56206a051ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1034833193 shutdown_connections 2024-05-21T15:21:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 -- 172.21.15.78:0/1034833193 wait complete. 2024-05-21T15:21:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 -- start start 2024-05-21T15:21:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 Processor -- start 2024-05-21T15:21:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c8a4d0 0x56206a064730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.598+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 0x56206a064c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d46ff6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 0x56206a0651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381ade70 con 0x562069c8a4d0 2024-05-21T15:21:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381ad7a0 con 0x562069cb3390 2024-05-21T15:21:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d46ff6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381aa9e0 con 0x562069c926b0 2024-05-21T15:21:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d46ff6740 10 client.0 _ll_register_callbacks cb 0x7f8d28005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:21:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c8a4d0 0x56206a064730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c8a4d0 0x56206a064730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35754/0 (socket says 172.21.15.78:35754) 2024-05-21T15:21:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 -- 172.21.15.78:0/4087136972 learned_addr learned my addr 172.21.15.78:0/4087136972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:21:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3e6e9640 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 0x56206a0651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 0x56206a064c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 -- 172.21.15.78:0/4087136972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 msgr2=0x56206a064c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 0x56206a064c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 -- 172.21.15.78:0/4087136972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 msgr2=0x56206a0651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:21:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 0x56206a0651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 -- 172.21.15.78:0/4087136972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206a0da5e0 con 0x562069c8a4d0 2024-05-21T15:21:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d3dee8640 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c8a4d0 0x56206a064730 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8d340988a0 tx=0x7f8d3406df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa32d19a540c037 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:21:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:21:45.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d217fa640 1 -- 172.21.15.78:0/4087136972 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3406be60 con 0x562069c8a4d0 2024-05-21T15:21:45.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d217fa640 1 -- 172.21.15.78:0/4087136972 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f8d340cd380 con 0x562069c8a4d0 2024-05-21T15:21:45.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d46ff6740 1 -- 172.21.15.78:0/4087136972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3004be50 con 0x562069c8a4d0 2024-05-21T15:21:45.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d217fa640 1 -- 172.21.15.78:0/4087136972 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8d340a4b40 con 0x562069c8a4d0 2024-05-21T15:21:45.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.599+0000 7f8d46ff6740 10 client.17295 fetch_fsmap learned FSMap version 37 2024-05-21T15:21:45.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d46ff6740 1 -- 172.21.15.78:0/4087136972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56206a0db8d0 con 0x562069c8a4d0 2024-05-21T15:21:45.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d217fa640 1 -- 172.21.15.78:0/4087136972 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3004be50 con 0x562069c8a4d0 2024-05-21T15:21:45.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d217fa640 1 -- 172.21.15.78:0/4087136972 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8d3409b320 con 0x562069c8a4d0 2024-05-21T15:21:45.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d46ff6740 10 client.17295 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:21:45.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d46ff6740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:21:45.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d46ff6740 20 client.17295 populate_metadata read hostname 'smithi078' 2024-05-21T15:21:45.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562069c473a0 0x56206a0ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:21:45.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d46ff6740 4 client.17295 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:21:45.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d46ff6740 1 -- 172.21.15.78:0/4087136972 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562069fbf780 con 0x562069c473a0 2024-05-21T15:21:45.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562069c473a0 0x56206a0ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:21:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d3d6e7640 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562069c473a0 0x56206a0ddfb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca64a484015a608 server_cookie=364ade20814f91e2 in_seq=0 out_seq=0 2024-05-21T15:21:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.600+0000 7f8d217fa640 10 client.17295 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d217fa640 1 -- 172.21.15.78:0/4087136972 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562069fbf780 con 0x562069c473a0 2024-05-21T15:21:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d217fa640 10 client.17295 handle_command_reply: tid=0 2024-05-21T15:21:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 client.17295 shutdown 2024-05-21T15:21:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 -- 172.21.15.78:0/4087136972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c8a4d0 msgr2=0x56206a064730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c8a4d0 0x56206a064730 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8d340988a0 tx=0x7f8d3406df50 comp rx=0 tx=0).stop 2024-05-21T15:21:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 -- 172.21.15.78:0/4087136972 shutdown_connections 2024-05-21T15:21:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562069c473a0 0x56206a0ddfb0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562069cb3390 0x56206a0651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562069c926b0 0x56206a064c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 --2- 172.21.15.78:0/4087136972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562069c8a4d0 0x56206a064730 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d217fa640 0 client.17295 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:21:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 -- 172.21.15.78:0/4087136972 >> 172.21.15.78:0/4087136972 conn(0x562069ca74e0 msgr2=0x56206a0524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 -- 172.21.15.78:0/4087136972 shutdown_connections 2024-05-21T15:21:45.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 1 -- 172.21.15.78:0/4087136972 wait complete. 2024-05-21T15:21:45.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d46ff6740 20 client.17295 trim_cache size 0 max 16384 2024-05-21T15:21:45.620 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:21:45.620 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:21:45.620 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:21:45.621 INFO:teuthology.orchestra.run.smithi078.stdout: "9240f5f8-3e44-4f3e-b9fa-019096e83413": { 2024-05-21T15:21:45.621 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:21:45.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "9240f5f8-3e44-4f3e-b9fa-019096e83413", 2024-05-21T15:21:45.621 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:21:45.621 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:21:45.621 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:21:45.621 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:21:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.601+0000 7f8d4014c640 1 -- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8d240a2d00 msgr2=0x7f8d240a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8d240a2d00 0x7f8d240a5210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d3002ade0 tx=0x7f8d300095c0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 -- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d38103c20 msgr2=0x7f8d381997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:21:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d38103c20 0x7f8d381997a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e9f0 tx=0x7f8d3400edf0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 -- 172.21.15.78:0/1008145563 shutdown_connections 2024-05-21T15:21:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d3810a680 0x7f8d38199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f8d240a2d00 0x7f8d240a5210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d3810d070 0x7f8d3819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 --2- 172.21.15.78:0/1008145563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d38103c20 0x7f8d381997a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:21:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 -- 172.21.15.78:0/1008145563 >> 172.21.15.78:0/1008145563 conn(0x7f8d380770c0 msgr2=0x7f8d38100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:21:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 -- 172.21.15.78:0/1008145563 shutdown_connections 2024-05-21T15:21:45.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:21:45.602+0000 7f8d4014c640 1 -- 172.21.15.78:0/1008145563 wait complete. 2024-05-21T15:21:45.625 INFO:tasks.cephfs.filesystem:scrub status for tag:9240f5f8-3e44-4f3e-b9fa-019096e83413 - {'path': '/', 'tag': '9240f5f8-3e44-4f3e-b9fa-019096e83413', 'options': 'recursive,force'} 2024-05-21T15:21:46.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: pgmap v173: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-21T15:21:46.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:21:46.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:46.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:46.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:46.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:46.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:46.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:21:46.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:46 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: pgmap v173: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-21T15:21:46.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:21:46.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:21:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:46 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: pgmap v173: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-05-21T15:21:46.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:21:46.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:46.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:46.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:46.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:46.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:46.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:21:46.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:46.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:46 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:21:48.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:48 smithi078 ceph-mon[31997]: pgmap v174: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 KiB/s wr, 36 op/s 2024-05-21T15:21:48.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:48 smithi084 ceph-mon[36334]: pgmap v174: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 KiB/s wr, 36 op/s 2024-05-21T15:21:48.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:48 smithi172 ceph-mon[36402]: pgmap v174: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 KiB/s wr, 36 op/s 2024-05-21T15:21:50.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:50 smithi078 ceph-mon[31997]: pgmap v175: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 2.9 KiB/s wr, 40 op/s 2024-05-21T15:21:50.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:50 smithi084 ceph-mon[36334]: pgmap v175: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 2.9 KiB/s wr, 40 op/s 2024-05-21T15:21:50.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:50 smithi172 ceph-mon[36402]: pgmap v175: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 2.9 KiB/s wr, 40 op/s 2024-05-21T15:21:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:52 smithi078 ceph-mon[31997]: pgmap v176: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 2.9 KiB/s wr, 42 op/s 2024-05-21T15:21:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:52 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:52.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:52 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:52.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:52 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:52 smithi084 ceph-mon[36334]: pgmap v176: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 2.9 KiB/s wr, 42 op/s 2024-05-21T15:21:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:52 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:52 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:52 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:52.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:52 smithi172 ceph-mon[36402]: pgmap v176: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 2.9 KiB/s wr, 42 op/s 2024-05-21T15:21:52.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:52 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:52.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:52 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:52.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:52 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:53.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:53 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:53.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:53 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:21:53.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:53 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:53.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:53 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:21:53.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:53 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:53.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:53 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:21:54.835 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:54 smithi172 ceph-mon[36402]: pgmap v177: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 43 KiB/s rd, 3.3 KiB/s wr, 45 op/s 2024-05-21T15:21:54.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:54 smithi078 ceph-mon[31997]: pgmap v177: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 43 KiB/s rd, 3.3 KiB/s wr, 45 op/s 2024-05-21T15:21:54.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:54 smithi084 ceph-mon[36334]: pgmap v177: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 43 KiB/s rd, 3.3 KiB/s wr, 45 op/s 2024-05-21T15:21:56.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:56 smithi078 ceph-mon[31997]: pgmap v178: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 2.9 KiB/s wr, 41 op/s 2024-05-21T15:21:56.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:56 smithi084 ceph-mon[36334]: pgmap v178: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 2.9 KiB/s wr, 41 op/s 2024-05-21T15:21:56.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:56 smithi172 ceph-mon[36402]: pgmap v178: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 40 KiB/s rd, 2.9 KiB/s wr, 41 op/s 2024-05-21T15:21:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:21:58 smithi078 ceph-mon[31997]: pgmap v179: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 42 KiB/s rd, 2.9 KiB/s wr, 45 op/s 2024-05-21T15:21:58.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:21:58 smithi084 ceph-mon[36334]: pgmap v179: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 42 KiB/s rd, 2.9 KiB/s wr, 45 op/s 2024-05-21T15:21:58.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:21:58 smithi172 ceph-mon[36402]: pgmap v179: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 42 KiB/s rd, 2.9 KiB/s wr, 45 op/s 2024-05-21T15:22:00.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:00 smithi078 ceph-mon[31997]: pgmap v180: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 426 B/s wr, 18 op/s 2024-05-21T15:22:00.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:00 smithi084 ceph-mon[36334]: pgmap v180: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 426 B/s wr, 18 op/s 2024-05-21T15:22:00.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:00 smithi172 ceph-mon[36402]: pgmap v180: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 426 B/s wr, 18 op/s 2024-05-21T15:22:02.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:02 smithi078 ceph-mon[31997]: pgmap v181: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:22:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:02 smithi084 ceph-mon[36334]: pgmap v181: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:22:02.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:02 smithi172 ceph-mon[36402]: pgmap v181: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-21T15:22:04.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:04 smithi078 ceph-mon[31997]: pgmap v182: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T15:22:04.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:04 smithi084 ceph-mon[36334]: pgmap v182: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T15:22:04.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:04 smithi172 ceph-mon[36402]: pgmap v182: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T15:22:06.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:06 smithi078 ceph-mon[31997]: pgmap v183: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:06.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:06 smithi084 ceph-mon[36334]: pgmap v183: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:06.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:06 smithi172 ceph-mon[36402]: pgmap v183: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:08 smithi078 ceph-mon[31997]: pgmap v184: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T15:22:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:08 smithi084 ceph-mon[36334]: pgmap v184: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T15:22:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:08 smithi172 ceph-mon[36402]: pgmap v184: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T15:22:10.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:10 smithi078 ceph-mon[31997]: pgmap v185: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:10.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:10 smithi084 ceph-mon[36334]: pgmap v185: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:10.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:10 smithi172 ceph-mon[36402]: pgmap v185: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:12.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:12 smithi078 ceph-mon[31997]: pgmap v186: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:12.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:12 smithi084 ceph-mon[36334]: pgmap v186: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:12.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:12 smithi172 ceph-mon[36402]: pgmap v186: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T15:22:13.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:13 smithi078 ceph-mon[31997]: pgmap v187: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T15:22:13.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:13 smithi084 ceph-mon[36334]: pgmap v187: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T15:22:13.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:13 smithi172 ceph-mon[36402]: pgmap v187: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T15:22:15.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:22:15.626 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:22:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.754+0000 7f2dfcf10640 1 -- 172.21.15.78:0/342281388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8108f60 msgr2=0x7f2df8102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.755+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/342281388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8108f60 0x7f2df8102ba0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009900 tx=0x7f2de002a410 comp rx=0 tx=0).stop 2024-05-21T15:22:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.755+0000 7f2dfcf10640 1 -- 172.21.15.78:0/342281388 shutdown_connections 2024-05-21T15:22:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.755+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/342281388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df8103210 0x7f2df810f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.755+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/342281388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8108f60 0x7f2df8102ba0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.755+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/342281388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8108ac0 0x7f2df8102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.755+0000 7f2dfcf10640 1 -- 172.21.15.78:0/342281388 >> 172.21.15.78:0/342281388 conn(0x7f2df80fe1e0 msgr2=0x7f2df8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.755+0000 7f2dfcf10640 1 -- 172.21.15.78:0/342281388 shutdown_connections 2024-05-21T15:22:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.755+0000 7f2dfcf10640 1 -- 172.21.15.78:0/342281388 wait complete. 2024-05-21T15:22:15.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2dfcf10640 1 Processor -- start 2024-05-21T15:22:15.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2dfcf10640 1 -- start start 2024-05-21T15:22:15.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2dfcf10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8103210 0x7f2df819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2dfcf10640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8108ac0 0x7f2df819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2dfcf10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df8108f60 0x7f2df8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2dfcf10640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8112e00 con 0x7f2df8103210 2024-05-21T15:22:15.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2dfcf10640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8112c80 con 0x7f2df8108f60 2024-05-21T15:22:15.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2dfcf10640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8112f80 con 0x7f2df8108ac0 2024-05-21T15:22:15.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8103210 0x7f2df819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8103210 0x7f2df819f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52584/0 (socket says 172.21.15.78:52584) 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df659c640 1 -- 172.21.15.78:0/1943579991 learned_addr learned my addr 172.21.15.78:0/1943579991 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df6d9d640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df8108f60 0x7f2df8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8108ac0 0x7f2df819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df659c640 1 -- 172.21.15.78:0/1943579991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8108ac0 msgr2=0x7f2df819fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df659c640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8108ac0 0x7f2df819fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df659c640 1 -- 172.21.15.78:0/1943579991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df8108f60 msgr2=0x7f2df8199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df659c640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df8108f60 0x7f2df8199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.756+0000 7f2df659c640 1 -- 172.21.15.78:0/1943579991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df8199ca0 con 0x7f2df8103210 2024-05-21T15:22:15.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.757+0000 7f2df659c640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8103210 0x7f2df819f4e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2df000d740 tx=0x7f2df000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:15.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.757+0000 7f2def7fe640 1 -- 172.21.15.78:0/1943579991 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0017070 con 0x7f2df8103210 2024-05-21T15:22:15.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.757+0000 7f2dfcf10640 1 -- 172.21.15.78:0/1943579991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df8199f90 con 0x7f2df8103210 2024-05-21T15:22:15.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.757+0000 7f2def7fe640 1 -- 172.21.15.78:0/1943579991 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df000de50 con 0x7f2df8103210 2024-05-21T15:22:15.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.757+0000 7f2def7fe640 1 -- 172.21.15.78:0/1943579991 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0004e00 con 0x7f2df8103210 2024-05-21T15:22:15.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.757+0000 7f2dfcf10640 1 -- 172.21.15.78:0/1943579991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df819a4b0 con 0x7f2df8103210 2024-05-21T15:22:15.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.759+0000 7f2e03dbc740 1 Processor -- start 2024-05-21T15:22:15.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2e03dbc740 1 -- start start 2024-05-21T15:22:15.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2def7fe640 1 -- 172.21.15.78:0/1943579991 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2df0010460 con 0x7f2df8103210 2024-05-21T15:22:15.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d337390 0x56244d6d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d3166b0 0x56244d6d67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56244d63bd00 con 0x56244d337390 2024-05-21T15:22:15.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56244d5e34c0 con 0x56244d3166b0 2024-05-21T15:22:15.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56244d637cc0 con 0x56244d30e4d0 2024-05-21T15:22:15.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2df6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d337390 0x56244d6d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2df6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d337390 0x56244d6d5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52590/0 (socket says 172.21.15.78:52590) 2024-05-21T15:22:15.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2df6d9d640 1 -- 172.21.15.78:0/2694975615 learned_addr learned my addr 172.21.15.78:0/2694975615 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:15.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d3166b0 0x56244d6d67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.760+0000 7f2df659c640 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2df5d9b640 1 -- 172.21.15.78:0/2694975615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 msgr2=0x56244d6d6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:15.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6d6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2df5d9b640 1 -- 172.21.15.78:0/2694975615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d337390 msgr2=0x56244d6d5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:15.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d337390 0x56244d6d5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2df5d9b640 1 -- 172.21.15.78:0/2694975615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56244d4197d0 con 0x56244d3166b0 2024-05-21T15:22:15.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2def7fe640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2de40a2d00 0x7f2de40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2df6d9d640 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d337390 0x56244d6d5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:22:15.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2df6d9d640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2de40a2d00 0x7f2de40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2def7fe640 1 -- 172.21.15.78:0/1943579991 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2df00cee00 con 0x7f2df8103210 2024-05-21T15:22:15.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d3166b0 0x56244d6d67b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2de002a930 tx=0x7f2de0032d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3010855459017c87 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:15.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2decff9640 1 -- 172.21.15.78:0/2694975615 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de003f070 con 0x56244d3166b0 2024-05-21T15:22:15.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2decff9640 1 -- 172.21.15.78:0/2694975615 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de00317b0 con 0x56244d3166b0 2024-05-21T15:22:15.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.761+0000 7f2decff9640 1 -- 172.21.15.78:0/2694975615 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0035070 con 0x56244d3166b0 2024-05-21T15:22:15.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2694975615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d3166b0 msgr2=0x56244d6d67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2df6d9d640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2de40a2d00 0x7f2de40a5210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x56244d6d2ea0 tx=0x7f2de8007520 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:15.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d3166b0 0x56244d6d67b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2de002a930 tx=0x7f2de0032d00 comp rx=0 tx=0).stop 2024-05-21T15:22:15.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2694975615 shutdown_connections 2024-05-21T15:22:15.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d3166b0 0x56244d6d67b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6d6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2694975615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d337390 0x56244d6d5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2694975615 >> 172.21.15.78:0/2694975615 conn(0x56244d32b4e0 msgr2=0x56244d6d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:15.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2694975615 shutdown_connections 2024-05-21T15:22:15.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.762+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2694975615 wait complete. 2024-05-21T15:22:15.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 1 -- start start 2024-05-21T15:22:15.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 1 Processor -- start 2024-05-21T15:22:15.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d30e4d0 0x56244d6e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d3166b0 0x56244d6e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56244d6e2fc0 con 0x56244d30e4d0 2024-05-21T15:22:15.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56244d5f6b90 con 0x56244d337390 2024-05-21T15:22:15.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56244d6e3140 con 0x56244d3166b0 2024-05-21T15:22:15.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2e03dbc740 10 client.0 _ll_register_callbacks cb 0x56244d632da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:15.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d30e4d0 0x56244d6e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d30e4d0 0x56244d6e1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52602/0 (socket says 172.21.15.78:52602) 2024-05-21T15:22:15.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df659c640 1 -- 172.21.15.78:0/695311409 learned_addr learned my addr 172.21.15.78:0/695311409 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:15.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d3166b0 0x56244d6e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df6d9d640 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df659c640 1 -- 172.21.15.78:0/695311409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d3166b0 msgr2=0x56244d6e23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:15.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df659c640 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d3166b0 0x56244d6e23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df659c640 1 -- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 msgr2=0x56244d6e0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:15.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df659c640 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df659c640 1 -- 172.21.15.78:0/695311409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56244d34f3f0 con 0x56244d30e4d0 2024-05-21T15:22:15.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.763+0000 7f2df6d9d640 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:15.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2df659c640 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d30e4d0 0x56244d6e1e60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2df009afc0 tx=0x7f2df001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc793d4d0571cd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:15.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:15.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2dddffb640 1 -- 172.21.15.78:0/695311409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0020070 con 0x56244d30e4d0 2024-05-21T15:22:15.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2dddffb640 1 -- 172.21.15.78:0/695311409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2df0097930 con 0x56244d30e4d0 2024-05-21T15:22:15.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 1 -- 172.21.15.78:0/695311409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de00317b0 con 0x56244d30e4d0 2024-05-21T15:22:15.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2dddffb640 1 -- 172.21.15.78:0/695311409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2df009ac20 con 0x56244d30e4d0 2024-05-21T15:22:15.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 10 client.17307 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:15.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 1 -- 172.21.15.78:0/695311409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56244d30f0f0 con 0x56244d30e4d0 2024-05-21T15:22:15.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2dddffb640 1 -- 172.21.15.78:0/695311409 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de00317b0 con 0x56244d30e4d0 2024-05-21T15:22:15.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2dddffb640 1 -- 172.21.15.78:0/695311409 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2df00b40f0 con 0x56244d30e4d0 2024-05-21T15:22:15.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 10 client.17307 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:15.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:15.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 20 client.17307 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:15.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56244d2f5f60 0x56244d6f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 4 client.17307 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:15.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.764+0000 7f2e03dbc740 1 -- 172.21.15.78:0/695311409 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56244d649430 con 0x56244d2f5f60 2024-05-21T15:22:15.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.765+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56244d2f5f60 0x56244d6f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.765+0000 7f2dddffb640 10 client.17307 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:15.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.765+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56244d2f5f60 0x56244d6f2c00 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0653fd56d064c5 server_cookie=95ad5b5479db0900 in_seq=0 out_seq=0 2024-05-21T15:22:15.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2dddffb640 1 -- 172.21.15.78:0/695311409 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56244d649430 con 0x56244d2f5f60 2024-05-21T15:22:15.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2dddffb640 10 client.17307 handle_command_reply: tid=0 2024-05-21T15:22:15.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 client.17307 shutdown 2024-05-21T15:22:15.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 -- 172.21.15.78:0/695311409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d30e4d0 msgr2=0x56244d6e1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d30e4d0 0x56244d6e1e60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2df009afc0 tx=0x7f2df001e000 comp rx=0 tx=0).stop 2024-05-21T15:22:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 -- 172.21.15.78:0/695311409 shutdown_connections 2024-05-21T15:22:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56244d2f5f60 0x56244d6f2c00 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d3166b0 0x56244d6e23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/695311409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d30e4d0 0x56244d6e1e60 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2dddffb640 0 client.17307 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:15.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 -- 172.21.15.78:0/695311409 >> 172.21.15.78:0/695311409 conn(0x56244d32b4e0 msgr2=0x56244d6d7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:15.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 -- 172.21.15.78:0/695311409 shutdown_connections 2024-05-21T15:22:15.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 1 -- 172.21.15.78:0/695311409 wait complete. 2024-05-21T15:22:15.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.766+0000 7f2e03dbc740 20 client.17307 trim_cache size 0 max 16384 2024-05-21T15:22:15.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2e03dbc740 1 Processor -- start 2024-05-21T15:22:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2e03dbc740 1 -- start start 2024-05-21T15:22:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81aea20 con 0x56244d3166b0 2024-05-21T15:22:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81b0560 con 0x56244d337390 2024-05-21T15:22:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81af0f0 con 0x56244d30e4d0 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2df5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2df5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6d24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52606/0 (socket says 172.21.15.78:52606) 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.784+0000 7f2df5d9b640 1 -- 172.21.15.78:0/3532294967 learned_addr learned my addr 172.21.15.78:0/3532294967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2df6d9d640 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2df659c640 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2df5d9b640 1 -- 172.21.15.78:0/3532294967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 msgr2=0x56244d6df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2df5d9b640 1 -- 172.21.15.78:0/3532294967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 msgr2=0x56244d6f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2df5d9b640 1 -- 172.21.15.78:0/3532294967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56244d6e0630 con 0x56244d3166b0 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6d24f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009900 tx=0x7f2de00097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869c828e000ebcab server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2ddffff640 1 -- 172.21.15.78:0/3532294967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0055070 con 0x56244d3166b0 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2ddffff640 1 -- 172.21.15.78:0/3532294967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0005040 con 0x56244d3166b0 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.785+0000 7f2ddffff640 1 -- 172.21.15.78:0/3532294967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de003f040 con 0x56244d3166b0 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 -- 172.21.15.78:0/3532294967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 msgr2=0x56244d6d24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6d24f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009900 tx=0x7f2de00097a0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 -- 172.21.15.78:0/3532294967 shutdown_connections 2024-05-21T15:22:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6d24f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/3532294967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 -- 172.21.15.78:0/3532294967 >> 172.21.15.78:0/3532294967 conn(0x56244d32b4e0 msgr2=0x56244d6d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 -- 172.21.15.78:0/3532294967 shutdown_connections 2024-05-21T15:22:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 -- 172.21.15.78:0/3532294967 wait complete. 2024-05-21T15:22:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 -- start start 2024-05-21T15:22:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.786+0000 7f2e03dbc740 1 Processor -- start 2024-05-21T15:22:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6e7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2e03dbc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81afe90 con 0x56244d3166b0 2024-05-21T15:22:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81af7c0 con 0x56244d337390 2024-05-21T15:22:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2e03dbc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81aca00 con 0x56244d30e4d0 2024-05-21T15:22:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2e03dbc740 10 client.0 _ll_register_callbacks cb 0x7f2df00d2640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6e7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52614/0 (socket says 172.21.15.78:52614) 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 -- 172.21.15.78:0/2778520495 learned_addr learned my addr 172.21.15.78:0/2778520495 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df659c640 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df6d9d640 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 -- 172.21.15.78:0/2778520495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 msgr2=0x56244d6e7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6e7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 -- 172.21.15.78:0/2778520495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 msgr2=0x56244d6e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 -- 172.21.15.78:0/2778520495 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56244d759090 con 0x56244d3166b0 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2df5d9b640 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6e7c80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2de002a450 tx=0x7f2de0005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986a3016654fd1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2dddffb640 1 -- 172.21.15.78:0/2778520495 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0055070 con 0x56244d3166b0 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2dddffb640 1 -- 172.21.15.78:0/2778520495 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2de00053f0 con 0x56244d3166b0 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.787+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2778520495 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de0005040 con 0x56244d3166b0 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2dddffb640 1 -- 172.21.15.78:0/2778520495 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2de003f040 con 0x56244d3166b0 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2e03dbc740 10 client.17313 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2778520495 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56244d75a2e0 con 0x56244d3166b0 2024-05-21T15:22:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2dddffb640 1 -- 172.21.15.78:0/2778520495 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0005040 con 0x56244d3166b0 2024-05-21T15:22:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2dddffb640 1 -- 172.21.15.78:0/2778520495 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2de004e600 con 0x56244d3166b0 2024-05-21T15:22:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2e03dbc740 10 client.17313 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2e03dbc740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2e03dbc740 20 client.17313 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56244d2cb3a0 0x56244d75c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2e03dbc740 4 client.17313 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:22:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2778520495 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56244d643780 con 0x56244d2cb3a0 2024-05-21T15:22:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.788+0000 7f2df659c640 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56244d2cb3a0 0x56244d75c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2dddffb640 10 client.17313 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2df659c640 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56244d2cb3a0 0x56244d75c560 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc5a9ca55082377 server_cookie=96525a0d97e55623 in_seq=0 out_seq=0 2024-05-21T15:22:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2dddffb640 1 -- 172.21.15.78:0/2778520495 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56244d643780 con 0x56244d2cb3a0 2024-05-21T15:22:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2dddffb640 10 client.17313 handle_command_reply: tid=0 2024-05-21T15:22:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2e03dbc740 1 client.17313 shutdown 2024-05-21T15:22:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2778520495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 msgr2=0x56244d6e7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6e7c80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2de002a450 tx=0x7f2de0005940 comp rx=0 tx=0).stop 2024-05-21T15:22:15.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2778520495 shutdown_connections 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56244d2cb3a0 0x56244d75c560 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2dddffb640 0 client.17313 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56244d337390 0x56244d6e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.789+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56244d3166b0 0x56244d6e7c80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2e03dbc740 1 --2- 172.21.15.78:0/2778520495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56244d30e4d0 0x56244d6e7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2778520495 >> 172.21.15.78:0/2778520495 conn(0x56244d32b4e0 msgr2=0x56244d6d99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2778520495 shutdown_connections 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2e03dbc740 1 -- 172.21.15.78:0/2778520495 wait complete. 2024-05-21T15:22:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2e03dbc740 20 client.17313 trim_cache size 0 max 16384 2024-05-21T15:22:15.804 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:22:15.804 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:22:15.804 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:22:15.804 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:22:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 -- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2de40a2d00 msgr2=0x7f2de40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2de40a2d00 0x7f2de40a5210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x56244d6d2ea0 tx=0x7f2de8007520 comp rx=0 tx=0).stop 2024-05-21T15:22:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 -- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8103210 msgr2=0x7f2df819f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8103210 0x7f2df819f4e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2df000d740 tx=0x7f2df000dc00 comp rx=0 tx=0).stop 2024-05-21T15:22:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 -- 172.21.15.78:0/1943579991 shutdown_connections 2024-05-21T15:22:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2df8108f60 0x7f2df8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2de40a2d00 0x7f2de40a5210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2df8108ac0 0x7f2df819fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 --2- 172.21.15.78:0/1943579991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2df8103210 0x7f2df819f4e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 -- 172.21.15.78:0/1943579991 >> 172.21.15.78:0/1943579991 conn(0x7f2df80fe1e0 msgr2=0x7f2df810e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 -- 172.21.15.78:0/1943579991 shutdown_connections 2024-05-21T15:22:15.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.790+0000 7f2dfcf10640 1 -- 172.21.15.78:0/1943579991 wait complete. 2024-05-21T15:22:15.807 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:22:15.808 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:22:15.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.915+0000 7f95e9656640 1 -- 172.21.15.78:0/1672770474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 msgr2=0x7f95e410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.915+0000 7f95e9656640 1 --2- 172.21.15.78:0/1672770474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 0x7f95e410ca20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009900 tx=0x7f95cc02a410 comp rx=0 tx=0).stop 2024-05-21T15:22:15.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 -- 172.21.15.78:0/1672770474 shutdown_connections 2024-05-21T15:22:15.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 --2- 172.21.15.78:0/1672770474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e410cff0 0x7f95e410f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 --2- 172.21.15.78:0/1672770474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 0x7f95e410ca20 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 --2- 172.21.15.78:0/1672770474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95e4103ba0 0x7f95e4103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 -- 172.21.15.78:0/1672770474 >> 172.21.15.78:0/1672770474 conn(0x7f95e40770c0 msgr2=0x7f95e4077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:15.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 -- 172.21.15.78:0/1672770474 shutdown_connections 2024-05-21T15:22:15.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 -- 172.21.15.78:0/1672770474 wait complete. 2024-05-21T15:22:15.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 Processor -- start 2024-05-21T15:22:15.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.916+0000 7f95e9656640 1 -- start start 2024-05-21T15:22:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e9656640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e4103ba0 0x7f95e4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e9656640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 0x7f95e4199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e9656640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95e410cff0 0x7f95e419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e9656640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4112da0 con 0x7f95e410a600 2024-05-21T15:22:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e9656640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4112c20 con 0x7f95e4103ba0 2024-05-21T15:22:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e9656640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4112f20 con 0x7f95e410cff0 2024-05-21T15:22:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 0x7f95e4199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 0x7f95e4199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52630/0 (socket says 172.21.15.78:52630) 2024-05-21T15:22:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 -- 172.21.15.78:0/1355123647 learned_addr learned my addr 172.21.15.78:0/1355123647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e37fe640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95e410cff0 0x7f95e419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e2ffd640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e4103ba0 0x7f95e4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 -- 172.21.15.78:0/1355123647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95e410cff0 msgr2=0x7f95e419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95e410cff0 0x7f95e419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 -- 172.21.15.78:0/1355123647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e4103ba0 msgr2=0x7f95e4199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e4103ba0 0x7f95e4199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 -- 172.21.15.78:0/1355123647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e419e5c0 con 0x7f95e410a600 2024-05-21T15:22:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e27fc640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 0x7f95e4199cc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f95cc006010 tx=0x7f95cc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95dbfff640 1 -- 172.21.15.78:0/1355123647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc03f030 con 0x7f95e410a600 2024-05-21T15:22:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95dbfff640 1 -- 172.21.15.78:0/1355123647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc004440 con 0x7f95e410a600 2024-05-21T15:22:15.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e419e850 con 0x7f95e410a600 2024-05-21T15:22:15.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95dbfff640 1 -- 172.21.15.78:0/1355123647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc035070 con 0x7f95e410a600 2024-05-21T15:22:15.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.917+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e419ed40 con 0x7f95e410a600 2024-05-21T15:22:15.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.918+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e4069720 con 0x7f95e410a600 2024-05-21T15:22:15.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.920+0000 7f95dbfff640 1 -- 172.21.15.78:0/1355123647 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f95cc042030 con 0x7f95e410a600 2024-05-21T15:22:15.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.920+0000 7f95dbfff640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95d00a2d00 0x7f95d00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:15.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.921+0000 7f95dbfff640 1 -- 172.21.15.78:0/1355123647 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f95cc0efaa0 con 0x7f95e410a600 2024-05-21T15:22:15.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.921+0000 7f95e2ffd640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95d00a2d00 0x7f95d00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:15.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.921+0000 7f95e2ffd640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95d00a2d00 0x7f95d00a5210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95d4005fd0 tx=0x7f95d4009500 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:15.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:15.924+0000 7f95dbfff640 1 -- 172.21.15.78:0/1355123647 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f95cc0bc010 con 0x7f95e410a600 2024-05-21T15:22:16.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:15 smithi078 ceph-mon[31997]: pgmap v188: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T15:22:16.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.137+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95e4069f50 con 0x7f95e410a600 2024-05-21T15:22:16.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.138+0000 7f95dbfff640 1 -- 172.21.15.78:0/1355123647 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7f95cc0c0310 con 0x7f95e410a600 2024-05-21T15:22:16.142 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:22:16.142 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:22:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:22:16.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95d00a2d00 msgr2=0x7f95d00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95d00a2d00 0x7f95d00a5210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f95d4005fd0 tx=0x7f95d4009500 comp rx=0 tx=0).stop 2024-05-21T15:22:16.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 msgr2=0x7f95e4199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 0x7f95e4199cc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f95cc006010 tx=0x7f95cc004040 comp rx=0 tx=0).stop 2024-05-21T15:22:16.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 shutdown_connections 2024-05-21T15:22:16.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f95d00a2d00 0x7f95d00a5210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95e410cff0 0x7f95e419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95e410a600 0x7f95e4199cc0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 --2- 172.21.15.78:0/1355123647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e4103ba0 0x7f95e4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 >> 172.21.15.78:0/1355123647 conn(0x7f95e40770c0 msgr2=0x7f95e4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 shutdown_connections 2024-05-21T15:22:16.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.140+0000 7f95e9656640 1 -- 172.21.15.78:0/1355123647 wait complete. 2024-05-21T15:22:16.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:15 smithi084 ceph-mon[36334]: pgmap v188: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T15:22:16.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:22:16.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:15 smithi172 ceph-mon[36402]: pgmap v188: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T15:22:16.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 -- 172.21.15.78:0/3865858280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 msgr2=0x7f179810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 --2- 172.21.15.78:0/3865858280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 0x7f179810e790 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f179002a410 comp rx=0 tx=0).stop 2024-05-21T15:22:16.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 -- 172.21.15.78:0/3865858280 shutdown_connections 2024-05-21T15:22:16.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 --2- 172.21.15.78:0/3865858280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 0x7f179810e790 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 --2- 172.21.15.78:0/3865858280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1798101b30 0x7f1798101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 --2- 172.21.15.78:0/3865858280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1798107a10 0x7f17981015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 -- 172.21.15.78:0/3865858280 >> 172.21.15.78:0/3865858280 conn(0x7f1798077130 msgr2=0x7f1798077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 -- 172.21.15.78:0/3865858280 shutdown_connections 2024-05-21T15:22:16.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 -- 172.21.15.78:0/3865858280 wait complete. 2024-05-21T15:22:16.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.265+0000 7f179ff9d640 1 Processor -- start 2024-05-21T15:22:16.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179ff9d640 1 -- start start 2024-05-21T15:22:16.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179ff9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1798101b30 0x7f1798110d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179ff9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1798107a10 0x7f179810f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179ff9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 0x7f179810f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179ff9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1798112e40 con 0x7f179810c370 2024-05-21T15:22:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179ff9d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1798112cc0 con 0x7f1798101b30 2024-05-21T15:22:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179ff9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1798112fc0 con 0x7f1798107a10 2024-05-21T15:22:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 0x7f179810f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 0x7f179810f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52658/0 (socket says 172.21.15.78:52658) 2024-05-21T15:22:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 -- 172.21.15.78:0/2300065787 learned_addr learned my addr 172.21.15.78:0/2300065787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179d538640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1798107a10 0x7f179810f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179dd39640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1798101b30 0x7f1798110d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 -- 172.21.15.78:0/2300065787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1798107a10 msgr2=0x7f179810f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1798107a10 0x7f179810f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 -- 172.21.15.78:0/2300065787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1798101b30 msgr2=0x7f1798110d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1798101b30 0x7f1798110d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 -- 172.21.15.78:0/2300065787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179810ffc0 con 0x7f179810c370 2024-05-21T15:22:16.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.266+0000 7f179e53a640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 0x7f179810f7d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1790009230 tx=0x7f17900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.267+0000 7f1796ffd640 1 -- 172.21.15.78:0/2300065787 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179003f030 con 0x7f179810c370 2024-05-21T15:22:16.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.267+0000 7f1796ffd640 1 -- 172.21.15.78:0/2300065787 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1790003ce0 con 0x7f179810c370 2024-05-21T15:22:16.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.267+0000 7f179ff9d640 1 -- 172.21.15.78:0/2300065787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17981ac1c0 con 0x7f179810c370 2024-05-21T15:22:16.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.267+0000 7f1796ffd640 1 -- 172.21.15.78:0/2300065787 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790035070 con 0x7f179810c370 2024-05-21T15:22:16.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.267+0000 7f179ff9d640 1 -- 172.21.15.78:0/2300065787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17981ac680 con 0x7f179810c370 2024-05-21T15:22:16.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.269+0000 7f1796ffd640 1 -- 172.21.15.78:0/2300065787 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1790042030 con 0x7f179810c370 2024-05-21T15:22:16.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.269+0000 7f17a6e47740 1 Processor -- start 2024-05-21T15:22:16.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.269+0000 7f17a6e47740 1 -- start start 2024-05-21T15:22:16.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b100390 0x564d5b49ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b49f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0df6b0 0x564d5b49f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f17a6e47740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d5b404c40 con 0x564d5b0df6b0 2024-05-21T15:22:16.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f17a6e47740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d5b3ac580 con 0x564d5b100390 2024-05-21T15:22:16.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f17a6e47740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d5b400c00 con 0x564d5b0d74d0 2024-05-21T15:22:16.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0df6b0 0x564d5b49f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f1796ffd640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f17800a2d00 0x7f17800a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0df6b0 0x564d5b49f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52670/0 (socket says 172.21.15.78:52670) 2024-05-21T15:22:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 -- 172.21.15.78:0/1217636556 learned_addr learned my addr 172.21.15.78:0/1217636556 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179dd39640 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b49f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b100390 0x564d5b49ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179e53a640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f17800a2d00 0x7f17800a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 -- 172.21.15.78:0/1217636556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 msgr2=0x564d5b49f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b49f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 -- 172.21.15.78:0/1217636556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b100390 msgr2=0x564d5b49ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b100390 0x564d5b49ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f1796ffd640 1 -- 172.21.15.78:0/2300065787 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f17900efd30 con 0x7f179810c370 2024-05-21T15:22:16.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 -- 172.21.15.78:0/1217636556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5b1183f0 con 0x564d5b0df6b0 2024-05-21T15:22:16.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179d538640 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0df6b0 0x564d5b49f7a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f178c00afd0 tx=0x7f178c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5642b997147923f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f179e53a640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f17800a2d00 0x7f17800a5210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17900b8c70 tx=0x7f179008f310 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.270+0000 7f177ffff640 1 -- 172.21.15.78:0/1217636556 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c004020 con 0x564d5b0df6b0 2024-05-21T15:22:16.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f177ffff640 1 -- 172.21.15.78:0/1217636556 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c005500 con 0x564d5b0df6b0 2024-05-21T15:22:16.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f177ffff640 1 -- 172.21.15.78:0/1217636556 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c004d00 con 0x564d5b0df6b0 2024-05-21T15:22:16.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f17a6e47740 1 -- 172.21.15.78:0/1217636556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0df6b0 msgr2=0x564d5b49f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f17a6e47740 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0df6b0 0x564d5b49f7a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f178c00afd0 tx=0x7f178c00a660 comp rx=0 tx=0).stop 2024-05-21T15:22:16.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f17a6e47740 1 -- 172.21.15.78:0/1217636556 shutdown_connections 2024-05-21T15:22:16.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f17a6e47740 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0df6b0 0x564d5b49f7a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f17a6e47740 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b49f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f17a6e47740 1 --2- 172.21.15.78:0/1217636556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b100390 0x564d5b49ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.271+0000 7f17a6e47740 1 -- 172.21.15.78:0/1217636556 >> 172.21.15.78:0/1217636556 conn(0x564d5b0f44e0 msgr2=0x564d5b49b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 -- 172.21.15.78:0/1217636556 shutdown_connections 2024-05-21T15:22:16.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 -- 172.21.15.78:0/1217636556 wait complete. 2024-05-21T15:22:16.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 -- start start 2024-05-21T15:22:16.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 Processor -- start 2024-05-21T15:22:16.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 0x564d5b4a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 0x564d5b4a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b100390 0x564d5b4af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d5b4ac070 con 0x564d5b100390 2024-05-21T15:22:16.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d5b3bfc40 con 0x564d5b0d74d0 2024-05-21T15:22:16.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.272+0000 7f17a6e47740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d5b4ac1f0 con 0x564d5b0df6b0 2024-05-21T15:22:16.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f17a6e47740 10 client.0 _ll_register_callbacks cb 0x564d5b117be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b100390 0x564d5b4af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b100390 0x564d5b4af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52682/0 (socket says 172.21.15.78:52682) 2024-05-21T15:22:16.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 -- 172.21.15.78:0/2031551002 learned_addr learned my addr 172.21.15.78:0/2031551002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 0x564d5b4a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179dd39640 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 0x564d5b4a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 -- 172.21.15.78:0/2031551002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 msgr2=0x564d5b4a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 0x564d5b4a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 -- 172.21.15.78:0/2031551002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 msgr2=0x564d5b4a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 0x564d5b4a93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 -- 172.21.15.78:0/2031551002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5b1e27d0 con 0x564d5b100390 2024-05-21T15:22:16.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f179d538640 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b100390 0x564d5b4af520 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f178c005c50 tx=0x7f178c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd763c8b854ce7e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f177d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f177d7fa640 1 -- 172.21.15.78:0/2031551002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c004dc0 con 0x564d5b100390 2024-05-21T15:22:16.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f177d7fa640 1 -- 172.21.15.78:0/2031551002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f178c004030 con 0x564d5b100390 2024-05-21T15:22:16.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f17a6e47740 1 -- 172.21.15.78:0/2031551002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178c005500 con 0x564d5b100390 2024-05-21T15:22:16.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f177d7fa640 1 -- 172.21.15.78:0/2031551002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f178c004230 con 0x564d5b100390 2024-05-21T15:22:16.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f17a6e47740 10 client.17331 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.273+0000 7f17a6e47740 1 -- 172.21.15.78:0/2031551002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d5b4b8810 con 0x564d5b100390 2024-05-21T15:22:16.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f177d7fa640 1 -- 172.21.15.78:0/2031551002 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c005500 con 0x564d5b100390 2024-05-21T15:22:16.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f177d7fa640 1 -- 172.21.15.78:0/2031551002 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f178c003710 con 0x564d5b100390 2024-05-21T15:22:16.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f17a6e47740 10 client.17331 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f17a6e47740 10 client.17331 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:22:16.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f17a6e47740 20 client.17331 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f17a6e47740 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564d5b0bef60 0x564d5b4bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f17a6e47740 4 client.17331 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:16.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f17a6e47740 1 -- 172.21.15.78:0/2031551002 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d5b412370 con 0x564d5b0bef60 2024-05-21T15:22:16.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f179dd39640 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564d5b0bef60 0x564d5b4bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f177d7fa640 10 client.17331 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:22:16.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.274+0000 7f179dd39640 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564d5b0bef60 0x564d5b4bbb70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f2c424b60b7f89c3 server_cookie=41f5a95e46af596d in_seq=0 out_seq=0 2024-05-21T15:22:16.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f177d7fa640 1 -- 172.21.15.78:0/2031551002 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d5b412370 con 0x564d5b0bef60 2024-05-21T15:22:16.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f177d7fa640 10 client.17331 handle_command_reply: tid=0 2024-05-21T15:22:16.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 client.17331 shutdown 2024-05-21T15:22:16.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 -- 172.21.15.78:0/2031551002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b100390 msgr2=0x564d5b4af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b100390 0x564d5b4af520 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f178c005c50 tx=0x7f178c004840 comp rx=0 tx=0).stop 2024-05-21T15:22:16.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 -- 172.21.15.78:0/2031551002 shutdown_connections 2024-05-21T15:22:16.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564d5b0bef60 0x564d5b4bbb70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b100390 0x564d5b4af520 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f177d7fa640 0 client.17331 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:22:16.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 0x564d5b4a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 --2- 172.21.15.78:0/2031551002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 0x564d5b4a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.275+0000 7f17a6e47740 1 -- 172.21.15.78:0/2031551002 >> 172.21.15.78:0/2031551002 conn(0x564d5b0f44e0 msgr2=0x564d5b4a0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.276+0000 7f17a6e47740 1 -- 172.21.15.78:0/2031551002 shutdown_connections 2024-05-21T15:22:16.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.276+0000 7f17a6e47740 1 -- 172.21.15.78:0/2031551002 wait complete. 2024-05-21T15:22:16.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.276+0000 7f17a6e47740 20 client.17331 trim_cache size 0 max 16384 2024-05-21T15:22:16.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f17a6e47740 1 Processor -- start 2024-05-21T15:22:16.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f17a6e47740 1 -- start start 2024-05-21T15:22:16.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b49b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 0x564d5b4af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 0x564d5b4a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f17a6e47740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b5250 con 0x564d5b0bef60 2024-05-21T15:22:16.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f17a6e47740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b6d90 con 0x564d5b0d74d0 2024-05-21T15:22:16.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f17a6e47740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b5920 con 0x564d5b0df6b0 2024-05-21T15:22:16.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f179d538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b49b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f179d538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b49b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52688/0 (socket says 172.21.15.78:52688) 2024-05-21T15:22:16.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f179d538640 1 -- 172.21.15.78:0/1907439351 learned_addr learned my addr 172.21.15.78:0/1907439351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.289+0000 7f179dd39640 1 --2- 172.21.15.78:0/1907439351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 0x564d5b4af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f179d538640 1 -- 172.21.15.78:0/1907439351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 msgr2=0x564d5b4a2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f179d538640 1 --2- 172.21.15.78:0/1907439351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 0x564d5b4a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f179d538640 1 -- 172.21.15.78:0/1907439351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 msgr2=0x564d5b4af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f179d538640 1 --2- 172.21.15.78:0/1907439351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 0x564d5b4af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f179d538640 1 -- 172.21.15.78:0/1907439351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d5b4a9120 con 0x564d5b0bef60 2024-05-21T15:22:16.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f179d538640 1 --2- 172.21.15.78:0/1907439351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b49b5a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f178c003c50 tx=0x7f178c00a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411865bbcd1da411 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f177f7fe640 1 -- 172.21.15.78:0/1907439351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0043d0 con 0x564d5b0bef60 2024-05-21T15:22:16.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f177f7fe640 1 -- 172.21.15.78:0/1907439351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c015040 con 0x564d5b0bef60 2024-05-21T15:22:16.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.290+0000 7f177f7fe640 1 -- 172.21.15.78:0/1907439351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c003850 con 0x564d5b0bef60 2024-05-21T15:22:16.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 -- 172.21.15.78:0/1907439351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 msgr2=0x564d5b49b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 --2- 172.21.15.78:0/1907439351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b49b5a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f178c003c50 tx=0x7f178c00a6a0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 -- 172.21.15.78:0/1907439351 shutdown_connections 2024-05-21T15:22:16.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 --2- 172.21.15.78:0/1907439351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0d74d0 0x564d5b4af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 --2- 172.21.15.78:0/1907439351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0df6b0 0x564d5b4a2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 --2- 172.21.15.78:0/1907439351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b49b5a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 -- 172.21.15.78:0/1907439351 >> 172.21.15.78:0/1907439351 conn(0x564d5b0f44e0 msgr2=0x564d5b49ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 -- 172.21.15.78:0/1907439351 shutdown_connections 2024-05-21T15:22:16.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 -- 172.21.15.78:0/1907439351 wait complete. 2024-05-21T15:22:16.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 -- start start 2024-05-21T15:22:16.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 Processor -- start 2024-05-21T15:22:16.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.291+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b4b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b4b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0df6b0 0x564d5b49a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b66c0 con 0x564d5b0bef60 2024-05-21T15:22:16.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b5ff0 con 0x564d5b0df6b0 2024-05-21T15:22:16.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17981b3230 con 0x564d5b0d74d0 2024-05-21T15:22:16.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 10 client.0 _ll_register_callbacks cb 0x7f178c00ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b4b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b4b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52696/0 (socket says 172.21.15.78:52696) 2024-05-21T15:22:16.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 -- 172.21.15.78:0/146418734 learned_addr learned my addr 172.21.15.78:0/146418734 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179dd39640 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b4b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 -- 172.21.15.78:0/146418734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 msgr2=0x564d5b4b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b4b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 -- 172.21.15.78:0/146418734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0df6b0 msgr2=0x564d5b49a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0df6b0 0x564d5b49a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 -- 172.21.15.78:0/146418734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d5b49a930 con 0x564d5b0bef60 2024-05-21T15:22:16.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179dd39640 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b4b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:16.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f179d538640 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b4b5790 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f178c000c00 tx=0x7f178c00a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897cfa97325e0d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f177d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f177d7fa640 1 -- 172.21.15.78:0/146418734 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c00a970 con 0x564d5b0bef60 2024-05-21T15:22:16.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f177d7fa640 1 -- 172.21.15.78:0/146418734 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f178c003cd0 con 0x564d5b0bef60 2024-05-21T15:22:16.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 1 -- 172.21.15.78:0/146418734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178c015040 con 0x564d5b0bef60 2024-05-21T15:22:16.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f177d7fa640 1 -- 172.21.15.78:0/146418734 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f178c003f10 con 0x564d5b0bef60 2024-05-21T15:22:16.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 10 client.17337 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.292+0000 7f17a6e47740 1 -- 172.21.15.78:0/146418734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d5b4b3b80 con 0x564d5b0bef60 2024-05-21T15:22:16.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f177d7fa640 1 -- 172.21.15.78:0/146418734 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178c015040 con 0x564d5b0bef60 2024-05-21T15:22:16.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f177d7fa640 1 -- 172.21.15.78:0/146418734 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f178c035030 con 0x564d5b0bef60 2024-05-21T15:22:16.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f17a6e47740 10 client.17337 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f17a6e47740 10 client.17337 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:22:16.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f17a6e47740 20 client.17337 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f17a6e47740 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564d5b100390 0x564d5b525cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f17a6e47740 4 client.17337 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f17a6e47740 1 -- 172.21.15.78:0/146418734 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d5b40c6c0 con 0x564d5b100390 2024-05-21T15:22:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.293+0000 7f179dd39640 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564d5b100390 0x564d5b525cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f179dd39640 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564d5b100390 0x564d5b525cd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a6914bc67fcd0bb server_cookie=bc7ff7acf8bded8d in_seq=0 out_seq=0 2024-05-21T15:22:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f177d7fa640 10 client.17337 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:22:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f177d7fa640 1 -- 172.21.15.78:0/146418734 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d5b40c6c0 con 0x564d5b100390 2024-05-21T15:22:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f177d7fa640 10 client.17337 handle_command_reply: tid=0 2024-05-21T15:22:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 client.17337 shutdown 2024-05-21T15:22:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 -- 172.21.15.78:0/146418734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 msgr2=0x564d5b4b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b4b5790 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f178c000c00 tx=0x7f178c00a470 comp rx=0 tx=0).stop 2024-05-21T15:22:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 -- 172.21.15.78:0/146418734 shutdown_connections 2024-05-21T15:22:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x564d5b100390 0x564d5b525cd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d5b0df6b0 0x564d5b49a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d5b0d74d0 0x564d5b4b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 --2- 172.21.15.78:0/146418734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d5b0bef60 0x564d5b4b5790 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f177d7fa640 0 client.17337 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:22:16.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 -- 172.21.15.78:0/146418734 >> 172.21.15.78:0/146418734 conn(0x564d5b0f44e0 msgr2=0x564d5b49eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 -- 172.21.15.78:0/146418734 shutdown_connections 2024-05-21T15:22:16.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 1 -- 172.21.15.78:0/146418734 wait complete. 2024-05-21T15:22:16.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.294+0000 7f17a6e47740 20 client.17337 trim_cache size 0 max 16384 2024-05-21T15:22:16.311 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 -- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f17800a2d00 msgr2=0x7f17800a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f17800a2d00 0x7f17800a5210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17900b8c70 tx=0x7f179008f310 comp rx=0 tx=0).stop 2024-05-21T15:22:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 -- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 msgr2=0x7f179810f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 0x7f179810f7d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1790009230 tx=0x7f17900042f0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 -- 172.21.15.78:0/2300065787 shutdown_connections 2024-05-21T15:22:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f17800a2d00 0x7f17800a5210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f179810c370 0x7f179810f7d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1798107a10 0x7f179810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 --2- 172.21.15.78:0/2300065787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1798101b30 0x7f1798110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 -- 172.21.15.78:0/2300065787 >> 172.21.15.78:0/2300065787 conn(0x7f1798077130 msgr2=0x7f17981030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 -- 172.21.15.78:0/2300065787 shutdown_connections 2024-05-21T15:22:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.295+0000 7f179ff9d640 1 -- 172.21.15.78:0/2300065787 wait complete. 2024-05-21T15:22:16.318 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:22:16.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 -- 172.21.15.78:0/2398383729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3768108b70 msgr2=0x7f3768100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 --2- 172.21.15.78:0/2398383729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3768108b70 0x7f3768100ae0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f3758009900 tx=0x7f375802a410 comp rx=0 tx=0).stop 2024-05-21T15:22:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 -- 172.21.15.78:0/2398383729 shutdown_connections 2024-05-21T15:22:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 --2- 172.21.15.78:0/2398383729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3768101020 0x7f3768101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 --2- 172.21.15.78:0/2398383729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f376810d520 0x7f376810f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 --2- 172.21.15.78:0/2398383729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3768108b70 0x7f3768100ae0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 -- 172.21.15.78:0/2398383729 >> 172.21.15.78:0/2398383729 conn(0x7f37680768d0 msgr2=0x7f3768076d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 -- 172.21.15.78:0/2398383729 shutdown_connections 2024-05-21T15:22:16.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.428+0000 7f376cb5e640 1 -- 172.21.15.78:0/2398383729 wait complete. 2024-05-21T15:22:16.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.429+0000 7f376cb5e640 1 Processor -- start 2024-05-21T15:22:16.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.429+0000 7f376cb5e640 1 -- start start 2024-05-21T15:22:16.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.429+0000 7f376cb5e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3768101020 0x7f376819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.429+0000 7f376cb5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768108b70 0x7f376819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.429+0000 7f376cb5e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376810d520 0x7f3768199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.429+0000 7f376cb5e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3768112e30 con 0x7f376810d520 2024-05-21T15:22:16.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.429+0000 7f376cb5e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3768112cb0 con 0x7f3768101020 2024-05-21T15:22:16.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.429+0000 7f376cb5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3768112fb0 con 0x7f3768108b70 2024-05-21T15:22:16.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376810d520 0x7f3768199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376810d520 0x7f3768199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52702/0 (socket says 172.21.15.78:52702) 2024-05-21T15:22:16.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 -- 172.21.15.78:0/3837896054 learned_addr learned my addr 172.21.15.78:0/3837896054 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3765d9b640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768108b70 0x7f376819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f376659c640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3768101020 0x7f376819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 -- 172.21.15.78:0/3837896054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768108b70 msgr2=0x7f376819fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768108b70 0x7f376819fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 -- 172.21.15.78:0/3837896054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3768101020 msgr2=0x7f376819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3768101020 0x7f376819f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 -- 172.21.15.78:0/3837896054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3768199d80 con 0x7f376810d520 2024-05-21T15:22:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3765d9b640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768108b70 0x7f376819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f3766d9d640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376810d520 0x7f3768199650 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f376000eb40 tx=0x7f376000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f37577fe640 1 -- 172.21.15.78:0/3837896054 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760004b20 con 0x7f376810d520 2024-05-21T15:22:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f376cb5e640 1 -- 172.21.15.78:0/3837896054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376819a010 con 0x7f376810d520 2024-05-21T15:22:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f37577fe640 1 -- 172.21.15.78:0/3837896054 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3760004cc0 con 0x7f376810d520 2024-05-21T15:22:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f37577fe640 1 -- 172.21.15.78:0/3837896054 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760010850 con 0x7f376810d520 2024-05-21T15:22:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.430+0000 7f376cb5e640 1 -- 172.21.15.78:0/3837896054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681a6270 con 0x7f376810d520 2024-05-21T15:22:16.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f37577fe640 1 -- 172.21.15.78:0/3837896054 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3760005020 con 0x7f376810d520 2024-05-21T15:22:16.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f3773a07740 1 Processor -- start 2024-05-21T15:22:16.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f3773a07740 1 -- start start 2024-05-21T15:22:16.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f3773a07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b965c390 0x5581b99fad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f3773a07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 0x5581b99fb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f3773a07740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b99fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f3773a07740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581b9960c40 con 0x5581b963b6b0 2024-05-21T15:22:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f3773a07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5581b9908580 con 0x5581b965c390 2024-05-21T15:22:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.433+0000 7f3773a07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5581b995cc00 con 0x5581b96334d0 2024-05-21T15:22:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b99fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f37577fe640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37500a2d00 0x7f37500a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b99fb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52706/0 (socket says 172.21.15.78:52706) 2024-05-21T15:22:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 -- 172.21.15.78:0/3018683686 learned_addr learned my addr 172.21.15.78:0/3018683686 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f376659c640 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 0x5581b99fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3766d9d640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37500a2d00 0x7f37500a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b965c390 0x5581b99fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 -- 172.21.15.78:0/3018683686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 msgr2=0x5581b99fb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 0x5581b99fb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 -- 172.21.15.78:0/3018683686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b965c390 msgr2=0x5581b99fad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b965c390 0x5581b99fad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 -- 172.21.15.78:0/3018683686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b96743f0 con 0x5581b963b6b0 2024-05-21T15:22:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f37577fe640 1 -- 172.21.15.78:0/3837896054 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f37600cfd80 con 0x7f376810d520 2024-05-21T15:22:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3765d9b640 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b99fb7a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f375c00bfd0 tx=0x7f375c00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7a518f711bda2c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3766d9d640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37500a2d00 0x7f37500a5210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37600986a0 tx=0x7f3760097e40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3754ff9640 1 -- 172.21.15.78:0/3018683686 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c018030 con 0x5581b963b6b0 2024-05-21T15:22:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3754ff9640 1 -- 172.21.15.78:0/3018683686 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c00e8b0 con 0x5581b963b6b0 2024-05-21T15:22:16.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.434+0000 7f3754ff9640 1 -- 172.21.15.78:0/3018683686 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c01c8c0 con 0x5581b963b6b0 2024-05-21T15:22:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 -- 172.21.15.78:0/3018683686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 msgr2=0x5581b99fb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b99fb7a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f375c00bfd0 tx=0x7f375c00b5a0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 -- 172.21.15.78:0/3018683686 shutdown_connections 2024-05-21T15:22:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b99fb7a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 0x5581b99fb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 --2- 172.21.15.78:0/3018683686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b965c390 0x5581b99fad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 -- 172.21.15.78:0/3018683686 >> 172.21.15.78:0/3018683686 conn(0x5581b96504e0 msgr2=0x5581b99f74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 -- 172.21.15.78:0/3018683686 shutdown_connections 2024-05-21T15:22:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.435+0000 7f3773a07740 1 -- 172.21.15.78:0/3018683686 wait complete. 2024-05-21T15:22:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 1 -- start start 2024-05-21T15:22:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 1 Processor -- start 2024-05-21T15:22:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 0x5581b9a05360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 0x5581b9a058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b965c390 0x5581b9a0b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581b9a08010 con 0x5581b965c390 2024-05-21T15:22:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5581b991bc40 con 0x5581b963b6b0 2024-05-21T15:22:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5581b9a08190 con 0x5581b96334d0 2024-05-21T15:22:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3773a07740 10 client.0 _ll_register_callbacks cb 0x5581b9673be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3765d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b965c390 0x5581b9a0b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3765d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b965c390 0x5581b9a0b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52720/0 (socket says 172.21.15.78:52720) 2024-05-21T15:22:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3765d9b640 1 -- 172.21.15.78:0/209716065 learned_addr learned my addr 172.21.15.78:0/209716065 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f376659c640 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 0x5581b9a058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.436+0000 7f3765d9b640 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 0x5581b9a05360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3765d9b640 1 -- 172.21.15.78:0/209716065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 msgr2=0x5581b9a05360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3765d9b640 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 0x5581b9a05360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3765d9b640 1 -- 172.21.15.78:0/209716065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 msgr2=0x5581b9a058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3765d9b640 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 0x5581b9a058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3765d9b640 1 -- 172.21.15.78:0/209716065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b973e7d0 con 0x5581b965c390 2024-05-21T15:22:16.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3765d9b640 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b965c390 0x5581b9a0b4f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f375c001eb0 tx=0x7f375c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0d26e982ba8e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f374dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f374dffb640 1 -- 172.21.15.78:0/209716065 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c018070 con 0x5581b965c390 2024-05-21T15:22:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f374dffb640 1 -- 172.21.15.78:0/209716065 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f375c01c5a0 con 0x5581b965c390 2024-05-21T15:22:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3773a07740 1 -- 172.21.15.78:0/209716065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375c00e8b0 con 0x5581b965c390 2024-05-21T15:22:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f374dffb640 1 -- 172.21.15.78:0/209716065 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f375c01c7a0 con 0x5581b965c390 2024-05-21T15:22:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3773a07740 10 client.17349 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3773a07740 1 -- 172.21.15.78:0/209716065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5581b9a14780 con 0x5581b965c390 2024-05-21T15:22:16.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f374dffb640 1 -- 172.21.15.78:0/209716065 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c00e8b0 con 0x5581b965c390 2024-05-21T15:22:16.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f374dffb640 1 -- 172.21.15.78:0/209716065 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f375c00f040 con 0x5581b965c390 2024-05-21T15:22:16.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.437+0000 7f3773a07740 10 client.17349 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.438+0000 7f3773a07740 10 client.17349 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:22:16.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.438+0000 7f3773a07740 20 client.17349 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.438+0000 7f3773a07740 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5581b961af60 0x5581b9a17b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.438+0000 7f3773a07740 4 client.17349 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:16.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.438+0000 7f3773a07740 1 -- 172.21.15.78:0/209716065 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581b996e370 con 0x5581b961af60 2024-05-21T15:22:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.438+0000 7f376659c640 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5581b961af60 0x5581b9a17b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.438+0000 7f374dffb640 10 client.17349 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:22:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.438+0000 7f376659c640 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5581b961af60 0x5581b9a17b40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4eaa923a06579a6 server_cookie=7ea98364db3e745d in_seq=0 out_seq=0 2024-05-21T15:22:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f374dffb640 1 -- 172.21.15.78:0/209716065 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5581b996e370 con 0x5581b961af60 2024-05-21T15:22:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f374dffb640 10 client.17349 handle_command_reply: tid=0 2024-05-21T15:22:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 client.17349 shutdown 2024-05-21T15:22:16.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 -- 172.21.15.78:0/209716065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b965c390 msgr2=0x5581b9a0b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b965c390 0x5581b9a0b4f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f375c001eb0 tx=0x7f375c014040 comp rx=0 tx=0).stop 2024-05-21T15:22:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 -- 172.21.15.78:0/209716065 shutdown_connections 2024-05-21T15:22:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5581b961af60 0x5581b9a17b40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 0x5581b9a058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f374dffb640 0 client.17349 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:22:16.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b965c390 0x5581b9a0b4f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 --2- 172.21.15.78:0/209716065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b96334d0 0x5581b9a05360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 -- 172.21.15.78:0/209716065 >> 172.21.15.78:0/209716065 conn(0x5581b96504e0 msgr2=0x5581b99fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 -- 172.21.15.78:0/209716065 shutdown_connections 2024-05-21T15:22:16.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 1 -- 172.21.15.78:0/209716065 wait complete. 2024-05-21T15:22:16.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.439+0000 7f3773a07740 20 client.17349 trim_cache size 0 max 16384 2024-05-21T15:22:16.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3773a07740 1 Processor -- start 2024-05-21T15:22:16.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3773a07740 1 -- start start 2024-05-21T15:22:16.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3773a07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 0x5581b9a072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3773a07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b96334d0 0x5581b9a04900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3773a07740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b9a1d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3773a07740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37681aed30 con 0x5581b963b6b0 2024-05-21T15:22:16.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3773a07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f37681b0870 con 0x5581b96334d0 2024-05-21T15:22:16.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3773a07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37681af400 con 0x5581b961af60 2024-05-21T15:22:16.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b9a1d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b9a1d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52724/0 (socket says 172.21.15.78:52724) 2024-05-21T15:22:16.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 -- 172.21.15.78:0/2235380891 learned_addr learned my addr 172.21.15.78:0/2235380891 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f376659c640 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b96334d0 0x5581b9a04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 0x5581b9a072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 -- 172.21.15.78:0/2235380891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 msgr2=0x5581b9a072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 0x5581b9a072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 -- 172.21.15.78:0/2235380891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b96334d0 msgr2=0x5581b9a04900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b96334d0 0x5581b9a04900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 -- 172.21.15.78:0/2235380891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581b99fc090 con 0x5581b963b6b0 2024-05-21T15:22:16.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f376659c640 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b96334d0 0x5581b9a04900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:16.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.453+0000 7f3765d9b640 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b9a1d1d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f375c00b9e0 tx=0x7f375c01cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f64d51acc6bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f374ffff640 1 -- 172.21.15.78:0/2235380891 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c018070 con 0x5581b963b6b0 2024-05-21T15:22:16.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f374ffff640 1 -- 172.21.15.78:0/2235380891 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c01de50 con 0x5581b963b6b0 2024-05-21T15:22:16.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f374ffff640 1 -- 172.21.15.78:0/2235380891 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c036040 con 0x5581b963b6b0 2024-05-21T15:22:16.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 -- 172.21.15.78:0/2235380891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 msgr2=0x5581b9a1d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b9a1d1d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f375c00b9e0 tx=0x7f375c01cfb0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 -- 172.21.15.78:0/2235380891 shutdown_connections 2024-05-21T15:22:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b96334d0 0x5581b9a04900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b963b6b0 0x5581b9a1d1d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 --2- 172.21.15.78:0/2235380891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 0x5581b9a072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 -- 172.21.15.78:0/2235380891 >> 172.21.15.78:0/2235380891 conn(0x5581b96504e0 msgr2=0x5581b99fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 -- 172.21.15.78:0/2235380891 shutdown_connections 2024-05-21T15:22:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.454+0000 7f3773a07740 1 -- 172.21.15.78:0/2235380891 wait complete. 2024-05-21T15:22:16.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 1 -- start start 2024-05-21T15:22:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 1 Processor -- start 2024-05-21T15:22:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 0x5581b9a0c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b96334d0 0x5581b9a0cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 0x5581b9a0d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37681b01a0 con 0x5581b96334d0 2024-05-21T15:22:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f37681afad0 con 0x5581b963b6b0 2024-05-21T15:22:16.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37681acd10 con 0x5581b961af60 2024-05-21T15:22:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3773a07740 10 client.0 _ll_register_callbacks cb 0x7f375c00beb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f376659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b96334d0 0x5581b9a0cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f376659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b96334d0 0x5581b9a0cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52736/0 (socket says 172.21.15.78:52736) 2024-05-21T15:22:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f376659c640 1 -- 172.21.15.78:0/2374815161 learned_addr learned my addr 172.21.15.78:0/2374815161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3765d9b640 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 0x5581b9a0c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f3765d9b640 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 0x5581b9a0d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f376659c640 1 -- 172.21.15.78:0/2374815161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 msgr2=0x5581b9a0c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.455+0000 7f376659c640 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 0x5581b9a0c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f376659c640 1 -- 172.21.15.78:0/2374815161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 msgr2=0x5581b9a0d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f376659c640 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 0x5581b9a0d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f376659c640 1 -- 172.21.15.78:0/2374815161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581b9a7df70 con 0x5581b96334d0 2024-05-21T15:22:16.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f376659c640 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b96334d0 0x5581b9a0cea0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3758009900 tx=0x7f3758004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef26089b97a5d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f374dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f374dffb640 1 -- 172.21.15.78:0/2374815161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758035040 con 0x5581b96334d0 2024-05-21T15:22:16.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f374dffb640 1 -- 172.21.15.78:0/2374815161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3758005040 con 0x5581b96334d0 2024-05-21T15:22:16.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 1 -- 172.21.15.78:0/2374815161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375c01de50 con 0x5581b96334d0 2024-05-21T15:22:16.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f374dffb640 1 -- 172.21.15.78:0/2374815161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3758051050 con 0x5581b96334d0 2024-05-21T15:22:16.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 10 client.17355 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 1 -- 172.21.15.78:0/2374815161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5581b9a7f260 con 0x5581b96334d0 2024-05-21T15:22:16.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f374dffb640 1 -- 172.21.15.78:0/2374815161 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c01de50 con 0x5581b96334d0 2024-05-21T15:22:16.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f374dffb640 1 -- 172.21.15.78:0/2374815161 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3758043ce0 con 0x5581b96334d0 2024-05-21T15:22:16.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 10 client.17355 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 10 client.17355 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:22:16.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 20 client.17355 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5581b965c390 0x5581b9a81a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 4 client.17355 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:16.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.456+0000 7f3773a07740 1 -- 172.21.15.78:0/2374815161 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581b99686c0 con 0x5581b965c390 2024-05-21T15:22:16.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.457+0000 7f3765d9b640 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5581b965c390 0x5581b9a81a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.457+0000 7f374dffb640 10 client.17355 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:22:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.457+0000 7f3765d9b640 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5581b965c390 0x5581b9a81a20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f04419aaccd5b505 server_cookie=20ab50ad1bce7dd1 in_seq=0 out_seq=0 2024-05-21T15:22:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f374dffb640 1 -- 172.21.15.78:0/2374815161 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581b99686c0 con 0x5581b965c390 2024-05-21T15:22:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f374dffb640 10 client.17355 handle_command_reply: tid=0 2024-05-21T15:22:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 client.17355 shutdown 2024-05-21T15:22:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 -- 172.21.15.78:0/2374815161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b96334d0 msgr2=0x5581b9a0cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b96334d0 0x5581b9a0cea0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3758009900 tx=0x7f3758004040 comp rx=0 tx=0).stop 2024-05-21T15:22:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 -- 172.21.15.78:0/2374815161 shutdown_connections 2024-05-21T15:22:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5581b965c390 0x5581b9a81a20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5581b963b6b0 0x5581b9a0d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f374dffb640 0 client.17355 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:22:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581b96334d0 0x5581b9a0cea0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 --2- 172.21.15.78:0/2374815161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581b961af60 0x5581b9a0c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 -- 172.21.15.78:0/2374815161 >> 172.21.15.78:0/2374815161 conn(0x5581b96504e0 msgr2=0x5581b9a070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 -- 172.21.15.78:0/2374815161 shutdown_connections 2024-05-21T15:22:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 1 -- 172.21.15.78:0/2374815161 wait complete. 2024-05-21T15:22:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f3773a07740 20 client.17355 trim_cache size 0 max 16384 2024-05-21T15:22:16.477 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:16.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 -- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37500a2d00 msgr2=0x7f37500a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37500a2d00 0x7f37500a5210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37600986a0 tx=0x7f3760097e40 comp rx=0 tx=0).stop 2024-05-21T15:22:16.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 -- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376810d520 msgr2=0x7f3768199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376810d520 0x7f3768199650 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f376000eb40 tx=0x7f376000c2d0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 -- 172.21.15.78:0/3837896054 shutdown_connections 2024-05-21T15:22:16.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37500a2d00 0x7f37500a5210 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376810d520 0x7f3768199650 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768108b70 0x7f376819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 --2- 172.21.15.78:0/3837896054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3768101020 0x7f376819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 -- 172.21.15.78:0/3837896054 >> 172.21.15.78:0/3837896054 conn(0x7f37680768d0 msgr2=0x7f37681094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 -- 172.21.15.78:0/3837896054 shutdown_connections 2024-05-21T15:22:16.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.458+0000 7f376cb5e640 1 -- 172.21.15.78:0/3837896054 wait complete. 2024-05-21T15:22:16.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:22:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 -- 172.21.15.78:0/2443617808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9324101b20 msgr2=0x7f9324101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 --2- 172.21.15.78:0/2443617808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9324101b20 0x7f9324101f30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f930c009900 tx=0x7f930c02a3f0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 -- 172.21.15.78:0/2443617808 shutdown_connections 2024-05-21T15:22:16.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 --2- 172.21.15.78:0/2443617808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f932410c360 0x7f932410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 --2- 172.21.15.78:0/2443617808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9324101b20 0x7f9324101f30 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 --2- 172.21.15.78:0/2443617808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9324107a00 0x7f93241015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 -- 172.21.15.78:0/2443617808 >> 172.21.15.78:0/2443617808 conn(0x7f9324077160 msgr2=0x7f93240775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 -- 172.21.15.78:0/2443617808 shutdown_connections 2024-05-21T15:22:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.589+0000 7f93292bc640 1 -- 172.21.15.78:0/2443617808 wait complete. 2024-05-21T15:22:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.590+0000 7f93292bc640 1 Processor -- start 2024-05-21T15:22:16.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.590+0000 7f93292bc640 1 -- start start 2024-05-21T15:22:16.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.590+0000 7f93292bc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9324101b20 0x7f93241986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.590+0000 7f93292bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9324107a00 0x7f9324198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.590+0000 7f93292bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932410c360 0x7f932419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.590+0000 7f93292bc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9324111d10 con 0x7f932410c360 2024-05-21T15:22:16.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.590+0000 7f93292bc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9324111b90 con 0x7f9324101b20 2024-05-21T15:22:16.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.590+0000 7f93292bc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9324111e90 con 0x7f9324107a00 2024-05-21T15:22:16.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932410c360 0x7f932419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932410c360 0x7f932419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52754/0 (socket says 172.21.15.78:52754) 2024-05-21T15:22:16.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 -- 172.21.15.78:0/3852169201 learned_addr learned my addr 172.21.15.78:0/3852169201 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f9322ffd640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9324101b20 0x7f93241986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f931a7fc640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9324107a00 0x7f9324198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 -- 172.21.15.78:0/3852169201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9324107a00 msgr2=0x7f9324198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9324107a00 0x7f9324198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 -- 172.21.15.78:0/3852169201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9324101b20 msgr2=0x7f93241986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9324101b20 0x7f93241986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 -- 172.21.15.78:0/3852169201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932419d490 con 0x7f932410c360 2024-05-21T15:22:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93237fe640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932410c360 0x7f932419ced0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f931c00ead0 tx=0x7f931c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f9320ff9640 1 -- 172.21.15.78:0/3852169201 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c004d20 con 0x7f932410c360 2024-05-21T15:22:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f9320ff9640 1 -- 172.21.15.78:0/3852169201 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f931c005020 con 0x7f932410c360 2024-05-21T15:22:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f9320ff9640 1 -- 172.21.15.78:0/3852169201 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0116d0 con 0x7f932410c360 2024-05-21T15:22:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93292bc640 1 -- 172.21.15.78:0/3852169201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932419d780 con 0x7f932410c360 2024-05-21T15:22:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.591+0000 7f93292bc640 1 -- 172.21.15.78:0/3852169201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932419dca0 con 0x7f932410c360 2024-05-21T15:22:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f9320ff9640 1 -- 172.21.15.78:0/3852169201 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f931c005560 con 0x7f932410c360 2024-05-21T15:22:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f932fdfd740 1 Processor -- start 2024-05-21T15:22:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f932fdfd740 1 -- start start 2024-05-21T15:22:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de63ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de63e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 0x55c3de63e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f932fdfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c3de5a1c40 con 0x55c3de29d390 2024-05-21T15:22:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f932fdfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c3de549580 con 0x55c3de27c6b0 2024-05-21T15:22:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f932fdfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c3de59dc00 con 0x55c3de2744d0 2024-05-21T15:22:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f931a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de63ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f931a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de63ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52762/0 (socket says 172.21.15.78:52762) 2024-05-21T15:22:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.594+0000 7f931a7fc640 1 -- 172.21.15.78:0/843736392 learned_addr learned my addr 172.21.15.78:0/843736392 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f9320ff9640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f93100a2d00 0x7f93100a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f931a7fc640 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 0x55c3de63e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f9322ffd640 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de63e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f93237fe640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f93100a2d00 0x7f93100a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f931a7fc640 1 -- 172.21.15.78:0/843736392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 msgr2=0x55c3de63e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f931a7fc640 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de63e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f931a7fc640 1 -- 172.21.15.78:0/843736392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 msgr2=0x55c3de63e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f931a7fc640 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 0x55c3de63e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f931a7fc640 1 -- 172.21.15.78:0/843736392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3de2b53f0 con 0x55c3de29d390 2024-05-21T15:22:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f9322ffd640 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de63e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f9320ff9640 1 -- 172.21.15.78:0/3852169201 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f931c0cf6c0 con 0x7f932410c360 2024-05-21T15:22:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f931a7fc640 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de63ddc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f930c000c00 tx=0x7f930c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1bd87589d41558 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f93237fe640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f93100a2d00 0x7f93100a5210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f931c0cce60 tx=0x7f931c0054d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f93197fa640 1 -- 172.21.15.78:0/843736392 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c035070 con 0x55c3de29d390 2024-05-21T15:22:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f93197fa640 1 -- 172.21.15.78:0/843736392 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c044070 con 0x55c3de29d390 2024-05-21T15:22:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.595+0000 7f93197fa640 1 -- 172.21.15.78:0/843736392 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c005550 con 0x55c3de29d390 2024-05-21T15:22:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 -- 172.21.15.78:0/843736392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 msgr2=0x55c3de63ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de63ddc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f930c000c00 tx=0x7f930c0058b0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 -- 172.21.15.78:0/843736392 shutdown_connections 2024-05-21T15:22:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 0x55c3de63e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de63e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 --2- 172.21.15.78:0/843736392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de63ddc0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 -- 172.21.15.78:0/843736392 >> 172.21.15.78:0/843736392 conn(0x55c3de2914e0 msgr2=0x55c3de63a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 -- 172.21.15.78:0/843736392 shutdown_connections 2024-05-21T15:22:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.596+0000 7f932fdfd740 1 -- 172.21.15.78:0/843736392 wait complete. 2024-05-21T15:22:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 1 -- start start 2024-05-21T15:22:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 1 Processor -- start 2024-05-21T15:22:16.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de64a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 0x55c3de6568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c3de64d450 con 0x55c3de27c6b0 2024-05-21T15:22:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c3de55cc40 con 0x55c3de2744d0 2024-05-21T15:22:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c3de64d5d0 con 0x55c3de29d390 2024-05-21T15:22:16.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f932fdfd740 10 client.0 _ll_register_callbacks cb 0x55c3de2b4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f9322ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f9322ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52778/0 (socket says 172.21.15.78:52778) 2024-05-21T15:22:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f9322ffd640 1 -- 172.21.15.78:0/3225446719 learned_addr learned my addr 172.21.15.78:0/3225446719 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f931a7fc640 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de64a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f9322ffd640 1 -- 172.21.15.78:0/3225446719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 msgr2=0x55c3de6568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f9322ffd640 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 0x55c3de6568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f9322ffd640 1 -- 172.21.15.78:0/3225446719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 msgr2=0x55c3de64a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f9322ffd640 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de64a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.597+0000 7f9322ffd640 1 -- 172.21.15.78:0/3225446719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3de37f7d0 con 0x55c3de27c6b0 2024-05-21T15:22:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f9322ffd640 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64ae40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9314011800 tx=0x7f9314011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c86067c14115d77 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f930a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f930a7fc640 1 -- 172.21.15.78:0/3225446719 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931401f060 con 0x55c3de27c6b0 2024-05-21T15:22:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f930a7fc640 1 -- 172.21.15.78:0/3225446719 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f931400fcf0 con 0x55c3de27c6b0 2024-05-21T15:22:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 1 -- 172.21.15.78:0/3225446719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c044070 con 0x55c3de27c6b0 2024-05-21T15:22:16.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f930a7fc640 1 -- 172.21.15.78:0/3225446719 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9314012450 con 0x55c3de27c6b0 2024-05-21T15:22:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 10 client.17367 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 1 -- 172.21.15.78:0/3225446719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c3de659b00 con 0x55c3de27c6b0 2024-05-21T15:22:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f930a7fc640 1 -- 172.21.15.78:0/3225446719 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c044070 con 0x55c3de27c6b0 2024-05-21T15:22:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f930a7fc640 1 -- 172.21.15.78:0/3225446719 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f93140226e0 con 0x55c3de27c6b0 2024-05-21T15:22:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 10 client.17367 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 10 client.17367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:22:16.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 20 client.17367 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55c3de25bf60 0x55c3de65d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 4 client.17367 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.598+0000 7f932fdfd740 1 -- 172.21.15.78:0/3225446719 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3de5af370 con 0x55c3de25bf60 2024-05-21T15:22:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.599+0000 7f931a7fc640 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55c3de25bf60 0x55c3de65d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.599+0000 7f930a7fc640 10 client.17367 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:22:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.599+0000 7f931a7fc640 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55c3de25bf60 0x55c3de65d030 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=940f514e230acdf6 server_cookie=c2bddf037404581c in_seq=0 out_seq=0 2024-05-21T15:22:16.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f930a7fc640 1 -- 172.21.15.78:0/3225446719 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3de5af370 con 0x55c3de25bf60 2024-05-21T15:22:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f930a7fc640 10 client.17367 handle_command_reply: tid=0 2024-05-21T15:22:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 client.17367 shutdown 2024-05-21T15:22:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 -- 172.21.15.78:0/3225446719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 msgr2=0x55c3de64ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64ae40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9314011800 tx=0x7f9314011cc0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 -- 172.21.15.78:0/3225446719 shutdown_connections 2024-05-21T15:22:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55c3de25bf60 0x55c3de65d030 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 0x55c3de6568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64ae40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f930a7fc640 0 client.17367 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:22:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 --2- 172.21.15.78:0/3225446719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de64a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 -- 172.21.15.78:0/3225446719 >> 172.21.15.78:0/3225446719 conn(0x55c3de2914e0 msgr2=0x55c3de63fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 -- 172.21.15.78:0/3225446719 shutdown_connections 2024-05-21T15:22:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 1 -- 172.21.15.78:0/3225446719 wait complete. 2024-05-21T15:22:16.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.600+0000 7f932fdfd740 20 client.17367 trim_cache size 0 max 16384 2024-05-21T15:22:16.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f932fdfd740 1 Processor -- start 2024-05-21T15:22:16.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f932fdfd740 1 -- start start 2024-05-21T15:22:16.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de649a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 0x55c3de63a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de6564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f932fdfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93241ac7d0 con 0x55c3de29d390 2024-05-21T15:22:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f932fdfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93241ae310 con 0x55c3de27c6b0 2024-05-21T15:22:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f932fdfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93241acea0 con 0x55c3de2744d0 2024-05-21T15:22:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de6564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de6564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52780/0 (socket says 172.21.15.78:52780) 2024-05-21T15:22:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 -- 172.21.15.78:0/45632445 learned_addr learned my addr 172.21.15.78:0/45632445 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de649a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f9322ffd640 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 0x55c3de63a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 -- 172.21.15.78:0/45632445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 msgr2=0x55c3de649a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de649a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 -- 172.21.15.78:0/45632445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 msgr2=0x55c3de63a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 0x55c3de63a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.614+0000 7f931a7fc640 1 -- 172.21.15.78:0/45632445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3de64cfd0 con 0x55c3de29d390 2024-05-21T15:22:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.615+0000 7f931a7fc640 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de6564d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f930c009900 tx=0x7f930c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a9aca92466b698 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.615+0000 7f9318ff9640 1 -- 172.21.15.78:0/45632445 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c035040 con 0x55c3de29d390 2024-05-21T15:22:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.615+0000 7f9318ff9640 1 -- 172.21.15.78:0/45632445 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c044040 con 0x55c3de29d390 2024-05-21T15:22:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.615+0000 7f9318ff9640 1 -- 172.21.15.78:0/45632445 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c003ab0 con 0x55c3de29d390 2024-05-21T15:22:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- 172.21.15.78:0/45632445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 msgr2=0x55c3de6564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de6564d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f930c009900 tx=0x7f930c03f040 comp rx=0 tx=0).stop 2024-05-21T15:22:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- 172.21.15.78:0/45632445 shutdown_connections 2024-05-21T15:22:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de27c6b0 0x55c3de63a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de29d390 0x55c3de6564d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 --2- 172.21.15.78:0/45632445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de2744d0 0x55c3de649a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- 172.21.15.78:0/45632445 >> 172.21.15.78:0/45632445 conn(0x55c3de2914e0 msgr2=0x55c3de639e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- 172.21.15.78:0/45632445 shutdown_connections 2024-05-21T15:22:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- 172.21.15.78:0/45632445 wait complete. 2024-05-21T15:22:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- start start 2024-05-21T15:22:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 Processor -- start 2024-05-21T15:22:16.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de652d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 0x55c3de64e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93241adc40 con 0x55c3de27c6b0 2024-05-21T15:22:16.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93241ad570 con 0x55c3de2744d0 2024-05-21T15:22:16.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93241aa7b0 con 0x55c3de29d390 2024-05-21T15:22:16.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.616+0000 7f932fdfd740 10 client.0 _ll_register_callbacks cb 0x7f931400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52788/0 (socket says 172.21.15.78:52788) 2024-05-21T15:22:16.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 -- 172.21.15.78:0/1564007236 learned_addr learned my addr 172.21.15.78:0/1564007236 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f931a7fc640 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 0x55c3de64e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f931a7fc640 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de652d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 -- 172.21.15.78:0/1564007236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 msgr2=0x55c3de64e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 0x55c3de64e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 -- 172.21.15.78:0/1564007236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 msgr2=0x55c3de652d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de652d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 -- 172.21.15.78:0/1564007236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3de64ecb0 con 0x55c3de27c6b0 2024-05-21T15:22:16.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f931a7fc640 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de652d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:16.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f9322ffd640 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64e0d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x55c3de63b3f0 tx=0x7f931400fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b68a502f8b1a03a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f930a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f930a7fc640 1 -- 172.21.15.78:0/1564007236 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931402f070 con 0x55c3de27c6b0 2024-05-21T15:22:16.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f930a7fc640 1 -- 172.21.15.78:0/1564007236 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f9314022820 con 0x55c3de27c6b0 2024-05-21T15:22:16.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f932fdfd740 1 -- 172.21.15.78:0/1564007236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c044040 con 0x55c3de27c6b0 2024-05-21T15:22:16.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f930a7fc640 1 -- 172.21.15.78:0/1564007236 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9314005040 con 0x55c3de27c6b0 2024-05-21T15:22:16.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f932fdfd740 10 client.17373 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.617+0000 7f932fdfd740 1 -- 172.21.15.78:0/1564007236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c3de6c5280 con 0x55c3de27c6b0 2024-05-21T15:22:16.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f930a7fc640 1 -- 172.21.15.78:0/1564007236 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c044040 con 0x55c3de27c6b0 2024-05-21T15:22:16.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f930a7fc640 1 -- 172.21.15.78:0/1564007236 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9314004c30 con 0x55c3de27c6b0 2024-05-21T15:22:16.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f932fdfd740 10 client.17373 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f932fdfd740 10 client.17373 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:22:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f932fdfd740 20 client.17373 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f932fdfd740 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55c3de2313a0 0x55c3de6c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f932fdfd740 4 client.17373 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f932fdfd740 1 -- 172.21.15.78:0/1564007236 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3de5a96c0 con 0x55c3de2313a0 2024-05-21T15:22:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f931a7fc640 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55c3de2313a0 0x55c3de6c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f930a7fc640 10 client.17373 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:22:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.618+0000 7f931a7fc640 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55c3de2313a0 0x55c3de6c7a70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6996e83ebe78d21e server_cookie=a6a947784c83c8d4 in_seq=0 out_seq=0 2024-05-21T15:22:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f930a7fc640 1 -- 172.21.15.78:0/1564007236 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3de5a96c0 con 0x55c3de2313a0 2024-05-21T15:22:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f930a7fc640 10 client.17373 handle_command_reply: tid=0 2024-05-21T15:22:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 client.17373 shutdown 2024-05-21T15:22:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 -- 172.21.15.78:0/1564007236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 msgr2=0x55c3de64e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64e0d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x55c3de63b3f0 tx=0x7f931400fda0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 -- 172.21.15.78:0/1564007236 shutdown_connections 2024-05-21T15:22:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55c3de2313a0 0x55c3de6c7a70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c3de29d390 0x55c3de64e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3de27c6b0 0x55c3de64e0d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f930a7fc640 0 client.17373 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:22:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 --2- 172.21.15.78:0/1564007236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c3de2744d0 0x55c3de652d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 -- 172.21.15.78:0/1564007236 >> 172.21.15.78:0/1564007236 conn(0x55c3de2914e0 msgr2=0x55c3de649770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 -- 172.21.15.78:0/1564007236 shutdown_connections 2024-05-21T15:22:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 1 -- 172.21.15.78:0/1564007236 wait complete. 2024-05-21T15:22:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f932fdfd740 20 client.17373 trim_cache size 0 max 16384 2024-05-21T15:22:16.638 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f93292bc640 1 -- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f93100a2d00 msgr2=0x7f93100a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f93292bc640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f93100a2d00 0x7f93100a5210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f931c0cce60 tx=0x7f931c0054d0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.619+0000 7f93292bc640 1 -- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932410c360 msgr2=0x7f932419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932410c360 0x7f932419ced0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f931c00ead0 tx=0x7f931c00eed0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 -- 172.21.15.78:0/3852169201 shutdown_connections 2024-05-21T15:22:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f93100a2d00 0x7f93100a5210 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f932410c360 0x7f932419ced0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9324107a00 0x7f9324198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 --2- 172.21.15.78:0/3852169201 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9324101b20 0x7f93241986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 -- 172.21.15.78:0/3852169201 >> 172.21.15.78:0/3852169201 conn(0x7f9324077160 msgr2=0x7f9324103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 -- 172.21.15.78:0/3852169201 shutdown_connections 2024-05-21T15:22:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.620+0000 7f93292bc640 1 -- 172.21.15.78:0/3852169201 wait complete. 2024-05-21T15:22:16.641 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:22:16.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 -- 172.21.15.78:0/848411999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b301079f0 msgr2=0x7f0b30101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/848411999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b301079f0 0x7f0b30101cd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009900 tx=0x7f0b2002a410 comp rx=0 tx=0).stop 2024-05-21T15:22:16.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 -- 172.21.15.78:0/848411999 shutdown_connections 2024-05-21T15:22:16.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/848411999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b3010c350 0x7f0b3010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/848411999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b30102210 0x7f0b30102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/848411999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b301079f0 0x7f0b30101cd0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 -- 172.21.15.78:0/848411999 >> 172.21.15.78:0/848411999 conn(0x7f0b30076830 msgr2=0x7f0b30076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 -- 172.21.15.78:0/848411999 shutdown_connections 2024-05-21T15:22:16.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.750+0000 7f0b34f8f640 1 -- 172.21.15.78:0/848411999 wait complete. 2024-05-21T15:22:16.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b34f8f640 1 Processor -- start 2024-05-21T15:22:16.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b34f8f640 1 -- start start 2024-05-21T15:22:16.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b34f8f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b30102210 0x7f0b30198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b34f8f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301079f0 0x7f0b30198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b34f8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b3010c350 0x7f0b3019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b34f8f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30111d50 con 0x7f0b3010c350 2024-05-21T15:22:16.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b34f8f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30111bd0 con 0x7f0b30102210 2024-05-21T15:22:16.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b34f8f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30111ed0 con 0x7f0b301079f0 2024-05-21T15:22:16.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b2ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b3010c350 0x7f0b3019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b2ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b3010c350 0x7f0b3019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52800/0 (socket says 172.21.15.78:52800) 2024-05-21T15:22:16.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b2ed9d640 1 -- 172.21.15.78:0/1270690225 learned_addr learned my addr 172.21.15.78:0/1270690225 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.751+0000 7f0b27fff640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301079f0 0x7f0b30198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b30102210 0x7f0b30198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b2ed9d640 1 -- 172.21.15.78:0/1270690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301079f0 msgr2=0x7f0b30198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b2ed9d640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301079f0 0x7f0b30198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b2ed9d640 1 -- 172.21.15.78:0/1270690225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b30102210 msgr2=0x7f0b30198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b2ed9d640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b30102210 0x7f0b30198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b2ed9d640 1 -- 172.21.15.78:0/1270690225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3019d5d0 con 0x7f0b3010c350 2024-05-21T15:22:16.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b27fff640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301079f0 0x7f0b30198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:16.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b2ed9d640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b3010c350 0x7f0b3019cf90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800eb10 tx=0x7f0b2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b277fe640 1 -- 172.21.15.78:0/1270690225 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28004b20 con 0x7f0b3010c350 2024-05-21T15:22:16.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b277fe640 1 -- 172.21.15.78:0/1270690225 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b28004cc0 con 0x7f0b3010c350 2024-05-21T15:22:16.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b34f8f640 1 -- 172.21.15.78:0/1270690225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3019d8c0 con 0x7f0b3010c350 2024-05-21T15:22:16.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b277fe640 1 -- 172.21.15.78:0/1270690225 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28010760 con 0x7f0b3010c350 2024-05-21T15:22:16.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.752+0000 7f0b34f8f640 1 -- 172.21.15.78:0/1270690225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b3019ddc0 con 0x7f0b3010c350 2024-05-21T15:22:16.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b277fe640 1 -- 172.21.15.78:0/1270690225 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0b28005020 con 0x7f0b3010c350 2024-05-21T15:22:16.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b3be3c740 1 Processor -- start 2024-05-21T15:22:16.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b3be3c740 1 -- start start 2024-05-21T15:22:16.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 0x55d7d34cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7d3431c40 con 0x55d7d31044d0 2024-05-21T15:22:16.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d7d33d9580 con 0x55d7d312d390 2024-05-21T15:22:16.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d7d342dc00 con 0x55d7d310c6b0 2024-05-21T15:22:16.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b2e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b2e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52814/0 (socket says 172.21.15.78:52814) 2024-05-21T15:22:16.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b2e59c640 1 -- 172.21.15.78:0/3947035802 learned_addr learned my addr 172.21.15.78:0/3947035802 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b277fe640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0b1c0a2d00 0x7f0b1c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.755+0000 7f0b27fff640 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b27fff640 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 0x55d7d34cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b2ed9d640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0b1c0a2d00 0x7f0b1c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b2e59c640 1 -- 172.21.15.78:0/3947035802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 msgr2=0x55d7d34cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b2e59c640 1 -- 172.21.15.78:0/3947035802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 msgr2=0x55d7d34cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 0x55d7d34cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b2e59c640 1 -- 172.21.15.78:0/3947035802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7d31453f0 con 0x55d7d31044d0 2024-05-21T15:22:16.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b27fff640 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:16.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b277fe640 1 -- 172.21.15.78:0/1270690225 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f0b280cfb50 con 0x7f0b3010c350 2024-05-21T15:22:16.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34cc0b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009900 tx=0x7f0b20005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e682225cb93702d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b2ed9d640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0b1c0a2d00 0x7f0b1c0a5210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b2809bfa0 tx=0x7f0b2800efd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b24ff9640 1 -- 172.21.15.78:0/3947035802 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20035030 con 0x55d7d31044d0 2024-05-21T15:22:16.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b24ff9640 1 -- 172.21.15.78:0/3947035802 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b20005520 con 0x55d7d31044d0 2024-05-21T15:22:16.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.756+0000 7f0b24ff9640 1 -- 172.21.15.78:0/3947035802 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20004dc0 con 0x55d7d31044d0 2024-05-21T15:22:16.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3947035802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 msgr2=0x55d7d34cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34cc0b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009900 tx=0x7f0b20005d50 comp rx=0 tx=0).stop 2024-05-21T15:22:16.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3947035802 shutdown_connections 2024-05-21T15:22:16.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34cc0b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/3947035802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 0x55d7d34cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3947035802 >> 172.21.15.78:0/3947035802 conn(0x55d7d31214e0 msgr2=0x55d7d34c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3947035802 shutdown_connections 2024-05-21T15:22:16.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3947035802 wait complete. 2024-05-21T15:22:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.757+0000 7f0b3be3c740 1 -- start start 2024-05-21T15:22:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b3be3c740 1 Processor -- start 2024-05-21T15:22:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34d89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 0x55d7d34e4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7d34db250 con 0x55d7d31044d0 2024-05-21T15:22:16.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d7d33ecc40 con 0x55d7d312d390 2024-05-21T15:22:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d7d34db3d0 con 0x55d7d310c6b0 2024-05-21T15:22:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b3be3c740 10 client.0 _ll_register_callbacks cb 0x55d7d3144be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34d84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52830/0 (socket says 172.21.15.78:52830) 2024-05-21T15:22:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 -- 172.21.15.78:0/4001019927 learned_addr learned my addr 172.21.15.78:0/4001019927 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34d89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 -- 172.21.15.78:0/4001019927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 msgr2=0x55d7d34d89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34d89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 -- 172.21.15.78:0/4001019927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 msgr2=0x55d7d34e4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 0x55d7d34e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 -- 172.21.15.78:0/4001019927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7d320f7d0 con 0x55d7d31044d0 2024-05-21T15:22:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b27fff640 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34d84b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0b180117b0 tx=0x7f0b18011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37fe7d9b6a00645 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.758+0000 7f0b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b15ffb640 1 -- 172.21.15.78:0/4001019927 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18021060 con 0x55d7d31044d0 2024-05-21T15:22:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b15ffb640 1 -- 172.21.15.78:0/4001019927 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f0b18010d60 con 0x55d7d31044d0 2024-05-21T15:22:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 1 -- 172.21.15.78:0/4001019927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b20005520 con 0x55d7d31044d0 2024-05-21T15:22:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b15ffb640 1 -- 172.21.15.78:0/4001019927 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0b1801c420 con 0x55d7d31044d0 2024-05-21T15:22:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 10 client.17385 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 1 -- 172.21.15.78:0/4001019927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d7d34e7900 con 0x55d7d31044d0 2024-05-21T15:22:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b15ffb640 1 -- 172.21.15.78:0/4001019927 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b20005520 con 0x55d7d31044d0 2024-05-21T15:22:16.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b15ffb640 1 -- 172.21.15.78:0/4001019927 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0b1800e4d0 con 0x55d7d31044d0 2024-05-21T15:22:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 10 client.17385 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 10 client.17385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:22:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 20 client.17385 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55d7d30ebf60 0x55d7d34eae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 4 client.17385 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.759+0000 7f0b3be3c740 1 -- 172.21.15.78:0/4001019927 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7d343f370 con 0x55d7d30ebf60 2024-05-21T15:22:16.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.760+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55d7d30ebf60 0x55d7d34eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.760+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55d7d30ebf60 0x55d7d34eae10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d27975207e0ee547 server_cookie=8afe9937f6024c4e in_seq=0 out_seq=0 2024-05-21T15:22:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.760+0000 7f0b15ffb640 10 client.17385 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:22:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b15ffb640 1 -- 172.21.15.78:0/4001019927 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7d343f370 con 0x55d7d30ebf60 2024-05-21T15:22:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b15ffb640 10 client.17385 handle_command_reply: tid=0 2024-05-21T15:22:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 client.17385 shutdown 2024-05-21T15:22:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 -- 172.21.15.78:0/4001019927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 msgr2=0x55d7d34d84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34d84b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0b180117b0 tx=0x7f0b18011c70 comp rx=0 tx=0).stop 2024-05-21T15:22:16.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 -- 172.21.15.78:0/4001019927 shutdown_connections 2024-05-21T15:22:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55d7d30ebf60 0x55d7d34eae10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d312d390 0x55d7d34e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d310c6b0 0x55d7d34d89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b15ffb640 0 client.17385 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:22:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/4001019927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34d84b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 -- 172.21.15.78:0/4001019927 >> 172.21.15.78:0/4001019927 conn(0x55d7d31214e0 msgr2=0x55d7d34cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 -- 172.21.15.78:0/4001019927 shutdown_connections 2024-05-21T15:22:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 1 -- 172.21.15.78:0/4001019927 wait complete. 2024-05-21T15:22:16.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.761+0000 7f0b3be3c740 20 client.17385 trim_cache size 0 max 16384 2024-05-21T15:22:16.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b3be3c740 1 Processor -- start 2024-05-21T15:22:16.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b3be3c740 1 -- start start 2024-05-21T15:22:16.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d31044d0 0x55d7d34da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d312d390 0x55d7d34e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301ac950 con 0x55d7d312d390 2024-05-21T15:22:16.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301ae490 con 0x55d7d310c6b0 2024-05-21T15:22:16.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301ad020 con 0x55d7d31044d0 2024-05-21T15:22:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d312d390 0x55d7d34e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d312d390 0x55d7d34e42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52832/0 (socket says 172.21.15.78:52832) 2024-05-21T15:22:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 -- 172.21.15.78:0/3732165056 learned_addr learned my addr 172.21.15.78:0/3732165056 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d31044d0 0x55d7d34da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 -- 172.21.15.78:0/3732165056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d31044d0 msgr2=0x55d7d34da3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d31044d0 0x55d7d34da3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 -- 172.21.15.78:0/3732165056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 msgr2=0x55d7d34d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 -- 172.21.15.78:0/3732165056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7d34ccac0 con 0x55d7d312d390 2024-05-21T15:22:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.775+0000 7f0b27fff640 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d312d390 0x55d7d34e42e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800f000 tx=0x7f0b180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345d80377ba42f8e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b17fff640 1 -- 172.21.15.78:0/3732165056 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800ed10 con 0x55d7d312d390 2024-05-21T15:22:16.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b17fff640 1 -- 172.21.15.78:0/3732165056 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1800e3b0 con 0x55d7d312d390 2024-05-21T15:22:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b17fff640 1 -- 172.21.15.78:0/3732165056 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18003850 con 0x55d7d312d390 2024-05-21T15:22:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3732165056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d312d390 msgr2=0x55d7d34e42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d312d390 0x55d7d34e42e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800f000 tx=0x7f0b180046c0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3732165056 shutdown_connections 2024-05-21T15:22:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d312d390 0x55d7d34e42e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/3732165056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d31044d0 0x55d7d34da3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3732165056 >> 172.21.15.78:0/3732165056 conn(0x55d7d31214e0 msgr2=0x55d7d34cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3732165056 shutdown_connections 2024-05-21T15:22:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.776+0000 7f0b3be3c740 1 -- 172.21.15.78:0/3732165056 wait complete. 2024-05-21T15:22:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 1 -- start start 2024-05-21T15:22:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 1 Processor -- start 2024-05-21T15:22:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d312d390 0x55d7d34c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301addc0 con 0x55d7d31044d0 2024-05-21T15:22:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301ad6f0 con 0x55d7d310c6b0 2024-05-21T15:22:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b301aa930 con 0x55d7d312d390 2024-05-21T15:22:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b3be3c740 10 client.0 _ll_register_callbacks cb 0x7f0b180099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52834/0 (socket says 172.21.15.78:52834) 2024-05-21T15:22:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b27fff640 1 -- 172.21.15.78:0/622436104 learned_addr learned my addr 172.21.15.78:0/622436104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b27fff640 1 -- 172.21.15.78:0/622436104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d312d390 msgr2=0x55d7d34c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b27fff640 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d312d390 0x55d7d34c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b27fff640 1 -- 172.21.15.78:0/622436104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 msgr2=0x55d7d34c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.777+0000 7f0b27fff640 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b27fff640 1 -- 172.21.15.78:0/622436104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7d34de950 con 0x55d7d31044d0 2024-05-21T15:22:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b27fff640 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34c74b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800a010 tx=0x7f0b180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5e3a0ea8f4924b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b15ffb640 1 -- 172.21.15.78:0/622436104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18035040 con 0x55d7d31044d0 2024-05-21T15:22:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b15ffb640 1 -- 172.21.15.78:0/622436104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f0b18010b60 con 0x55d7d31044d0 2024-05-21T15:22:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 1 -- 172.21.15.78:0/622436104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b1800e3b0 con 0x55d7d31044d0 2024-05-21T15:22:16.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b15ffb640 1 -- 172.21.15.78:0/622436104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0b18021030 con 0x55d7d31044d0 2024-05-21T15:22:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 10 client.17391 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 1 -- 172.21.15.78:0/622436104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d7d3553310 con 0x55d7d31044d0 2024-05-21T15:22:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b15ffb640 1 -- 172.21.15.78:0/622436104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1800e3b0 con 0x55d7d31044d0 2024-05-21T15:22:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b15ffb640 1 -- 172.21.15.78:0/622436104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0b180048f0 con 0x55d7d31044d0 2024-05-21T15:22:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 10 client.17391 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 10 client.17391 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:22:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 20 client.17391 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55d7d30c13a0 0x55d7d3555930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 4 client.17391 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.778+0000 7f0b3be3c740 1 -- 172.21.15.78:0/622436104 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7d34396c0 con 0x55d7d30c13a0 2024-05-21T15:22:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55d7d30c13a0 0x55d7d3555930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b2e59c640 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55d7d30c13a0 0x55d7d3555930 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=239c4b5cc5bec610 server_cookie=e352c406903fb8b1 in_seq=0 out_seq=0 2024-05-21T15:22:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b15ffb640 10 client.17391 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:22:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b15ffb640 1 -- 172.21.15.78:0/622436104 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7d34396c0 con 0x55d7d30c13a0 2024-05-21T15:22:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b15ffb640 10 client.17391 handle_command_reply: tid=0 2024-05-21T15:22:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 client.17391 shutdown 2024-05-21T15:22:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 -- 172.21.15.78:0/622436104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 msgr2=0x55d7d34c74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34c74b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800a010 tx=0x7f0b180107a0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 -- 172.21.15.78:0/622436104 shutdown_connections 2024-05-21T15:22:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55d7d30c13a0 0x55d7d3555930 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d7d310c6b0 0x55d7d34c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d312d390 0x55d7d34c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 --2- 172.21.15.78:0/622436104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d7d31044d0 0x55d7d34c74b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 -- 172.21.15.78:0/622436104 >> 172.21.15.78:0/622436104 conn(0x55d7d31214e0 msgr2=0x55d7d34cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b15ffb640 0 client.17391 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:22:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 -- 172.21.15.78:0/622436104 shutdown_connections 2024-05-21T15:22:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.779+0000 7f0b3be3c740 1 -- 172.21.15.78:0/622436104 wait complete. 2024-05-21T15:22:16.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b3be3c740 20 client.17391 trim_cache size 0 max 16384 2024-05-21T15:22:16.800 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 -- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0b1c0a2d00 msgr2=0x7f0b1c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0b1c0a2d00 0x7f0b1c0a5210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b2809bfa0 tx=0x7f0b2800efd0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 -- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b3010c350 msgr2=0x7f0b3019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b3010c350 0x7f0b3019cf90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800eb10 tx=0x7f0b2800c2d0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 -- 172.21.15.78:0/1270690225 shutdown_connections 2024-05-21T15:22:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0b1c0a2d00 0x7f0b1c0a5210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b3010c350 0x7f0b3019cf90 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301079f0 0x7f0b30198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 --2- 172.21.15.78:0/1270690225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b30102210 0x7f0b30198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 -- 172.21.15.78:0/1270690225 >> 172.21.15.78:0/1270690225 conn(0x7f0b30076830 msgr2=0x7f0b301030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 -- 172.21.15.78:0/1270690225 shutdown_connections 2024-05-21T15:22:16.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.780+0000 7f0b34f8f640 1 -- 172.21.15.78:0/1270690225 wait complete. 2024-05-21T15:22:16.803 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:22:16.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 -- 172.21.15.78:0/2228635657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c010d520 msgr2=0x7fb3c010f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/2228635657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c010d520 0x7fb3c010f940 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009ca0 tx=0x7fb3b802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 -- 172.21.15.78:0/2228635657 shutdown_connections 2024-05-21T15:22:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/2228635657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c010d520 0x7fb3c010f940 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/2228635657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0101020 0x7fb3c0101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/2228635657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3c0108b70 0x7fb3c0100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 -- 172.21.15.78:0/2228635657 >> 172.21.15.78:0/2228635657 conn(0x7fb3c00768d0 msgr2=0x7fb3c0076d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 -- 172.21.15.78:0/2228635657 shutdown_connections 2024-05-21T15:22:16.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.912+0000 7fb3c64a2640 1 -- 172.21.15.78:0/2228635657 wait complete. 2024-05-21T15:22:16.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.913+0000 7fb3c64a2640 1 Processor -- start 2024-05-21T15:22:16.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.913+0000 7fb3c64a2640 1 -- start start 2024-05-21T15:22:16.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.913+0000 7fb3c64a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0101020 0x7fb3c0199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.913+0000 7fb3c64a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c0108b70 0x7fb3c0199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.913+0000 7fb3c64a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3c010d520 0x7fb3c019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.913+0000 7fb3c64a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c0112e30 con 0x7fb3c0108b70 2024-05-21T15:22:16.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.913+0000 7fb3c64a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c0112cb0 con 0x7fb3c010d520 2024-05-21T15:22:16.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.913+0000 7fb3c64a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c0112fb0 con 0x7fb3c0101020 2024-05-21T15:22:16.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c0108b70 0x7fb3c0199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c0108b70 0x7fb3c0199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52852/0 (socket says 172.21.15.78:52852) 2024-05-21T15:22:16.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/754123287 learned_addr learned my addr 172.21.15.78:0/754123287 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3c4a3f640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3c010d520 0x7fb3c019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bffff640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0101020 0x7fb3c0199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/754123287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0101020 msgr2=0x7fb3c0199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0101020 0x7fb3c0199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/754123287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3c010d520 msgr2=0x7fb3c019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3c010d520 0x7fb3c019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/754123287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c019e530 con 0x7fb3c0108b70 2024-05-21T15:22:16.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c0108b70 0x7fb3c0199cb0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400d950 tx=0x7fb3b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bd7fa640 1 -- 172.21.15.78:0/754123287 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4017070 con 0x7fb3c0108b70 2024-05-21T15:22:16.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bd7fa640 1 -- 172.21.15.78:0/754123287 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b4009c30 con 0x7fb3c0108b70 2024-05-21T15:22:16.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3c64a2640 1 -- 172.21.15.78:0/754123287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c019e820 con 0x7fb3c0108b70 2024-05-21T15:22:16.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3bd7fa640 1 -- 172.21.15.78:0/754123287 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4010460 con 0x7fb3c0108b70 2024-05-21T15:22:16.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.914+0000 7fb3c64a2640 1 -- 172.21.15.78:0/754123287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c019ed70 con 0x7fb3c0108b70 2024-05-21T15:22:16.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3cd423740 1 Processor -- start 2024-05-21T15:22:16.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bd7fa640 1 -- 172.21.15.78:0/754123287 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb3b4010600 con 0x7fb3c0108b70 2024-05-21T15:22:16.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3cd423740 1 -- start start 2024-05-21T15:22:16.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 0x55acc9512d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc914b4d0 0x55acc9513260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc95137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3cd423740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55acc9478c40 con 0x55acc914b4d0 2024-05-21T15:22:16.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3cd423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55acc9420580 con 0x55acc9174390 2024-05-21T15:22:16.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3cd423740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55acc9474c00 con 0x55acc91536b0 2024-05-21T15:22:16.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc914b4d0 0x55acc9513260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc914b4d0 0x55acc9513260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52860/0 (socket says 172.21.15.78:52860) 2024-05-21T15:22:16.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bffff640 1 -- 172.21.15.78:0/1729925914 learned_addr learned my addr 172.21.15.78:0/1729925914 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3c4a3f640 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 0x55acc9512d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc95137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bd7fa640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3a80a2d00 0x7fb3a80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bffff640 1 -- 172.21.15.78:0/1729925914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 msgr2=0x55acc95137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bffff640 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc95137a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bffff640 1 -- 172.21.15.78:0/1729925914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 msgr2=0x55acc9512d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bffff640 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 0x55acc9512d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bffff640 1 -- 172.21.15.78:0/1729925914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc918c3f0 con 0x55acc914b4d0 2024-05-21T15:22:16.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.917+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc95137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:16.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.918+0000 7fb3c4a3f640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3a80a2d00 0x7fb3a80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.918+0000 7fb3bd7fa640 1 -- 172.21.15.78:0/754123287 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb3b40cf100 con 0x7fb3c0108b70 2024-05-21T15:22:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.918+0000 7fb3bffff640 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc914b4d0 0x55acc9513260 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000abc0 tx=0x7fb3b000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99973c1ec97da70d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.918+0000 7fb3a67fc640 1 -- 172.21.15.78:0/1729925914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0005220 con 0x55acc914b4d0 2024-05-21T15:22:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.918+0000 7fb3a67fc640 1 -- 172.21.15.78:0/1729925914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b0004c80 con 0x55acc914b4d0 2024-05-21T15:22:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.918+0000 7fb3a67fc640 1 -- 172.21.15.78:0/1729925914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b00187c0 con 0x55acc914b4d0 2024-05-21T15:22:16.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 -- 172.21.15.78:0/1729925914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc914b4d0 msgr2=0x55acc9513260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3c4a3f640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3a80a2d00 0x7fb3a80a5210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8005e30 tx=0x7fb3b80095c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc914b4d0 0x55acc9513260 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000abc0 tx=0x7fb3b000f790 comp rx=0 tx=0).stop 2024-05-21T15:22:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 -- 172.21.15.78:0/1729925914 shutdown_connections 2024-05-21T15:22:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc95137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc914b4d0 0x55acc9513260 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 --2- 172.21.15.78:0/1729925914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 0x55acc9512d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 -- 172.21.15.78:0/1729925914 >> 172.21.15.78:0/1729925914 conn(0x55acc91684e0 msgr2=0x55acc950f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 -- 172.21.15.78:0/1729925914 shutdown_connections 2024-05-21T15:22:16.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 -- 172.21.15.78:0/1729925914 wait complete. 2024-05-21T15:22:16.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 -- start start 2024-05-21T15:22:16.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.919+0000 7fb3cd423740 1 Processor -- start 2024-05-21T15:22:16.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc914b4d0 0x55acc951d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc951d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 0x55acc9523460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3cd423740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55acc951ffb0 con 0x55acc91536b0 2024-05-21T15:22:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3cd423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55acc9433c40 con 0x55acc9174390 2024-05-21T15:22:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3cd423740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55acc9520130 con 0x55acc914b4d0 2024-05-21T15:22:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3cd423740 10 client.0 _ll_register_callbacks cb 0x55acc918bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc951d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc951d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52868/0 (socket says 172.21.15.78:52868) 2024-05-21T15:22:16.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/2345146894 learned_addr learned my addr 172.21.15.78:0/2345146894 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bffff640 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc914b4d0 0x55acc951d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3c4a3f640 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 0x55acc9523460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/2345146894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc914b4d0 msgr2=0x55acc951d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc914b4d0 0x55acc951d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/2345146894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 msgr2=0x55acc9523460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 0x55acc9523460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/2345146894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc92567d0 con 0x55acc91536b0 2024-05-21T15:22:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc951d920 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40ccdf0 tx=0x7fb3b4099ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934f3db2217c4e19 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb39f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb39f7fe640 1 -- 172.21.15.78:0/2345146894 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4009de0 con 0x55acc91536b0 2024-05-21T15:22:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb39f7fe640 1 -- 172.21.15.78:0/2345146894 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb3b409aa10 con 0x55acc91536b0 2024-05-21T15:22:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.920+0000 7fb3cd423740 1 -- 172.21.15.78:0/2345146894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b0004c80 con 0x55acc91536b0 2024-05-21T15:22:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb39f7fe640 1 -- 172.21.15.78:0/2345146894 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb3b409bcc0 con 0x55acc91536b0 2024-05-21T15:22:16.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3cd423740 10 client.17403 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3cd423740 1 -- 172.21.15.78:0/2345146894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55acc952c770 con 0x55acc91536b0 2024-05-21T15:22:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb39f7fe640 1 -- 172.21.15.78:0/2345146894 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b0004c80 con 0x55acc91536b0 2024-05-21T15:22:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb39f7fe640 1 -- 172.21.15.78:0/2345146894 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b40afae0 con 0x55acc91536b0 2024-05-21T15:22:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3cd423740 10 client.17403 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3cd423740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3cd423740 20 client.17403 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3cd423740 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55acc9132f60 0x55acc952fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3cd423740 4 client.17403 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3cd423740 1 -- 172.21.15.78:0/2345146894 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acc9486370 con 0x55acc9132f60 2024-05-21T15:22:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.921+0000 7fb3bffff640 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55acc9132f60 0x55acc952fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.922+0000 7fb3bffff640 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55acc9132f60 0x55acc952fc40 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57ccaab59c85999f server_cookie=b40e3cab552c11f5 in_seq=0 out_seq=0 2024-05-21T15:22:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.922+0000 7fb39f7fe640 10 client.17403 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.922+0000 7fb39f7fe640 1 -- 172.21.15.78:0/2345146894 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55acc9486370 con 0x55acc9132f60 2024-05-21T15:22:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.922+0000 7fb39f7fe640 10 client.17403 handle_command_reply: tid=0 2024-05-21T15:22:16.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.922+0000 7fb3cd423740 1 client.17403 shutdown 2024-05-21T15:22:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 -- 172.21.15.78:0/2345146894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 msgr2=0x55acc951d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc951d920 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40ccdf0 tx=0x7fb3b4099ad0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 -- 172.21.15.78:0/2345146894 shutdown_connections 2024-05-21T15:22:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55acc9132f60 0x55acc952fc40 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc9174390 0x55acc9523460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc951d920 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb39f7fe640 0 client.17403 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:16.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 --2- 172.21.15.78:0/2345146894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc914b4d0 0x55acc951d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 -- 172.21.15.78:0/2345146894 >> 172.21.15.78:0/2345146894 conn(0x55acc91684e0 msgr2=0x55acc9514a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 -- 172.21.15.78:0/2345146894 shutdown_connections 2024-05-21T15:22:16.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 1 -- 172.21.15.78:0/2345146894 wait complete. 2024-05-21T15:22:16.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.923+0000 7fb3cd423740 20 client.17403 trim_cache size 0 max 16384 2024-05-21T15:22:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3cd423740 1 Processor -- start 2024-05-21T15:22:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3cd423740 1 -- start start 2024-05-21T15:22:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9517050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc9523360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc9174390 0x55acc951c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3cd423740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01aec00 con 0x55acc91536b0 2024-05-21T15:22:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3cd423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01b0740 con 0x55acc914b4d0 2024-05-21T15:22:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3cd423740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01af2d0 con 0x55acc9174390 2024-05-21T15:22:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc9523360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc9523360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52882/0 (socket says 172.21.15.78:52882) 2024-05-21T15:22:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.936+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/362356665 learned_addr learned my addr 172.21.15.78:0/362356665 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/362356665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc9174390 msgr2=0x55acc951c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3bffff640 1 --2- 172.21.15.78:0/362356665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9517050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/362356665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc9174390 0x55acc951c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/362356665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 msgr2=0x55acc9517050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/362356665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9517050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3bf7fe640 1 -- 172.21.15.78:0/362356665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc9518d70 con 0x55acc91536b0 2024-05-21T15:22:16.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/362356665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc9523360 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4099870 tx=0x7fb3b409ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf6fdca8344bdde server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3a5ffb640 1 -- 172.21.15.78:0/362356665 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4009de0 con 0x55acc91536b0 2024-05-21T15:22:16.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3a5ffb640 1 -- 172.21.15.78:0/362356665 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b409b5c0 con 0x55acc91536b0 2024-05-21T15:22:16.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.937+0000 7fb3a5ffb640 1 -- 172.21.15.78:0/362356665 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4097310 con 0x55acc91536b0 2024-05-21T15:22:16.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 -- 172.21.15.78:0/362356665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 msgr2=0x55acc9523360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 --2- 172.21.15.78:0/362356665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc9523360 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4099870 tx=0x7fb3b409ad60 comp rx=0 tx=0).stop 2024-05-21T15:22:16.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 -- 172.21.15.78:0/362356665 shutdown_connections 2024-05-21T15:22:16.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 --2- 172.21.15.78:0/362356665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc9174390 0x55acc951c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 --2- 172.21.15.78:0/362356665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc91536b0 0x55acc9523360 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 --2- 172.21.15.78:0/362356665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9517050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 -- 172.21.15.78:0/362356665 >> 172.21.15.78:0/362356665 conn(0x55acc91684e0 msgr2=0x55acc951baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 -- 172.21.15.78:0/362356665 shutdown_connections 2024-05-21T15:22:16.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 -- 172.21.15.78:0/362356665 wait complete. 2024-05-21T15:22:16.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 -- start start 2024-05-21T15:22:16.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.938+0000 7fb3cd423740 1 Processor -- start 2024-05-21T15:22:16.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9524a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc9524f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc9174390 0x55acc9595fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01b0070 con 0x55acc9174390 2024-05-21T15:22:16.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01af9a0 con 0x55acc914b4d0 2024-05-21T15:22:16.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c01acbe0 con 0x55acc91536b0 2024-05-21T15:22:16.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 10 client.0 _ll_register_callbacks cb 0x7fb3b40971a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:16.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc9174390 0x55acc9595fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc9174390 0x55acc9595fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52884/0 (socket says 172.21.15.78:52884) 2024-05-21T15:22:16.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 -- 172.21.15.78:0/4199229096 learned_addr learned my addr 172.21.15.78:0/4199229096 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:16.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3bf7fe640 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc9524f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3bffff640 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9524a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 -- 172.21.15.78:0/4199229096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 msgr2=0x55acc9524f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc9524f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 -- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 msgr2=0x55acc9524a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:16.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9524a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 -- 172.21.15.78:0/4199229096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc9525670 con 0x55acc9174390 2024-05-21T15:22:16.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3bffff640 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9524a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:16.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3c4a3f640 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc9174390 0x55acc9595fa0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009ca0 tx=0x7fb3b8004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5735e16a47ee0857 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:16.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb39f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:16.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb39f7fe640 1 -- 172.21.15.78:0/4199229096 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8004000 con 0x55acc9174390 2024-05-21T15:22:16.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb39f7fe640 1 -- 172.21.15.78:0/4199229096 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fb3b80041a0 con 0x55acc9174390 2024-05-21T15:22:16.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 1 -- 172.21.15.78:0/4199229096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b409b5c0 con 0x55acc9174390 2024-05-21T15:22:16.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb39f7fe640 1 -- 172.21.15.78:0/4199229096 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb3b80043a0 con 0x55acc9174390 2024-05-21T15:22:16.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 10 client.17409 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:16.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.939+0000 7fb3cd423740 1 -- 172.21.15.78:0/4199229096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55acc9597560 con 0x55acc9174390 2024-05-21T15:22:16.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb39f7fe640 1 -- 172.21.15.78:0/4199229096 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b409b5c0 con 0x55acc9174390 2024-05-21T15:22:16.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb39f7fe640 1 -- 172.21.15.78:0/4199229096 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b80305c0 con 0x55acc9174390 2024-05-21T15:22:16.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb3cd423740 10 client.17409 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:16.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb3cd423740 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:16.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb3cd423740 20 client.17409 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:16.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb3cd423740 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55acc91083a0 0x55acc9599cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:16.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb3cd423740 4 client.17409 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:16.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb3cd423740 1 -- 172.21.15.78:0/4199229096 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55acc94806c0 con 0x55acc91083a0 2024-05-21T15:22:16.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.940+0000 7fb3bffff640 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55acc91083a0 0x55acc9599cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:16.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb39f7fe640 10 client.17409 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:16.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3bffff640 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55acc91083a0 0x55acc9599cd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdde3292161f6c64 server_cookie=2a58e7c63c4dfff8 in_seq=0 out_seq=0 2024-05-21T15:22:16.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb39f7fe640 1 -- 172.21.15.78:0/4199229096 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55acc94806c0 con 0x55acc91083a0 2024-05-21T15:22:16.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb39f7fe640 10 client.17409 handle_command_reply: tid=0 2024-05-21T15:22:16.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 client.17409 shutdown 2024-05-21T15:22:16.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 -- 172.21.15.78:0/4199229096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc9174390 msgr2=0x55acc9595fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc9174390 0x55acc9595fa0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009ca0 tx=0x7fb3b8004a80 comp rx=0 tx=0).stop 2024-05-21T15:22:16.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 -- 172.21.15.78:0/4199229096 shutdown_connections 2024-05-21T15:22:16.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55acc91083a0 0x55acc9599cd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55acc9174390 0x55acc9595fa0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acc91536b0 0x55acc9524f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb39f7fe640 0 client.17409 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:16.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 --2- 172.21.15.78:0/4199229096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55acc914b4d0 0x55acc9524a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 -- 172.21.15.78:0/4199229096 >> 172.21.15.78:0/4199229096 conn(0x55acc91684e0 msgr2=0x55acc951e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 -- 172.21.15.78:0/4199229096 shutdown_connections 2024-05-21T15:22:16.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 1 -- 172.21.15.78:0/4199229096 wait complete. 2024-05-21T15:22:16.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.941+0000 7fb3cd423740 20 client.17409 trim_cache size 0 max 16384 2024-05-21T15:22:16.965 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 -- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3a80a2d00 msgr2=0x7fb3a80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3a80a2d00 0x7fb3a80a5210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8005e30 tx=0x7fb3b80095c0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 -- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c0108b70 msgr2=0x7fb3c0199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c0108b70 0x7fb3c0199cb0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400d950 tx=0x7fb3b400de10 comp rx=0 tx=0).stop 2024-05-21T15:22:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 -- 172.21.15.78:0/754123287 shutdown_connections 2024-05-21T15:22:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3c010d520 0x7fb3c019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3a80a2d00 0x7fb3a80a5210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3c0108b70 0x7fb3c0199cb0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 --2- 172.21.15.78:0/754123287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0101020 0x7fb3c0199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 -- 172.21.15.78:0/754123287 >> 172.21.15.78:0/754123287 conn(0x7fb3c00768d0 msgr2=0x7fb3c01094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 -- 172.21.15.78:0/754123287 shutdown_connections 2024-05-21T15:22:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:16.942+0000 7fb3c64a2640 1 -- 172.21.15.78:0/754123287 wait complete. 2024-05-21T15:22:17.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:16 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1355123647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:22:17.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:16 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1355123647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:22:17.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:16 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1355123647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:22:17.969 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:22:17.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:22:17.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:22:18.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 -- 172.21.15.78:0/170338550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108ac0 msgr2=0x7f86d0102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 --2- 172.21.15.78:0/170338550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108ac0 0x7f86d0102660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009900 tx=0x7f86cc02a410 comp rx=0 tx=0).stop 2024-05-21T15:22:18.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 -- 172.21.15.78:0/170338550 shutdown_connections 2024-05-21T15:22:18.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 --2- 172.21.15.78:0/170338550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86d0108f60 0x7f86d0102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 --2- 172.21.15.78:0/170338550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d0103210 0x7f86d010f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 --2- 172.21.15.78:0/170338550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108ac0 0x7f86d0102660 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 -- 172.21.15.78:0/170338550 >> 172.21.15.78:0/170338550 conn(0x7f86d00fe1c0 msgr2=0x7f86d0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 -- 172.21.15.78:0/170338550 shutdown_connections 2024-05-21T15:22:18.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.097+0000 7f86d7709640 1 -- 172.21.15.78:0/170338550 wait complete. 2024-05-21T15:22:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.098+0000 7f86d7709640 1 Processor -- start 2024-05-21T15:22:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.098+0000 7f86d7709640 1 -- start start 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.098+0000 7f86d7709640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86d0103210 0x7f86d01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.098+0000 7f86d7709640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d0108ac0 0x7f86d0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.098+0000 7f86d7709640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108f60 0x7f86d019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.098+0000 7f86d7709640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86d0112e00 con 0x7f86d0108f60 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.098+0000 7f86d7709640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86d0112c80 con 0x7f86d0103210 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.098+0000 7f86d7709640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86d0112f80 con 0x7f86d0108ac0 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108f60 0x7f86d019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108f60 0x7f86d019dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52904/0 (socket says 172.21.15.78:52904) 2024-05-21T15:22:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 -- 172.21.15.78:0/2527498430 learned_addr learned my addr 172.21.15.78:0/2527498430 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d0108ac0 0x7f86d0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d54a5640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86d0103210 0x7f86d01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 -- 172.21.15.78:0/2527498430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d0108ac0 msgr2=0x7f86d0199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d0108ac0 0x7f86d0199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 -- 172.21.15.78:0/2527498430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86d0103210 msgr2=0x7f86d01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86d0103210 0x7f86d01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 -- 172.21.15.78:0/2527498430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d019e5a0 con 0x7f86d0108f60 2024-05-21T15:22:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d5ca6640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108f60 0x7f86d019dfe0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f86c800cb00 tx=0x7f86c800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86c67fc640 1 -- 172.21.15.78:0/2527498430 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8004b20 con 0x7f86d0108f60 2024-05-21T15:22:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86c67fc640 1 -- 172.21.15.78:0/2527498430 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c8004cc0 con 0x7f86d0108f60 2024-05-21T15:22:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d7709640 1 -- 172.21.15.78:0/2527498430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d019e890 con 0x7f86d0108f60 2024-05-21T15:22:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86c67fc640 1 -- 172.21.15.78:0/2527498430 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800f740 con 0x7f86d0108f60 2024-05-21T15:22:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.099+0000 7f86d7709640 1 -- 172.21.15.78:0/2527498430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d019ede0 con 0x7f86d0108f60 2024-05-21T15:22:18.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.102+0000 7f86de5b1740 1 Processor -- start 2024-05-21T15:22:18.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.102+0000 7f86c67fc640 1 -- 172.21.15.78:0/2527498430 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f86c800f8e0 con 0x7f86d0108f60 2024-05-21T15:22:18.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.102+0000 7f86de5b1740 1 -- start start 2024-05-21T15:22:18.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d155510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d155a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d155f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86de5b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55814d0ba990 con 0x55814cdb6410 2024-05-21T15:22:18.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86de5b1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55814d0625b0 con 0x55814cd95730 2024-05-21T15:22:18.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86de5b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55814d0b6950 con 0x55814cd8d550 2024-05-21T15:22:18.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52918/0 (socket says 172.21.15.78:52918) 2024-05-21T15:22:18.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 -- 172.21.15.78:0/1130066994 learned_addr learned my addr 172.21.15.78:0/1130066994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d54a5640 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86c67fc640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f86bc0a2d00 0x7f86bc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d5ca6640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f86bc0a2d00 0x7f86bc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 -- 172.21.15.78:0/1130066994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 msgr2=0x55814d155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 -- 172.21.15.78:0/1130066994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 msgr2=0x55814d155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 -- 172.21.15.78:0/1130066994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814ce967e0 con 0x55814cdb6410 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86c67fc640 1 -- 172.21.15.78:0/2527498430 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f86c80cf440 con 0x7f86d0108f60 2024-05-21T15:22:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d155510 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f86c0013510 tx=0x7f86c000a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537ded0b92b4c410 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.103+0000 7f86bb7fe640 1 -- 172.21.15.78:0/1130066994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0019030 con 0x55814cdb6410 2024-05-21T15:22:18.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.104+0000 7f86bb7fe640 1 -- 172.21.15.78:0/1130066994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c0005500 con 0x55814cdb6410 2024-05-21T15:22:18.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.104+0000 7f86bb7fe640 1 -- 172.21.15.78:0/1130066994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0004d90 con 0x55814cdb6410 2024-05-21T15:22:18.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.104+0000 7f86de5b1740 1 -- 172.21.15.78:0/1130066994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 msgr2=0x55814d155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.104+0000 7f86d5ca6640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f86bc0a2d00 0x7f86bc0a5210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ccbe0 tx=0x7f86c8097cd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.104+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d155510 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f86c0013510 tx=0x7f86c000a700 comp rx=0 tx=0).stop 2024-05-21T15:22:18.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 -- 172.21.15.78:0/1130066994 shutdown_connections 2024-05-21T15:22:18.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1130066994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d155510 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 -- 172.21.15.78:0/1130066994 >> 172.21.15.78:0/1130066994 conn(0x55814cdaa560 msgr2=0x55814d151dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 -- 172.21.15.78:0/1130066994 shutdown_connections 2024-05-21T15:22:18.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 -- 172.21.15.78:0/1130066994 wait complete. 2024-05-21T15:22:18.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 -- start start 2024-05-21T15:22:18.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 Processor -- start 2024-05-21T15:22:18.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d1640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.105+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d164610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86de5b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55814d166d80 con 0x55814cdb6410 2024-05-21T15:22:18.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86de5b1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55814d075b40 con 0x55814cd95730 2024-05-21T15:22:18.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86de5b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55814d166f00 con 0x55814cd8d550 2024-05-21T15:22:18.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86de5b1740 10 client.0 _ll_register_callbacks cb 0x55814d06a0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:18.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52922/0 (socket says 172.21.15.78:52922) 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 -- 172.21.15.78:0/3370928173 learned_addr learned my addr 172.21.15.78:0/3370928173 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d1640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d54a5640 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 -- 172.21.15.78:0/3370928173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 msgr2=0x55814d1640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d1640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 -- 172.21.15.78:0/3370928173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 msgr2=0x55814d164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 -- 172.21.15.78:0/3370928173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814cef6f70 con 0x55814cdb6410 2024-05-21T15:22:18.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a270 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005d50 tx=0x7f86c0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e02402e13802271 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86b8ff9640 1 -- 172.21.15.78:0/3370928173 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0019040 con 0x55814cdb6410 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86b8ff9640 1 -- 172.21.15.78:0/3370928173 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f86c00110d0 con 0x55814cdb6410 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86de5b1740 1 -- 172.21.15.78:0/3370928173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c0005500 con 0x55814cdb6410 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86b8ff9640 1 -- 172.21.15.78:0/3370928173 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f86c0004b10 con 0x55814cdb6410 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86de5b1740 10 client.17421 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.106+0000 7f86de5b1740 1 -- 172.21.15.78:0/3370928173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55814cd8e170 con 0x55814cdb6410 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86b8ff9640 1 -- 172.21.15.78:0/3370928173 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c0005500 con 0x55814cdb6410 2024-05-21T15:22:18.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86b8ff9640 1 -- 172.21.15.78:0/3370928173 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f86c000f420 con 0x55814cdb6410 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86de5b1740 10 client.17421 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86de5b1740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86de5b1740 20 client.17421 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55814cd751f0 0x55814d176470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86de5b1740 4 client.17421 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86de5b1740 1 -- 172.21.15.78:0/3370928173 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55814d0c80c0 con 0x55814cd751f0 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.107+0000 7f86d54a5640 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55814cd751f0 0x55814d176470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.108+0000 7f86b8ff9640 10 client.17421 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:18.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.108+0000 7f86d54a5640 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55814cd751f0 0x55814d176470 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c44b418741a508 server_cookie=de7ba1e79399f9c4 in_seq=0 out_seq=0 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.108+0000 7f86b8ff9640 1 -- 172.21.15.78:0/3370928173 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55814d0c80c0 con 0x55814cd751f0 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.108+0000 7f86b8ff9640 10 client.17421 handle_command_reply: tid=0 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.108+0000 7f86de5b1740 1 client.17421 shutdown 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 -- 172.21.15.78:0/3370928173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 msgr2=0x55814d16a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a270 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005d50 tx=0x7f86c0004970 comp rx=0 tx=0).stop 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 -- 172.21.15.78:0/3370928173 shutdown_connections 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55814cd751f0 0x55814d176470 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86b8ff9640 0 client.17421 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:18.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a270 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3370928173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d1640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 -- 172.21.15.78:0/3370928173 >> 172.21.15.78:0/3370928173 conn(0x55814cdaa560 msgr2=0x55814d1553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 -- 172.21.15.78:0/3370928173 shutdown_connections 2024-05-21T15:22:18.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 1 -- 172.21.15.78:0/3370928173 wait complete. 2024-05-21T15:22:18.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.109+0000 7f86de5b1740 20 client.17421 trim_cache size 0 max 16384 2024-05-21T15:22:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86de5b1740 1 Processor -- start 2024-05-21T15:22:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86de5b1740 1 -- start start 2024-05-21T15:22:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d164180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86de5b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01ac9f0 con 0x55814cdb6410 2024-05-21T15:22:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86de5b1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01ae530 con 0x55814cd95730 2024-05-21T15:22:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86de5b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01ad0c0 con 0x55814cd8d550 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52932/0 (socket says 172.21.15.78:52932) 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 -- 172.21.15.78:0/3974678678 learned_addr learned my addr 172.21.15.78:0/3974678678 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d54a5640 1 --2- 172.21.15.78:0/3974678678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 -- 172.21.15.78:0/3974678678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 msgr2=0x55814d164180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/3974678678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d164180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 -- 172.21.15.78:0/3974678678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 msgr2=0x55814d16a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/3974678678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 -- 172.21.15.78:0/3974678678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814d1668b0 con 0x55814cdb6410 2024-05-21T15:22:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/3974678678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a6b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005d20 tx=0x7f86c0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5291dc828235ef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.127+0000 7f86baffd640 1 -- 172.21.15.78:0/3974678678 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0019070 con 0x55814cdb6410 2024-05-21T15:22:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86baffd640 1 -- 172.21.15.78:0/3974678678 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c000ac20 con 0x55814cdb6410 2024-05-21T15:22:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86baffd640 1 -- 172.21.15.78:0/3974678678 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c000fa10 con 0x55814cdb6410 2024-05-21T15:22:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 -- 172.21.15.78:0/3974678678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 msgr2=0x55814d16a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3974678678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a6b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005d20 tx=0x7f86c0005f90 comp rx=0 tx=0).stop 2024-05-21T15:22:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 -- 172.21.15.78:0/3974678678 shutdown_connections 2024-05-21T15:22:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3974678678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3974678678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cdb6410 0x55814d16a6b0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 --2- 172.21.15.78:0/3974678678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d164180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 -- 172.21.15.78:0/3974678678 >> 172.21.15.78:0/3974678678 conn(0x55814cdaa560 msgr2=0x55814d157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 -- 172.21.15.78:0/3974678678 shutdown_connections 2024-05-21T15:22:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.128+0000 7f86de5b1740 1 -- 172.21.15.78:0/3974678678 wait complete. 2024-05-21T15:22:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 1 -- start start 2024-05-21T15:22:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 1 Processor -- start 2024-05-21T15:22:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d170a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cd4a630 0x55814d16e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01ade60 con 0x55814cd4a630 2024-05-21T15:22:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01ad790 con 0x55814cd95730 2024-05-21T15:22:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86d01aa9f0 con 0x55814cd8d550 2024-05-21T15:22:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86de5b1740 10 client.0 _ll_register_callbacks cb 0x7f86cc0314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86d4ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cd4a630 0x55814d16e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86d4ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cd4a630 0x55814d16e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52946/0 (socket says 172.21.15.78:52946) 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86d4ca4640 1 -- 172.21.15.78:0/1290010787 learned_addr learned my addr 172.21.15.78:0/1290010787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d170a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.129+0000 7f86d54a5640 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86d4ca4640 1 -- 172.21.15.78:0/1290010787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 msgr2=0x55814d170a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d170a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86d4ca4640 1 -- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 msgr2=0x55814d16ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86d4ca4640 1 -- 172.21.15.78:0/1290010787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814d16e870 con 0x55814cd4a630 2024-05-21T15:22:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86d54a5640 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86d4ca4640 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cd4a630 0x55814d16e330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f86c000f8c0 tx=0x7f86c0004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47edd29d7d86ca68 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86b8ff9640 1 -- 172.21.15.78:0/1290010787 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0019070 con 0x55814cd4a630 2024-05-21T15:22:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86b8ff9640 1 -- 172.21.15.78:0/1290010787 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f86c0005540 con 0x55814cd4a630 2024-05-21T15:22:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 1 -- 172.21.15.78:0/1290010787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c000ac20 con 0x55814cd4a630 2024-05-21T15:22:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86b8ff9640 1 -- 172.21.15.78:0/1290010787 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f86c0005740 con 0x55814cd4a630 2024-05-21T15:22:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 10 client.17427 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 1 -- 172.21.15.78:0/1290010787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55814d1de220 con 0x55814cd4a630 2024-05-21T15:22:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86b8ff9640 1 -- 172.21.15.78:0/1290010787 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c000ac20 con 0x55814cd4a630 2024-05-21T15:22:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86b8ff9640 1 -- 172.21.15.78:0/1290010787 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f86c0035030 con 0x55814cd4a630 2024-05-21T15:22:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 10 client.17427 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 20 client.17427 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55814d1de560 0x55814d1e0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 4 client.17427 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:22:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.130+0000 7f86de5b1740 1 -- 172.21.15.78:0/1290010787 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55814d0c2410 con 0x55814d1de560 2024-05-21T15:22:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.131+0000 7f86d54a5640 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55814d1de560 0x55814d1e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.131+0000 7f86d54a5640 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55814d1de560 0x55814d1e0980 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc560767b6bdc20 server_cookie=846dbda6efc79aa8 in_seq=0 out_seq=0 2024-05-21T15:22:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.131+0000 7f86b8ff9640 10 client.17427 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86b8ff9640 1 -- 172.21.15.78:0/1290010787 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55814d0c2410 con 0x55814d1de560 2024-05-21T15:22:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86b8ff9640 10 client.17427 handle_command_reply: tid=0 2024-05-21T15:22:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 client.17427 shutdown 2024-05-21T15:22:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 -- 172.21.15.78:0/1290010787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cd4a630 msgr2=0x55814d16e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cd4a630 0x55814d16e330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f86c000f8c0 tx=0x7f86c0004a70 comp rx=0 tx=0).stop 2024-05-21T15:22:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 -- 172.21.15.78:0/1290010787 shutdown_connections 2024-05-21T15:22:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55814d1de560 0x55814d1e0980 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55814cd95730 0x55814d16ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55814cd4a630 0x55814d16e330 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 --2- 172.21.15.78:0/1290010787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55814cd8d550 0x55814d170a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 -- 172.21.15.78:0/1290010787 >> 172.21.15.78:0/1290010787 conn(0x55814cdaa560 msgr2=0x55814d157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86b8ff9640 0 client.17427 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 -- 172.21.15.78:0/1290010787 shutdown_connections 2024-05-21T15:22:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 1 -- 172.21.15.78:0/1290010787 wait complete. 2024-05-21T15:22:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.133+0000 7f86de5b1740 20 client.17427 trim_cache size 0 max 16384 2024-05-21T15:22:18.148 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:22:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:22:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "1d1fce06-4a05-4f04-a921-585ceca7262c", 2024-05-21T15:22:18.148 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:22:18.149 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:22:18.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 -- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f86bc0a2d00 msgr2=0x7f86bc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f86bc0a2d00 0x7f86bc0a5210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ccbe0 tx=0x7f86c8097cd0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 -- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108f60 msgr2=0x7f86d019dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108f60 0x7f86d019dfe0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f86c800cb00 tx=0x7f86c800a250 comp rx=0 tx=0).stop 2024-05-21T15:22:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 -- 172.21.15.78:0/2527498430 shutdown_connections 2024-05-21T15:22:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f86bc0a2d00 0x7f86bc0a5210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86d0108f60 0x7f86d019dfe0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86d0108ac0 0x7f86d0199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 --2- 172.21.15.78:0/2527498430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86d0103210 0x7f86d01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 -- 172.21.15.78:0/2527498430 >> 172.21.15.78:0/2527498430 conn(0x7f86d00fe1c0 msgr2=0x7f86d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 -- 172.21.15.78:0/2527498430 shutdown_connections 2024-05-21T15:22:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.134+0000 7f86d7709640 1 -- 172.21.15.78:0/2527498430 wait complete. 2024-05-21T15:22:18.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:22:18.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:22:18.176 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:17 smithi078 ceph-mon[31997]: pgmap v189: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T15:22:18.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:17 smithi172 ceph-mon[36402]: pgmap v189: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T15:22:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 -- 172.21.15.78:0/723660446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c101ab0 msgr2=0x7f673c101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 --2- 172.21.15.78:0/723660446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c101ab0 0x7f673c101ec0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6724009900 tx=0x7f672402a410 comp rx=0 tx=0).stop 2024-05-21T15:22:18.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 -- 172.21.15.78:0/723660446 shutdown_connections 2024-05-21T15:22:18.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 --2- 172.21.15.78:0/723660446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c10c340 0x7f673c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 --2- 172.21.15.78:0/723660446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c101ab0 0x7f673c101ec0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 --2- 172.21.15.78:0/723660446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c1079e0 0x7f673c101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 -- 172.21.15.78:0/723660446 >> 172.21.15.78:0/723660446 conn(0x7f673c0fd210 msgr2=0x7f673c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 -- 172.21.15.78:0/723660446 shutdown_connections 2024-05-21T15:22:18.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.265+0000 7f6740a61640 1 -- 172.21.15.78:0/723660446 wait complete. 2024-05-21T15:22:18.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f6740a61640 1 Processor -- start 2024-05-21T15:22:18.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f6740a61640 1 -- start start 2024-05-21T15:22:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f6740a61640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c101ab0 0x7f673c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f6740a61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c1079e0 0x7f673c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f6740a61640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c10c340 0x7f673c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f6740a61640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111d40 con 0x7f673c10c340 2024-05-21T15:22:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f6740a61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111bc0 con 0x7f673c1079e0 2024-05-21T15:22:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f6740a61640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f673c111ec0 con 0x7f673c101ab0 2024-05-21T15:22:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f673ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c10c340 0x7f673c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f673ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c10c340 0x7f673c19b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52968/0 (socket says 172.21.15.78:52968) 2024-05-21T15:22:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.266+0000 7f673ad9d640 1 -- 172.21.15.78:0/1511884591 learned_addr learned my addr 172.21.15.78:0/1511884591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f6731d9b640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c1079e0 0x7f673c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f673a59c640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c101ab0 0x7f673c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f673ad9d640 1 -- 172.21.15.78:0/1511884591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c101ab0 msgr2=0x7f673c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f673ad9d640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c101ab0 0x7f673c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f673ad9d640 1 -- 172.21.15.78:0/1511884591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c1079e0 msgr2=0x7f673c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f673ad9d640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c1079e0 0x7f673c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f673ad9d640 1 -- 172.21.15.78:0/1511884591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c1a5960 con 0x7f673c10c340 2024-05-21T15:22:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f673ad9d640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c10c340 0x7f673c19b0c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f673400eb10 tx=0x7f673400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f6733fff640 1 -- 172.21.15.78:0/1511884591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734004b20 con 0x7f673c10c340 2024-05-21T15:22:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f6733fff640 1 -- 172.21.15.78:0/1511884591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6734004cc0 con 0x7f673c10c340 2024-05-21T15:22:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f6740a61640 1 -- 172.21.15.78:0/1511884591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c1a5c50 con 0x7f673c10c340 2024-05-21T15:22:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f6733fff640 1 -- 172.21.15.78:0/1511884591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734010760 con 0x7f673c10c340 2024-05-21T15:22:18.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.267+0000 7f6740a61640 1 -- 172.21.15.78:0/1511884591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c1a6170 con 0x7f673c10c340 2024-05-21T15:22:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f6733fff640 1 -- 172.21.15.78:0/1511884591 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6734010900 con 0x7f673c10c340 2024-05-21T15:22:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f67479e5740 1 Processor -- start 2024-05-21T15:22:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f67479e5740 1 -- start start 2024-05-21T15:22:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f67479e5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e869d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f67479e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e86a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f67479e5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e86a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f67479e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c7e7cfd00 con 0x558c7e4a24d0 2024-05-21T15:22:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f67479e5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558c7e7774c0 con 0x558c7e4aa6b0 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f67479e5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c7e7cbcc0 con 0x558c7e4cb390 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f673a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e86a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f673a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e86a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52976/0 (socket says 172.21.15.78:52976) 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f673a59c640 1 -- 172.21.15.78:0/624352394 learned_addr learned my addr 172.21.15.78:0/624352394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.270+0000 7f6731d9b640 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e869d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f6731d9b640 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e86a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f6733fff640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67280a2d00 0x7f67280a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f673a59c640 1 -- 172.21.15.78:0/624352394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 msgr2=0x558c7e869d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f673a59c640 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e869d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f673a59c640 1 -- 172.21.15.78:0/624352394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 msgr2=0x558c7e86a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f673a59c640 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e86a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f673a59c640 1 -- 172.21.15.78:0/624352394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7e5ad7d0 con 0x558c7e4a24d0 2024-05-21T15:22:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f6731d9b640 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e86a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f673ad9d640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67280a2d00 0x7f67280a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f6733fff640 1 -- 172.21.15.78:0/1511884591 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f67340cfcb0 con 0x7f673c10c340 2024-05-21T15:22:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f673a59c640 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e86a270 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f672c009c10 tx=0x7f672c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980a5a293f253df server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f6730d99640 1 -- 172.21.15.78:0/624352394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c012450 con 0x558c7e4a24d0 2024-05-21T15:22:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f6730d99640 1 -- 172.21.15.78:0/624352394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c012ad0 con 0x558c7e4a24d0 2024-05-21T15:22:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.271+0000 7f6730d99640 1 -- 172.21.15.78:0/624352394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c00fb00 con 0x558c7e4a24d0 2024-05-21T15:22:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 -- 172.21.15.78:0/624352394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 msgr2=0x558c7e86a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e86a270 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f672c009c10 tx=0x7f672c0117b0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f673ad9d640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67280a2d00 0x7f67280a5210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6734098570 tx=0x7f6734005b40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 -- 172.21.15.78:0/624352394 shutdown_connections 2024-05-21T15:22:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e86a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e86a270 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 --2- 172.21.15.78:0/624352394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e869d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 -- 172.21.15.78:0/624352394 >> 172.21.15.78:0/624352394 conn(0x558c7e4bf4e0 msgr2=0x558c7e8664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 -- 172.21.15.78:0/624352394 shutdown_connections 2024-05-21T15:22:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.272+0000 7f67479e5740 1 -- 172.21.15.78:0/624352394 wait complete. 2024-05-21T15:22:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 1 -- start start 2024-05-21T15:22:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 1 Processor -- start 2024-05-21T15:22:18.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e875e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e8763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e874310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c7e877020 con 0x558c7e4a24d0 2024-05-21T15:22:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558c7e78ab90 con 0x558c7e4aa6b0 2024-05-21T15:22:18.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c7e8771a0 con 0x558c7e4cb390 2024-05-21T15:22:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f67479e5740 10 client.0 _ll_register_callbacks cb 0x558c7e7c6da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e875e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e875e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52988/0 (socket says 172.21.15.78:52988) 2024-05-21T15:22:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 -- 172.21.15.78:0/2271006361 learned_addr learned my addr 172.21.15.78:0/2271006361 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 -- 172.21.15.78:0/2271006361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 msgr2=0x558c7e874310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e874310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 -- 172.21.15.78:0/2271006361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 msgr2=0x558c7e8763d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e8763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 -- 172.21.15.78:0/2271006361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7e4e33f0 con 0x558c7e4a24d0 2024-05-21T15:22:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6731d9b640 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e875e90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6724006010 tx=0x7f672402ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4ddb9a2fe4ab98 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.273+0000 7f6721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f6721ffb640 1 -- 172.21.15.78:0/2271006361 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724035040 con 0x558c7e4a24d0 2024-05-21T15:22:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f6721ffb640 1 -- 172.21.15.78:0/2271006361 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f6724031bf0 con 0x558c7e4a24d0 2024-05-21T15:22:18.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 1 -- 172.21.15.78:0/2271006361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672c012ad0 con 0x558c7e4a24d0 2024-05-21T15:22:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f6721ffb640 1 -- 172.21.15.78:0/2271006361 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6724031df0 con 0x558c7e4a24d0 2024-05-21T15:22:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 10 client.17439 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 1 -- 172.21.15.78:0/2271006361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558c7e4a30f0 con 0x558c7e4a24d0 2024-05-21T15:22:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f6721ffb640 1 -- 172.21.15.78:0/2271006361 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c012ad0 con 0x558c7e4a24d0 2024-05-21T15:22:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f6721ffb640 1 -- 172.21.15.78:0/2271006361 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f672403a040 con 0x558c7e4a24d0 2024-05-21T15:22:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 10 client.17439 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:18.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 20 client.17439 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558c7e489f60 0x558c7e886c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 4 client.17439 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.274+0000 7f67479e5740 1 -- 172.21.15.78:0/2271006361 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c7e7dd430 con 0x558c7e489f60 2024-05-21T15:22:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.275+0000 7f673a59c640 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558c7e489f60 0x558c7e886c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.275+0000 7f6721ffb640 10 client.17439 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.275+0000 7f673a59c640 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558c7e489f60 0x558c7e886c90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8efeaa9f26ef0d2e server_cookie=6566297d16c4a563 in_seq=0 out_seq=0 2024-05-21T15:22:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f6721ffb640 1 -- 172.21.15.78:0/2271006361 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558c7e7dd430 con 0x558c7e489f60 2024-05-21T15:22:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f6721ffb640 10 client.17439 handle_command_reply: tid=0 2024-05-21T15:22:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 client.17439 shutdown 2024-05-21T15:22:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 -- 172.21.15.78:0/2271006361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 msgr2=0x558c7e875e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e875e90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6724006010 tx=0x7f672402ab10 comp rx=0 tx=0).stop 2024-05-21T15:22:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 -- 172.21.15.78:0/2271006361 shutdown_connections 2024-05-21T15:22:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558c7e489f60 0x558c7e886c90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e8763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f6721ffb640 0 client.17439 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e874310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 --2- 172.21.15.78:0/2271006361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e875e90 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 -- 172.21.15.78:0/2271006361 >> 172.21.15.78:0/2271006361 conn(0x558c7e4bf4e0 msgr2=0x558c7e86ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 -- 172.21.15.78:0/2271006361 shutdown_connections 2024-05-21T15:22:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 1 -- 172.21.15.78:0/2271006361 wait complete. 2024-05-21T15:22:18.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.276+0000 7f67479e5740 20 client.17439 trim_cache size 0 max 16384 2024-05-21T15:22:18.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f67479e5740 1 Processor -- start 2024-05-21T15:22:18.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f67479e5740 1 -- start start 2024-05-21T15:22:18.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f67479e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f67479e5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4aa6b0 0x558c7e88c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f67479e5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4cb390 0x558c7e88c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f67479e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1aed60 con 0x558c7e4a24d0 2024-05-21T15:22:18.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f67479e5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1b08a0 con 0x558c7e4cb390 2024-05-21T15:22:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f67479e5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1af430 con 0x558c7e4aa6b0 2024-05-21T15:22:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f6731d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f6731d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52996/0 (socket says 172.21.15.78:52996) 2024-05-21T15:22:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f6731d9b640 1 -- 172.21.15.78:0/1763379764 learned_addr learned my addr 172.21.15.78:0/1763379764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f673a59c640 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4aa6b0 0x558c7e88c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f6731d9b640 1 -- 172.21.15.78:0/1763379764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4aa6b0 msgr2=0x558c7e88c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f6731d9b640 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4aa6b0 0x558c7e88c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f6731d9b640 1 -- 172.21.15.78:0/1763379764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4cb390 msgr2=0x558c7e88c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f6731d9b640 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4cb390 0x558c7e88c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f6731d9b640 1 -- 172.21.15.78:0/1763379764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7e876f60 con 0x558c7e4a24d0 2024-05-21T15:22:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.294+0000 7f673a59c640 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4aa6b0 0x558c7e88c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f6731d9b640 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8742b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6724004650 tx=0x7f6724009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88700cdcc721df5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f6723fff640 1 -- 172.21.15.78:0/1763379764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724053030 con 0x558c7e4a24d0 2024-05-21T15:22:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f6723fff640 1 -- 172.21.15.78:0/1763379764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724005500 con 0x558c7e4a24d0 2024-05-21T15:22:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f6723fff640 1 -- 172.21.15.78:0/1763379764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724035070 con 0x558c7e4a24d0 2024-05-21T15:22:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 -- 172.21.15.78:0/1763379764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 msgr2=0x558c7e8742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8742b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6724004650 tx=0x7f6724009800 comp rx=0 tx=0).stop 2024-05-21T15:22:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 -- 172.21.15.78:0/1763379764 shutdown_connections 2024-05-21T15:22:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4cb390 0x558c7e88c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4aa6b0 0x558c7e88c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 --2- 172.21.15.78:0/1763379764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8742b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 -- 172.21.15.78:0/1763379764 >> 172.21.15.78:0/1763379764 conn(0x558c7e4bf4e0 msgr2=0x558c7e873ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 -- 172.21.15.78:0/1763379764 shutdown_connections 2024-05-21T15:22:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.295+0000 7f67479e5740 1 -- 172.21.15.78:0/1763379764 wait complete. 2024-05-21T15:22:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 1 -- start start 2024-05-21T15:22:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 1 Processor -- start 2024-05-21T15:22:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8eb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e8eba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e8e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1b01d0 con 0x558c7e4a24d0 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1afb00 con 0x558c7e4aa6b0 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1acd40 con 0x558c7e4cb390 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f67479e5740 10 client.0 _ll_register_callbacks cb 0x7f67340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f6731d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f6731d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8eb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53008/0 (socket says 172.21.15.78:53008) 2024-05-21T15:22:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.296+0000 7f6731d9b640 1 -- 172.21.15.78:0/3044623097 learned_addr learned my addr 172.21.15.78:0/3044623097 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f673a59c640 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e8eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6731d9b640 1 -- 172.21.15.78:0/3044623097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 msgr2=0x558c7e8e5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6731d9b640 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e8e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6731d9b640 1 -- 172.21.15.78:0/3044623097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 msgr2=0x558c7e8eba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6731d9b640 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e8eba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6731d9b640 1 -- 172.21.15.78:0/3044623097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7e8e5f40 con 0x558c7e4a24d0 2024-05-21T15:22:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6731d9b640 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8eb540 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f67240057c0 tx=0x7f6724031d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9f099ee7e5e68 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:18.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6721ffb640 1 -- 172.21.15.78:0/3044623097 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672405d020 con 0x558c7e4a24d0 2024-05-21T15:22:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6721ffb640 1 -- 172.21.15.78:0/3044623097 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f67240037f0 con 0x558c7e4a24d0 2024-05-21T15:22:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f67479e5740 1 -- 172.21.15.78:0/3044623097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6724005500 con 0x558c7e4a24d0 2024-05-21T15:22:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6721ffb640 1 -- 172.21.15.78:0/3044623097 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6724053040 con 0x558c7e4a24d0 2024-05-21T15:22:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f67479e5740 10 client.17445 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f67479e5740 1 -- 172.21.15.78:0/3044623097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558c7e8f2b60 con 0x558c7e4a24d0 2024-05-21T15:22:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6721ffb640 1 -- 172.21.15.78:0/3044623097 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6724005500 con 0x558c7e4a24d0 2024-05-21T15:22:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f6721ffb640 1 -- 172.21.15.78:0/3044623097 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6724058c30 con 0x558c7e4a24d0 2024-05-21T15:22:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f67479e5740 10 client.17445 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f67479e5740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f67479e5740 20 client.17445 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.297+0000 7f67479e5740 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558c7e45f3a0 0x558c7e8f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 4 client.17445 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:22:18.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 -- 172.21.15.78:0/3044623097 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c7e7d7780 con 0x558c7e45f3a0 2024-05-21T15:22:18.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f673a59c640 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558c7e45f3a0 0x558c7e8f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:18.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f6721ffb640 10 client.17445 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:18.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f673a59c640 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558c7e45f3a0 0x558c7e8f51a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b982e29f9d7bc6 server_cookie=5e3d772d31c222eb in_seq=0 out_seq=0 2024-05-21T15:22:18.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f6721ffb640 1 -- 172.21.15.78:0/3044623097 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558c7e7d7780 con 0x558c7e45f3a0 2024-05-21T15:22:18.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f6721ffb640 10 client.17445 handle_command_reply: tid=0 2024-05-21T15:22:18.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 client.17445 shutdown 2024-05-21T15:22:18.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 -- 172.21.15.78:0/3044623097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 msgr2=0x558c7e8eb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8eb540 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f67240057c0 tx=0x7f6724031d60 comp rx=0 tx=0).stop 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 -- 172.21.15.78:0/3044623097 shutdown_connections 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x558c7e45f3a0 0x558c7e8f51a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558c7e4aa6b0 0x558c7e8eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f6721ffb640 0 client.17445 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c7e4cb390 0x558c7e8e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 --2- 172.21.15.78:0/3044623097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c7e4a24d0 0x558c7e8eb540 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.298+0000 7f67479e5740 1 -- 172.21.15.78:0/3044623097 >> 172.21.15.78:0/3044623097 conn(0x558c7e4bf4e0 msgr2=0x558c7e875980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f67479e5740 1 -- 172.21.15.78:0/3044623097 shutdown_connections 2024-05-21T15:22:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f67479e5740 1 -- 172.21.15.78:0/3044623097 wait complete. 2024-05-21T15:22:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f67479e5740 20 client.17445 trim_cache size 0 max 16384 2024-05-21T15:22:18.314 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:22:18.314 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:22:18.314 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:22:18.315 INFO:teuthology.orchestra.run.smithi078.stdout: "1d1fce06-4a05-4f04-a921-585ceca7262c": { 2024-05-21T15:22:18.315 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:22:18.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "1d1fce06-4a05-4f04-a921-585ceca7262c", 2024-05-21T15:22:18.315 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:22:18.315 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:22:18.315 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:22:18.315 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:22:18.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 -- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67280a2d00 msgr2=0x7f67280a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67280a2d00 0x7f67280a5210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6734098570 tx=0x7f6734005b40 comp rx=0 tx=0).stop 2024-05-21T15:22:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 -- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c10c340 msgr2=0x7f673c19b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c10c340 0x7f673c19b0c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f673400eb10 tx=0x7f673400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 -- 172.21.15.78:0/1511884591 shutdown_connections 2024-05-21T15:22:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c1079e0 0x7f673c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67280a2d00 0x7f67280a5210 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f673c10c340 0x7f673c19b0c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 --2- 172.21.15.78:0/1511884591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f673c101ab0 0x7f673c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:18.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 -- 172.21.15.78:0/1511884591 >> 172.21.15.78:0/1511884591 conn(0x7f673c0fd210 msgr2=0x7f673c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:18.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 -- 172.21.15.78:0/1511884591 shutdown_connections 2024-05-21T15:22:18.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:18.299+0000 7f6740a61640 1 -- 172.21.15.78:0/1511884591 wait complete. 2024-05-21T15:22:18.318 INFO:tasks.cephfs.filesystem:scrub status for tag:1d1fce06-4a05-4f04-a921-585ceca7262c - {'path': '/', 'tag': '1d1fce06-4a05-4f04-a921-585ceca7262c', 'options': 'recursive,force'} 2024-05-21T15:22:18.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:17 smithi084 ceph-mon[36334]: pgmap v189: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T15:22:19.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:18 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:19.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:18 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:19.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:18 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:19.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:18 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:19.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:18 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:19.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:18 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:19.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:18 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:18 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:18 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:18 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:18 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:19.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:18 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:19.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:18 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:19.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:18 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:19.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:18 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:19.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:18 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:19.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:18 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:19.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:18 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:22:20.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:20.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:20.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:20.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.185 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:19 smithi172 ceph-mon[36402]: pgmap v190: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T15:22:20.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:22:20.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:20.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:20.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:20.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:19 smithi078 ceph-mon[31997]: pgmap v190: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T15:22:20.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:22:20.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:20.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:20.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:20.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:20.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:19 smithi084 ceph-mon[36334]: pgmap v190: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T15:22:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:22 smithi084 ceph-mon[36334]: pgmap v191: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-21T15:22:22.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:22:22.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:22 smithi172 ceph-mon[36402]: pgmap v191: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-21T15:22:22.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:22:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:22 smithi078 ceph-mon[31997]: pgmap v191: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-21T15:22:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:22:23.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:23 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:22:23.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:23 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:22:23.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:23 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:22:23.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:23 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:22:23.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:23 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:22:23.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:23 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:22:23.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:23 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:22:23.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:23 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:22:23.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:23 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:22:24.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:24 smithi084 ceph-mon[36334]: pgmap v192: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 85 B/s wr, 45 op/s 2024-05-21T15:22:24.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:24 smithi172 ceph-mon[36402]: pgmap v192: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 85 B/s wr, 45 op/s 2024-05-21T15:22:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:24 smithi078 ceph-mon[31997]: pgmap v192: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 85 B/s wr, 45 op/s 2024-05-21T15:22:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:26 smithi084 ceph-mon[36334]: pgmap v193: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:26.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:26 smithi172 ceph-mon[36402]: pgmap v193: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:26.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:26 smithi078 ceph-mon[31997]: pgmap v193: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:27.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:27 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:27.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:27 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:27.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:27 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:27.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:27 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:27.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:27 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:27.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:27 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:28 smithi084 ceph-mon[36334]: pgmap v194: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:22:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:28 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:28 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:28.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:28 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:28 smithi172 ceph-mon[36402]: pgmap v194: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:22:28.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:28 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:28.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:28 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:28.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:28 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:28.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:28 smithi078 ceph-mon[31997]: pgmap v194: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:22:28.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:28 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:28.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:28 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:28.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:28 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:30.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:30 smithi172 ceph-mon[36402]: pgmap v195: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:30.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:30 smithi078 ceph-mon[31997]: pgmap v195: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:30.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:30 smithi084 ceph-mon[36334]: pgmap v195: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:32.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:32 smithi172 ceph-mon[36402]: pgmap v196: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:32.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:32 smithi078 ceph-mon[31997]: pgmap v196: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:32 smithi084 ceph-mon[36334]: pgmap v196: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:22:34.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:34 smithi172 ceph-mon[36402]: pgmap v197: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:22:34.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:34 smithi078 ceph-mon[31997]: pgmap v197: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:22:34.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:34 smithi084 ceph-mon[36334]: pgmap v197: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:22:36.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:36 smithi172 ceph-mon[36402]: pgmap v198: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:36.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:36 smithi078 ceph-mon[31997]: pgmap v198: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:36 smithi084 ceph-mon[36334]: pgmap v198: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:38.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:38 smithi172 ceph-mon[36402]: pgmap v199: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:22:38.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:38 smithi078 ceph-mon[31997]: pgmap v199: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:22:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:38 smithi084 ceph-mon[36334]: pgmap v199: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:22:40.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:40 smithi172 ceph-mon[36402]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:40.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:40 smithi078 ceph-mon[31997]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:40 smithi084 ceph-mon[36334]: pgmap v200: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:42.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:42 smithi172 ceph-mon[36402]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:42.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:42 smithi078 ceph-mon[31997]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:42 smithi084 ceph-mon[36334]: pgmap v201: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:44.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:44 smithi172 ceph-mon[36402]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:22:44.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:44 smithi078 ceph-mon[31997]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:22:44.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:44 smithi084 ceph-mon[36334]: pgmap v202: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:22:46.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:46 smithi172 ceph-mon[36402]: pgmap v203: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:46.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:46 smithi078 ceph-mon[31997]: pgmap v203: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:46.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:46 smithi084 ceph-mon[36334]: pgmap v203: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:22:48.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:22:48.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:22:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 -- 172.21.15.78:0/23655450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4401016d0 msgr2=0x7fe44010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 --2- 172.21.15.78:0/23655450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4401016d0 0x7fe44010f520 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ca0 tx=0x7fe43802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 -- 172.21.15.78:0/23655450 shutdown_connections 2024-05-21T15:22:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 --2- 172.21.15.78:0/23655450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe440108f40 0x7fe440101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 --2- 172.21.15.78:0/23655450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4401016d0 0x7fe44010f520 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 --2- 172.21.15.78:0/23655450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe440108aa0 0x7fe440100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 -- 172.21.15.78:0/23655450 >> 172.21.15.78:0/23655450 conn(0x7fe440076830 msgr2=0x7fe440076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 -- 172.21.15.78:0/23655450 shutdown_connections 2024-05-21T15:22:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.442+0000 7fe446d02640 1 -- 172.21.15.78:0/23655450 wait complete. 2024-05-21T15:22:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.443+0000 7fe446d02640 1 Processor -- start 2024-05-21T15:22:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.443+0000 7fe446d02640 1 -- start start 2024-05-21T15:22:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.443+0000 7fe446d02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4401016d0 0x7fe440199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.443+0000 7fe446d02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe440108aa0 0x7fe440199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.443+0000 7fe446d02640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe440108f40 0x7fe44019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.443+0000 7fe446d02640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe440112de0 con 0x7fe440108aa0 2024-05-21T15:22:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.443+0000 7fe446d02640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe440112c60 con 0x7fe440108f40 2024-05-21T15:22:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.443+0000 7fe446d02640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe440112f60 con 0x7fe4401016d0 2024-05-21T15:22:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe440108aa0 0x7fe440199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe440108aa0 0x7fe440199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33378/0 (socket says 172.21.15.78:33378) 2024-05-21T15:22:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 -- 172.21.15.78:0/3162259295 learned_addr learned my addr 172.21.15.78:0/3162259295 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe444a9e640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4401016d0 0x7fe440199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe44529f640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe440108f40 0x7fe44019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 -- 172.21.15.78:0/3162259295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4401016d0 msgr2=0x7fe440199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4401016d0 0x7fe440199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 -- 172.21.15.78:0/3162259295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe440108f40 msgr2=0x7fe44019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe440108f40 0x7fe44019df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 -- 172.21.15.78:0/3162259295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44019e5a0 con 0x7fe440108aa0 2024-05-21T15:22:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43ffff640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe440108aa0 0x7fe440199ca0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe42800d770 tx=0x7fe42800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43dffb640 1 -- 172.21.15.78:0/3162259295 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428017070 con 0x7fe440108aa0 2024-05-21T15:22:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43dffb640 1 -- 172.21.15.78:0/3162259295 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe428009c30 con 0x7fe440108aa0 2024-05-21T15:22:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe43dffb640 1 -- 172.21.15.78:0/3162259295 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42800f620 con 0x7fe440108aa0 2024-05-21T15:22:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe446d02640 1 -- 172.21.15.78:0/3162259295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44019e890 con 0x7fe440108aa0 2024-05-21T15:22:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.444+0000 7fe446d02640 1 -- 172.21.15.78:0/3162259295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4401a6210 con 0x7fe440108aa0 2024-05-21T15:22:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe43dffb640 1 -- 172.21.15.78:0/3162259295 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe42801a070 con 0x7fe440108aa0 2024-05-21T15:22:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44dbaf740 1 Processor -- start 2024-05-21T15:22:48.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44dbaf740 1 -- start start 2024-05-21T15:22:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e68a390 0x56241ea28d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6614d0 0x56241ea29270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56241e98ed00 con 0x56241e6614d0 2024-05-21T15:22:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56241e9364c0 con 0x56241e6696b0 2024-05-21T15:22:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56241e98acc0 con 0x56241e68a390 2024-05-21T15:22:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe43dffb640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe42c0a2d80 0x7fe42c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe444a9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6614d0 0x56241ea29270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe444a9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6614d0 0x56241ea29270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33390/0 (socket says 172.21.15.78:33390) 2024-05-21T15:22:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe444a9e640 1 -- 172.21.15.78:0/2699183168 learned_addr learned my addr 172.21.15.78:0/2699183168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.447+0000 7fe44529f640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe42c0a2d80 0x7fe42c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe43dffb640 1 -- 172.21.15.78:0/3162259295 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4280cef40 con 0x7fe440108aa0 2024-05-21T15:22:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe44529f640 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e68a390 0x56241ea28d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe43ffff640 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe444a9e640 1 -- 172.21.15.78:0/2699183168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e68a390 msgr2=0x56241ea28d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe444a9e640 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e68a390 0x56241ea28d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe444a9e640 1 -- 172.21.15.78:0/2699183168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 msgr2=0x56241ea297b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe444a9e640 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea297b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe444a9e640 1 -- 172.21.15.78:0/2699183168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241e76c7d0 con 0x56241e6614d0 2024-05-21T15:22:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe43ffff640 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe444a9e640 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6614d0 0x56241ea29270 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe43000bd10 tx=0x7fe4300107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66de83d39a006b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe44529f640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe42c0a2d80 0x7fe42c0a5290 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe438004af0 tx=0x7fe438009a20 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe436ffd640 1 -- 172.21.15.78:0/2699183168 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430004030 con 0x56241e6614d0 2024-05-21T15:22:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe436ffd640 1 -- 172.21.15.78:0/2699183168 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4300041d0 con 0x56241e6614d0 2024-05-21T15:22:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.448+0000 7fe436ffd640 1 -- 172.21.15.78:0/2699183168 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430011800 con 0x56241e6614d0 2024-05-21T15:22:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 -- 172.21.15.78:0/2699183168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6614d0 msgr2=0x56241ea29270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6614d0 0x56241ea29270 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe43000bd10 tx=0x7fe4300107a0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 -- 172.21.15.78:0/2699183168 shutdown_connections 2024-05-21T15:22:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea297b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6614d0 0x56241ea29270 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/2699183168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e68a390 0x56241ea28d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 -- 172.21.15.78:0/2699183168 >> 172.21.15.78:0/2699183168 conn(0x56241e67e4e0 msgr2=0x56241ea254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 -- 172.21.15.78:0/2699183168 shutdown_connections 2024-05-21T15:22:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 -- 172.21.15.78:0/2699183168 wait complete. 2024-05-21T15:22:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.449+0000 7fe44dbaf740 1 -- start start 2024-05-21T15:22:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44dbaf740 1 Processor -- start 2024-05-21T15:22:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea34f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e68a390 0x56241ea333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56241ea35fc0 con 0x56241e6696b0 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56241e949b90 con 0x56241e68a390 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56241ea36140 con 0x56241e6614d0 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44dbaf740 10 client.0 _ll_register_callbacks cb 0x56241e985da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe43ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea35470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe43ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea35470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33404/0 (socket says 172.21.15.78:33404) 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe43ffff640 1 -- 172.21.15.78:0/66170945 learned_addr learned my addr 172.21.15.78:0/66170945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe444a9e640 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea34f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe44529f640 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e68a390 0x56241ea333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe43ffff640 1 -- 172.21.15.78:0/66170945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 msgr2=0x56241ea34f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe43ffff640 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea34f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe43ffff640 1 -- 172.21.15.78:0/66170945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e68a390 msgr2=0x56241ea333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.450+0000 7fe43ffff640 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e68a390 0x56241ea333d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe43ffff640 1 -- 172.21.15.78:0/66170945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241e6a23f0 con 0x56241e6696b0 2024-05-21T15:22:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe444a9e640 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea34f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe43ffff640 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea35470 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe428097860 tx=0x7fe42809b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd22f4058f37e96 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe41ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:48.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe41ffff640 1 -- 172.21.15.78:0/66170945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428021050 con 0x56241e6696b0 2024-05-21T15:22:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe41ffff640 1 -- 172.21.15.78:0/66170945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4280b4310 con 0x56241e6696b0 2024-05-21T15:22:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 1 -- 172.21.15.78:0/66170945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4300041d0 con 0x56241e6696b0 2024-05-21T15:22:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe41ffff640 1 -- 172.21.15.78:0/66170945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe42809bb00 con 0x56241e6696b0 2024-05-21T15:22:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 10 client.17457 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 1 -- 172.21.15.78:0/66170945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56241e6620f0 con 0x56241e6696b0 2024-05-21T15:22:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe41ffff640 1 -- 172.21.15.78:0/66170945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4300041d0 con 0x56241e6696b0 2024-05-21T15:22:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe41ffff640 1 -- 172.21.15.78:0/66170945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe42809a850 con 0x56241e6696b0 2024-05-21T15:22:48.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 10 client.17457 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 20 client.17457 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56241e648f60 0x56241ea45c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 4 client.17457 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.451+0000 7fe44dbaf740 1 -- 172.21.15.78:0/66170945 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56241e99c430 con 0x56241e648f60 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.452+0000 7fe444a9e640 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56241e648f60 0x56241ea45c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.452+0000 7fe41ffff640 10 client.17457 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.452+0000 7fe444a9e640 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56241e648f60 0x56241ea45c20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c62f3bab78e6e9 server_cookie=3857c5c8b395c432 in_seq=0 out_seq=0 2024-05-21T15:22:48.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe41ffff640 1 -- 172.21.15.78:0/66170945 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56241e99c430 con 0x56241e648f60 2024-05-21T15:22:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe41ffff640 10 client.17457 handle_command_reply: tid=0 2024-05-21T15:22:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 client.17457 shutdown 2024-05-21T15:22:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 -- 172.21.15.78:0/66170945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 msgr2=0x56241ea35470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea35470 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe428097860 tx=0x7fe42809b000 comp rx=0 tx=0).stop 2024-05-21T15:22:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 -- 172.21.15.78:0/66170945 shutdown_connections 2024-05-21T15:22:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56241e648f60 0x56241ea45c20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e68a390 0x56241ea333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe41ffff640 0 client.17457 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea35470 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/66170945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea34f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 -- 172.21.15.78:0/66170945 >> 172.21.15.78:0/66170945 conn(0x56241e67e4e0 msgr2=0x56241ea2aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 -- 172.21.15.78:0/66170945 shutdown_connections 2024-05-21T15:22:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 1 -- 172.21.15.78:0/66170945 wait complete. 2024-05-21T15:22:48.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.453+0000 7fe44dbaf740 20 client.17457 trim_cache size 0 max 16384 2024-05-21T15:22:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe44dbaf740 1 Processor -- start 2024-05-21T15:22:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe44dbaf740 1 -- start start 2024-05-21T15:22:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e648f60 0x56241ea2c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea2cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea2a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401aebf0 con 0x56241e648f60 2024-05-21T15:22:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401b0730 con 0x56241e6696b0 2024-05-21T15:22:48.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401af2c0 con 0x56241e6614d0 2024-05-21T15:22:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe444a9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e648f60 0x56241ea2c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe444a9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e648f60 0x56241ea2c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33414/0 (socket says 172.21.15.78:33414) 2024-05-21T15:22:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.471+0000 7fe444a9e640 1 -- 172.21.15.78:0/1466823438 learned_addr learned my addr 172.21.15.78:0/1466823438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe44529f640 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea2a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe444a9e640 1 -- 172.21.15.78:0/1466823438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 msgr2=0x56241ea2cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe43ffff640 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea2cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe444a9e640 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea2cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe444a9e640 1 -- 172.21.15.78:0/1466823438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 msgr2=0x56241ea2a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe444a9e640 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea2a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe444a9e640 1 -- 172.21.15.78:0/1466823438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241ea26630 con 0x56241e648f60 2024-05-21T15:22:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe44529f640 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea2a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe444a9e640 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e648f60 0x56241ea2c9e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe43000baa0 tx=0x7fe43000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c30bac7b5aac4cb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe4367fc640 1 -- 172.21.15.78:0/1466823438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300037f0 con 0x56241e648f60 2024-05-21T15:22:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe4367fc640 1 -- 172.21.15.78:0/1466823438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43002e070 con 0x56241e648f60 2024-05-21T15:22:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.472+0000 7fe4367fc640 1 -- 172.21.15.78:0/1466823438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300295d0 con 0x56241e648f60 2024-05-21T15:22:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 -- 172.21.15.78:0/1466823438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e648f60 msgr2=0x56241ea2c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e648f60 0x56241ea2c9e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe43000baa0 tx=0x7fe43000b390 comp rx=0 tx=0).stop 2024-05-21T15:22:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 -- 172.21.15.78:0/1466823438 shutdown_connections 2024-05-21T15:22:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e6696b0 0x56241ea2a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea2cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/1466823438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e648f60 0x56241ea2c9e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 -- 172.21.15.78:0/1466823438 >> 172.21.15.78:0/1466823438 conn(0x56241e67e4e0 msgr2=0x56241ea2e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 -- 172.21.15.78:0/1466823438 shutdown_connections 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 -- 172.21.15.78:0/1466823438 wait complete. 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 -- start start 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.473+0000 7fe44dbaf740 1 Processor -- start 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e648f60 0x56241ea3ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea3d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44dbaf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea3d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401b0060 con 0x56241e6696b0 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401af990 con 0x56241e648f60 2024-05-21T15:22:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44dbaf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401acbd0 con 0x56241e6614d0 2024-05-21T15:22:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44dbaf740 10 client.0 _ll_register_callbacks cb 0x7fe42800de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea3d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea3d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33420/0 (socket says 172.21.15.78:33420) 2024-05-21T15:22:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 -- 172.21.15.78:0/4213375161 learned_addr learned my addr 172.21.15.78:0/4213375161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe444a9e640 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e648f60 0x56241ea3ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe43ffff640 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea3d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 -- 172.21.15.78:0/4213375161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 msgr2=0x56241ea3d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea3d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 -- 172.21.15.78:0/4213375161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e648f60 msgr2=0x56241ea3ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e648f60 0x56241ea3ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 -- 172.21.15.78:0/4213375161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241ea28ac0 con 0x56241e6696b0 2024-05-21T15:22:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44529f640 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea3d7b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ca0 tx=0x7fe438005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae492239cdbbc2e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe41ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe41ffff640 1 -- 172.21.15.78:0/4213375161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438004030 con 0x56241e6696b0 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe41ffff640 1 -- 172.21.15.78:0/4213375161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fe4380041d0 con 0x56241e6696b0 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.474+0000 7fe44dbaf740 1 -- 172.21.15.78:0/4213375161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43002e070 con 0x56241e6696b0 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe41ffff640 1 -- 172.21.15.78:0/4213375161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe4380043d0 con 0x56241e6696b0 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe44dbaf740 10 client.17463 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe44dbaf740 1 -- 172.21.15.78:0/4213375161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56241eaada70 con 0x56241e6696b0 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe41ffff640 1 -- 172.21.15.78:0/4213375161 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43002e070 con 0x56241e6696b0 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe41ffff640 1 -- 172.21.15.78:0/4213375161 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe4380305c0 con 0x56241e6696b0 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe44dbaf740 10 client.17463 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe44dbaf740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe44dbaf740 20 client.17463 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56241e68a390 0x56241eab0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe44dbaf740 4 client.17463 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:22:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe44dbaf740 1 -- 172.21.15.78:0/4213375161 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56241e996780 con 0x56241e68a390 2024-05-21T15:22:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.475+0000 7fe444a9e640 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56241e68a390 0x56241eab0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe41ffff640 10 client.17463 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe444a9e640 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56241e68a390 0x56241eab0290 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49847f600d7c43c2 server_cookie=c9430e1d7d9e8bf6 in_seq=0 out_seq=0 2024-05-21T15:22:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe41ffff640 1 -- 172.21.15.78:0/4213375161 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56241e996780 con 0x56241e68a390 2024-05-21T15:22:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe41ffff640 10 client.17463 handle_command_reply: tid=0 2024-05-21T15:22:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 client.17463 shutdown 2024-05-21T15:22:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 -- 172.21.15.78:0/4213375161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 msgr2=0x56241ea3d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea3d7b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ca0 tx=0x7fe438005fd0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 -- 172.21.15.78:0/4213375161 shutdown_connections 2024-05-21T15:22:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56241e68a390 0x56241eab0290 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56241e6696b0 0x56241ea3d7b0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe41ffff640 0 client.17463 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56241e6614d0 0x56241ea3d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 --2- 172.21.15.78:0/4213375161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56241e648f60 0x56241ea3ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 -- 172.21.15.78:0/4213375161 >> 172.21.15.78:0/4213375161 conn(0x56241e67e4e0 msgr2=0x56241ea2e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 -- 172.21.15.78:0/4213375161 shutdown_connections 2024-05-21T15:22:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 1 -- 172.21.15.78:0/4213375161 wait complete. 2024-05-21T15:22:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.476+0000 7fe44dbaf740 20 client.17463 trim_cache size 0 max 16384 2024-05-21T15:22:48.491 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:22:48.491 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:22:48.491 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:22:48.491 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:22:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 -- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe42c0a2d80 msgr2=0x7fe42c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe42c0a2d80 0x7fe42c0a5290 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe438004af0 tx=0x7fe438009a20 comp rx=0 tx=0).stop 2024-05-21T15:22:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 -- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe440108aa0 msgr2=0x7fe440199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe440108aa0 0x7fe440199ca0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe42800d770 tx=0x7fe42800dc30 comp rx=0 tx=0).stop 2024-05-21T15:22:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 -- 172.21.15.78:0/3162259295 shutdown_connections 2024-05-21T15:22:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe440108f40 0x7fe44019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe42c0a2d80 0x7fe42c0a5290 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe440108aa0 0x7fe440199ca0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 --2- 172.21.15.78:0/3162259295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4401016d0 0x7fe440199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 -- 172.21.15.78:0/3162259295 >> 172.21.15.78:0/3162259295 conn(0x7fe440076830 msgr2=0x7fe4400ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 -- 172.21.15.78:0/3162259295 shutdown_connections 2024-05-21T15:22:48.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.477+0000 7fe446d02640 1 -- 172.21.15.78:0/3162259295 wait complete. 2024-05-21T15:22:48.494 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:22:48.494 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:22:48.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:48 smithi078 ceph-mon[31997]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:22:48.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 -- 172.21.15.78:0/992130913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 msgr2=0x7f12ac102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 --2- 172.21.15.78:0/992130913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 0x7f12ac102680 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009900 tx=0x7f12a802a410 comp rx=0 tx=0).stop 2024-05-21T15:22:48.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 -- 172.21.15.78:0/992130913 shutdown_connections 2024-05-21T15:22:48.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 --2- 172.21.15.78:0/992130913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ac108f80 0x7f12ac102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 --2- 172.21.15.78:0/992130913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac103230 0x7f12ac10f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 --2- 172.21.15.78:0/992130913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 0x7f12ac102680 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 -- 172.21.15.78:0/992130913 >> 172.21.15.78:0/992130913 conn(0x7f12ac0fe1c0 msgr2=0x7f12ac100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 -- 172.21.15.78:0/992130913 shutdown_connections 2024-05-21T15:22:48.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.602+0000 7f12b36e7640 1 -- 172.21.15.78:0/992130913 wait complete. 2024-05-21T15:22:48.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b36e7640 1 Processor -- start 2024-05-21T15:22:48.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b36e7640 1 -- start start 2024-05-21T15:22:48.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b36e7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ac103230 0x7f12ac1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b36e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 0x7f12ac199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b36e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac108f80 0x7f12ac19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b36e7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac112e70 con 0x7f12ac108ae0 2024-05-21T15:22:48.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b36e7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac112cf0 con 0x7f12ac103230 2024-05-21T15:22:48.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b36e7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac112ff0 con 0x7f12ac108f80 2024-05-21T15:22:48.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b0c82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 0x7f12ac199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b0c82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 0x7f12ac199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50124/0 (socket says 172.21.15.78:50124) 2024-05-21T15:22:48.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.603+0000 7f12b0c82640 1 -- 172.21.15.78:0/2908011326 learned_addr learned my addr 172.21.15.78:0/2908011326 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b1483640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ac103230 0x7f12ac1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b1c84640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac108f80 0x7f12ac19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b0c82640 1 -- 172.21.15.78:0/2908011326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac108f80 msgr2=0x7f12ac19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b0c82640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac108f80 0x7f12ac19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b0c82640 1 -- 172.21.15.78:0/2908011326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ac103230 msgr2=0x7f12ac1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b0c82640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ac103230 0x7f12ac1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b0c82640 1 -- 172.21.15.78:0/2908011326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac19e610 con 0x7f12ac108ae0 2024-05-21T15:22:48.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b0c82640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 0x7f12ac199d10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f129c00e9f0 tx=0x7f129c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12a27fc640 1 -- 172.21.15.78:0/2908011326 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c00ccc0 con 0x7f12ac108ae0 2024-05-21T15:22:48.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12a27fc640 1 -- 172.21.15.78:0/2908011326 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129c0035f0 con 0x7f12ac108ae0 2024-05-21T15:22:48.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12a27fc640 1 -- 172.21.15.78:0/2908011326 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c00ce60 con 0x7f12ac108ae0 2024-05-21T15:22:48.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac19e900 con 0x7f12ac108ae0 2024-05-21T15:22:48.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.604+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac19ee00 con 0x7f12ac108ae0 2024-05-21T15:22:48.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.605+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac069140 con 0x7f12ac108ae0 2024-05-21T15:22:48.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.606+0000 7f12a27fc640 1 -- 172.21.15.78:0/2908011326 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f129c0037a0 con 0x7f12ac108ae0 2024-05-21T15:22:48.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.607+0000 7f12a27fc640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12980a2d80 0x7f12980a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.607+0000 7f12b1483640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12980a2d80 0x7f12980a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.607+0000 7f12a27fc640 1 -- 172.21.15.78:0/2908011326 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f129c0cf7e0 con 0x7f12ac108ae0 2024-05-21T15:22:48.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.608+0000 7f12b1483640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12980a2d80 0x7f12980a5290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12a8006010 tx=0x7f12a8000bc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.612+0000 7f12a27fc640 1 -- 172.21.15.78:0/2908011326 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f12ac19ee00 con 0x7f12ac108ae0 2024-05-21T15:22:48.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:48 smithi172 ceph-mon[36402]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:22:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.825+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12ac1a6490 con 0x7f12ac108ae0 2024-05-21T15:22:48.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.825+0000 7f12a27fc640 1 -- 172.21.15.78:0/2908011326 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7f129c09c3a0 con 0x7f12ac108ae0 2024-05-21T15:22:48.829 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:22:48.829 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:22:48.829 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:22:48.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12980a2d80 msgr2=0x7f12980a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12980a2d80 0x7f12980a5290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12a8006010 tx=0x7f12a8000bc0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 msgr2=0x7f12ac199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 0x7f12ac199d10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f129c00e9f0 tx=0x7f129c00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 shutdown_connections 2024-05-21T15:22:48.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12980a2d80 0x7f12980a5290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12ac108f80 0x7f12ac19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12ac108ae0 0x7f12ac199d10 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 --2- 172.21.15.78:0/2908011326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12ac103230 0x7f12ac1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.827+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 >> 172.21.15.78:0/2908011326 conn(0x7f12ac0fe1c0 msgr2=0x7f12ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.828+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 shutdown_connections 2024-05-21T15:22:48.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.828+0000 7f12b36e7640 1 -- 172.21.15.78:0/2908011326 wait complete. 2024-05-21T15:22:48.842 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:22:48.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:48 smithi084 ceph-mon[36334]: pgmap v204: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:22:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 -- 172.21.15.78:0/2570666774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 msgr2=0x7f3808101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 --2- 172.21.15.78:0/2570666774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 0x7f3808101560 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f802a410 comp rx=0 tx=0).stop 2024-05-21T15:22:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 -- 172.21.15.78:0/2570666774 shutdown_connections 2024-05-21T15:22:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 --2- 172.21.15.78:0/2570666774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3808101aa0 0x7f3808101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 --2- 172.21.15.78:0/2570666774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f380810c330 0x7f380810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 --2- 172.21.15.78:0/2570666774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 0x7f3808101560 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 -- 172.21.15.78:0/2570666774 >> 172.21.15.78:0/2570666774 conn(0x7f38080770b0 msgr2=0x7f38080774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 -- 172.21.15.78:0/2570666774 shutdown_connections 2024-05-21T15:22:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.952+0000 7f380fb57640 1 -- 172.21.15.78:0/2570666774 wait complete. 2024-05-21T15:22:48.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380fb57640 1 Processor -- start 2024-05-21T15:22:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380fb57640 1 -- start start 2024-05-21T15:22:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380fb57640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3808101aa0 0x7f3808199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380fb57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 0x7f3808199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380fb57640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f380810c330 0x7f380819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380fb57640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3808111ce0 con 0x7f38081079d0 2024-05-21T15:22:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380fb57640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3808111b60 con 0x7f380810c330 2024-05-21T15:22:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380fb57640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3808111e60 con 0x7f3808101aa0 2024-05-21T15:22:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d0f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 0x7f3808199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d0f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 0x7f3808199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50148/0 (socket says 172.21.15.78:50148) 2024-05-21T15:22:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d0f2640 1 -- 172.21.15.78:0/3249413395 learned_addr learned my addr 172.21.15.78:0/3249413395 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f380810c330 0x7f380819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d8f3640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3808101aa0 0x7f3808199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d0f2640 1 -- 172.21.15.78:0/3249413395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3808101aa0 msgr2=0x7f3808199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d0f2640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3808101aa0 0x7f3808199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d0f2640 1 -- 172.21.15.78:0/3249413395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f380810c330 msgr2=0x7f380819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d0f2640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f380810c330 0x7f380819df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380d0f2640 1 -- 172.21.15.78:0/3249413395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380819e5d0 con 0x7f38081079d0 2024-05-21T15:22:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.953+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f380810c330 0x7f380819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.954+0000 7f380d0f2640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 0x7f3808199cd0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3804005fd0 tx=0x7f380400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.954+0000 7f37feffd640 1 -- 172.21.15.78:0/3249413395 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804017070 con 0x7f38081079d0 2024-05-21T15:22:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.954+0000 7f380fb57640 1 -- 172.21.15.78:0/3249413395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380819e8c0 con 0x7f38081079d0 2024-05-21T15:22:48.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.954+0000 7f37feffd640 1 -- 172.21.15.78:0/3249413395 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3804009c30 con 0x7f38081079d0 2024-05-21T15:22:48.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.954+0000 7f37feffd640 1 -- 172.21.15.78:0/3249413395 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804003e40 con 0x7f38081079d0 2024-05-21T15:22:48.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.954+0000 7f380fb57640 1 -- 172.21.15.78:0/3249413395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38081a6240 con 0x7f38081079d0 2024-05-21T15:22:48.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f37feffd640 1 -- 172.21.15.78:0/3249413395 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3804004000 con 0x7f38081079d0 2024-05-21T15:22:48.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f3816a03740 1 Processor -- start 2024-05-21T15:22:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f3816a03740 1 -- start start 2024-05-21T15:22:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f3816a03740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4c4390 0x559f9e862bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f3816a03740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e8630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f3816a03740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4a36b0 0x559f9e863630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f3816a03740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f9e7c8c40 con 0x559f9e49b4d0 2024-05-21T15:22:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f3816a03740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559f9e770580 con 0x559f9e4c4390 2024-05-21T15:22:48.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f3816a03740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559f9e7c4c00 con 0x559f9e4a36b0 2024-05-21T15:22:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f380d8f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e8630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f380d8f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e8630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50152/0 (socket says 172.21.15.78:50152) 2024-05-21T15:22:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f380d8f3640 1 -- 172.21.15.78:0/3645369759 learned_addr learned my addr 172.21.15.78:0/3645369759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4c4390 0x559f9e862bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.957+0000 7f37feffd640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37f40a2d00 0x7f37f40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37f40a2d00 0x7f37f40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380d8f3640 1 -- 172.21.15.78:0/3645369759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4a36b0 msgr2=0x559f9e863630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380d8f3640 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4a36b0 0x559f9e863630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380d8f3640 1 -- 172.21.15.78:0/3645369759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4c4390 msgr2=0x559f9e862bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380d8f3640 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4c4390 0x559f9e862bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380d8f3640 1 -- 172.21.15.78:0/3645369759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f9e4dc3f0 con 0x559f9e49b4d0 2024-05-21T15:22:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f37feffd640 1 -- 172.21.15.78:0/3249413395 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f38040cf250 con 0x7f38081079d0 2024-05-21T15:22:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4c4390 0x559f9e862bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380d8f3640 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e8630f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f8005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98256f8822e77e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37f40a2d00 0x7f37f40a5210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3800009fd0 tx=0x7f3800004ab0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f37f3fff640 1 -- 172.21.15.78:0/3645369759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f803f070 con 0x559f9e49b4d0 2024-05-21T15:22:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f37f3fff640 1 -- 172.21.15.78:0/3645369759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f8004030 con 0x559f9e49b4d0 2024-05-21T15:22:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.958+0000 7f37f3fff640 1 -- 172.21.15.78:0/3645369759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8035070 con 0x559f9e49b4d0 2024-05-21T15:22:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 -- 172.21.15.78:0/3645369759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 msgr2=0x559f9e8630f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e8630f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f8005e00 comp rx=0 tx=0).stop 2024-05-21T15:22:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 -- 172.21.15.78:0/3645369759 shutdown_connections 2024-05-21T15:22:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4a36b0 0x559f9e863630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e8630f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 --2- 172.21.15.78:0/3645369759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4c4390 0x559f9e862bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 -- 172.21.15.78:0/3645369759 >> 172.21.15.78:0/3645369759 conn(0x559f9e4b84e0 msgr2=0x559f9e85f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 -- 172.21.15.78:0/3645369759 shutdown_connections 2024-05-21T15:22:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.959+0000 7f3816a03740 1 -- 172.21.15.78:0/3645369759 wait complete. 2024-05-21T15:22:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 1 -- start start 2024-05-21T15:22:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 1 Processor -- start 2024-05-21T15:22:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e873170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e8736b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 0x559f9e871640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559f9e874420 con 0x559f9e49b4d0 2024-05-21T15:22:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559f9e783c40 con 0x559f9e4a36b0 2024-05-21T15:22:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559f9e8745a0 con 0x559f9e4c4390 2024-05-21T15:22:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f3816a03740 10 client.0 _ll_register_callbacks cb 0x559f9e4dbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f380d8f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e873170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f380d8f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e873170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50168/0 (socket says 172.21.15.78:50168) 2024-05-21T15:22:48.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f380d8f3640 1 -- 172.21.15.78:0/5174710 learned_addr learned my addr 172.21.15.78:0/5174710 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f380d0f2640 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e8736b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f380d8f3640 1 -- 172.21.15.78:0/5174710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 msgr2=0x559f9e871640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.960+0000 7f380e0f4640 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 0x559f9e871640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f380d8f3640 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 0x559f9e871640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f380d8f3640 1 -- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 msgr2=0x559f9e8736b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f380d8f3640 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e8736b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f380d8f3640 1 -- 172.21.15.78:0/5174710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f9e5a67d0 con 0x559f9e49b4d0 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f380d0f2640 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e8736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:48.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f380d8f3640 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e873170 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f37f802a930 tx=0x7f37f8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd4b3910d6176ea server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f37f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f37f17fa640 1 -- 172.21.15.78:0/5174710 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8035070 con 0x559f9e49b4d0 2024-05-21T15:22:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f37f17fa640 1 -- 172.21.15.78:0/5174710 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f37f80315b0 con 0x559f9e49b4d0 2024-05-21T15:22:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f3816a03740 1 -- 172.21.15.78:0/5174710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f8004030 con 0x559f9e49b4d0 2024-05-21T15:22:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f37f17fa640 1 -- 172.21.15.78:0/5174710 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f37f8055050 con 0x559f9e49b4d0 2024-05-21T15:22:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f3816a03740 10 client.17481 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f3816a03740 1 -- 172.21.15.78:0/5174710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559f9e880c20 con 0x559f9e49b4d0 2024-05-21T15:22:48.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f37f17fa640 1 -- 172.21.15.78:0/5174710 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f8004030 con 0x559f9e49b4d0 2024-05-21T15:22:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f37f17fa640 1 -- 172.21.15.78:0/5174710 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f37f8003710 con 0x559f9e49b4d0 2024-05-21T15:22:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f3816a03740 10 client.17481 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.961+0000 7f3816a03740 10 client.17481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:22:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.962+0000 7f3816a03740 20 client.17481 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.962+0000 7f3816a03740 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x559f9e482f60 0x559f9e883f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.962+0000 7f3816a03740 4 client.17481 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.962+0000 7f3816a03740 1 -- 172.21.15.78:0/5174710 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f9e7d6370 con 0x559f9e482f60 2024-05-21T15:22:48.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.962+0000 7f380d0f2640 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x559f9e482f60 0x559f9e883f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.962+0000 7f37f17fa640 10 client.17481 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:22:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.962+0000 7f380d0f2640 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x559f9e482f60 0x559f9e883f80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf6a9c1d82b79e46 server_cookie=c0d958487703b31d in_seq=0 out_seq=0 2024-05-21T15:22:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f37f17fa640 1 -- 172.21.15.78:0/5174710 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f9e7d6370 con 0x559f9e482f60 2024-05-21T15:22:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f37f17fa640 10 client.17481 handle_command_reply: tid=0 2024-05-21T15:22:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 client.17481 shutdown 2024-05-21T15:22:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 -- 172.21.15.78:0/5174710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 msgr2=0x559f9e873170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e873170 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f37f802a930 tx=0x7f37f8004610 comp rx=0 tx=0).stop 2024-05-21T15:22:48.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 -- 172.21.15.78:0/5174710 shutdown_connections 2024-05-21T15:22:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x559f9e482f60 0x559f9e883f80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e8736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 0x559f9e871640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 --2- 172.21.15.78:0/5174710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e49b4d0 0x559f9e873170 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f37f17fa640 0 client.17481 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:22:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.963+0000 7f3816a03740 1 -- 172.21.15.78:0/5174710 >> 172.21.15.78:0/5174710 conn(0x559f9e4b84e0 msgr2=0x559f9e864a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.964+0000 7f3816a03740 1 -- 172.21.15.78:0/5174710 shutdown_connections 2024-05-21T15:22:48.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.964+0000 7f3816a03740 1 -- 172.21.15.78:0/5174710 wait complete. 2024-05-21T15:22:48.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.964+0000 7f3816a03740 20 client.17481 trim_cache size 0 max 16384 2024-05-21T15:22:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f3816a03740 1 Processor -- start 2024-05-21T15:22:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f3816a03740 1 -- start start 2024-05-21T15:22:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f3816a03740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e49b4d0 0x559f9e870620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f3816a03740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e86af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f3816a03740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4c4390 0x559f9e86b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f3816a03740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38081ac870 con 0x559f9e4c4390 2024-05-21T15:22:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f3816a03740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38081ae3b0 con 0x559f9e4a36b0 2024-05-21T15:22:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f3816a03740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38081acf40 con 0x559f9e49b4d0 2024-05-21T15:22:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f380e0f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4c4390 0x559f9e86b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f380e0f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4c4390 0x559f9e86b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50174/0 (socket says 172.21.15.78:50174) 2024-05-21T15:22:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f380e0f4640 1 -- 172.21.15.78:0/3519273122 learned_addr learned my addr 172.21.15.78:0/3519273122 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.977+0000 7f380d0f2640 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e86af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f380d8f3640 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e49b4d0 0x559f9e870620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f380e0f4640 1 -- 172.21.15.78:0/3519273122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e49b4d0 msgr2=0x559f9e870620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e49b4d0 0x559f9e870620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f380e0f4640 1 -- 172.21.15.78:0/3519273122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 msgr2=0x559f9e86af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e86af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f380e0f4640 1 -- 172.21.15.78:0/3519273122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f9e8642b0 con 0x559f9e4c4390 2024-05-21T15:22:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f380d0f2640 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e86af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f380e0f4640 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4c4390 0x559f9e86b4a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3800016fd0 tx=0x7f3800009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ede82b2e9351ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f37f37fe640 1 -- 172.21.15.78:0/3519273122 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800025070 con 0x559f9e4c4390 2024-05-21T15:22:48.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f37f37fe640 1 -- 172.21.15.78:0/3519273122 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380001edc0 con 0x559f9e4c4390 2024-05-21T15:22:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.978+0000 7f37f37fe640 1 -- 172.21.15.78:0/3519273122 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800005800 con 0x559f9e4c4390 2024-05-21T15:22:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 -- 172.21.15.78:0/3519273122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4c4390 msgr2=0x559f9e86b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4c4390 0x559f9e86b4a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3800016fd0 tx=0x7f3800009700 comp rx=0 tx=0).stop 2024-05-21T15:22:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 -- 172.21.15.78:0/3519273122 shutdown_connections 2024-05-21T15:22:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e4a36b0 0x559f9e86af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4c4390 0x559f9e86b4a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 --2- 172.21.15.78:0/3519273122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e49b4d0 0x559f9e870620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 -- 172.21.15.78:0/3519273122 >> 172.21.15.78:0/3519273122 conn(0x559f9e4b84e0 msgr2=0x559f9e86ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 -- 172.21.15.78:0/3519273122 shutdown_connections 2024-05-21T15:22:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 -- 172.21.15.78:0/3519273122 wait complete. 2024-05-21T15:22:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 -- start start 2024-05-21T15:22:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.979+0000 7f3816a03740 1 Processor -- start 2024-05-21T15:22:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f3816a03740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e49b4d0 0x559f9e887190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f3816a03740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4a36b0 0x559f9e8876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f3816a03740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 0x559f9e884660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f3816a03740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38081adce0 con 0x559f9e4a36b0 2024-05-21T15:22:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f3816a03740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38081ad610 con 0x559f9e49b4d0 2024-05-21T15:22:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f3816a03740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38081aa850 con 0x559f9e4c4390 2024-05-21T15:22:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f3816a03740 10 client.0 _ll_register_callbacks cb 0x7f37f8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4a36b0 0x559f9e8876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4a36b0 0x559f9e8876d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50188/0 (socket says 172.21.15.78:50188) 2024-05-21T15:22:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 -- 172.21.15.78:0/2569008707 learned_addr learned my addr 172.21.15.78:0/2569008707 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d8f3640 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e49b4d0 0x559f9e887190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380e0f4640 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 0x559f9e884660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 -- 172.21.15.78:0/2569008707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 msgr2=0x559f9e884660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 0x559f9e884660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 -- 172.21.15.78:0/2569008707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e49b4d0 msgr2=0x559f9e887190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e49b4d0 0x559f9e887190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 -- 172.21.15.78:0/2569008707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f9e884dd0 con 0x559f9e4a36b0 2024-05-21T15:22:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f380d0f2640 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4a36b0 0x559f9e8876d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f38040cc9f0 tx=0x7f3804017eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3078c99f52179ba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f37f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:48.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f37f17fa640 1 -- 172.21.15.78:0/2569008707 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38040042f0 con 0x559f9e4a36b0 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f37f17fa640 1 -- 172.21.15.78:0/2569008707 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f38040ccbc0 con 0x559f9e4a36b0 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f3816a03740 1 -- 172.21.15.78:0/2569008707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f380001edc0 con 0x559f9e4a36b0 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.980+0000 7f37f17fa640 1 -- 172.21.15.78:0/2569008707 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f38040035f0 con 0x559f9e4a36b0 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f3816a03740 10 client.17487 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f3816a03740 1 -- 172.21.15.78:0/2569008707 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559f9e8ebc50 con 0x559f9e4a36b0 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f37f17fa640 1 -- 172.21.15.78:0/2569008707 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380001edc0 con 0x559f9e4a36b0 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f37f17fa640 1 -- 172.21.15.78:0/2569008707 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3804097d80 con 0x559f9e4a36b0 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f3816a03740 10 client.17487 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:48.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f3816a03740 10 client.17487 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:22:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f3816a03740 20 client.17487 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f3816a03740 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x559f9e4583a0 0x559f9e8ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f3816a03740 4 client.17487 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f3816a03740 1 -- 172.21.15.78:0/2569008707 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f9e7d06c0 con 0x559f9e4583a0 2024-05-21T15:22:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.981+0000 7f380d8f3640 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x559f9e4583a0 0x559f9e8ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f37f17fa640 10 client.17487 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:22:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f380d8f3640 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x559f9e4583a0 0x559f9e8ee390 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bcbce0453573f8f server_cookie=105252a658176a06 in_seq=0 out_seq=0 2024-05-21T15:22:48.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f37f17fa640 1 -- 172.21.15.78:0/2569008707 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f9e7d06c0 con 0x559f9e4583a0 2024-05-21T15:22:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f37f17fa640 10 client.17487 handle_command_reply: tid=0 2024-05-21T15:22:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 client.17487 shutdown 2024-05-21T15:22:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 -- 172.21.15.78:0/2569008707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4a36b0 msgr2=0x559f9e8876d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4a36b0 0x559f9e8876d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f38040cc9f0 tx=0x7f3804017eb0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 -- 172.21.15.78:0/2569008707 shutdown_connections 2024-05-21T15:22:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x559f9e4583a0 0x559f9e8ee390 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559f9e4c4390 0x559f9e884660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559f9e4a36b0 0x559f9e8876d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 --2- 172.21.15.78:0/2569008707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f9e49b4d0 0x559f9e887190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f37f17fa640 0 client.17487 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:22:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 -- 172.21.15.78:0/2569008707 >> 172.21.15.78:0/2569008707 conn(0x559f9e4b84e0 msgr2=0x559f9e8706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 -- 172.21.15.78:0/2569008707 shutdown_connections 2024-05-21T15:22:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 1 -- 172.21.15.78:0/2569008707 wait complete. 2024-05-21T15:22:48.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.982+0000 7f3816a03740 20 client.17487 trim_cache size 0 max 16384 2024-05-21T15:22:48.997 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 -- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37f40a2d00 msgr2=0x7f37f40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37f40a2d00 0x7f37f40a5210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3800009fd0 tx=0x7f3800004ab0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 -- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 msgr2=0x7f3808199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 0x7f3808199cd0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3804005fd0 tx=0x7f380400dde0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 -- 172.21.15.78:0/3249413395 shutdown_connections 2024-05-21T15:22:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f380810c330 0x7f380819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f37f40a2d00 0x7f37f40a5210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38081079d0 0x7f3808199cd0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 --2- 172.21.15.78:0/3249413395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3808101aa0 0x7f3808199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 -- 172.21.15.78:0/3249413395 >> 172.21.15.78:0/3249413395 conn(0x7f38080770b0 msgr2=0x7f3808068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 -- 172.21.15.78:0/3249413395 shutdown_connections 2024-05-21T15:22:48.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:48.983+0000 7f380fb57640 1 -- 172.21.15.78:0/3249413395 wait complete. 2024-05-21T15:22:49.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:22:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.110+0000 7f689a861640 1 -- 172.21.15.78:0/3578080788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 msgr2=0x7f689410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.110+0000 7f689a861640 1 --2- 172.21.15.78:0/3578080788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 0x7f689410e780 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ca0 tx=0x7f688c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.110+0000 7f689a861640 1 -- 172.21.15.78:0/3578080788 shutdown_connections 2024-05-21T15:22:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 --2- 172.21.15.78:0/3578080788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6894101b20 0x7f6894101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 --2- 172.21.15.78:0/3578080788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 0x7f689410e780 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 --2- 172.21.15.78:0/3578080788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6894107a00 0x7f68941015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 -- 172.21.15.78:0/3578080788 >> 172.21.15.78:0/3578080788 conn(0x7f6894077160 msgr2=0x7f68940775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 -- 172.21.15.78:0/3578080788 shutdown_connections 2024-05-21T15:22:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 -- 172.21.15.78:0/3578080788 wait complete. 2024-05-21T15:22:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 Processor -- start 2024-05-21T15:22:49.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 -- start start 2024-05-21T15:22:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6894101b20 0x7f689419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.111+0000 7f689a861640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6894107a00 0x7f689419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f689a861640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 0x7f68941a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f689a861640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6894111d30 con 0x7f689410c360 2024-05-21T15:22:49.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f689a861640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6894111bb0 con 0x7f6894101b20 2024-05-21T15:22:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f689a861640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6894111eb0 con 0x7f6894107a00 2024-05-21T15:22:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 0x7f68941a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 0x7f68941a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50200/0 (socket says 172.21.15.78:50200) 2024-05-21T15:22:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 -- 172.21.15.78:0/837051367 learned_addr learned my addr 172.21.15.78:0/837051367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6893fff640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6894101b20 0x7f689419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f688bfff640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6894107a00 0x7f689419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 -- 172.21.15.78:0/837051367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6894107a00 msgr2=0x7f689419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6894107a00 0x7f689419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 -- 172.21.15.78:0/837051367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6894101b20 msgr2=0x7f689419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6894101b20 0x7f689419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 -- 172.21.15.78:0/837051367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689419b2f0 con 0x7f689410c360 2024-05-21T15:22:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f688bfff640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6894107a00 0x7f689419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6898dfe640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 0x7f68941a55b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f688c031a20 tx=0x7f688c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6891ffb640 1 -- 172.21.15.78:0/837051367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c003c30 con 0x7f689410c360 2024-05-21T15:22:49.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f689a861640 1 -- 172.21.15.78:0/837051367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68941a5b50 con 0x7f689410c360 2024-05-21T15:22:49.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6891ffb640 1 -- 172.21.15.78:0/837051367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688c003dd0 con 0x7f689410c360 2024-05-21T15:22:49.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f6891ffb640 1 -- 172.21.15.78:0/837051367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c02fc40 con 0x7f689410c360 2024-05-21T15:22:49.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.112+0000 7f689a861640 1 -- 172.21.15.78:0/837051367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941a60f0 con 0x7f689410c360 2024-05-21T15:22:49.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.115+0000 7f6891ffb640 1 -- 172.21.15.78:0/837051367 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f688c004050 con 0x7f689410c360 2024-05-21T15:22:49.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.115+0000 7f68a17e9740 1 Processor -- start 2024-05-21T15:22:49.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.115+0000 7f68a17e9740 1 -- start start 2024-05-21T15:22:49.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.115+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edff2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edff3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edff37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f68a17e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5564edf58c40 con 0x5564edc336b0 2024-05-21T15:22:49.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f68a17e9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5564edf00580 con 0x5564edc2b4d0 2024-05-21T15:22:49.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f68a17e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5564edf54c00 con 0x5564edc54390 2024-05-21T15:22:49.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edff37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edff37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50214/0 (socket says 172.21.15.78:50214) 2024-05-21T15:22:49.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 -- 172.21.15.78:0/1894180986 learned_addr learned my addr 172.21.15.78:0/1894180986 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f6893fff640 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edff3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edff2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f6891ffb640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f68800a2d00 0x7f68800a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 -- 172.21.15.78:0/1894180986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 msgr2=0x5564edff2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edff2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 -- 172.21.15.78:0/1894180986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 msgr2=0x5564edff3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edff3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 -- 172.21.15.78:0/1894180986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564edc6c3f0 con 0x5564edc336b0 2024-05-21T15:22:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f6893fff640 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edff3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f6898dfe640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f68800a2d00 0x7f68800a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688bfff640 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edff37a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f687c009460 tx=0x7f687c0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7883658792f17387 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f6891ffb640 1 -- 172.21.15.78:0/837051367 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f688c0f05d0 con 0x7f689410c360 2024-05-21T15:22:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.116+0000 7f688a7fc640 1 -- 172.21.15.78:0/1894180986 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c019070 con 0x5564edc336b0 2024-05-21T15:22:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f688a7fc640 1 -- 172.21.15.78:0/1894180986 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c0047a0 con 0x5564edc336b0 2024-05-21T15:22:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f688a7fc640 1 -- 172.21.15.78:0/1894180986 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c011960 con 0x5564edc336b0 2024-05-21T15:22:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 -- 172.21.15.78:0/1894180986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 msgr2=0x5564edff37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edff37a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f687c009460 tx=0x7f687c0096e0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f6898dfe640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f68800a2d00 0x7f68800a5210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f688c0bcce0 tx=0x7f688c02a8c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 -- 172.21.15.78:0/1894180986 shutdown_connections 2024-05-21T15:22:49.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edff3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edff37a0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1894180986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edff2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 -- 172.21.15.78:0/1894180986 >> 172.21.15.78:0/1894180986 conn(0x5564edc484e0 msgr2=0x5564edfef4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 -- 172.21.15.78:0/1894180986 shutdown_connections 2024-05-21T15:22:49.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.117+0000 7f68a17e9740 1 -- 172.21.15.78:0/1894180986 wait complete. 2024-05-21T15:22:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 1 -- start start 2024-05-21T15:22:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 1 Processor -- start 2024-05-21T15:22:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edffed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edffd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5564ee000010 con 0x5564edc336b0 2024-05-21T15:22:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5564edf13c40 con 0x5564edc2b4d0 2024-05-21T15:22:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5564ee000190 con 0x5564edc54390 2024-05-21T15:22:49.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f68a17e9740 10 client.0 _ll_register_callbacks cb 0x5564edc6bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f6893fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f6893fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50222/0 (socket says 172.21.15.78:50222) 2024-05-21T15:22:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.118+0000 7f6893fff640 1 -- 172.21.15.78:0/1382466685 learned_addr learned my addr 172.21.15.78:0/1382466685 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f688bfff640 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edffd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f688bfff640 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edffed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f6893fff640 1 -- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 msgr2=0x5564edffd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f6893fff640 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edffd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f6893fff640 1 -- 172.21.15.78:0/1382466685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 msgr2=0x5564edffed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f6893fff640 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edffed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f6893fff640 1 -- 172.21.15.78:0/1382466685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564edd367d0 con 0x5564edc336b0 2024-05-21T15:22:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f688bfff640 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edffd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f6893fff640 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfff270 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f688400b970 tx=0x7f688400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdb6eb75719598d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f68737fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f68737fe640 1 -- 172.21.15.78:0/1382466685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884005020 con 0x5564edc336b0 2024-05-21T15:22:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f68737fe640 1 -- 172.21.15.78:0/1382466685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f68840051c0 con 0x5564edc336b0 2024-05-21T15:22:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f68a17e9740 1 -- 172.21.15.78:0/1382466685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c0047a0 con 0x5564edc336b0 2024-05-21T15:22:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f68737fe640 1 -- 172.21.15.78:0/1382466685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f68840053c0 con 0x5564edc336b0 2024-05-21T15:22:49.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f68a17e9740 10 client.17499 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f68a17e9740 1 -- 172.21.15.78:0/1382466685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5564ee00c8c0 con 0x5564edc336b0 2024-05-21T15:22:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.119+0000 7f68737fe640 1 -- 172.21.15.78:0/1382466685 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c0047a0 con 0x5564edc336b0 2024-05-21T15:22:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f68737fe640 1 -- 172.21.15.78:0/1382466685 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6884019030 con 0x5564edc336b0 2024-05-21T15:22:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f68a17e9740 10 client.17499 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f68a17e9740 10 client.17499 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:22:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f68a17e9740 20 client.17499 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5564edc12f60 0x5564ee00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f68a17e9740 4 client.17499 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f68a17e9740 1 -- 172.21.15.78:0/1382466685 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564edf66370 con 0x5564edc12f60 2024-05-21T15:22:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f688bfff640 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5564edc12f60 0x5564ee00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f68737fe640 10 client.17499 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:22:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.120+0000 7f688bfff640 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5564edc12f60 0x5564ee00fdd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=204f44b967152cea server_cookie=d07da90511d3d0cb in_seq=0 out_seq=0 2024-05-21T15:22:49.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.121+0000 7f68737fe640 1 -- 172.21.15.78:0/1382466685 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564edf66370 con 0x5564edc12f60 2024-05-21T15:22:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.121+0000 7f68737fe640 10 client.17499 handle_command_reply: tid=0 2024-05-21T15:22:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.121+0000 7f68a17e9740 1 client.17499 shutdown 2024-05-21T15:22:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.121+0000 7f68a17e9740 1 -- 172.21.15.78:0/1382466685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 msgr2=0x5564edfff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.121+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfff270 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f688400b970 tx=0x7f688400be30 comp rx=0 tx=0).stop 2024-05-21T15:22:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.121+0000 7f68a17e9740 1 -- 172.21.15.78:0/1382466685 shutdown_connections 2024-05-21T15:22:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.121+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5564edc12f60 0x5564ee00fdd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.121+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc54390 0x5564edffd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.122+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfff270 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.122+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1382466685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edffed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.122+0000 7f68737fe640 0 client.17499 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:22:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.122+0000 7f68a17e9740 1 -- 172.21.15.78:0/1382466685 >> 172.21.15.78:0/1382466685 conn(0x5564edc484e0 msgr2=0x5564edff4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.122+0000 7f68a17e9740 1 -- 172.21.15.78:0/1382466685 shutdown_connections 2024-05-21T15:22:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.122+0000 7f68a17e9740 1 -- 172.21.15.78:0/1382466685 wait complete. 2024-05-21T15:22:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.122+0000 7f68a17e9740 20 client.17499 trim_cache size 0 max 16384 2024-05-21T15:22:49.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f68a17e9740 1 Processor -- start 2024-05-21T15:22:49.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f68a17e9740 1 -- start start 2024-05-21T15:22:49.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc12f60 0x5564edffcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edffcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfeeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f68a17e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68941aebc0 con 0x5564edc336b0 2024-05-21T15:22:49.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f68a17e9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68941b0700 con 0x5564edc2b4d0 2024-05-21T15:22:49.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f68a17e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68941af290 con 0x5564edc12f60 2024-05-21T15:22:49.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.135+0000 7f688bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfeeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfeeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50230/0 (socket says 172.21.15.78:50230) 2024-05-21T15:22:49.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 -- 172.21.15.78:0/545755458 learned_addr learned my addr 172.21.15.78:0/545755458 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f6893fff640 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edffcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc12f60 0x5564edffcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 -- 172.21.15.78:0/545755458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc12f60 msgr2=0x5564edffcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc12f60 0x5564edffcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 -- 172.21.15.78:0/545755458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 msgr2=0x5564edffcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edffcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 -- 172.21.15.78:0/545755458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564edffff50 con 0x5564edc336b0 2024-05-21T15:22:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f688bfff640 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfeeb50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e461cd0890345a5b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f6889ffb640 1 -- 172.21.15.78:0/545755458 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c019040 con 0x5564edc336b0 2024-05-21T15:22:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f6889ffb640 1 -- 172.21.15.78:0/545755458 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c030030 con 0x5564edc336b0 2024-05-21T15:22:49.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.136+0000 7f6889ffb640 1 -- 172.21.15.78:0/545755458 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c02b520 con 0x5564edc336b0 2024-05-21T15:22:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 -- 172.21.15.78:0/545755458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 msgr2=0x5564edfeeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfeeb50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c015040 comp rx=0 tx=0).stop 2024-05-21T15:22:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 -- 172.21.15.78:0/545755458 shutdown_connections 2024-05-21T15:22:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564edffcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc336b0 0x5564edfeeb50 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 --2- 172.21.15.78:0/545755458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc12f60 0x5564edffcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 -- 172.21.15.78:0/545755458 >> 172.21.15.78:0/545755458 conn(0x5564edc484e0 msgr2=0x5564edff6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 -- 172.21.15.78:0/545755458 shutdown_connections 2024-05-21T15:22:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 -- 172.21.15.78:0/545755458 wait complete. 2024-05-21T15:22:49.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.137+0000 7f68a17e9740 1 -- start start 2024-05-21T15:22:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68a17e9740 1 Processor -- start 2024-05-21T15:22:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc12f60 0x5564ee005390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564ee0058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68a17e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc336b0 0x5564ee002820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68a17e9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68941b0030 con 0x5564edc12f60 2024-05-21T15:22:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68a17e9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68941af960 con 0x5564edc2b4d0 2024-05-21T15:22:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68a17e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68941acba0 con 0x5564edc336b0 2024-05-21T15:22:49.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68a17e9740 10 client.0 _ll_register_callbacks cb 0x7f6884004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc12f60 0x5564ee005390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc12f60 0x5564ee005390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50236/0 (socket says 172.21.15.78:50236) 2024-05-21T15:22:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 -- 172.21.15.78:0/1478826231 learned_addr learned my addr 172.21.15.78:0/1478826231 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f6893fff640 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564ee0058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc336b0 0x5564ee002820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 -- 172.21.15.78:0/1478826231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc336b0 msgr2=0x5564ee002820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc336b0 0x5564ee002820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 -- 172.21.15.78:0/1478826231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 msgr2=0x5564ee0058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564ee0058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 -- 172.21.15.78:0/1478826231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ee002e60 con 0x5564edc12f60 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f688bfff640 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc12f60 0x5564ee005390 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f687c009460 tx=0x7f687c004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5bbeb8af1f954f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.138+0000 7f68737fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68737fe640 1 -- 172.21.15.78:0/1478826231 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c019040 con 0x5564edc12f60 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68737fe640 1 -- 172.21.15.78:0/1478826231 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f687c02b6a0 con 0x5564edc12f60 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 1 -- 172.21.15.78:0/1478826231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c030030 con 0x5564edc12f60 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68737fe640 1 -- 172.21.15.78:0/1478826231 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f687c044050 con 0x5564edc12f60 2024-05-21T15:22:49.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 10 client.17505 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 1 -- 172.21.15.78:0/1478826231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5564ee077af0 con 0x5564edc12f60 2024-05-21T15:22:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68737fe640 1 -- 172.21.15.78:0/1478826231 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c030030 con 0x5564edc12f60 2024-05-21T15:22:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68737fe640 1 -- 172.21.15.78:0/1478826231 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f687c035470 con 0x5564edc12f60 2024-05-21T15:22:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 10 client.17505 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 10 client.17505 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:22:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 20 client.17505 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5564edc54390 0x5564ee079fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 4 client.17505 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f68a17e9740 1 -- 172.21.15.78:0/1478826231 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564edf606c0 con 0x5564edc54390 2024-05-21T15:22:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.139+0000 7f6893fff640 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5564edc54390 0x5564ee079fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.140+0000 7f68737fe640 10 client.17505 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:22:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.140+0000 7f6893fff640 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5564edc54390 0x5564ee079fc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c10d7c1b346b0751 server_cookie=1bc66d73915e5dad in_seq=0 out_seq=0 2024-05-21T15:22:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.140+0000 7f68737fe640 1 -- 172.21.15.78:0/1478826231 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564edf606c0 con 0x5564edc54390 2024-05-21T15:22:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.140+0000 7f68737fe640 10 client.17505 handle_command_reply: tid=0 2024-05-21T15:22:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.140+0000 7f68a17e9740 1 client.17505 shutdown 2024-05-21T15:22:49.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.140+0000 7f68a17e9740 1 -- 172.21.15.78:0/1478826231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc12f60 msgr2=0x5564ee005390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.140+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc12f60 0x5564ee005390 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f687c009460 tx=0x7f687c004bf0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 1 -- 172.21.15.78:0/1478826231 shutdown_connections 2024-05-21T15:22:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5564edc54390 0x5564ee079fc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5564edc2b4d0 0x5564ee0058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564edc336b0 0x5564ee002820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68737fe640 0 client.17505 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:22:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 1 --2- 172.21.15.78:0/1478826231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564edc12f60 0x5564ee005390 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 1 -- 172.21.15.78:0/1478826231 >> 172.21.15.78:0/1478826231 conn(0x5564edc484e0 msgr2=0x5564edffca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 1 -- 172.21.15.78:0/1478826231 shutdown_connections 2024-05-21T15:22:49.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 1 -- 172.21.15.78:0/1478826231 wait complete. 2024-05-21T15:22:49.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f68a17e9740 20 client.17505 trim_cache size 0 max 16384 2024-05-21T15:22:49.179 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:49.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 -- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f68800a2d00 msgr2=0x7f68800a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f68800a2d00 0x7f68800a5210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f688c0bcce0 tx=0x7f688c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 -- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 msgr2=0x7f68941a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 0x7f68941a55b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f688c031a20 tx=0x7f688c031a50 comp rx=0 tx=0).stop 2024-05-21T15:22:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 -- 172.21.15.78:0/837051367 shutdown_connections 2024-05-21T15:22:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f68800a2d00 0x7f68800a5210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410c360 0x7f68941a55b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6894107a00 0x7f689419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 --2- 172.21.15.78:0/837051367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6894101b20 0x7f689419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 -- 172.21.15.78:0/837051367 >> 172.21.15.78:0/837051367 conn(0x7f6894077160 msgr2=0x7f6894103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 -- 172.21.15.78:0/837051367 shutdown_connections 2024-05-21T15:22:49.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.141+0000 7f689a861640 1 -- 172.21.15.78:0/837051367 wait complete. 2024-05-21T15:22:49.182 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:22:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.292+0000 7fd691073640 1 -- 172.21.15.78:0/1654193217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 msgr2=0x7fd68c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.292+0000 7fd691073640 1 --2- 172.21.15.78:0/1654193217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 0x7fd68c101eb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd674009900 tx=0x7fd67402a410 comp rx=0 tx=0).stop 2024-05-21T15:22:49.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.292+0000 7fd691073640 1 -- 172.21.15.78:0/1654193217 shutdown_connections 2024-05-21T15:22:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.292+0000 7fd691073640 1 --2- 172.21.15.78:0/1654193217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c10c330 0x7fd68c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.292+0000 7fd691073640 1 --2- 172.21.15.78:0/1654193217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 0x7fd68c101eb0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.292+0000 7fd691073640 1 --2- 172.21.15.78:0/1654193217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd68c1079d0 0x7fd68c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.292+0000 7fd691073640 1 -- 172.21.15.78:0/1654193217 >> 172.21.15.78:0/1654193217 conn(0x7fd68c0770b0 msgr2=0x7fd68c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.293+0000 7fd691073640 1 -- 172.21.15.78:0/1654193217 shutdown_connections 2024-05-21T15:22:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.293+0000 7fd691073640 1 -- 172.21.15.78:0/1654193217 wait complete. 2024-05-21T15:22:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.293+0000 7fd691073640 1 Processor -- start 2024-05-21T15:22:49.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.293+0000 7fd691073640 1 -- start start 2024-05-21T15:22:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd691073640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 0x7fd68c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd691073640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c1079d0 0x7fd68c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd691073640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd68c10c330 0x7fd68c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd691073640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c111d30 con 0x7fd68c101aa0 2024-05-21T15:22:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd691073640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c111bb0 con 0x7fd68c1079d0 2024-05-21T15:22:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd691073640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c111eb0 con 0x7fd68c10c330 2024-05-21T15:22:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 0x7fd68c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 0x7fd68c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50254/0 (socket says 172.21.15.78:50254) 2024-05-21T15:22:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 -- 172.21.15.78:0/366127899 learned_addr learned my addr 172.21.15.78:0/366127899 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd683fff640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c1079d0 0x7fd68c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68b59e640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd68c10c330 0x7fd68c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 -- 172.21.15.78:0/366127899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd68c10c330 msgr2=0x7fd68c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd68c10c330 0x7fd68c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 -- 172.21.15.78:0/366127899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c1079d0 msgr2=0x7fd68c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c1079d0 0x7fd68c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 -- 172.21.15.78:0/366127899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c19d590 con 0x7fd68c101aa0 2024-05-21T15:22:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 0x7fd68c198730 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d740 tx=0x7fd67c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd688d99640 1 -- 172.21.15.78:0/366127899 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c017070 con 0x7fd68c101aa0 2024-05-21T15:22:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd688d99640 1 -- 172.21.15.78:0/366127899 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c00de50 con 0x7fd68c101aa0 2024-05-21T15:22:49.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd691073640 1 -- 172.21.15.78:0/366127899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c19d880 con 0x7fd68c101aa0 2024-05-21T15:22:49.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.294+0000 7fd688d99640 1 -- 172.21.15.78:0/366127899 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c004e00 con 0x7fd68c101aa0 2024-05-21T15:22:49.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.295+0000 7fd691073640 1 -- 172.21.15.78:0/366127899 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1a5200 con 0x7fd68c101aa0 2024-05-21T15:22:49.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.297+0000 7fd688d99640 1 -- 172.21.15.78:0/366127899 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd67c004fa0 con 0x7fd68c101aa0 2024-05-21T15:22:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.297+0000 7fd697ff8740 1 Processor -- start 2024-05-21T15:22:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.297+0000 7fd697ff8740 1 -- start start 2024-05-21T15:22:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87f5390 0x5594d8b95de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 0x5594d8b96320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8b96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd697ff8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594d8af9c40 con 0x5594d87d46b0 2024-05-21T15:22:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd697ff8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5594d8aa1580 con 0x5594d87cc4d0 2024-05-21T15:22:49.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd697ff8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594d8af5c00 con 0x5594d87f5390 2024-05-21T15:22:49.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8b96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8b96860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50256/0 (socket says 172.21.15.78:50256) 2024-05-21T15:22:49.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 -- 172.21.15.78:0/144031704 learned_addr learned my addr 172.21.15.78:0/144031704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd68b59e640 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87f5390 0x5594d8b95de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd688d99640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd6780a2d80 0x7fd6780a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 0x5594d8b96320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd68b59e640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd6780a2d80 0x7fd6780a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 -- 172.21.15.78:0/144031704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87f5390 msgr2=0x5594d8b95de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87f5390 0x5594d8b95de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 -- 172.21.15.78:0/144031704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 msgr2=0x5594d8b96320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 0x5594d8b96320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 -- 172.21.15.78:0/144031704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d880d3f0 con 0x5594d87d46b0 2024-05-21T15:22:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd68b59e640 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87f5390 0x5594d8b95de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:22:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd688d99640 1 -- 172.21.15.78:0/366127899 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd67c0ced70 con 0x7fd68c101aa0 2024-05-21T15:22:49.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.298+0000 7fd683fff640 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8b96860 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd68c06b480 tx=0x7fd674005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509609941247e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.299+0000 7fd68b59e640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd6780a2d80 0x7fd6780a5290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd684009fd0 tx=0x7fd684004ab0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.299+0000 7fd6817fa640 1 -- 172.21.15.78:0/144031704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674035040 con 0x5594d87d46b0 2024-05-21T15:22:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.299+0000 7fd6817fa640 1 -- 172.21.15.78:0/144031704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd674005520 con 0x5594d87d46b0 2024-05-21T15:22:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.299+0000 7fd6817fa640 1 -- 172.21.15.78:0/144031704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6740035f0 con 0x5594d87d46b0 2024-05-21T15:22:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 -- 172.21.15.78:0/144031704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 msgr2=0x5594d8b96860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8b96860 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd68c06b480 tx=0x7fd674005fd0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 -- 172.21.15.78:0/144031704 shutdown_connections 2024-05-21T15:22:49.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 0x5594d8b96320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8b96860 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 --2- 172.21.15.78:0/144031704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87f5390 0x5594d8b95de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 -- 172.21.15.78:0/144031704 >> 172.21.15.78:0/144031704 conn(0x5594d87e94e0 msgr2=0x5594d8b926e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 -- 172.21.15.78:0/144031704 shutdown_connections 2024-05-21T15:22:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 -- 172.21.15.78:0/144031704 wait complete. 2024-05-21T15:22:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 -- start start 2024-05-21T15:22:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 Processor -- start 2024-05-21T15:22:49.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8ba2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87d46b0 0x5594d8ba2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.300+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87f5390 0x5594d8bae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd697ff8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594d8ba5450 con 0x5594d87cc4d0 2024-05-21T15:22:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd697ff8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5594d8ab4c40 con 0x5594d87f5390 2024-05-21T15:22:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd697ff8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594d8ba55d0 con 0x5594d87d46b0 2024-05-21T15:22:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd697ff8740 10 client.0 _ll_register_callbacks cb 0x5594d880cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8ba2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8ba2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50266/0 (socket says 172.21.15.78:50266) 2024-05-21T15:22:49.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 -- 172.21.15.78:0/610637993 learned_addr learned my addr 172.21.15.78:0/610637993 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68b59e640 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87f5390 0x5594d8bae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd683fff640 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87d46b0 0x5594d8ba2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 -- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87d46b0 msgr2=0x5594d8ba2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87d46b0 0x5594d8ba2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 -- 172.21.15.78:0/610637993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87f5390 msgr2=0x5594d8bae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87f5390 0x5594d8bae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 -- 172.21.15.78:0/610637993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d88d77d0 con 0x5594d87cc4d0 2024-05-21T15:22:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd683fff640 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87d46b0 0x5594d8ba2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8ba2930 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd67c09b270 tx=0x7fd67c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3124d2080c1f8405 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd6727fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd6727fc640 1 -- 172.21.15.78:0/610637993 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0b4060 con 0x5594d87cc4d0 2024-05-21T15:22:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd6727fc640 1 -- 172.21.15.78:0/610637993 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd67c097950 con 0x5594d87cc4d0 2024-05-21T15:22:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd697ff8740 1 -- 172.21.15.78:0/610637993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd674005520 con 0x5594d87cc4d0 2024-05-21T15:22:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd6727fc640 1 -- 172.21.15.78:0/610637993 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd67c020040 con 0x5594d87cc4d0 2024-05-21T15:22:49.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd697ff8740 10 client.17517 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd697ff8740 1 -- 172.21.15.78:0/610637993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5594d8bb1b10 con 0x5594d87cc4d0 2024-05-21T15:22:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.301+0000 7fd6727fc640 1 -- 172.21.15.78:0/610637993 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd674005520 con 0x5594d87cc4d0 2024-05-21T15:22:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.302+0000 7fd6727fc640 1 -- 172.21.15.78:0/610637993 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd67c0ccbb0 con 0x5594d87cc4d0 2024-05-21T15:22:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.302+0000 7fd697ff8740 10 client.17517 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.302+0000 7fd697ff8740 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:22:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.302+0000 7fd697ff8740 20 client.17517 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.302+0000 7fd697ff8740 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594d87b3f60 0x5594d8bb4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.302+0000 7fd697ff8740 4 client.17517 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.302+0000 7fd697ff8740 1 -- 172.21.15.78:0/610637993 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594d8b07370 con 0x5594d87b3f60 2024-05-21T15:22:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.302+0000 7fd683fff640 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594d87b3f60 0x5594d8bb4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd6727fc640 10 client.17517 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:22:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd683fff640 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594d87b3f60 0x5594d8bb4fe0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88e1388fd10e1fed server_cookie=591704c9881be0ba in_seq=0 out_seq=0 2024-05-21T15:22:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd6727fc640 1 -- 172.21.15.78:0/610637993 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594d8b07370 con 0x5594d87b3f60 2024-05-21T15:22:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd6727fc640 10 client.17517 handle_command_reply: tid=0 2024-05-21T15:22:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd697ff8740 1 client.17517 shutdown 2024-05-21T15:22:49.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd697ff8740 1 -- 172.21.15.78:0/610637993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 msgr2=0x5594d8ba2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd697ff8740 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8ba2930 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd67c09b270 tx=0x7fd67c0048a0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd697ff8740 1 -- 172.21.15.78:0/610637993 shutdown_connections 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd697ff8740 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594d87b3f60 0x5594d8bb4fe0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd697ff8740 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87f5390 0x5594d8bae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd6727fc640 0 client.17517 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.303+0000 7fd697ff8740 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87d46b0 0x5594d8ba2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.304+0000 7fd697ff8740 1 --2- 172.21.15.78:0/610637993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8ba2930 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.304+0000 7fd697ff8740 1 -- 172.21.15.78:0/610637993 >> 172.21.15.78:0/610637993 conn(0x5594d87e94e0 msgr2=0x5594d8b97be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.304+0000 7fd697ff8740 1 -- 172.21.15.78:0/610637993 shutdown_connections 2024-05-21T15:22:49.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.304+0000 7fd697ff8740 1 -- 172.21.15.78:0/610637993 wait complete. 2024-05-21T15:22:49.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.304+0000 7fd697ff8740 20 client.17517 trim_cache size 0 max 16384 2024-05-21T15:22:49.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.317+0000 7fd697ff8740 1 Processor -- start 2024-05-21T15:22:49.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.317+0000 7fd697ff8740 1 -- start start 2024-05-21T15:22:49.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8ba1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 0x5594d8b92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8bae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd697ff8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1ac860 con 0x5594d87d46b0 2024-05-21T15:22:49.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd697ff8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1ae3a0 con 0x5594d87cc4d0 2024-05-21T15:22:49.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd697ff8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1acf30 con 0x5594d87b3f60 2024-05-21T15:22:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8bae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8bae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50270/0 (socket says 172.21.15.78:50270) 2024-05-21T15:22:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 -- 172.21.15.78:0/1861479500 learned_addr learned my addr 172.21.15.78:0/1861479500 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd683fff640 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 0x5594d8b92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8ba1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 -- 172.21.15.78:0/1861479500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 msgr2=0x5594d8ba1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8ba1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 -- 172.21.15.78:0/1861479500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 msgr2=0x5594d8b92830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 0x5594d8b92830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 -- 172.21.15.78:0/1861479500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d8b8a870 con 0x5594d87d46b0 2024-05-21T15:22:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd68b59e640 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8bae4a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd6840049d0 tx=0x7fd684009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2881c5541e5392 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd680ff9640 1 -- 172.21.15.78:0/1861479500 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684003b70 con 0x5594d87d46b0 2024-05-21T15:22:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd680ff9640 1 -- 172.21.15.78:0/1861479500 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684028070 con 0x5594d87d46b0 2024-05-21T15:22:49.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.318+0000 7fd680ff9640 1 -- 172.21.15.78:0/1861479500 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684023bc0 con 0x5594d87d46b0 2024-05-21T15:22:49.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 -- 172.21.15.78:0/1861479500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 msgr2=0x5594d8bae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8bae4a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd6840049d0 tx=0x7fd684009920 comp rx=0 tx=0).stop 2024-05-21T15:22:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 -- 172.21.15.78:0/1861479500 shutdown_connections 2024-05-21T15:22:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87cc4d0 0x5594d8b92830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87d46b0 0x5594d8bae4a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 --2- 172.21.15.78:0/1861479500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8ba1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 -- 172.21.15.78:0/1861479500 >> 172.21.15.78:0/1861479500 conn(0x5594d87e94e0 msgr2=0x5594d8b980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 -- 172.21.15.78:0/1861479500 shutdown_connections 2024-05-21T15:22:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.319+0000 7fd697ff8740 1 -- 172.21.15.78:0/1861479500 wait complete. 2024-05-21T15:22:49.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 1 -- start start 2024-05-21T15:22:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 1 Processor -- start 2024-05-21T15:22:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8baa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8baac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87d46b0 0x5594d8ba5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1adcd0 con 0x5594d87cc4d0 2024-05-21T15:22:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1ad600 con 0x5594d87d46b0 2024-05-21T15:22:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1aa840 con 0x5594d87b3f60 2024-05-21T15:22:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd697ff8740 10 client.0 _ll_register_callbacks cb 0x7fd67c09ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd683fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8baac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd683fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8baac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50282/0 (socket says 172.21.15.78:50282) 2024-05-21T15:22:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd683fff640 1 -- 172.21.15.78:0/49111937 learned_addr learned my addr 172.21.15.78:0/49111937 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8baa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd683fff640 1 -- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 msgr2=0x5594d8baa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd683fff640 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8baa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd683fff640 1 -- 172.21.15.78:0/49111937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87d46b0 msgr2=0x5594d8ba5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd683fff640 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87d46b0 0x5594d8ba5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd683fff640 1 -- 172.21.15.78:0/49111937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d8ba6260 con 0x5594d87cc4d0 2024-05-21T15:22:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.320+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8baa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd683fff640 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8baac20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd674004ab0 tx=0x7fd6740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed92b950c15819b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd6727fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd6727fc640 1 -- 172.21.15.78:0/49111937 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674035040 con 0x5594d87cc4d0 2024-05-21T15:22:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd6727fc640 1 -- 172.21.15.78:0/49111937 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7fd674050380 con 0x5594d87cc4d0 2024-05-21T15:22:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 1 -- 172.21.15.78:0/49111937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd684028070 con 0x5594d87cc4d0 2024-05-21T15:22:49.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd6727fc640 1 -- 172.21.15.78:0/49111937 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd674050580 con 0x5594d87cc4d0 2024-05-21T15:22:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 10 client.17523 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 1 -- 172.21.15.78:0/49111937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5594d8c1cbb0 con 0x5594d87cc4d0 2024-05-21T15:22:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd6727fc640 1 -- 172.21.15.78:0/49111937 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684028070 con 0x5594d87cc4d0 2024-05-21T15:22:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd6727fc640 1 -- 172.21.15.78:0/49111937 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd674031c80 con 0x5594d87cc4d0 2024-05-21T15:22:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 10 client.17523 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 10 client.17523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:22:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 20 client.17523 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:49.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594d87f5390 0x5594d8c1efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 4 client.17523 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.321+0000 7fd697ff8740 1 -- 172.21.15.78:0/49111937 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594d8b016c0 con 0x5594d87f5390 2024-05-21T15:22:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.322+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594d87f5390 0x5594d8c1efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.322+0000 7fd6727fc640 10 client.17523 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:22:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.322+0000 7fd68ad9d640 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594d87f5390 0x5594d8c1efa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee5382899ff4623 server_cookie=bc856549c166f3f5 in_seq=0 out_seq=0 2024-05-21T15:22:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.322+0000 7fd6727fc640 1 -- 172.21.15.78:0/49111937 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594d8b016c0 con 0x5594d87f5390 2024-05-21T15:22:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.322+0000 7fd6727fc640 10 client.17523 handle_command_reply: tid=0 2024-05-21T15:22:49.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.322+0000 7fd697ff8740 1 client.17523 shutdown 2024-05-21T15:22:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 -- 172.21.15.78:0/49111937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 msgr2=0x5594d8baac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8baac20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd674004ab0 tx=0x7fd6740046c0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 -- 172.21.15.78:0/49111937 shutdown_connections 2024-05-21T15:22:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5594d87f5390 0x5594d8c1efa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5594d87d46b0 0x5594d8ba5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594d87cc4d0 0x5594d8baac20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd6727fc640 0 client.17523 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:22:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 --2- 172.21.15.78:0/49111937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d87b3f60 0x5594d8baa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 -- 172.21.15.78:0/49111937 >> 172.21.15.78:0/49111937 conn(0x5594d87e94e0 msgr2=0x5594d8b95cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 -- 172.21.15.78:0/49111937 shutdown_connections 2024-05-21T15:22:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 1 -- 172.21.15.78:0/49111937 wait complete. 2024-05-21T15:22:49.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd697ff8740 20 client.17523 trim_cache size 0 max 16384 2024-05-21T15:22:49.336 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 -- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd6780a2d80 msgr2=0x7fd6780a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd6780a2d80 0x7fd6780a5290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd684009fd0 tx=0x7fd684004ab0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 -- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 msgr2=0x7fd68c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 0x7fd68c198730 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d740 tx=0x7fd67c00dc00 comp rx=0 tx=0).stop 2024-05-21T15:22:49.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 -- 172.21.15.78:0/366127899 shutdown_connections 2024-05-21T15:22:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd68c1079d0 0x7fd68c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd6780a2d80 0x7fd6780a5290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd68c10c330 0x7fd68c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 --2- 172.21.15.78:0/366127899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd68c101aa0 0x7fd68c198730 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 -- 172.21.15.78:0/366127899 >> 172.21.15.78:0/366127899 conn(0x7fd68c0770b0 msgr2=0x7fd68c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 -- 172.21.15.78:0/366127899 shutdown_connections 2024-05-21T15:22:49.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.323+0000 7fd691073640 1 -- 172.21.15.78:0/366127899 wait complete. 2024-05-21T15:22:49.340 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:22:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 -- 172.21.15.78:0/3409286296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2354103c20 msgr2=0x7f2354104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 --2- 172.21.15.78:0/3409286296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2354103c20 0x7f2354104030 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2344009900 tx=0x7f234402a410 comp rx=0 tx=0).stop 2024-05-21T15:22:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 -- 172.21.15.78:0/3409286296 shutdown_connections 2024-05-21T15:22:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 --2- 172.21.15.78:0/3409286296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235410a680 0x7f235410caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 --2- 172.21.15.78:0/3409286296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f235410d070 0x7f235410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 --2- 172.21.15.78:0/3409286296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2354103c20 0x7f2354104030 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 -- 172.21.15.78:0/3409286296 >> 172.21.15.78:0/3409286296 conn(0x7f23540770c0 msgr2=0x7f2354077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 -- 172.21.15.78:0/3409286296 shutdown_connections 2024-05-21T15:22:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.450+0000 7f2359b02640 1 -- 172.21.15.78:0/3409286296 wait complete. 2024-05-21T15:22:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.451+0000 7f2359b02640 1 Processor -- start 2024-05-21T15:22:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.451+0000 7f2359b02640 1 -- start start 2024-05-21T15:22:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.451+0000 7f2359b02640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2354103c20 0x7f235419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.451+0000 7f2359b02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f235410a680 0x7f235419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2359b02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235410d070 0x7f23541996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2359b02640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2354112de0 con 0x7f235410d070 2024-05-21T15:22:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2359b02640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2354112c60 con 0x7f2354103c20 2024-05-21T15:22:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2359b02640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2354112f60 con 0x7f235410a680 2024-05-21T15:22:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235410d070 0x7f23541996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235410d070 0x7f23541996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50298/0 (socket says 172.21.15.78:50298) 2024-05-21T15:22:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 -- 172.21.15.78:0/2312684319 learned_addr learned my addr 172.21.15.78:0/2312684319 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f23537fe640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2354103c20 0x7f235419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2352ffd640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f235410a680 0x7f235419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 -- 172.21.15.78:0/2312684319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f235410a680 msgr2=0x7f235419fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f235410a680 0x7f235419fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 -- 172.21.15.78:0/2312684319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2354103c20 msgr2=0x7f235419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2354103c20 0x7f235419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 -- 172.21.15.78:0/2312684319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2354199ce0 con 0x7f235410d070 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f23537fe640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2354103c20 0x7f235419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2352ffd640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f235410a680 0x7f235419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2353fff640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235410d070 0x7f23541996a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f234c00eaa0 tx=0x7f234c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2350ff9640 1 -- 172.21.15.78:0/2312684319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c00cdb0 con 0x7f235410d070 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2350ff9640 1 -- 172.21.15.78:0/2312684319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234c005020 con 0x7f235410d070 2024-05-21T15:22:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2350ff9640 1 -- 172.21.15.78:0/2312684319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c0116b0 con 0x7f235410d070 2024-05-21T15:22:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2359b02640 1 -- 172.21.15.78:0/2312684319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2354199fd0 con 0x7f235410d070 2024-05-21T15:22:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.452+0000 7f2359b02640 1 -- 172.21.15.78:0/2312684319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23541a62c0 con 0x7f235410d070 2024-05-21T15:22:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f23609af740 1 Processor -- start 2024-05-21T15:22:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f2350ff9640 1 -- 172.21.15.78:0/2312684319 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f234c011850 con 0x7f235410d070 2024-05-21T15:22:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f23609af740 1 -- start start 2024-05-21T15:22:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f23609af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5cf390 0x55cf2a96dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f23609af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f23609af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a96e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f23609af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cf2a8d3c40 con 0x55cf2a5cf390 2024-05-21T15:22:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f23609af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cf2a87b580 con 0x55cf2a5a64d0 2024-05-21T15:22:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.455+0000 7f23609af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cf2a8cfc00 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5cf390 0x55cf2a96dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5cf390 0x55cf2a96dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50312/0 (socket says 172.21.15.78:50312) 2024-05-21T15:22:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 -- 172.21.15.78:0/1932600399 learned_addr learned my addr 172.21.15.78:0/1932600399 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f23537fe640 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a96e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2350ff9640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f233c0a2d80 0x7f233c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 -- 172.21.15.78:0/1932600399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5ae6b0 msgr2=0x55cf2a96e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a96e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 -- 172.21.15.78:0/1932600399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 msgr2=0x55cf2a96e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 -- 172.21.15.78:0/1932600399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2a5e73f0 con 0x55cf2a5cf390 2024-05-21T15:22:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f23537fe640 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2353fff640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f233c0a2d80 0x7f233c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2352ffd640 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5cf390 0x55cf2a96dd20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f234800bfc0 tx=0x7f234800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4306db6be3a4d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2350ff9640 1 -- 172.21.15.78:0/2312684319 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f234c0cf6b0 con 0x7f235410d070 2024-05-21T15:22:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2339ffb640 1 -- 172.21.15.78:0/1932600399 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348005220 con 0x55cf2a5cf390 2024-05-21T15:22:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2339ffb640 1 -- 172.21.15.78:0/1932600399 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2348005e40 con 0x55cf2a5cf390 2024-05-21T15:22:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.456+0000 7f2339ffb640 1 -- 172.21.15.78:0/1932600399 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23480127e0 con 0x55cf2a5cf390 2024-05-21T15:22:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 -- 172.21.15.78:0/1932600399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5cf390 msgr2=0x55cf2a96dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f2353fff640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f233c0a2d80 0x7f233c0a5290 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f234c0cd3a0 tx=0x7f234c0bd040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5cf390 0x55cf2a96dd20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f234800bfc0 tx=0x7f234800b5b0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 -- 172.21.15.78:0/1932600399 shutdown_connections 2024-05-21T15:22:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a96e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 --2- 172.21.15.78:0/1932600399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5cf390 0x55cf2a96dd20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 -- 172.21.15.78:0/1932600399 >> 172.21.15.78:0/1932600399 conn(0x55cf2a5c34e0 msgr2=0x55cf2a96a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 -- 172.21.15.78:0/1932600399 shutdown_connections 2024-05-21T15:22:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.457+0000 7f23609af740 1 -- 172.21.15.78:0/1932600399 wait complete. 2024-05-21T15:22:49.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 1 -- start start 2024-05-21T15:22:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 1 Processor -- start 2024-05-21T15:22:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a979d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5cf390 0x55cf2a978240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cf2a97afb0 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cf2a88ec40 con 0x55cf2a5a64d0 2024-05-21T15:22:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cf2a97b130 con 0x55cf2a5cf390 2024-05-21T15:22:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23609af740 10 client.0 _ll_register_callbacks cb 0x55cf2a5e6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50318/0 (socket says 172.21.15.78:50318) 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.458+0000 7f23537fe640 1 -- 172.21.15.78:0/4018817040 learned_addr learned my addr 172.21.15.78:0/4018817040 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f2352ffd640 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5cf390 0x55cf2a978240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f2352ffd640 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a979d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23537fe640 1 -- 172.21.15.78:0/4018817040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5cf390 msgr2=0x55cf2a978240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23537fe640 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5cf390 0x55cf2a978240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23537fe640 1 -- 172.21.15.78:0/4018817040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 msgr2=0x55cf2a979d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23537fe640 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a979d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23537fe640 1 -- 172.21.15.78:0/4018817040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2a6b17d0 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f2352ffd640 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a979d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23537fe640 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97a2e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2344009900 tx=0x7f2344031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15673d9104595685 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f2332ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f2332ffd640 1 -- 172.21.15.78:0/4018817040 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234403f070 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f2332ffd640 1 -- 172.21.15.78:0/4018817040 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2344004450 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23609af740 1 -- 172.21.15.78:0/4018817040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2348005e40 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f2332ffd640 1 -- 172.21.15.78:0/4018817040 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2344035070 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23609af740 10 client.17535 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.459+0000 7f23609af740 1 -- 172.21.15.78:0/4018817040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cf2a987770 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f2332ffd640 1 -- 172.21.15.78:0/4018817040 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2348005e40 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f2332ffd640 1 -- 172.21.15.78:0/4018817040 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2344004c60 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f23609af740 10 client.17535 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f23609af740 10 client.17535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:22:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f23609af740 20 client.17535 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:49.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f23609af740 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55cf2a58df60 0x55cf2a98aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f23609af740 4 client.17535 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f23609af740 1 -- 172.21.15.78:0/4018817040 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf2a8e1370 con 0x55cf2a58df60 2024-05-21T15:22:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.460+0000 7f2352ffd640 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55cf2a58df60 0x55cf2a98aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.461+0000 7f2332ffd640 10 client.17535 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:22:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.461+0000 7f2352ffd640 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55cf2a58df60 0x55cf2a98aca0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f809547578ee5b06 server_cookie=1d3884825e2574d9 in_seq=0 out_seq=0 2024-05-21T15:22:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.461+0000 7f2332ffd640 1 -- 172.21.15.78:0/4018817040 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf2a8e1370 con 0x55cf2a58df60 2024-05-21T15:22:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.461+0000 7f2332ffd640 10 client.17535 handle_command_reply: tid=0 2024-05-21T15:22:49.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.461+0000 7f23609af740 1 client.17535 shutdown 2024-05-21T15:22:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.461+0000 7f23609af740 1 -- 172.21.15.78:0/4018817040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 msgr2=0x55cf2a97a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.461+0000 7f23609af740 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97a2e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2344009900 tx=0x7f2344031920 comp rx=0 tx=0).stop 2024-05-21T15:22:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 1 -- 172.21.15.78:0/4018817040 shutdown_connections 2024-05-21T15:22:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55cf2a58df60 0x55cf2a98aca0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5cf390 0x55cf2a978240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97a2e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 1 --2- 172.21.15.78:0/4018817040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5a64d0 0x55cf2a979d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f2332ffd640 0 client.17535 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:22:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 1 -- 172.21.15.78:0/4018817040 >> 172.21.15.78:0/4018817040 conn(0x55cf2a5c34e0 msgr2=0x55cf2a96fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 1 -- 172.21.15.78:0/4018817040 shutdown_connections 2024-05-21T15:22:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 1 -- 172.21.15.78:0/4018817040 wait complete. 2024-05-21T15:22:49.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.462+0000 7f23609af740 20 client.17535 trim_cache size 0 max 16384 2024-05-21T15:22:49.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23609af740 1 Processor -- start 2024-05-21T15:22:49.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23609af740 1 -- start start 2024-05-21T15:22:49.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23609af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23609af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a990460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23609af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23609af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23541aecb0 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23609af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f23541b07f0 con 0x55cf2a5cf390 2024-05-21T15:22:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23609af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23541af380 con 0x55cf2a5a64d0 2024-05-21T15:22:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a990460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a990460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50322/0 (socket says 172.21.15.78:50322) 2024-05-21T15:22:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.475+0000 7f23537fe640 1 -- 172.21.15.78:0/2855441260 learned_addr learned my addr 172.21.15.78:0/2855441260 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23537fe640 1 -- 172.21.15.78:0/2855441260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 msgr2=0x55cf2a96a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f2352ffd640 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f2352ffd640 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23537fe640 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23537fe640 1 -- 172.21.15.78:0/2855441260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 msgr2=0x55cf2a9909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23537fe640 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23537fe640 1 -- 172.21.15.78:0/2855441260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2a972f10 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23537fe640 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a990460 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2344006010 tx=0x7f234402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832ebb8bd57ceaed server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23397fa640 1 -- 172.21.15.78:0/2855441260 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234403f040 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23397fa640 1 -- 172.21.15.78:0/2855441260 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2344003a60 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.476+0000 7f23397fa640 1 -- 172.21.15.78:0/2855441260 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440054a0 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 -- 172.21.15.78:0/2855441260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 msgr2=0x55cf2a990460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a990460 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2344006010 tx=0x7f234402a410 comp rx=0 tx=0).stop 2024-05-21T15:22:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 -- 172.21.15.78:0/2855441260 shutdown_connections 2024-05-21T15:22:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a990460 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 --2- 172.21.15.78:0/2855441260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 0x55cf2a96a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 -- 172.21.15.78:0/2855441260 >> 172.21.15.78:0/2855441260 conn(0x55cf2a5c34e0 msgr2=0x55cf2a969e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 -- 172.21.15.78:0/2855441260 shutdown_connections 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 -- 172.21.15.78:0/2855441260 wait complete. 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 -- start start 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.477+0000 7f23609af740 1 Processor -- start 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23609af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 0x55cf2a97b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23609af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23609af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9f1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23609af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23541b0120 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23609af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f23541afa50 con 0x55cf2a5cf390 2024-05-21T15:22:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23609af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23541acc90 con 0x55cf2a5a64d0 2024-05-21T15:22:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23609af740 10 client.0 _ll_register_callbacks cb 0x7f2344009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50334/0 (socket says 172.21.15.78:50334) 2024-05-21T15:22:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 -- 172.21.15.78:0/2585629978 learned_addr learned my addr 172.21.15.78:0/2585629978 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f2352ffd640 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 -- 172.21.15.78:0/2585629978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 msgr2=0x55cf2a97b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 0x55cf2a97b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 -- 172.21.15.78:0/2585629978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 msgr2=0x55cf2a9f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 -- 172.21.15.78:0/2585629978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2a97c170 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f2352ffd640 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23537fe640 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97b880 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f23440037e0 tx=0x7f2344005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4d8d2191a6b6c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f2332ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f2332ffd640 1 -- 172.21.15.78:0/2585629978 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234405d060 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f2332ffd640 1 -- 172.21.15.78:0/2585629978 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f2344005430 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.478+0000 7f23609af740 1 -- 172.21.15.78:0/2585629978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2344003a60 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f2332ffd640 1 -- 172.21.15.78:0/2585629978 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f234403f070 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f23609af740 10 client.17541 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f23609af740 1 -- 172.21.15.78:0/2585629978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cf2a9f25f0 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f2332ffd640 1 -- 172.21.15.78:0/2585629978 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2344003a60 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f2332ffd640 1 -- 172.21.15.78:0/2585629978 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f234403b700 con 0x55cf2a5ae6b0 2024-05-21T15:22:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f23609af740 10 client.17541 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:49.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f23609af740 10 client.17541 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:22:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f23609af740 20 client.17541 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f23609af740 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55cf2a5633a0 0x55cf2a9f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f23609af740 4 client.17541 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f23609af740 1 -- 172.21.15.78:0/2585629978 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf2a8db6c0 con 0x55cf2a5633a0 2024-05-21T15:22:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.479+0000 7f2352ffd640 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55cf2a5633a0 0x55cf2a9f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f2352ffd640 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55cf2a5633a0 0x55cf2a9f4c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44032be64862aed8 server_cookie=2da9a8af045d01b9 in_seq=0 out_seq=0 2024-05-21T15:22:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f2332ffd640 10 client.17541 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:22:49.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f2332ffd640 1 -- 172.21.15.78:0/2585629978 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf2a8db6c0 con 0x55cf2a5633a0 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f2332ffd640 10 client.17541 handle_command_reply: tid=0 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 client.17541 shutdown 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 -- 172.21.15.78:0/2585629978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 msgr2=0x55cf2a97b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97b880 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f23440037e0 tx=0x7f2344005000 comp rx=0 tx=0).stop 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 -- 172.21.15.78:0/2585629978 shutdown_connections 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55cf2a5633a0 0x55cf2a9f4c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cf2a5cf390 0x55cf2a9f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cf2a5ae6b0 0x55cf2a97b880 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 --2- 172.21.15.78:0/2585629978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf2a5a64d0 0x55cf2a97b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f2332ffd640 0 client.17541 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:22:49.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 -- 172.21.15.78:0/2585629978 >> 172.21.15.78:0/2585629978 conn(0x55cf2a5c34e0 msgr2=0x55cf2a979700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 -- 172.21.15.78:0/2585629978 shutdown_connections 2024-05-21T15:22:49.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 1 -- 172.21.15.78:0/2585629978 wait complete. 2024-05-21T15:22:49.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.480+0000 7f23609af740 20 client.17541 trim_cache size 0 max 16384 2024-05-21T15:22:49.493 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 -- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f233c0a2d80 msgr2=0x7f233c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f233c0a2d80 0x7f233c0a5290 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f234c0cd3a0 tx=0x7f234c0bd040 comp rx=0 tx=0).stop 2024-05-21T15:22:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 -- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235410d070 msgr2=0x7f23541996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235410d070 0x7f23541996a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f234c00eaa0 tx=0x7f234c00ef60 comp rx=0 tx=0).stop 2024-05-21T15:22:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 -- 172.21.15.78:0/2312684319 shutdown_connections 2024-05-21T15:22:49.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f233c0a2d80 0x7f233c0a5290 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f235410d070 0x7f23541996a0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f235410a680 0x7f235419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 --2- 172.21.15.78:0/2312684319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2354103c20 0x7f235419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 -- 172.21.15.78:0/2312684319 >> 172.21.15.78:0/2312684319 conn(0x7f23540770c0 msgr2=0x7f2354100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 -- 172.21.15.78:0/2312684319 shutdown_connections 2024-05-21T15:22:49.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.481+0000 7f2359b02640 1 -- 172.21.15.78:0/2312684319 wait complete. 2024-05-21T15:22:49.497 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:22:49.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 -- 172.21.15.78:0/1198396171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a7410c330 msgr2=0x7f7a7410e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 --2- 172.21.15.78:0/1198396171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a7410c330 0x7f7a7410e750 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009ca0 tx=0x7f7a6c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 -- 172.21.15.78:0/1198396171 shutdown_connections 2024-05-21T15:22:49.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 --2- 172.21.15.78:0/1198396171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a7410c330 0x7f7a7410e750 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 --2- 172.21.15.78:0/1198396171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a74101aa0 0x7f7a74101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 --2- 172.21.15.78:0/1198396171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a741079d0 0x7f7a74101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 -- 172.21.15.78:0/1198396171 >> 172.21.15.78:0/1198396171 conn(0x7f7a740770b0 msgr2=0x7f7a740774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 -- 172.21.15.78:0/1198396171 shutdown_connections 2024-05-21T15:22:49.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.607+0000 7f7a796fe640 1 -- 172.21.15.78:0/1198396171 wait complete. 2024-05-21T15:22:49.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a796fe640 1 Processor -- start 2024-05-21T15:22:49.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a796fe640 1 -- start start 2024-05-21T15:22:49.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a796fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101aa0 0x7f7a7419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a796fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a741079d0 0x7f7a7419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a796fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a7410c330 0x7f7a7419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a796fe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74111d00 con 0x7f7a74101aa0 2024-05-21T15:22:49.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a796fe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74111b80 con 0x7f7a7410c330 2024-05-21T15:22:49.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a796fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74111e80 con 0x7f7a741079d0 2024-05-21T15:22:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101aa0 0x7f7a7419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101aa0 0x7f7a7419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50354/0 (socket says 172.21.15.78:50354) 2024-05-21T15:22:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.608+0000 7f7a72ffd640 1 -- 172.21.15.78:0/3375134136 learned_addr learned my addr 172.21.15.78:0/3375134136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a737fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a7410c330 0x7f7a7419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a72ffd640 1 -- 172.21.15.78:0/3375134136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a741079d0 msgr2=0x7f7a7419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a741079d0 0x7f7a7419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a72ffd640 1 -- 172.21.15.78:0/3375134136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a7410c330 msgr2=0x7f7a7419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a7410c330 0x7f7a7419b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a72ffd640 1 -- 172.21.15.78:0/3375134136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a741a5960 con 0x7f7a74101aa0 2024-05-21T15:22:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101aa0 0x7f7a7419a620 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7a64005fd0 tx=0x7f7a6400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a70ff9640 1 -- 172.21.15.78:0/3375134136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64017070 con 0x7f7a74101aa0 2024-05-21T15:22:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a796fe640 1 -- 172.21.15.78:0/3375134136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a741a5c50 con 0x7f7a74101aa0 2024-05-21T15:22:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a70ff9640 1 -- 172.21.15.78:0/3375134136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a64009c30 con 0x7f7a74101aa0 2024-05-21T15:22:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a70ff9640 1 -- 172.21.15.78:0/3375134136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64010460 con 0x7f7a74101aa0 2024-05-21T15:22:49.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.609+0000 7f7a796fe640 1 -- 172.21.15.78:0/3375134136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a741a6170 con 0x7f7a74101aa0 2024-05-21T15:22:49.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.612+0000 7f7a70ff9640 1 -- 172.21.15.78:0/3375134136 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7a64003e70 con 0x7f7a74101aa0 2024-05-21T15:22:49.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.612+0000 7f7a805a2740 1 Processor -- start 2024-05-21T15:22:49.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.612+0000 7f7a805a2740 1 -- start start 2024-05-21T15:22:49.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639ab2390 0x559639e50d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e51260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a916b0 0x559639e517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a805a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559639db6c40 con 0x559639a894d0 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a805a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559639d5e580 con 0x559639a916b0 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a805a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559639db2c00 con 0x559639ab2390 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e51260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e51260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50366/0 (socket says 172.21.15.78:50366) 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1330478462 learned_addr learned my addr 172.21.15.78:0/1330478462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a737fe640 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639ab2390 0x559639e50d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a916b0 0x559639e517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a70ff9640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a600a2d00 0x7f7a600a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a737fe640 1 -- 172.21.15.78:0/1330478462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a916b0 msgr2=0x559639e517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a737fe640 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a916b0 0x559639e517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a737fe640 1 -- 172.21.15.78:0/1330478462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 msgr2=0x559639e51260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a737fe640 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e51260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a737fe640 1 -- 172.21.15.78:0/1330478462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559639aca3f0 con 0x559639ab2390 2024-05-21T15:22:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a916b0 0x559639e517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:22:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e51260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:22:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a70ff9640 1 -- 172.21.15.78:0/3375134136 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7a640cf0b0 con 0x7f7a74101aa0 2024-05-21T15:22:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.613+0000 7f7a737fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a600a2d00 0x7f7a600a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.614+0000 7f7a737fe640 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639ab2390 0x559639e50d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a7419b600 tx=0x7f7a6c02f4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=748c3c69737532ca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.614+0000 7f7a737fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a600a2d00 0x7f7a600a5210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c049000 tx=0x7f7a6c04e000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.614+0000 7f7a697fa640 1 -- 172.21.15.78:0/1330478462 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c042040 con 0x559639ab2390 2024-05-21T15:22:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.614+0000 7f7a697fa640 1 -- 172.21.15.78:0/1330478462 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a6c036040 con 0x559639ab2390 2024-05-21T15:22:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.614+0000 7f7a697fa640 1 -- 172.21.15.78:0/1330478462 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c02f810 con 0x559639ab2390 2024-05-21T15:22:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 -- 172.21.15.78:0/1330478462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639ab2390 msgr2=0x559639e50d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639ab2390 0x559639e50d20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a7419b600 tx=0x7f7a6c02f4e0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 -- 172.21.15.78:0/1330478462 shutdown_connections 2024-05-21T15:22:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a916b0 0x559639e517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e51260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1330478462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639ab2390 0x559639e50d20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 -- 172.21.15.78:0/1330478462 >> 172.21.15.78:0/1330478462 conn(0x559639aa64e0 msgr2=0x559639e4d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 -- 172.21.15.78:0/1330478462 shutdown_connections 2024-05-21T15:22:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 -- 172.21.15.78:0/1330478462 wait complete. 2024-05-21T15:22:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 -- start start 2024-05-21T15:22:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.615+0000 7f7a805a2740 1 Processor -- start 2024-05-21T15:22:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639e5b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a805a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559639e5e010 con 0x559639a894d0 2024-05-21T15:22:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a805a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559639d71c40 con 0x559639ab2390 2024-05-21T15:22:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a805a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559639e5e190 con 0x559639a916b0 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a805a2740 10 client.0 _ll_register_callbacks cb 0x559639ac9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e5cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50380/0 (socket says 172.21.15.78:50380) 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1384317795 learned_addr learned my addr 172.21.15.78:0/1384317795 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a737fe640 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639e5b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1384317795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 msgr2=0x559639e5d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1384317795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 msgr2=0x559639e5b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639e5b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1384317795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559639b947d0 con 0x559639a894d0 2024-05-21T15:22:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e5cd30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7a6409b500 tx=0x7f7a64097210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615683b66869e0b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1384317795 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6409bc70 con 0x559639a894d0 2024-05-21T15:22:49.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1384317795 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7a6409f150 con 0x559639a894d0 2024-05-21T15:22:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.616+0000 7f7a805a2740 1 -- 172.21.15.78:0/1384317795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6c036040 con 0x559639a894d0 2024-05-21T15:22:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1384317795 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7a64009de0 con 0x559639a894d0 2024-05-21T15:22:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a805a2740 10 client.17553 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a805a2740 1 -- 172.21.15.78:0/1384317795 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559639e6a7d0 con 0x559639a894d0 2024-05-21T15:22:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1384317795 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a6c036040 con 0x559639a894d0 2024-05-21T15:22:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1384317795 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7a640cca20 con 0x559639a894d0 2024-05-21T15:22:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a805a2740 10 client.17553 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a805a2740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a805a2740 20 client.17553 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559639a70f60 0x559639e6dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a805a2740 4 client.17553 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a805a2740 1 -- 172.21.15.78:0/1384317795 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559639dc4370 con 0x559639a70f60 2024-05-21T15:22:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.617+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559639a70f60 0x559639e6dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.618+0000 7f7a5a7fc640 10 client.17553 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:49.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.618+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559639a70f60 0x559639e6dd20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71282f5c45dbc28e server_cookie=6cfc22ba86a93172 in_seq=0 out_seq=0 2024-05-21T15:22:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.618+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1384317795 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559639dc4370 con 0x559639a70f60 2024-05-21T15:22:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.618+0000 7f7a5a7fc640 10 client.17553 handle_command_reply: tid=0 2024-05-21T15:22:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 client.17553 shutdown 2024-05-21T15:22:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 -- 172.21.15.78:0/1384317795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 msgr2=0x559639e5cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e5cd30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7a6409b500 tx=0x7f7a64097210 comp rx=0 tx=0).stop 2024-05-21T15:22:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 -- 172.21.15.78:0/1384317795 shutdown_connections 2024-05-21T15:22:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559639a70f60 0x559639e6dd20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639e5b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a5a7fc640 0 client.17553 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1384317795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e5cd30 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 -- 172.21.15.78:0/1384317795 >> 172.21.15.78:0/1384317795 conn(0x559639aa64e0 msgr2=0x559639e52a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 -- 172.21.15.78:0/1384317795 shutdown_connections 2024-05-21T15:22:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 1 -- 172.21.15.78:0/1384317795 wait complete. 2024-05-21T15:22:49.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.619+0000 7f7a805a2740 20 client.17553 trim_cache size 0 max 16384 2024-05-21T15:22:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.632+0000 7f7a805a2740 1 Processor -- start 2024-05-21T15:22:49.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.632+0000 7f7a805a2740 1 -- start start 2024-05-21T15:22:49.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a894d0 0x559639e5b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639ab2390 0x559639e633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a805a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741aeaa0 con 0x559639ab2390 2024-05-21T15:22:49.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a805a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b05e0 con 0x559639a894d0 2024-05-21T15:22:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a805a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741af170 con 0x559639a916b0 2024-05-21T15:22:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639ab2390 0x559639e633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639ab2390 0x559639e633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50384/0 (socket says 172.21.15.78:50384) 2024-05-21T15:22:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 -- 172.21.15.78:0/476860994 learned_addr learned my addr 172.21.15.78:0/476860994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 -- 172.21.15.78:0/476860994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 msgr2=0x559639e5a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 -- 172.21.15.78:0/476860994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a894d0 msgr2=0x559639e5b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a894d0 0x559639e5b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a894d0 0x559639e5b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 -- 172.21.15.78:0/476860994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559639e4e060 con 0x559639ab2390 2024-05-21T15:22:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a894d0 0x559639e5b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a737fe640 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639ab2390 0x559639e633e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009ca0 tx=0x7f7a6c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd101759029cad8f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a68ff9640 1 -- 172.21.15.78:0/476860994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c042070 con 0x559639ab2390 2024-05-21T15:22:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a68ff9640 1 -- 172.21.15.78:0/476860994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a6c036030 con 0x559639ab2390 2024-05-21T15:22:49.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.633+0000 7f7a68ff9640 1 -- 172.21.15.78:0/476860994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c030820 con 0x559639ab2390 2024-05-21T15:22:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 -- 172.21.15.78:0/476860994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639ab2390 msgr2=0x559639e633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639ab2390 0x559639e633e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009ca0 tx=0x7f7a6c004a80 comp rx=0 tx=0).stop 2024-05-21T15:22:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 -- 172.21.15.78:0/476860994 shutdown_connections 2024-05-21T15:22:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639ab2390 0x559639e633e0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e5a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 --2- 172.21.15.78:0/476860994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639a894d0 0x559639e5b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 -- 172.21.15.78:0/476860994 >> 172.21.15.78:0/476860994 conn(0x559639aa64e0 msgr2=0x559639e59af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 -- 172.21.15.78:0/476860994 shutdown_connections 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.634+0000 7f7a805a2740 1 -- 172.21.15.78:0/476860994 wait complete. 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 1 -- start start 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 1 Processor -- start 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e62800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e62d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639ece070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741aff10 con 0x559639a894d0 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741af840 con 0x559639ab2390 2024-05-21T15:22:49.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741aca80 con 0x559639a916b0 2024-05-21T15:22:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a805a2740 10 client.0 _ll_register_callbacks cb 0x7f7a64005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e62800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e62800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50388/0 (socket says 172.21.15.78:50388) 2024-05-21T15:22:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1030753230 learned_addr learned my addr 172.21.15.78:0/1030753230 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e62d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a737fe640 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639ece070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1030753230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 msgr2=0x559639e62d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e62d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 msgr2=0x559639ece070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639ece070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a72ffd640 1 -- 172.21.15.78:0/1030753230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559639ece5b0 con 0x559639a894d0 2024-05-21T15:22:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.635+0000 7f7a737fe640 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639ece070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a72ffd640 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e62800 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7a64097be0 tx=0x7f7a6409f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d165285b4ae7e43a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1030753230 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6409b0b0 con 0x559639a894d0 2024-05-21T15:22:49.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1030753230 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f7a6409abc0 con 0x559639a894d0 2024-05-21T15:22:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 1 -- 172.21.15.78:0/1030753230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6c036030 con 0x559639a894d0 2024-05-21T15:22:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1030753230 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7a64065050 con 0x559639a894d0 2024-05-21T15:22:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 10 client.17559 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 1 -- 172.21.15.78:0/1030753230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559639ed5810 con 0x559639a894d0 2024-05-21T15:22:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1030753230 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a6c036030 con 0x559639a894d0 2024-05-21T15:22:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1030753230 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7a640972c0 con 0x559639a894d0 2024-05-21T15:22:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 10 client.17559 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:49.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 20 client.17559 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559639a463a0 0x559639ed7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 4 client.17559 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.636+0000 7f7a805a2740 1 -- 172.21.15.78:0/1030753230 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559639dbe6c0 con 0x559639a463a0 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559639a463a0 0x559639ed7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a5a7fc640 10 client.17559 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a6a7fc640 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559639a463a0 0x559639ed7da0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bd43fe0a8473ba server_cookie=9084ebe3cb243bb1 in_seq=0 out_seq=0 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a5a7fc640 1 -- 172.21.15.78:0/1030753230 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559639dbe6c0 con 0x559639a463a0 2024-05-21T15:22:49.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a5a7fc640 10 client.17559 handle_command_reply: tid=0 2024-05-21T15:22:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 client.17559 shutdown 2024-05-21T15:22:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 -- 172.21.15.78:0/1030753230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 msgr2=0x559639e62800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e62800 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7a64097be0 tx=0x7f7a6409f380 comp rx=0 tx=0).stop 2024-05-21T15:22:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 -- 172.21.15.78:0/1030753230 shutdown_connections 2024-05-21T15:22:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x559639a463a0 0x559639ed7da0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559639ab2390 0x559639ece070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a5a7fc640 0 client.17559 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:49.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559639a916b0 0x559639e62d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 --2- 172.21.15.78:0/1030753230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559639a894d0 0x559639e62800 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 -- 172.21.15.78:0/1030753230 >> 172.21.15.78:0/1030753230 conn(0x559639aa64e0 msgr2=0x559639e5a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 -- 172.21.15.78:0/1030753230 shutdown_connections 2024-05-21T15:22:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 1 -- 172.21.15.78:0/1030753230 wait complete. 2024-05-21T15:22:49.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.637+0000 7f7a805a2740 20 client.17559 trim_cache size 0 max 16384 2024-05-21T15:22:49.651 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:22:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 -- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a600a2d00 msgr2=0x7f7a600a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a600a2d00 0x7f7a600a5210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c049000 tx=0x7f7a6c04e000 comp rx=0 tx=0).stop 2024-05-21T15:22:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 -- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101aa0 msgr2=0x7f7a7419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101aa0 0x7f7a7419a620 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7a64005fd0 tx=0x7f7a6400dde0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 -- 172.21.15.78:0/3375134136 shutdown_connections 2024-05-21T15:22:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a7410c330 0x7f7a7419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7a600a2d00 0x7f7a600a5210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a741079d0 0x7f7a7419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 --2- 172.21.15.78:0/3375134136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a74101aa0 0x7f7a7419a620 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 -- 172.21.15.78:0/3375134136 >> 172.21.15.78:0/3375134136 conn(0x7f7a740770b0 msgr2=0x7f7a74103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 -- 172.21.15.78:0/3375134136 shutdown_connections 2024-05-21T15:22:49.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:49.638+0000 7f7a796fe640 1 -- 172.21.15.78:0/3375134136 wait complete. 2024-05-21T15:22:49.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:49 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2908011326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:22:49.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:49 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2908011326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:22:49.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:49 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2908011326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:22:50.655 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:22:50.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:22:50.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:22:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.778+0000 7f6274026640 1 -- 172.21.15.78:0/1988203681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c101b20 msgr2=0x7f626c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.778+0000 7f6274026640 1 --2- 172.21.15.78:0/1988203681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c101b20 0x7f626c101f30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f625c009900 tx=0x7f625c02a410 comp rx=0 tx=0).stop 2024-05-21T15:22:50.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 -- 172.21.15.78:0/1988203681 shutdown_connections 2024-05-21T15:22:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 --2- 172.21.15.78:0/1988203681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626c10c360 0x7f626c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 --2- 172.21.15.78:0/1988203681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c101b20 0x7f626c101f30 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 --2- 172.21.15.78:0/1988203681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f626c107a00 0x7f626c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 -- 172.21.15.78:0/1988203681 >> 172.21.15.78:0/1988203681 conn(0x7f626c0770c0 msgr2=0x7f626c077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 -- 172.21.15.78:0/1988203681 shutdown_connections 2024-05-21T15:22:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 -- 172.21.15.78:0/1988203681 wait complete. 2024-05-21T15:22:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 Processor -- start 2024-05-21T15:22:50.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.779+0000 7f6274026640 1 -- start start 2024-05-21T15:22:50.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f6274026640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f626c101b20 0x7f626c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f6274026640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c107a00 0x7f626c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f6274026640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626c10c360 0x7f626c19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f6274026640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111d10 con 0x7f626c107a00 2024-05-21T15:22:50.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f6274026640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111b90 con 0x7f626c101b20 2024-05-21T15:22:50.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f6274026640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f626c111e90 con 0x7f626c10c360 2024-05-21T15:22:50.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c107a00 0x7f626c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c107a00 0x7f626c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50392/0 (socket says 172.21.15.78:50392) 2024-05-21T15:22:50.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 -- 172.21.15.78:0/3135638410 learned_addr learned my addr 172.21.15.78:0/3135638410 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f6271dc2640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f626c101b20 0x7f626c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62725c3640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626c10c360 0x7f626c19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 -- 172.21.15.78:0/3135638410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626c10c360 msgr2=0x7f626c19b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626c10c360 0x7f626c19b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 -- 172.21.15.78:0/3135638410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f626c101b20 msgr2=0x7f626c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f626c101b20 0x7f626c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 -- 172.21.15.78:0/3135638410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c1a5920 con 0x7f626c107a00 2024-05-21T15:22:50.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.780+0000 7f62715c1640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c107a00 0x7f626c19ab50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f625c005940 tx=0x7f625c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.781+0000 7f6262ffd640 1 -- 172.21.15.78:0/3135638410 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c03f030 con 0x7f626c107a00 2024-05-21T15:22:50.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.781+0000 7f6274026640 1 -- 172.21.15.78:0/3135638410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c1a5bb0 con 0x7f626c107a00 2024-05-21T15:22:50.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.781+0000 7f6262ffd640 1 -- 172.21.15.78:0/3135638410 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c003ce0 con 0x7f626c107a00 2024-05-21T15:22:50.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.781+0000 7f6262ffd640 1 -- 172.21.15.78:0/3135638410 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c035070 con 0x7f626c107a00 2024-05-21T15:22:50.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.781+0000 7f6274026640 1 -- 172.21.15.78:0/3135638410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1a60a0 con 0x7f626c107a00 2024-05-21T15:22:50.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.783+0000 7f6262ffd640 1 -- 172.21.15.78:0/3135638410 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f625c042030 con 0x7f626c107a00 2024-05-21T15:22:50.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.783+0000 7f627afaa740 1 Processor -- start 2024-05-21T15:22:50.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.783+0000 7f627afaa740 1 -- start start 2024-05-21T15:22:50.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.783+0000 7f627afaa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984ba2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f627afaa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 0x561984ba2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f627afaa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f627afaa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561984b07990 con 0x5619847e2730 2024-05-21T15:22:50.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f627afaa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561984aaf5b0 con 0x5619847da550 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f627afaa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561984b03950 con 0x561984803410 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f6262ffd640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f62580a2d00 0x7f62580a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50404/0 (socket says 172.21.15.78:50404) 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 -- 172.21.15.78:0/3523257988 learned_addr learned my addr 172.21.15.78:0/3523257988 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f6271dc2640 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 0x561984ba2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62725c3640 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984ba2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62725c3640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f62580a2d00 0x7f62580a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f6262ffd640 1 -- 172.21.15.78:0/3135638410 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f625c0ef950 con 0x7f626c107a00 2024-05-21T15:22:50.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 -- 172.21.15.78:0/3523257988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 msgr2=0x561984ba2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984ba2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 -- 172.21.15.78:0/3523257988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 msgr2=0x561984ba2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 0x561984ba2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 -- 172.21.15.78:0/3523257988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619848e37e0 con 0x5619847e2730 2024-05-21T15:22:50.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f6271dc2640 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 0x561984ba2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:22:50.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62715c1640 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba30a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f625c004770 tx=0x7f625c0bbdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63c70589af991cb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f62725c3640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f62580a2d00 0x7f62580a5210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x561984b9f790 tx=0x7f6264009600 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.784+0000 7f6257fff640 1 -- 172.21.15.78:0/3523257988 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c049070 con 0x5619847e2730 2024-05-21T15:22:50.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f6257fff640 1 -- 172.21.15.78:0/3523257988 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c0bbec0 con 0x5619847e2730 2024-05-21T15:22:50.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f6257fff640 1 -- 172.21.15.78:0/3523257988 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0ed2c0 con 0x5619847e2730 2024-05-21T15:22:50.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f627afaa740 1 -- 172.21.15.78:0/3523257988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 msgr2=0x561984ba30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f627afaa740 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba30a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f625c004770 tx=0x7f625c0bbdc0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f627afaa740 1 -- 172.21.15.78:0/3523257988 shutdown_connections 2024-05-21T15:22:50.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f627afaa740 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 0x561984ba2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f627afaa740 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba30a0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f627afaa740 1 --2- 172.21.15.78:0/3523257988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984ba2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.785+0000 7f627afaa740 1 -- 172.21.15.78:0/3523257988 >> 172.21.15.78:0/3523257988 conn(0x5619847f7560 msgr2=0x561984b9edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 -- 172.21.15.78:0/3523257988 shutdown_connections 2024-05-21T15:22:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 -- 172.21.15.78:0/3523257988 wait complete. 2024-05-21T15:22:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 -- start start 2024-05-21T15:22:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 Processor -- start 2024-05-21T15:22:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847da550 0x561984bae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847e2730 0x561984baec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984bacbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561984baf7f0 con 0x5619847da550 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561984ac2b40 con 0x5619847e2730 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.786+0000 7f627afaa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561984baf970 con 0x561984803410 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f627afaa740 10 client.0 _ll_register_callbacks cb 0x561984ab70f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847da550 0x561984bae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847da550 0x561984bae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50410/0 (socket says 172.21.15.78:50410) 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 -- 172.21.15.78:0/1905475057 learned_addr learned my addr 172.21.15.78:0/1905475057 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f62715c1640 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847e2730 0x561984baec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f62725c3640 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984bacbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 -- 172.21.15.78:0/1905475057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 msgr2=0x561984bacbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984bacbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 -- 172.21.15.78:0/1905475057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847e2730 msgr2=0x561984baec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847e2730 0x561984baec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 -- 172.21.15.78:0/1905475057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561984943f70 con 0x5619847da550 2024-05-21T15:22:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f62725c3640 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984bacbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:50.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f6271dc2640 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847da550 0x561984bae710 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f626c102a00 tx=0x7f626800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d141552c6b7bcb7f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f62557fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f62557fa640 1 -- 172.21.15.78:0/1905475057 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268004020 con 0x5619847da550 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f62557fa640 1 -- 172.21.15.78:0/1905475057 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f62680041c0 con 0x5619847da550 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f627afaa740 1 -- 172.21.15.78:0/1905475057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561984b150c0 con 0x5619847da550 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f62557fa640 1 -- 172.21.15.78:0/1905475057 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f62680043c0 con 0x5619847da550 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f627afaa740 10 client.17571 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.787+0000 7f627afaa740 1 -- 172.21.15.78:0/1905475057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5619847db170 con 0x5619847da550 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f62557fa640 1 -- 172.21.15.78:0/1905475057 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561984b150c0 con 0x5619847da550 2024-05-21T15:22:50.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f62557fa640 1 -- 172.21.15.78:0/1905475057 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6268003a70 con 0x5619847da550 2024-05-21T15:22:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f627afaa740 10 client.17571 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f627afaa740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f627afaa740 20 client.17571 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f627afaa740 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619847c21f0 0x561984bbf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f627afaa740 4 client.17571 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f627afaa740 1 -- 172.21.15.78:0/1905475057 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561984b0f410 con 0x5619847c21f0 2024-05-21T15:22:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.788+0000 7f62715c1640 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619847c21f0 0x561984bbf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.789+0000 7f62715c1640 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619847c21f0 0x561984bbf100 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1fb12da4d3cbaf3 server_cookie=eedb1353a63b17c7 in_seq=0 out_seq=0 2024-05-21T15:22:50.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.789+0000 7f62557fa640 10 client.17571 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:50.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.789+0000 7f62557fa640 1 -- 172.21.15.78:0/1905475057 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561984b0f410 con 0x5619847c21f0 2024-05-21T15:22:50.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.789+0000 7f62557fa640 10 client.17571 handle_command_reply: tid=0 2024-05-21T15:22:50.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 client.17571 shutdown 2024-05-21T15:22:50.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 -- 172.21.15.78:0/1905475057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847da550 msgr2=0x561984bae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847da550 0x561984bae710 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f626c102a00 tx=0x7f626800bf90 comp rx=0 tx=0).stop 2024-05-21T15:22:50.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 -- 172.21.15.78:0/1905475057 shutdown_connections 2024-05-21T15:22:50.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5619847c21f0 0x561984bbf100 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847e2730 0x561984baec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984bacbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 --2- 172.21.15.78:0/1905475057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847da550 0x561984bae710 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f62557fa640 0 client.17571 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:50.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 -- 172.21.15.78:0/1905475057 >> 172.21.15.78:0/1905475057 conn(0x5619847f7560 msgr2=0x561984ba4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 -- 172.21.15.78:0/1905475057 shutdown_connections 2024-05-21T15:22:50.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 1 -- 172.21.15.78:0/1905475057 wait complete. 2024-05-21T15:22:50.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.790+0000 7f627afaa740 20 client.17571 trim_cache size 0 max 16384 2024-05-21T15:22:50.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f627afaa740 1 Processor -- start 2024-05-21T15:22:50.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f627afaa740 1 -- start start 2024-05-21T15:22:50.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f627afaa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619847da550 0x561984baed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f627afaa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f627afaa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561984803410 0x561984ba6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f627afaa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1aeb40 con 0x5619847e2730 2024-05-21T15:22:50.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f627afaa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1b0680 con 0x561984803410 2024-05-21T15:22:50.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f627afaa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1af210 con 0x5619847da550 2024-05-21T15:22:50.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62715c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62715c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50420/0 (socket says 172.21.15.78:50420) 2024-05-21T15:22:50.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62715c1640 1 -- 172.21.15.78:0/1863744505 learned_addr learned my addr 172.21.15.78:0/1863744505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f6271dc2640 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619847da550 0x561984baed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62725c3640 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561984803410 0x561984ba6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62715c1640 1 -- 172.21.15.78:0/1863744505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619847da550 msgr2=0x561984baed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62715c1640 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619847da550 0x561984baed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62715c1640 1 -- 172.21.15.78:0/1863744505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561984803410 msgr2=0x561984ba6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62715c1640 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561984803410 0x561984ba6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62715c1640 1 -- 172.21.15.78:0/1863744505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561984ba1fe0 con 0x5619847e2730 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.808+0000 7f62725c3640 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561984803410 0x561984ba6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f62715c1640 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba62d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f625c0c0300 tx=0x7f625c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b7280bac7134c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f62577fe640 1 -- 172.21.15.78:0/1863744505 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c049040 con 0x5619847e2730 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f62577fe640 1 -- 172.21.15.78:0/1863744505 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c0ba1c0 con 0x5619847e2730 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f62577fe640 1 -- 172.21.15.78:0/1863744505 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c03bc50 con 0x5619847e2730 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f627afaa740 1 -- 172.21.15.78:0/1863744505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 msgr2=0x561984ba62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f627afaa740 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba62d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f625c0c0300 tx=0x7f625c02a410 comp rx=0 tx=0).stop 2024-05-21T15:22:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f627afaa740 1 -- 172.21.15.78:0/1863744505 shutdown_connections 2024-05-21T15:22:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f627afaa740 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561984803410 0x561984ba6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f627afaa740 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984ba62d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f627afaa740 1 --2- 172.21.15.78:0/1863744505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619847da550 0x561984baed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f627afaa740 1 -- 172.21.15.78:0/1863744505 >> 172.21.15.78:0/1863744505 conn(0x5619847f7560 msgr2=0x561984ba47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.809+0000 7f627afaa740 1 -- 172.21.15.78:0/1863744505 shutdown_connections 2024-05-21T15:22:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 -- 172.21.15.78:0/1863744505 wait complete. 2024-05-21T15:22:50.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 -- start start 2024-05-21T15:22:50.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 Processor -- start 2024-05-21T15:22:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 0x561984bb4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984bb4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984bb1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1affb0 con 0x5619847e2730 2024-05-21T15:22:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1af8e0 con 0x5619847da550 2024-05-21T15:22:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f626c1acb20 con 0x561984803410 2024-05-21T15:22:50.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f627afaa740 10 client.0 _ll_register_callbacks cb 0x7f626800bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.810+0000 7f62715c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984bb4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62715c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984bb4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50426/0 (socket says 172.21.15.78:50426) 2024-05-21T15:22:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62715c1640 1 -- 172.21.15.78:0/646301946 learned_addr learned my addr 172.21.15.78:0/646301946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62715c1640 1 -- 172.21.15.78:0/646301946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 msgr2=0x561984bb1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62715c1640 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984bb1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62715c1640 1 -- 172.21.15.78:0/646301946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 msgr2=0x561984bb4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62715c1640 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 0x561984bb4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62715c1640 1 -- 172.21.15.78:0/646301946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561984bb2530 con 0x5619847e2730 2024-05-21T15:22:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62715c1640 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984bb4f60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f625c0bbe50 tx=0x7f625c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dd7a0e757c6fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62557fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62557fa640 1 -- 172.21.15.78:0/646301946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0c4a80 con 0x5619847e2730 2024-05-21T15:22:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62557fa640 1 -- 172.21.15.78:0/646301946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f625c0b8530 con 0x5619847e2730 2024-05-21T15:22:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f627afaa740 1 -- 172.21.15.78:0/646301946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f625c0ba1c0 con 0x5619847e2730 2024-05-21T15:22:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62557fa640 1 -- 172.21.15.78:0/646301946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f625c03bc50 con 0x5619847e2730 2024-05-21T15:22:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f627afaa740 10 client.17577 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:50.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f627afaa740 1 -- 172.21.15.78:0/646301946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561984c26d40 con 0x5619847e2730 2024-05-21T15:22:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.811+0000 7f62557fa640 1 -- 172.21.15.78:0/646301946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f625c0ba1c0 con 0x5619847e2730 2024-05-21T15:22:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f62557fa640 1 -- 172.21.15.78:0/646301946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f625c09ace0 con 0x5619847e2730 2024-05-21T15:22:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f627afaa740 10 client.17577 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f627afaa740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f627afaa740 20 client.17577 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f627afaa740 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561984797630 0x561984c28e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f627afaa740 4 client.17577 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:22:50.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f627afaa740 1 -- 172.21.15.78:0/646301946 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561984a8cda0 con 0x561984797630 2024-05-21T15:22:50.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f6271dc2640 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561984797630 0x561984c28e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f62557fa640 10 client.17577 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:50.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.812+0000 7f6271dc2640 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561984797630 0x561984c28e10 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fdf56fb60132fba server_cookie=91cd40b1ad71e8e8 in_seq=0 out_seq=0 2024-05-21T15:22:50.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.813+0000 7f62557fa640 1 -- 172.21.15.78:0/646301946 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561984a8cda0 con 0x561984797630 2024-05-21T15:22:50.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.813+0000 7f62557fa640 10 client.17577 handle_command_reply: tid=0 2024-05-21T15:22:50.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.813+0000 7f627afaa740 1 client.17577 shutdown 2024-05-21T15:22:50.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.813+0000 7f627afaa740 1 -- 172.21.15.78:0/646301946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 msgr2=0x561984bb4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.813+0000 7f627afaa740 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984bb4f60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f625c0bbe50 tx=0x7f625c009680 comp rx=0 tx=0).stop 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 1 -- 172.21.15.78:0/646301946 shutdown_connections 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x561984797630 0x561984c28e10 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561984803410 0x561984bb1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619847e2730 0x561984bb4f60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 1 --2- 172.21.15.78:0/646301946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5619847da550 0x561984bb4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f62557fa640 0 client.17577 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 1 -- 172.21.15.78:0/646301946 >> 172.21.15.78:0/646301946 conn(0x5619847f7560 msgr2=0x561984bae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 1 -- 172.21.15.78:0/646301946 shutdown_connections 2024-05-21T15:22:50.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 1 -- 172.21.15.78:0/646301946 wait complete. 2024-05-21T15:22:50.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f627afaa740 20 client.17577 trim_cache size 0 max 16384 2024-05-21T15:22:50.826 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:22:50.826 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:22:50.826 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c7b26c4d-9942-4a23-ac2f-157a6fdc04a9", 2024-05-21T15:22:50.826 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:22:50.827 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:22:50.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 -- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f62580a2d00 msgr2=0x7f62580a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f62580a2d00 0x7f62580a5210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x561984b9f790 tx=0x7f6264009600 comp rx=0 tx=0).stop 2024-05-21T15:22:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 -- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c107a00 msgr2=0x7f626c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c107a00 0x7f626c19ab50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f625c005940 tx=0x7f625c0042f0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 -- 172.21.15.78:0/3135638410 shutdown_connections 2024-05-21T15:22:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f62580a2d00 0x7f62580a5210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f626c10c360 0x7f626c19b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f626c107a00 0x7f626c19ab50 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 --2- 172.21.15.78:0/3135638410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f626c101b20 0x7f626c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 -- 172.21.15.78:0/3135638410 >> 172.21.15.78:0/3135638410 conn(0x7f626c0770c0 msgr2=0x7f626c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 -- 172.21.15.78:0/3135638410 shutdown_connections 2024-05-21T15:22:50.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.814+0000 7f6274026640 1 -- 172.21.15.78:0/3135638410 wait complete. 2024-05-21T15:22:50.829 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:50 smithi078 ceph-mon[31997]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:22:50.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:22:50.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:22:50.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:50 smithi084 ceph-mon[36334]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:22:50.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:50 smithi172 ceph-mon[36402]: pgmap v205: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:22:50.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 -- 172.21.15.78:0/1254291026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410d000 msgr2=0x7f3e8410f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 --2- 172.21.15.78:0/1254291026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410d000 0x7f3e8410f4f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009900 tx=0x7f3e7c02a480 comp rx=0 tx=0).stop 2024-05-21T15:22:50.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 -- 172.21.15.78:0/1254291026 shutdown_connections 2024-05-21T15:22:50.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 --2- 172.21.15.78:0/1254291026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410d000 0x7f3e8410f4f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 --2- 172.21.15.78:0/1254291026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e8410a6a0 0x7f3e8410cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 --2- 172.21.15.78:0/1254291026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e84103c40 0x7f3e84104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 -- 172.21.15.78:0/1254291026 >> 172.21.15.78:0/1254291026 conn(0x7f3e840770b0 msgr2=0x7f3e840774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 -- 172.21.15.78:0/1254291026 shutdown_connections 2024-05-21T15:22:50.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 -- 172.21.15.78:0/1254291026 wait complete. 2024-05-21T15:22:50.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.944+0000 7f3e890f7640 1 Processor -- start 2024-05-21T15:22:50.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e890f7640 1 -- start start 2024-05-21T15:22:50.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e890f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e84103c40 0x7f3e84199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e890f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410a6a0 0x7f3e84199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e890f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e8410d000 0x7f3e8419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e890f7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84112db0 con 0x7f3e8410a6a0 2024-05-21T15:22:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e890f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84112c30 con 0x7f3e84103c40 2024-05-21T15:22:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e890f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84112f30 con 0x7f3e8410d000 2024-05-21T15:22:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410a6a0 0x7f3e84199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410a6a0 0x7f3e84199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50450/0 (socket says 172.21.15.78:50450) 2024-05-21T15:22:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 -- 172.21.15.78:0/678300235 learned_addr learned my addr 172.21.15.78:0/678300235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8359e640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e8410d000 0x7f3e8419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e84103c40 0x7f3e84199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 -- 172.21.15.78:0/678300235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e8410d000 msgr2=0x7f3e8419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e8410d000 0x7f3e8419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 -- 172.21.15.78:0/678300235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e84103c40 msgr2=0x7f3e84199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e84103c40 0x7f3e84199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 -- 172.21.15.78:0/678300235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8419e590 con 0x7f3e8410a6a0 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.945+0000 7f3e8259c640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410a6a0 0x7f3e84199c90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800ebc0 tx=0x7f3e7800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.946+0000 7f3e6bfff640 1 -- 172.21.15.78:0/678300235 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78010630 con 0x7f3e8410a6a0 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.946+0000 7f3e890f7640 1 -- 172.21.15.78:0/678300235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8419e880 con 0x7f3e8410a6a0 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.946+0000 7f3e6bfff640 1 -- 172.21.15.78:0/678300235 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e78010cb0 con 0x7f3e8410a6a0 2024-05-21T15:22:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.946+0000 7f3e6bfff640 1 -- 172.21.15.78:0/678300235 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78018bd0 con 0x7f3e8410a6a0 2024-05-21T15:22:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.946+0000 7f3e890f7640 1 -- 172.21.15.78:0/678300235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e841a6200 con 0x7f3e8410a6a0 2024-05-21T15:22:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.948+0000 7f3e6bfff640 1 -- 172.21.15.78:0/678300235 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3e780107d0 con 0x7f3e8410a6a0 2024-05-21T15:22:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.948+0000 7f3e8ff9e740 1 Processor -- start 2024-05-21T15:22:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.948+0000 7f3e8ff9e740 1 -- start start 2024-05-21T15:22:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeecfad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeecfb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeecfb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563aeec60d00 con 0x563aee93b6b0 2024-05-21T15:22:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563aeec084c0 con 0x563aee95c390 2024-05-21T15:22:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563aeec5ccc0 con 0x563aee9334d0 2024-05-21T15:22:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeecfb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeecfb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50456/0 (socket says 172.21.15.78:50456) 2024-05-21T15:22:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 -- 172.21.15.78:0/3939857384 learned_addr learned my addr 172.21.15.78:0/3939857384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8359e640 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeecfad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e6bfff640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3e6c0a2d00 0x7f3e6c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeecfb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8359e640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3e6c0a2d00 0x7f3e6c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e6bfff640 1 -- 172.21.15.78:0/678300235 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3e780cfb40 con 0x7f3e8410a6a0 2024-05-21T15:22:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 -- 172.21.15.78:0/3939857384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 msgr2=0x563aeecfb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeecfb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 -- 172.21.15.78:0/3939857384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 msgr2=0x563aeecfad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeecfad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 -- 172.21.15.78:0/3939857384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aeea3e7d0 con 0x563aee93b6b0 2024-05-21T15:22:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.949+0000 7f3e8259c640 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeecfb7b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3e7809bf90 tx=0x7f3e78018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dab24d1a77461a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.950+0000 7f3e8359e640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3e6c0a2d00 0x7f3e6c0a5210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005ab0 tx=0x7f3e7c02a480 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.950+0000 7f3e697fa640 1 -- 172.21.15.78:0/3939857384 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78010e60 con 0x563aee93b6b0 2024-05-21T15:22:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.950+0000 7f3e697fa640 1 -- 172.21.15.78:0/3939857384 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e7809c700 con 0x563aee93b6b0 2024-05-21T15:22:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.950+0000 7f3e697fa640 1 -- 172.21.15.78:0/3939857384 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7809a300 con 0x563aee93b6b0 2024-05-21T15:22:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.950+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3939857384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 msgr2=0x563aeecfb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.950+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeecfb7b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3e7809bf90 tx=0x7f3e78018670 comp rx=0 tx=0).stop 2024-05-21T15:22:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3939857384 shutdown_connections 2024-05-21T15:22:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeecfb7b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeecfb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/3939857384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeecfad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3939857384 >> 172.21.15.78:0/3939857384 conn(0x563aee9504e0 msgr2=0x563aeecf74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3939857384 shutdown_connections 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3939857384 wait complete. 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 -- start start 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 Processor -- start 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeed06f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeed07480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeed05350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563aeed07fc0 con 0x563aee93b6b0 2024-05-21T15:22:50.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563aeec1bb90 con 0x563aee95c390 2024-05-21T15:22:50.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563aeed08140 con 0x563aee9334d0 2024-05-21T15:22:50.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.951+0000 7f3e8ff9e740 10 client.0 _ll_register_callbacks cb 0x563aeec57da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:50.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeed07480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeed07480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50464/0 (socket says 172.21.15.78:50464) 2024-05-21T15:22:50.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 -- 172.21.15.78:0/2449306053 learned_addr learned my addr 172.21.15.78:0/2449306053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 -- 172.21.15.78:0/2449306053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 msgr2=0x563aeed06f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeed06f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8359e640 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeed05350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 -- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 msgr2=0x563aeed05350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeed05350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 -- 172.21.15.78:0/2449306053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aee9743f0 con 0x563aee93b6b0 2024-05-21T15:22:50.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8359e640 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeed05350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:50.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8259c640 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeed07480 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f3e7809bd00 tx=0x7f3e780a04f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a722b3de92159272 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:50.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e66ff5640 1 -- 172.21.15.78:0/2449306053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78010e60 con 0x563aee93b6b0 2024-05-21T15:22:50.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e66ff5640 1 -- 172.21.15.78:0/2449306053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3e78096b70 con 0x563aee93b6b0 2024-05-21T15:22:50.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/2449306053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7809c700 con 0x563aee93b6b0 2024-05-21T15:22:50.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e66ff5640 1 -- 172.21.15.78:0/2449306053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3e7809c000 con 0x563aee93b6b0 2024-05-21T15:22:50.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8ff9e740 10 client.17589 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:50.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/2449306053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563aee9340f0 con 0x563aee93b6b0 2024-05-21T15:22:50.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.952+0000 7f3e66ff5640 1 -- 172.21.15.78:0/2449306053 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e7809c700 con 0x563aee93b6b0 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e66ff5640 1 -- 172.21.15.78:0/2449306053 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3e780b8050 con 0x563aee93b6b0 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e8ff9e740 10 client.17589 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e8ff9e740 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e8ff9e740 20 client.17589 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563aee91af60 0x563aeed179a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e8ff9e740 4 client.17589 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/2449306053 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aeec6e430 con 0x563aee91af60 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563aee91af60 0x563aeed179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e66ff5640 10 client.17589 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:50.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.953+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563aee91af60 0x563aeed179a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5de5b0912dc0e579 server_cookie=47ceeae2a4031d3a in_seq=0 out_seq=0 2024-05-21T15:22:50.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e66ff5640 1 -- 172.21.15.78:0/2449306053 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563aeec6e430 con 0x563aee91af60 2024-05-21T15:22:50.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e66ff5640 10 client.17589 handle_command_reply: tid=0 2024-05-21T15:22:50.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 client.17589 shutdown 2024-05-21T15:22:50.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/2449306053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 msgr2=0x563aeed07480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeed07480 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f3e7809bd00 tx=0x7f3e780a04f0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/2449306053 shutdown_connections 2024-05-21T15:22:50.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563aee91af60 0x563aeed179a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee95c390 0x563aeed05350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e66ff5640 0 client.17589 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:50.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee93b6b0 0x563aeed07480 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/2449306053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeed06f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/2449306053 >> 172.21.15.78:0/2449306053 conn(0x563aee9504e0 msgr2=0x563aeecfca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/2449306053 shutdown_connections 2024-05-21T15:22:50.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/2449306053 wait complete. 2024-05-21T15:22:50.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.954+0000 7f3e8ff9e740 20 client.17589 trim_cache size 0 max 16384 2024-05-21T15:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8ff9e740 1 Processor -- start 2024-05-21T15:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8ff9e740 1 -- start start 2024-05-21T15:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee9334d0 0x563aeed074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeecfb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee95c390 0x563aeecfb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841aecd0 con 0x563aee9334d0 2024-05-21T15:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841b0810 con 0x563aee93b6b0 2024-05-21T15:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841af3a0 con 0x563aee95c390 2024-05-21T15:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e82d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee9334d0 0x563aeed074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e82d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee9334d0 0x563aeed074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50470/0 (socket says 172.21.15.78:50470) 2024-05-21T15:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e82d9d640 1 -- 172.21.15.78:0/3812335187 learned_addr learned my addr 172.21.15.78:0/3812335187 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8359e640 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee95c390 0x563aeecfb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8259c640 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeecfb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e82d9d640 1 -- 172.21.15.78:0/3812335187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee95c390 msgr2=0x563aeecfb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee95c390 0x563aeecfb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e82d9d640 1 -- 172.21.15.78:0/3812335187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 msgr2=0x563aeecfb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeecfb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e82d9d640 1 -- 172.21.15.78:0/3812335187 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aeecf8630 con 0x563aee9334d0 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8259c640 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeecfb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.972+0000 7f3e8359e640 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee95c390 0x563aeecfb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.973+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee9334d0 0x563aeed074a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x563aeecf8470 tx=0x7f3e7400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3d41de4c6fa82 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.973+0000 7f3e68ff9640 1 -- 172.21.15.78:0/3812335187 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74012e70 con 0x563aee9334d0 2024-05-21T15:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.973+0000 7f3e68ff9640 1 -- 172.21.15.78:0/3812335187 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e74005500 con 0x563aee9334d0 2024-05-21T15:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.973+0000 7f3e68ff9640 1 -- 172.21.15.78:0/3812335187 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74019590 con 0x563aee9334d0 2024-05-21T15:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3812335187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee9334d0 msgr2=0x563aeed074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee9334d0 0x563aeed074a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x563aeecf8470 tx=0x7f3e7400c750 comp rx=0 tx=0).stop 2024-05-21T15:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3812335187 shutdown_connections 2024-05-21T15:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeecfb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee95c390 0x563aeecfb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/3812335187 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee9334d0 0x563aeed074a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3812335187 >> 172.21.15.78:0/3812335187 conn(0x563aee9504e0 msgr2=0x563aeecfaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3812335187 shutdown_connections 2024-05-21T15:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/3812335187 wait complete. 2024-05-21T15:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- start start 2024-05-21T15:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 Processor -- start 2024-05-21T15:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeed19c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeed1a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee95c390 0x563aeed7dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841b0140 con 0x563aee95c390 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841afa70 con 0x563aee93b6b0 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841accb0 con 0x563aee9334d0 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8ff9e740 10 client.0 _ll_register_callbacks cb 0x7f3e780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee95c390 0x563aeed7dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee95c390 0x563aeed7dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50480/0 (socket says 172.21.15.78:50480) 2024-05-21T15:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.974+0000 7f3e8359e640 1 -- 172.21.15.78:0/523459179 learned_addr learned my addr 172.21.15.78:0/523459179 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeed19c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8259c640 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeed1a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8359e640 1 -- 172.21.15.78:0/523459179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 msgr2=0x563aeed19c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8359e640 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeed19c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8359e640 1 -- 172.21.15.78:0/523459179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 msgr2=0x563aeed1a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8359e640 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeed1a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8359e640 1 -- 172.21.15.78:0/523459179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aeed7e010 con 0x563aee95c390 2024-05-21T15:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8359e640 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee95c390 0x563aeed7dad0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c02a9a0 tx=0x7f3e7c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cac160f5945b66 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e66ff5640 1 -- 172.21.15.78:0/523459179 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c050020 con 0x563aee95c390 2024-05-21T15:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e66ff5640 1 -- 172.21.15.78:0/523459179 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8614+0+0 (secure 0 0 0) 0x7f3e7c005310 con 0x563aee95c390 2024-05-21T15:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/523459179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e74005500 con 0x563aee95c390 2024-05-21T15:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e66ff5640 1 -- 172.21.15.78:0/523459179 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3e7c05b050 con 0x563aee95c390 2024-05-21T15:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8ff9e740 10 client.17595 fetch_fsmap learned FSMap version 37 2024-05-21T15:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/523459179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563aeed7f050 con 0x563aee95c390 2024-05-21T15:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.975+0000 7f3e66ff5640 1 -- 172.21.15.78:0/523459179 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e74005500 con 0x563aee95c390 2024-05-21T15:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e66ff5640 1 -- 172.21.15.78:0/523459179 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3e7c003710 con 0x563aee95c390 2024-05-21T15:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e8ff9e740 10 client.17595 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e8ff9e740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e8ff9e740 20 client.17595 populate_metadata read hostname 'smithi078' 2024-05-21T15:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563aee8f03a0 0x563aeed81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e8ff9e740 4 client.17595 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/523459179 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563aeec68780 con 0x563aee8f03a0 2024-05-21T15:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563aee8f03a0 0x563aeed81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e66ff5640 10 client.17595 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e82d9d640 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563aee8f03a0 0x563aeed81890 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324d1953fe40052d server_cookie=f3d05ca099a4b41f in_seq=0 out_seq=0 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e66ff5640 1 -- 172.21.15.78:0/523459179 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563aeec68780 con 0x563aee8f03a0 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e66ff5640 10 client.17595 handle_command_reply: tid=0 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.976+0000 7f3e8ff9e740 1 client.17595 shutdown 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/523459179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee95c390 msgr2=0x563aeed7dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee95c390 0x563aeed7dad0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c02a9a0 tx=0x7f3e7c009260 comp rx=0 tx=0).stop 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/523459179 shutdown_connections 2024-05-21T15:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x563aee8f03a0 0x563aeed81890 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563aee93b6b0 0x563aeed1a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e66ff5640 0 client.17595 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563aee95c390 0x563aeed7dad0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 --2- 172.21.15.78:0/523459179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563aee9334d0 0x563aeed19c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/523459179 >> 172.21.15.78:0/523459179 conn(0x563aee9504e0 msgr2=0x563aeecfb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/523459179 shutdown_connections 2024-05-21T15:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 1 -- 172.21.15.78:0/523459179 wait complete. 2024-05-21T15:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e8ff9e740 20 client.17595 trim_cache size 0 max 16384 2024-05-21T15:22:50.991 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:22:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:22:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:22:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "c7b26c4d-9942-4a23-ac2f-157a6fdc04a9": { 2024-05-21T15:22:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:22:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c7b26c4d-9942-4a23-ac2f-157a6fdc04a9", 2024-05-21T15:22:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:22:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:22:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:22:50.992 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:22:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 -- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3e6c0a2d00 msgr2=0x7f3e6c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3e6c0a2d00 0x7f3e6c0a5210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005ab0 tx=0x7f3e7c02a480 comp rx=0 tx=0).stop 2024-05-21T15:22:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 -- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410a6a0 msgr2=0x7f3e84199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:22:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410a6a0 0x7f3e84199c90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800ebc0 tx=0x7f3e7800c6a0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 -- 172.21.15.78:0/678300235 shutdown_connections 2024-05-21T15:22:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f3e6c0a2d00 0x7f3e6c0a5210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e8410d000 0x7f3e8419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e8410a6a0 0x7f3e84199c90 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 --2- 172.21.15.78:0/678300235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e84103c40 0x7f3e84199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:22:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 -- 172.21.15.78:0/678300235 >> 172.21.15.78:0/678300235 conn(0x7f3e840770b0 msgr2=0x7f3e84100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:22:50.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 -- 172.21.15.78:0/678300235 shutdown_connections 2024-05-21T15:22:50.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:22:50.977+0000 7f3e890f7640 1 -- 172.21.15.78:0/678300235 wait complete. 2024-05-21T15:22:50.995 INFO:tasks.cephfs.filesystem:scrub status for tag:c7b26c4d-9942-4a23-ac2f-157a6fdc04a9 - {'path': '/', 'tag': 'c7b26c4d-9942-4a23-ac2f-157a6fdc04a9', 'options': 'recursive,force'} 2024-05-21T15:22:51.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:22:51.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:51.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:51.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:51.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:51.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:51.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:51 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:22:51.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:51.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:51.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:51.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:51.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:51.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:51 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:22:51.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:51.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:51.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:51.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:51.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:51.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:51.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:51 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:52 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:22:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:52 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:52 smithi078 ceph-mon[31997]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:22:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:52 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:22:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:52 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:52 smithi084 ceph-mon[36334]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:22:52.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:52 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:22:52.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:52 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:22:52.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:52 smithi172 ceph-mon[36402]: pgmap v206: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T15:22:54.736 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:54 smithi172 ceph-mon[36402]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:22:54.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:54 smithi078 ceph-mon[31997]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:22:54.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:54 smithi084 ceph-mon[36334]: pgmap v207: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:22:56.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:56 smithi078 ceph-mon[31997]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 39 op/s 2024-05-21T15:22:56.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:56 smithi084 ceph-mon[36334]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 39 op/s 2024-05-21T15:22:56.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:56 smithi172 ceph-mon[36402]: pgmap v208: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 39 op/s 2024-05-21T15:22:57.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:57 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:57.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:57 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:57.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:57 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:57.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:57 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:57.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:57 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:57.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:57 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:57.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:57 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:57.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:57 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:57.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:57 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:58.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:58 smithi084 ceph-mon[36334]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:22:58.798 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:58 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:58.798 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:22:58 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:22:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:58 smithi078 ceph-mon[31997]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:22:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:58 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:22:58 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:22:58.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:58 smithi172 ceph-mon[36402]: pgmap v209: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:22:58.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:58 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:22:58.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:22:58 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:23:00.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:00 smithi078 ceph-mon[31997]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:00.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:00 smithi084 ceph-mon[36334]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:00.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:00 smithi172 ceph-mon[36402]: pgmap v210: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:02.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:02 smithi078 ceph-mon[31997]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:02 smithi084 ceph-mon[36334]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:02.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:02 smithi172 ceph-mon[36402]: pgmap v211: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:04.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:04 smithi078 ceph-mon[31997]: pgmap v212: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:23:04.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:04 smithi084 ceph-mon[36334]: pgmap v212: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:23:04.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:04 smithi172 ceph-mon[36402]: pgmap v212: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:23:06.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:06 smithi078 ceph-mon[31997]: pgmap v213: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-05-21T15:23:06.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:06 smithi084 ceph-mon[36334]: pgmap v213: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-05-21T15:23:06.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:06 smithi172 ceph-mon[36402]: pgmap v213: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-05-21T15:23:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:08 smithi078 ceph-mon[31997]: pgmap v214: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-21T15:23:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:08 smithi078 ceph-mon[31997]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T15:23:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:08 smithi084 ceph-mon[36334]: pgmap v214: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-21T15:23:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:08 smithi084 ceph-mon[36334]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T15:23:08.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:08 smithi172 ceph-mon[36402]: pgmap v214: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-21T15:23:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:08 smithi172 ceph-mon[36402]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T15:23:10.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:10 smithi078 ceph-mon[31997]: pgmap v216: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:23:10.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:10 smithi084 ceph-mon[36334]: pgmap v216: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:23:10.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:10 smithi172 ceph-mon[36402]: pgmap v216: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:23:12.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:12 smithi078 ceph-mon[31997]: pgmap v217: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:23:12.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:12 smithi084 ceph-mon[36334]: pgmap v217: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:23:12.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:12 smithi172 ceph-mon[36402]: pgmap v217: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:23:14.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:14 smithi078 ceph-mon[31997]: pgmap v218: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:14.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:14 smithi084 ceph-mon[36334]: pgmap v218: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:14.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:14 smithi172 ceph-mon[36402]: pgmap v218: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:16.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:16 smithi078 ceph-mon[31997]: pgmap v219: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:16.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:16 smithi078 ceph-mon[31997]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T15:23:16.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:16 smithi084 ceph-mon[36334]: pgmap v219: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:16.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:16 smithi084 ceph-mon[36334]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T15:23:16.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:16 smithi172 ceph-mon[36402]: pgmap v219: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:16.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:16 smithi172 ceph-mon[36402]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T15:23:18.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:18 smithi078 ceph-mon[31997]: pgmap v221: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 218 B/s wr, 10 op/s 2024-05-21T15:23:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:18 smithi084 ceph-mon[36334]: pgmap v221: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 218 B/s wr, 10 op/s 2024-05-21T15:23:18.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:18 smithi172 ceph-mon[36402]: pgmap v221: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 218 B/s wr, 10 op/s 2024-05-21T15:23:20.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:20 smithi078 ceph-mon[31997]: pgmap v222: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:20.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:20 smithi084 ceph-mon[36334]: pgmap v222: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:20.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:20 smithi172 ceph-mon[36402]: pgmap v222: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:20.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:23:20.997 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:23:21.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 -- 172.21.15.78:0/1585590351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704101aa0 msgr2=0x7fb704101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 --2- 172.21.15.78:0/1585590351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704101aa0 0x7fb704101eb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009900 tx=0x7fb6f402a410 comp rx=0 tx=0).stop 2024-05-21T15:23:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 -- 172.21.15.78:0/1585590351 shutdown_connections 2024-05-21T15:23:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 --2- 172.21.15.78:0/1585590351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb70410c330 0x7fb70410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 --2- 172.21.15.78:0/1585590351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704101aa0 0x7fb704101eb0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 --2- 172.21.15.78:0/1585590351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7041079d0 0x7fb704101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 -- 172.21.15.78:0/1585590351 >> 172.21.15.78:0/1585590351 conn(0x7fb7040770b0 msgr2=0x7fb7040774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 -- 172.21.15.78:0/1585590351 shutdown_connections 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.124+0000 7fb70c006640 1 -- 172.21.15.78:0/1585590351 wait complete. 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb70c006640 1 Processor -- start 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb70c006640 1 -- start start 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb70c006640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb704101aa0 0x7fb70419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb70c006640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7041079d0 0x7fb70419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb70c006640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb70410c330 0x7fb70419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb70c006640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb704111ce0 con 0x7fb7041079d0 2024-05-21T15:23:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb70c006640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb704111b60 con 0x7fb704101aa0 2024-05-21T15:23:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb70c006640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb704111e60 con 0x7fb70410c330 2024-05-21T15:23:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb7095a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7041079d0 0x7fb70419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb7095a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7041079d0 0x7fb70419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35360/0 (socket says 172.21.15.78:35360) 2024-05-21T15:23:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb7095a1640 1 -- 172.21.15.78:0/3752627972 learned_addr learned my addr 172.21.15.78:0/3752627972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.125+0000 7fb709da2640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb704101aa0 0x7fb70419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb70410c330 0x7fb70419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb7095a1640 1 -- 172.21.15.78:0/3752627972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb70410c330 msgr2=0x7fb70419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb7095a1640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb70410c330 0x7fb70419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb7095a1640 1 -- 172.21.15.78:0/3752627972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb704101aa0 msgr2=0x7fb70419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb7095a1640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb704101aa0 0x7fb70419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb7095a1640 1 -- 172.21.15.78:0/3752627972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7041a5970 con 0x7fb7041079d0 2024-05-21T15:23:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb7095a1640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7041079d0 0x7fb70419aba0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005940 tx=0x7fb6f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb6faffd640 1 -- 172.21.15.78:0/3752627972 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f403f030 con 0x7fb7041079d0 2024-05-21T15:23:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb6faffd640 1 -- 172.21.15.78:0/3752627972 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f4003ce0 con 0x7fb7041079d0 2024-05-21T15:23:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb6faffd640 1 -- 172.21.15.78:0/3752627972 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4035070 con 0x7fb7041079d0 2024-05-21T15:23:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb70c006640 1 -- 172.21.15.78:0/3752627972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7041a5c00 con 0x7fb7041079d0 2024-05-21T15:23:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.126+0000 7fb70c006640 1 -- 172.21.15.78:0/3752627972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7041a60f0 con 0x7fb7041079d0 2024-05-21T15:23:21.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.128+0000 7fb6faffd640 1 -- 172.21.15.78:0/3752627972 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb6f4042030 con 0x7fb7041079d0 2024-05-21T15:23:21.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb712f88740 1 Processor -- start 2024-05-21T15:23:21.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb712f88740 1 -- start start 2024-05-21T15:23:21.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb712f88740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b65ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb712f88740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b65b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb712f88740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b65b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb6faffd640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb6f00a2d00 0x7fb6f00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb712f88740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b22b5c0d00 con 0x55b22b2bc390 2024-05-21T15:23:21.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb712f88740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b22b5684c0 con 0x55b22b2934d0 2024-05-21T15:23:21.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb712f88740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b22b5bccc0 con 0x55b22b29b6b0 2024-05-21T15:23:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.129+0000 7fb70a5a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b65ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb7095a1640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb6f00a2d00 0x7fb6f00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b65ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35376/0 (socket says 172.21.15.78:35376) 2024-05-21T15:23:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 -- 172.21.15.78:0/2506024524 learned_addr learned my addr 172.21.15.78:0/2506024524 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb6faffd640 1 -- 172.21.15.78:0/3752627972 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fb6f40efa30 con 0x7fb7041079d0 2024-05-21T15:23:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b65b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb709da2640 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b65b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 -- 172.21.15.78:0/2506024524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 msgr2=0x55b22b65b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b65b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 -- 172.21.15.78:0/2506024524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 msgr2=0x55b22b65b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b65b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 -- 172.21.15.78:0/2506024524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22b39e7d0 con 0x55b22b2bc390 2024-05-21T15:23:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb709da2640 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b65b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb7095a1640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb6f00a2d00 0x7fb6f00a5210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40ed1a0 tx=0x7fb6f408de80 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b65ad30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00dfd0 tx=0x7fb6fc00d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a023c34a174802d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb6effff640 1 -- 172.21.15.78:0/2506024524 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc019020 con 0x55b22b2bc390 2024-05-21T15:23:21.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb6effff640 1 -- 172.21.15.78:0/2506024524 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc004230 con 0x55b22b2bc390 2024-05-21T15:23:21.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.130+0000 7fb6effff640 1 -- 172.21.15.78:0/2506024524 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0107e0 con 0x55b22b2bc390 2024-05-21T15:23:21.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 -- 172.21.15.78:0/2506024524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 msgr2=0x55b22b65ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b65ad30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00dfd0 tx=0x7fb6fc00d5b0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 -- 172.21.15.78:0/2506024524 shutdown_connections 2024-05-21T15:23:21.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b65b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b65b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 --2- 172.21.15.78:0/2506024524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b65ad30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 -- 172.21.15.78:0/2506024524 >> 172.21.15.78:0/2506024524 conn(0x55b22b2b04e0 msgr2=0x55b22b6574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 -- 172.21.15.78:0/2506024524 shutdown_connections 2024-05-21T15:23:21.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.131+0000 7fb712f88740 1 -- 172.21.15.78:0/2506024524 wait complete. 2024-05-21T15:23:21.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 1 -- start start 2024-05-21T15:23:21.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 1 Processor -- start 2024-05-21T15:23:21.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b666f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b667480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2bc390 0x55b22b665350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b22b667fc0 con 0x55b22b29b6b0 2024-05-21T15:23:21.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b22b57bb90 con 0x55b22b2934d0 2024-05-21T15:23:21.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b22b668140 con 0x55b22b2bc390 2024-05-21T15:23:21.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb712f88740 10 client.0 _ll_register_callbacks cb 0x55b22b5b7da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:21.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb709da2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b667480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb709da2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b667480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35386/0 (socket says 172.21.15.78:35386) 2024-05-21T15:23:21.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.132+0000 7fb709da2640 1 -- 172.21.15.78:0/659707558 learned_addr learned my addr 172.21.15.78:0/659707558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2bc390 0x55b22b665350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b666f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb709da2640 1 -- 172.21.15.78:0/659707558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2bc390 msgr2=0x55b22b665350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb709da2640 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2bc390 0x55b22b665350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb709da2640 1 -- 172.21.15.78:0/659707558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 msgr2=0x55b22b666f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb709da2640 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b666f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb709da2640 1 -- 172.21.15.78:0/659707558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22b2d43f0 con 0x55b22b29b6b0 2024-05-21T15:23:21.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2bc390 0x55b22b665350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:21.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb709da2640 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b667480 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb7000126f0 tx=0x7fb700012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c4a3006a41be38 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb6ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:21.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/659707558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700003d80 con 0x55b22b29b6b0 2024-05-21T15:23:21.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/659707558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fb700010950 con 0x55b22b29b6b0 2024-05-21T15:23:21.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb712f88740 1 -- 172.21.15.78:0/659707558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6fc004230 con 0x55b22b29b6b0 2024-05-21T15:23:21.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/659707558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb700010b50 con 0x55b22b29b6b0 2024-05-21T15:23:21.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb712f88740 10 client.17607 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:21.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb712f88740 1 -- 172.21.15.78:0/659707558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b22b2940f0 con 0x55b22b29b6b0 2024-05-21T15:23:21.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.133+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/659707558 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc004230 con 0x55b22b29b6b0 2024-05-21T15:23:21.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/659707558 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb700019070 con 0x55b22b29b6b0 2024-05-21T15:23:21.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb712f88740 10 client.17607 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:21.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb712f88740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:21.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb712f88740 20 client.17607 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb712f88740 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b22b27af60 0x55b22b677cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb712f88740 4 client.17607 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb712f88740 1 -- 172.21.15.78:0/659707558 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22b5ce430 con 0x55b22b27af60 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b22b27af60 0x55b22b677cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb6ed7fa640 10 client.17607 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.134+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b22b27af60 0x55b22b677cd0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4b92f7900e8649 server_cookie=dcc2334eb067b4c5 in_seq=0 out_seq=0 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/659707558 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b22b5ce430 con 0x55b22b27af60 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb6ed7fa640 10 client.17607 handle_command_reply: tid=0 2024-05-21T15:23:21.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 client.17607 shutdown 2024-05-21T15:23:21.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 -- 172.21.15.78:0/659707558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 msgr2=0x55b22b667480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b667480 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb7000126f0 tx=0x7fb700012bb0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 -- 172.21.15.78:0/659707558 shutdown_connections 2024-05-21T15:23:21.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b22b27af60 0x55b22b677cd0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2bc390 0x55b22b665350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b667480 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 --2- 172.21.15.78:0/659707558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b666f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb6ed7fa640 0 client.17607 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:21.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 -- 172.21.15.78:0/659707558 >> 172.21.15.78:0/659707558 conn(0x55b22b2b04e0 msgr2=0x55b22b65ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 -- 172.21.15.78:0/659707558 shutdown_connections 2024-05-21T15:23:21.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.135+0000 7fb712f88740 1 -- 172.21.15.78:0/659707558 wait complete. 2024-05-21T15:23:21.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.136+0000 7fb712f88740 20 client.17607 trim_cache size 0 max 16384 2024-05-21T15:23:21.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.153+0000 7fb712f88740 1 Processor -- start 2024-05-21T15:23:21.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.153+0000 7fb712f88740 1 -- start start 2024-05-21T15:23:21.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb712f88740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b664290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb712f88740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b660760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb712f88740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b664fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb712f88740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041aed10 con 0x55b22b2bc390 2024-05-21T15:23:21.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb712f88740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041b0850 con 0x55b22b2934d0 2024-05-21T15:23:21.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb712f88740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041af3e0 con 0x55b22b29b6b0 2024-05-21T15:23:21.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b664fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b664fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35400/0 (socket says 172.21.15.78:35400) 2024-05-21T15:23:21.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 -- 172.21.15.78:0/3495784081 learned_addr learned my addr 172.21.15.78:0/3495784081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b664290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb709da2640 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b660760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 -- 172.21.15.78:0/3495784081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 msgr2=0x55b22b660760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b660760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 -- 172.21.15.78:0/3495784081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 msgr2=0x55b22b664290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b664290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 -- 172.21.15.78:0/3495784081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22b660020 con 0x55b22b2bc390 2024-05-21T15:23:21.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb709da2640 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b660760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:21.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b664fa0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc004040 tx=0x7fb6fc00d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5503c1c17f70efa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb6ef7fe640 1 -- 172.21.15.78:0/3495784081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc019030 con 0x55b22b2bc390 2024-05-21T15:23:21.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb6ef7fe640 1 -- 172.21.15.78:0/3495784081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc02f070 con 0x55b22b2bc390 2024-05-21T15:23:21.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.154+0000 7fb6ef7fe640 1 -- 172.21.15.78:0/3495784081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0282b0 con 0x55b22b2bc390 2024-05-21T15:23:21.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 -- 172.21.15.78:0/3495784081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 msgr2=0x55b22b664fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b664fa0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc004040 tx=0x7fb6fc00d380 comp rx=0 tx=0).stop 2024-05-21T15:23:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 -- 172.21.15.78:0/3495784081 shutdown_connections 2024-05-21T15:23:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b2bc390 0x55b22b664fa0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b29b6b0 0x55b22b660760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 --2- 172.21.15.78:0/3495784081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2934d0 0x55b22b664290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 -- 172.21.15.78:0/3495784081 >> 172.21.15.78:0/3495784081 conn(0x55b22b2b04e0 msgr2=0x55b22b664890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 -- 172.21.15.78:0/3495784081 shutdown_connections 2024-05-21T15:23:21.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 -- 172.21.15.78:0/3495784081 wait complete. 2024-05-21T15:23:21.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 -- start start 2024-05-21T15:23:21.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.155+0000 7fb712f88740 1 Processor -- start 2024-05-21T15:23:21.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb712f88740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2934d0 0x55b22b671dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb712f88740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b66d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb712f88740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2bc390 0x55b22b66d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb712f88740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041b0180 con 0x55b22b29b6b0 2024-05-21T15:23:21.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb712f88740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041afab0 con 0x55b22b2bc390 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb712f88740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041acd10 con 0x55b22b2934d0 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb712f88740 10 client.0 _ll_register_callbacks cb 0x7fb700009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b66d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b66d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35410/0 (socket says 172.21.15.78:35410) 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 -- 172.21.15.78:0/3396153406 learned_addr learned my addr 172.21.15.78:0/3396153406 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2934d0 0x55b22b671dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2bc390 0x55b22b66d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 -- 172.21.15.78:0/3396153406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2934d0 msgr2=0x55b22b671dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2934d0 0x55b22b671dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 -- 172.21.15.78:0/3396153406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2bc390 msgr2=0x55b22b66d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2bc390 0x55b22b66d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 -- 172.21.15.78:0/3396153406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22b66dd70 con 0x55b22b29b6b0 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb709da2640 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b66d120 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x55b22b666920 tx=0x7fb700010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df46166667bc62a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.156+0000 7fb6ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/3396153406 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70001ba80 con 0x55b22b29b6b0 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/3396153406 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fb700003b80 con 0x55b22b29b6b0 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 1 -- 172.21.15.78:0/3396153406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6fc02f070 con 0x55b22b29b6b0 2024-05-21T15:23:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/3396153406 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb700032050 con 0x55b22b29b6b0 2024-05-21T15:23:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 10 client.17613 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 1 -- 172.21.15.78:0/3396153406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b22b6dfe80 con 0x55b22b29b6b0 2024-05-21T15:23:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/3396153406 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc02f070 con 0x55b22b29b6b0 2024-05-21T15:23:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/3396153406 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb70002a020 con 0x55b22b29b6b0 2024-05-21T15:23:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 10 client.17613 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 20 client.17613 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b22b2503a0 0x55b22b6e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 4 client.17613 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb712f88740 1 -- 172.21.15.78:0/3396153406 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b22b660f30 con 0x55b22b2503a0 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.157+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b22b2503a0 0x55b22b6e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb70a5a3640 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b22b2503a0 0x55b22b6e2690 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ea4073feb21923c server_cookie=72f4607147ffb378 in_seq=0 out_seq=0 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb6ed7fa640 10 client.17613 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb6ed7fa640 1 -- 172.21.15.78:0/3396153406 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b22b660f30 con 0x55b22b2503a0 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb6ed7fa640 10 client.17613 handle_command_reply: tid=0 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 client.17613 shutdown 2024-05-21T15:23:21.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 -- 172.21.15.78:0/3396153406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 msgr2=0x55b22b66d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b66d120 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x55b22b666920 tx=0x7fb700010830 comp rx=0 tx=0).stop 2024-05-21T15:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 -- 172.21.15.78:0/3396153406 shutdown_connections 2024-05-21T15:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55b22b2503a0 0x55b22b6e2690 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b22b2bc390 0x55b22b66d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b22b29b6b0 0x55b22b66d120 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 --2- 172.21.15.78:0/3396153406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b22b2934d0 0x55b22b671dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb6ed7fa640 0 client.17613 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 -- 172.21.15.78:0/3396153406 >> 172.21.15.78:0/3396153406 conn(0x55b22b2b04e0 msgr2=0x55b22b664890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 -- 172.21.15.78:0/3396153406 shutdown_connections 2024-05-21T15:23:21.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 1 -- 172.21.15.78:0/3396153406 wait complete. 2024-05-21T15:23:21.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.158+0000 7fb712f88740 20 client.17613 trim_cache size 0 max 16384 2024-05-21T15:23:21.173 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:23:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:23:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:23:21.174 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:23:21.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 -- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb6f00a2d00 msgr2=0x7fb6f00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb6f00a2d00 0x7fb6f00a5210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40ed1a0 tx=0x7fb6f408de80 comp rx=0 tx=0).stop 2024-05-21T15:23:21.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 -- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7041079d0 msgr2=0x7fb70419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7041079d0 0x7fb70419aba0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005940 tx=0x7fb6f40042f0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 -- 172.21.15.78:0/3752627972 shutdown_connections 2024-05-21T15:23:21.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb6f00a2d00 0x7fb6f00a5210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb70410c330 0x7fb70419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7041079d0 0x7fb70419aba0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 --2- 172.21.15.78:0/3752627972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb704101aa0 0x7fb70419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 -- 172.21.15.78:0/3752627972 >> 172.21.15.78:0/3752627972 conn(0x7fb7040770b0 msgr2=0x7fb704103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 -- 172.21.15.78:0/3752627972 shutdown_connections 2024-05-21T15:23:21.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.159+0000 7fb70c006640 1 -- 172.21.15.78:0/3752627972 wait complete. 2024-05-21T15:23:21.177 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:23:21.177 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:23:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 -- 172.21.15.78:0/568904017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364108ae0 msgr2=0x7fb364102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 --2- 172.21.15.78:0/568904017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364108ae0 0x7fb364102680 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb354009ca0 tx=0x7fb35402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 -- 172.21.15.78:0/568904017 shutdown_connections 2024-05-21T15:23:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 --2- 172.21.15.78:0/568904017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb364108f80 0x7fb364102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 --2- 172.21.15.78:0/568904017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb364103230 0x7fb36410f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 --2- 172.21.15.78:0/568904017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364108ae0 0x7fb364102680 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 -- 172.21.15.78:0/568904017 >> 172.21.15.78:0/568904017 conn(0x7fb3640fe1c0 msgr2=0x7fb364100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 -- 172.21.15.78:0/568904017 shutdown_connections 2024-05-21T15:23:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.284+0000 7fb368aa2640 1 -- 172.21.15.78:0/568904017 wait complete. 2024-05-21T15:23:21.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb368aa2640 1 Processor -- start 2024-05-21T15:23:21.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb368aa2640 1 -- start start 2024-05-21T15:23:21.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb368aa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364103230 0x7fb364074f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb368aa2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb364108ae0 0x7fb3640754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb368aa2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb364108f80 0x7fb3640759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb368aa2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb364112e20 con 0x7fb364103230 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb368aa2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb364112ca0 con 0x7fb364108ae0 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb368aa2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb364112fa0 con 0x7fb364108f80 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb36259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364103230 0x7fb364074f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb36259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364103230 0x7fb364074f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35418/0 (socket says 172.21.15.78:35418) 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb36259c640 1 -- 172.21.15.78:0/2495598011 learned_addr learned my addr 172.21.15.78:0/2495598011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb362d9d640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb364108f80 0x7fb3640759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb36259c640 1 -- 172.21.15.78:0/2495598011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb364108f80 msgr2=0x7fb3640759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.285+0000 7fb361d9b640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb364108ae0 0x7fb3640754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb36259c640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb364108f80 0x7fb3640759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb36259c640 1 -- 172.21.15.78:0/2495598011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb364108ae0 msgr2=0x7fb3640754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb36259c640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb364108ae0 0x7fb3640754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb36259c640 1 -- 172.21.15.78:0/2495598011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3641a7b60 con 0x7fb364103230 2024-05-21T15:23:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb362d9d640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb364108f80 0x7fb3640759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb36259c640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364103230 0x7fb364074f60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb354000f80 tx=0x7fb3540319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb35b7fe640 1 -- 172.21.15.78:0/2495598011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35402fa20 con 0x7fb364103230 2024-05-21T15:23:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3641a7df0 con 0x7fb364103230 2024-05-21T15:23:21.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb35b7fe640 1 -- 172.21.15.78:0/2495598011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35402fbc0 con 0x7fb364103230 2024-05-21T15:23:21.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb35b7fe640 1 -- 172.21.15.78:0/2495598011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3540466e0 con 0x7fb364103230 2024-05-21T15:23:21.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.286+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3641a8260 con 0x7fb364103230 2024-05-21T15:23:21.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.287+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb364069140 con 0x7fb364103230 2024-05-21T15:23:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.288+0000 7fb35b7fe640 1 -- 172.21.15.78:0/2495598011 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb3540305c0 con 0x7fb364103230 2024-05-21T15:23:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.289+0000 7fb35b7fe640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2d00 0x7fb3500a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.289+0000 7fb361d9b640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2d00 0x7fb3500a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.289+0000 7fb35b7fe640 1 -- 172.21.15.78:0/2495598011 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fb3540f0a50 con 0x7fb364103230 2024-05-21T15:23:21.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.289+0000 7fb361d9b640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2d00 0x7fb3500a5210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb364100f10 tx=0x7fb34c005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.292+0000 7fb35b7fe640 1 -- 172.21.15.78:0/2495598011 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3540bd6f0 con 0x7fb364103230 2024-05-21T15:23:21.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.508+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3641a8560 con 0x7fb364103230 2024-05-21T15:23:21.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.509+0000 7fb35b7fe640 1 -- 172.21.15.78:0/2495598011 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7fb3540bcff0 con 0x7fb364103230 2024-05-21T15:23:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:23:21.513 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:23:21.513 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:23:21.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2d00 msgr2=0x7fb3500a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2d00 0x7fb3500a5210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb364100f10 tx=0x7fb34c005f90 comp rx=0 tx=0).stop 2024-05-21T15:23:21.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364103230 msgr2=0x7fb364074f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364103230 0x7fb364074f60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb354000f80 tx=0x7fb3540319f0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 shutdown_connections 2024-05-21T15:23:21.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb364108ae0 0x7fb3640754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb3500a2d00 0x7fb3500a5210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb364108f80 0x7fb3640759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 --2- 172.21.15.78:0/2495598011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb364103230 0x7fb364074f60 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 >> 172.21.15.78:0/2495598011 conn(0x7fb3640fe1c0 msgr2=0x7fb3640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 shutdown_connections 2024-05-21T15:23:21.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.511+0000 7fb368aa2640 1 -- 172.21.15.78:0/2495598011 wait complete. 2024-05-21T15:23:21.527 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:23:21.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 -- 172.21.15.78:0/846082000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541079c0 msgr2=0x7fcc54101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 --2- 172.21.15.78:0/846082000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541079c0 0x7fcc54101c30 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009900 tx=0x7fcc4402a410 comp rx=0 tx=0).stop 2024-05-21T15:23:21.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 -- 172.21.15.78:0/846082000 shutdown_connections 2024-05-21T15:23:21.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 --2- 172.21.15.78:0/846082000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc54107e60 0x7fcc54102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 --2- 172.21.15.78:0/846082000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc541027e0 0x7fcc5410e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 --2- 172.21.15.78:0/846082000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541079c0 0x7fcc54101c30 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 -- 172.21.15.78:0/846082000 >> 172.21.15.78:0/846082000 conn(0x7fcc54076830 msgr2=0x7fcc54076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 -- 172.21.15.78:0/846082000 shutdown_connections 2024-05-21T15:23:21.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.636+0000 7fcc59b85640 1 -- 172.21.15.78:0/846082000 wait complete. 2024-05-21T15:23:21.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc59b85640 1 Processor -- start 2024-05-21T15:23:21.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc59b85640 1 -- start start 2024-05-21T15:23:21.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc59b85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541027e0 0x7fcc54198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc59b85640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc541079c0 0x7fcc54198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc59b85640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc54107e60 0x7fcc5419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc59b85640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54111d00 con 0x7fcc541027e0 2024-05-21T15:23:21.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc59b85640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54111b80 con 0x7fcc541079c0 2024-05-21T15:23:21.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc59b85640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54111e80 con 0x7fcc54107e60 2024-05-21T15:23:21.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541027e0 0x7fcc54198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541027e0 0x7fcc54198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35424/0 (socket says 172.21.15.78:35424) 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.637+0000 7fcc537fe640 1 -- 172.21.15.78:0/1459829689 learned_addr learned my addr 172.21.15.78:0/1459829689 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc541079c0 0x7fcc54198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc537fe640 1 -- 172.21.15.78:0/1459829689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc54107e60 msgr2=0x7fcc5419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc53fff640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc54107e60 0x7fcc5419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc537fe640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc54107e60 0x7fcc5419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc537fe640 1 -- 172.21.15.78:0/1459829689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc541079c0 msgr2=0x7fcc54198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc537fe640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc541079c0 0x7fcc54198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc537fe640 1 -- 172.21.15.78:0/1459829689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5419d560 con 0x7fcc541027e0 2024-05-21T15:23:21.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc541079c0 0x7fcc54198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:21.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc537fe640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541027e0 0x7fcc54198700 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fcc44005940 tx=0x7fcc44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc50ff9640 1 -- 172.21.15.78:0/1459829689 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4403f030 con 0x7fcc541027e0 2024-05-21T15:23:21.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc50ff9640 1 -- 172.21.15.78:0/1459829689 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc44003ad0 con 0x7fcc541027e0 2024-05-21T15:23:21.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc59b85640 1 -- 172.21.15.78:0/1459829689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5419d7f0 con 0x7fcc541027e0 2024-05-21T15:23:21.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc50ff9640 1 -- 172.21.15.78:0/1459829689 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44035070 con 0x7fcc541027e0 2024-05-21T15:23:21.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.638+0000 7fcc59b85640 1 -- 172.21.15.78:0/1459829689 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5419dce0 con 0x7fcc541027e0 2024-05-21T15:23:21.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc60a34740 1 Processor -- start 2024-05-21T15:23:21.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc50ff9640 1 -- 172.21.15.78:0/1459829689 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcc44042030 con 0x7fcc541027e0 2024-05-21T15:23:21.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc60a34740 1 -- start start 2024-05-21T15:23:21.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 0x555db6376bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db63770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 0x555db6377630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc60a34740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555db62dcc40 con 0x555db5faf4d0 2024-05-21T15:23:21.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc60a34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555db6284580 con 0x555db5fd8390 2024-05-21T15:23:21.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.641+0000 7fcc60a34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555db62d8c00 con 0x555db5fb76b0 2024-05-21T15:23:21.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db63770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db63770f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35438/0 (socket says 172.21.15.78:35438) 2024-05-21T15:23:21.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 -- 172.21.15.78:0/1887776596 learned_addr learned my addr 172.21.15.78:0/1887776596 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc53fff640 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 0x555db6376bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 0x555db6377630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 -- 172.21.15.78:0/1887776596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 msgr2=0x555db6377630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 0x555db6377630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 -- 172.21.15.78:0/1887776596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 msgr2=0x555db6376bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 0x555db6376bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc50ff9640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcc3c0a2d00 0x7fcc3c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 -- 172.21.15.78:0/1887776596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db5ff03f0 con 0x555db5faf4d0 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc53fff640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcc3c0a2d00 0x7fcc3c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc537fe640 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db63770f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcc440ed340 tx=0x7fcc440f3f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216922f7754925f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc50ff9640 1 -- 172.21.15.78:0/1459829689 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fcc440efd10 con 0x7fcc541027e0 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc39ffb640 1 -- 172.21.15.78:0/1887776596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44049070 con 0x555db5faf4d0 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc39ffb640 1 -- 172.21.15.78:0/1887776596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc440bc8a0 con 0x555db5faf4d0 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.642+0000 7fcc39ffb640 1 -- 172.21.15.78:0/1887776596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440bbce0 con 0x555db5faf4d0 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 -- 172.21.15.78:0/1887776596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 msgr2=0x555db63770f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc53fff640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcc3c0a2d00 0x7fcc3c0a5210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x555db6373ec0 tx=0x7fcc4c009500 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db63770f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcc440ed340 tx=0x7fcc440f3f80 comp rx=0 tx=0).stop 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 -- 172.21.15.78:0/1887776596 shutdown_connections 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 0x555db6377630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db63770f0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 --2- 172.21.15.78:0/1887776596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 0x555db6376bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 -- 172.21.15.78:0/1887776596 >> 172.21.15.78:0/1887776596 conn(0x555db5fcc4e0 msgr2=0x555db63734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 -- 172.21.15.78:0/1887776596 shutdown_connections 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.643+0000 7fcc60a34740 1 -- 172.21.15.78:0/1887776596 wait complete. 2024-05-21T15:23:21.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 1 -- start start 2024-05-21T15:23:21.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 1 Processor -- start 2024-05-21T15:23:21.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db6383590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 0x555db6383ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 0x555db6389440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555db6386040 con 0x555db5faf4d0 2024-05-21T15:23:21.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555db6297c40 con 0x555db5fd8390 2024-05-21T15:23:21.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555db63861c0 con 0x555db5fb76b0 2024-05-21T15:23:21.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc60a34740 10 client.0 _ll_register_callbacks cb 0x555db5fefbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:21.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db6383590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db6383590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35450/0 (socket says 172.21.15.78:35450) 2024-05-21T15:23:21.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.644+0000 7fcc537fe640 1 -- 172.21.15.78:0/3690973006 learned_addr learned my addr 172.21.15.78:0/3690973006 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc53fff640 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 0x555db6389440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 0x555db6383ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc537fe640 1 -- 172.21.15.78:0/3690973006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 msgr2=0x555db6383ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc537fe640 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 0x555db6383ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc537fe640 1 -- 172.21.15.78:0/3690973006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 msgr2=0x555db6389440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc537fe640 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 0x555db6389440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc537fe640 1 -- 172.21.15.78:0/3690973006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db60ba7d0 con 0x555db5faf4d0 2024-05-21T15:23:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc537fe640 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db6383590 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcc440ed480 tx=0x7fcc440bbfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0b4b6e18ddb6d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3690973006 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44049040 con 0x555db5faf4d0 2024-05-21T15:23:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3690973006 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fcc44003c80 con 0x555db5faf4d0 2024-05-21T15:23:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc60a34740 1 -- 172.21.15.78:0/3690973006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc440bc8a0 con 0x555db5faf4d0 2024-05-21T15:23:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3690973006 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcc440bbb00 con 0x555db5faf4d0 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc60a34740 10 client.17631 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc60a34740 1 -- 172.21.15.78:0/3690973006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555db6386040 con 0x555db5faf4d0 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3690973006 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc440bc8a0 con 0x555db5faf4d0 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3690973006 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fcc44086020 con 0x555db5faf4d0 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc60a34740 10 client.17631 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc60a34740 10 client.17631 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.645+0000 7fcc60a34740 20 client.17631 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.646+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x555db5f96f60 0x555db6395a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.646+0000 7fcc60a34740 4 client.17631 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.646+0000 7fcc60a34740 1 -- 172.21.15.78:0/3690973006 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db62ea370 con 0x555db5f96f60 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.646+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x555db5f96f60 0x555db6395a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.646+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x555db5f96f60 0x555db6395a30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=11b251435d23691 server_cookie=67066d4591c7d883 in_seq=0 out_seq=0 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.646+0000 7fcc32ffd640 10 client.17631 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3690973006 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555db62ea370 con 0x555db5f96f60 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc32ffd640 10 client.17631 handle_command_reply: tid=0 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 client.17631 shutdown 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 -- 172.21.15.78:0/3690973006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 msgr2=0x555db6383590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db6383590 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcc440ed480 tx=0x7fcc440bbfe0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 -- 172.21.15.78:0/3690973006 shutdown_connections 2024-05-21T15:23:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x555db5f96f60 0x555db6395a30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fd8390 0x555db6389440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fb76b0 0x555db6383ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3690973006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db6383590 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc32ffd640 0 client.17631 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:23:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 -- 172.21.15.78:0/3690973006 >> 172.21.15.78:0/3690973006 conn(0x555db5fcc4e0 msgr2=0x555db6378a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 -- 172.21.15.78:0/3690973006 shutdown_connections 2024-05-21T15:23:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 1 -- 172.21.15.78:0/3690973006 wait complete. 2024-05-21T15:23:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.647+0000 7fcc60a34740 20 client.17631 trim_cache size 0 max 16384 2024-05-21T15:23:21.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc60a34740 1 Processor -- start 2024-05-21T15:23:21.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc60a34740 1 -- start start 2024-05-21T15:23:21.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5faf4d0 0x555db63893f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db6382550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5fd8390 0x555db63771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc60a34740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541acb70 con 0x555db5fd8390 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc60a34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541ae6b0 con 0x555db5fb76b0 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc60a34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541ad240 con 0x555db5faf4d0 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5fd8390 0x555db63771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5fd8390 0x555db63771c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35462/0 (socket says 172.21.15.78:35462) 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc53fff640 1 -- 172.21.15.78:0/311351514 learned_addr learned my addr 172.21.15.78:0/311351514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db6382550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc537fe640 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5faf4d0 0x555db63893f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc53fff640 1 -- 172.21.15.78:0/311351514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5faf4d0 msgr2=0x555db63893f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc53fff640 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5faf4d0 0x555db63893f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc53fff640 1 -- 172.21.15.78:0/311351514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 msgr2=0x555db6382550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc53fff640 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db6382550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc53fff640 1 -- 172.21.15.78:0/311351514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db63784e0 con 0x555db5fd8390 2024-05-21T15:23:21.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db6382550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:21.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.661+0000 7fcc537fe640 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5faf4d0 0x555db63893f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:21.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.662+0000 7fcc53fff640 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5fd8390 0x555db63771c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c012800 tx=0x7fcc4c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feada2fa789b2348 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.662+0000 7fcc397fa640 1 -- 172.21.15.78:0/311351514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c01fb40 con 0x555db5fd8390 2024-05-21T15:23:21.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.662+0000 7fcc397fa640 1 -- 172.21.15.78:0/311351514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc4c012e40 con 0x555db5fd8390 2024-05-21T15:23:21.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.662+0000 7fcc397fa640 1 -- 172.21.15.78:0/311351514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c028630 con 0x555db5fd8390 2024-05-21T15:23:21.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.662+0000 7fcc60a34740 1 -- 172.21.15.78:0/311351514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5fd8390 msgr2=0x555db63771c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.662+0000 7fcc60a34740 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5fd8390 0x555db63771c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c012800 tx=0x7fcc4c014000 comp rx=0 tx=0).stop 2024-05-21T15:23:21.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 -- 172.21.15.78:0/311351514 shutdown_connections 2024-05-21T15:23:21.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db6382550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5fd8390 0x555db63771c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 --2- 172.21.15.78:0/311351514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5faf4d0 0x555db63893f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 -- 172.21.15.78:0/311351514 >> 172.21.15.78:0/311351514 conn(0x555db5fcc4e0 msgr2=0x555db6376ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 -- 172.21.15.78:0/311351514 shutdown_connections 2024-05-21T15:23:21.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 -- 172.21.15.78:0/311351514 wait complete. 2024-05-21T15:23:21.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 -- start start 2024-05-21T15:23:21.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 Processor -- start 2024-05-21T15:23:21.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db638f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db638ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fd8390 0x555db638b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541adfe0 con 0x555db5faf4d0 2024-05-21T15:23:21.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541ad910 con 0x555db5fb76b0 2024-05-21T15:23:21.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.663+0000 7fcc60a34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541aab70 con 0x555db5fd8390 2024-05-21T15:23:21.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc60a34740 10 client.0 _ll_register_callbacks cb 0x7fcc440f3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:21.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db638f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db638f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35464/0 (socket says 172.21.15.78:35464) 2024-05-21T15:23:21.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 -- 172.21.15.78:0/3543786018 learned_addr learned my addr 172.21.15.78:0/3543786018 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db638ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 -- 172.21.15.78:0/3543786018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fd8390 msgr2=0x555db638b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc53fff640 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fd8390 0x555db638b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fd8390 0x555db638b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 -- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 msgr2=0x555db638ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db638ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 -- 172.21.15.78:0/3543786018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db638b730 con 0x555db5faf4d0 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db638ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc537fe640 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db638f890 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcc440ba450 tx=0x7fcc440dc340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d955ff81d4a57be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:21.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3543786018 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44049070 con 0x555db5faf4d0 2024-05-21T15:23:21.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3543786018 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fcc440ed650 con 0x555db5faf4d0 2024-05-21T15:23:21.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc60a34740 1 -- 172.21.15.78:0/3543786018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555db62e46c0 con 0x555db5faf4d0 2024-05-21T15:23:21.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3543786018 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcc440bbce0 con 0x555db5faf4d0 2024-05-21T15:23:21.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc60a34740 10 client.17637 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:21.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc60a34740 1 -- 172.21.15.78:0/3543786018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555db638b730 con 0x555db5faf4d0 2024-05-21T15:23:21.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.664+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3543786018 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555db62e46c0 con 0x555db5faf4d0 2024-05-21T15:23:21.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3543786018 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fcc44086030 con 0x555db5faf4d0 2024-05-21T15:23:21.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc60a34740 10 client.17637 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:21.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc60a34740 10 client.17637 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:23:21.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc60a34740 20 client.17637 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:21.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x555db5f6c3a0 0x555db63ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc60a34740 4 client.17637 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:21.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc60a34740 1 -- 172.21.15.78:0/3543786018 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555db6261820 con 0x555db5f6c3a0 2024-05-21T15:23:21.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x555db5f6c3a0 0x555db63ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc32ffd640 10 client.17637 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:23:21.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc52ffd640 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x555db5f6c3a0 0x555db63ffd90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42cdd593a3cb8242 server_cookie=522faa4ba5ed3739 in_seq=0 out_seq=0 2024-05-21T15:23:21.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc32ffd640 1 -- 172.21.15.78:0/3543786018 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555db6261820 con 0x555db5f6c3a0 2024-05-21T15:23:21.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.665+0000 7fcc32ffd640 10 client.17637 handle_command_reply: tid=0 2024-05-21T15:23:21.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 client.17637 shutdown 2024-05-21T15:23:21.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 -- 172.21.15.78:0/3543786018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 msgr2=0x555db638f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db638f890 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcc440ba450 tx=0x7fcc440dc340 comp rx=0 tx=0).stop 2024-05-21T15:23:21.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 -- 172.21.15.78:0/3543786018 shutdown_connections 2024-05-21T15:23:21.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x555db5f6c3a0 0x555db63ffd90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555db5fb76b0 0x555db638ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db5fd8390 0x555db638b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc32ffd640 0 client.17637 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:23:21.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 --2- 172.21.15.78:0/3543786018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555db5faf4d0 0x555db638f890 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 -- 172.21.15.78:0/3543786018 >> 172.21.15.78:0/3543786018 conn(0x555db5fcc4e0 msgr2=0x555db63822b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 -- 172.21.15.78:0/3543786018 shutdown_connections 2024-05-21T15:23:21.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 1 -- 172.21.15.78:0/3543786018 wait complete. 2024-05-21T15:23:21.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc60a34740 20 client.17637 trim_cache size 0 max 16384 2024-05-21T15:23:21.681 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:21.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 -- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcc3c0a2d00 msgr2=0x7fcc3c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcc3c0a2d00 0x7fcc3c0a5210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x555db6373ec0 tx=0x7fcc4c009500 comp rx=0 tx=0).stop 2024-05-21T15:23:21.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 -- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541027e0 msgr2=0x7fcc54198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541027e0 0x7fcc54198700 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fcc44005940 tx=0x7fcc44004270 comp rx=0 tx=0).stop 2024-05-21T15:23:21.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 -- 172.21.15.78:0/1459829689 shutdown_connections 2024-05-21T15:23:21.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc541079c0 0x7fcc54198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fcc3c0a2d00 0x7fcc3c0a5210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc54107e60 0x7fcc5419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 --2- 172.21.15.78:0/1459829689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc541027e0 0x7fcc54198700 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.666+0000 7fcc59b85640 1 -- 172.21.15.78:0/1459829689 >> 172.21.15.78:0/1459829689 conn(0x7fcc54076830 msgr2=0x7fcc54103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.667+0000 7fcc59b85640 1 -- 172.21.15.78:0/1459829689 shutdown_connections 2024-05-21T15:23:21.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.667+0000 7fcc59b85640 1 -- 172.21.15.78:0/1459829689 wait complete. 2024-05-21T15:23:21.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:23:21.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 -- 172.21.15.78:0/3456632827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010ca70 msgr2=0x7f6e4010ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 --2- 172.21.15.78:0/3456632827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010ca70 0x7f6e4010ee90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009900 tx=0x7f6e3002a410 comp rx=0 tx=0).stop 2024-05-21T15:23:21.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 -- 172.21.15.78:0/3456632827 shutdown_connections 2024-05-21T15:23:21.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 --2- 172.21.15.78:0/3456632827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4010f3d0 0x7f6e401117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 --2- 172.21.15.78:0/3456632827 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010ca70 0x7f6e4010ee90 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 --2- 172.21.15.78:0/3456632827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e40101830 0x7f6e40101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 -- 172.21.15.78:0/3456632827 >> 172.21.15.78:0/3456632827 conn(0x7f6e400770c0 msgr2=0x7f6e40077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 -- 172.21.15.78:0/3456632827 shutdown_connections 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.793+0000 7f6e48092640 1 -- 172.21.15.78:0/3456632827 wait complete. 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e48092640 1 Processor -- start 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e48092640 1 -- start start 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e48092640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e40101830 0x7f6e401a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e48092640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4010ca70 0x7f6e401a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e48092640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010f3d0 0x7f6e4019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e48092640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40115030 con 0x7f6e4010f3d0 2024-05-21T15:23:21.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e48092640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40114eb0 con 0x7f6e40101830 2024-05-21T15:23:21.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e48092640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401151b0 con 0x7f6e4010ca70 2024-05-21T15:23:21.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010f3d0 0x7f6e4019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010f3d0 0x7f6e4019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35480/0 (socket says 172.21.15.78:35480) 2024-05-21T15:23:21.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 -- 172.21.15.78:0/3625913584 learned_addr learned my addr 172.21.15.78:0/3625913584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e40101830 0x7f6e401a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4562d640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4010ca70 0x7f6e401a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 -- 172.21.15.78:0/3625913584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4010ca70 msgr2=0x7f6e401a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4010ca70 0x7f6e401a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 -- 172.21.15.78:0/3625913584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e40101830 msgr2=0x7f6e401a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e40101830 0x7f6e401a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 -- 172.21.15.78:0/3625913584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4019be80 con 0x7f6e4010f3d0 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.794+0000 7f6e4662f640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010f3d0 0x7f6e4019b810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800cce0 tx=0x7f6e3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.795+0000 7f6e36ffd640 1 -- 172.21.15.78:0/3625913584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800f450 con 0x7f6e4010f3d0 2024-05-21T15:23:21.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.795+0000 7f6e48092640 1 -- 172.21.15.78:0/3625913584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4019c170 con 0x7f6e4010f3d0 2024-05-21T15:23:21.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.795+0000 7f6e36ffd640 1 -- 172.21.15.78:0/3625913584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3800fad0 con 0x7f6e4010f3d0 2024-05-21T15:23:21.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.795+0000 7f6e36ffd640 1 -- 172.21.15.78:0/3625913584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38018c10 con 0x7f6e4010f3d0 2024-05-21T15:23:21.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.795+0000 7f6e48092640 1 -- 172.21.15.78:0/3625913584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4019c640 con 0x7f6e4010f3d0 2024-05-21T15:23:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.797+0000 7f6e36ffd640 1 -- 172.21.15.78:0/3625913584 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6e3800f5f0 con 0x7f6e4010f3d0 2024-05-21T15:23:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.797+0000 7f6e4f018740 1 Processor -- start 2024-05-21T15:23:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.797+0000 7f6e4f018740 1 -- start start 2024-05-21T15:23:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.797+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1630a390 0x563b166a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.797+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b162e14d0 0x563b166a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.797+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e4f018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b1660ec40 con 0x563b162e14d0 2024-05-21T15:23:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e4f018740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b165b6580 con 0x563b1630a390 2024-05-21T15:23:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e4f018740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563b1660ac00 con 0x563b162e96b0 2024-05-21T15:23:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b162e14d0 0x563b166a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b162e14d0 0x563b166a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35490/0 (socket says 172.21.15.78:35490) 2024-05-21T15:23:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 -- 172.21.15.78:0/2460199582 learned_addr learned my addr 172.21.15.78:0/2460199582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e36ffd640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6e2c0a2d00 0x7f6e2c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e4562d640 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1630a390 0x563b166a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e4562d640 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 -- 172.21.15.78:0/2460199582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 msgr2=0x563b166a97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166a97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 -- 172.21.15.78:0/2460199582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1630a390 msgr2=0x563b166a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1630a390 0x563b166a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e4662f640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6e2c0a2d00 0x7f6e2c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 -- 172.21.15.78:0/2460199582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b163223f0 con 0x563b162e14d0 2024-05-21T15:23:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e4562d640 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166a97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e36ffd640 1 -- 172.21.15.78:0/3625913584 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f6e380cfb10 con 0x7f6e4010f3d0 2024-05-21T15:23:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b162e14d0 0x563b166a9260 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009c90 tx=0x7f6e3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b4650c35dc0a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e2bfff640 1 -- 172.21.15.78:0/2460199582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c005220 con 0x563b162e14d0 2024-05-21T15:23:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e2bfff640 1 -- 172.21.15.78:0/2460199582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3c004c80 con 0x563b162e14d0 2024-05-21T15:23:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.798+0000 7f6e2bfff640 1 -- 172.21.15.78:0/2460199582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c019710 con 0x563b162e14d0 2024-05-21T15:23:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 -- 172.21.15.78:0/2460199582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b162e14d0 msgr2=0x563b166a9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4662f640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6e2c0a2d00 0x7f6e2c0a5210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e380cd280 tx=0x7f6e3806f010 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b162e14d0 0x563b166a9260 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009c90 tx=0x7f6e3c011810 comp rx=0 tx=0).stop 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 -- 172.21.15.78:0/2460199582 shutdown_connections 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166a97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b162e14d0 0x563b166a9260 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 --2- 172.21.15.78:0/2460199582 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1630a390 0x563b166a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 -- 172.21.15.78:0/2460199582 >> 172.21.15.78:0/2460199582 conn(0x563b162fe4e0 msgr2=0x563b166a54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 -- 172.21.15.78:0/2460199582 shutdown_connections 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.799+0000 7f6e4f018740 1 -- 172.21.15.78:0/2460199582 wait complete. 2024-05-21T15:23:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 1 -- start start 2024-05-21T15:23:21.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 1 Processor -- start 2024-05-21T15:23:21.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 0x563b166b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b166b5fb0 con 0x563b1630a390 2024-05-21T15:23:21.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b165c9c40 con 0x563b162e14d0 2024-05-21T15:23:21.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563b166b6130 con 0x563b162e96b0 2024-05-21T15:23:21.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4f018740 10 client.0 _ll_register_callbacks cb 0x563b16321be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:21.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4562d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4562d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35498/0 (socket says 172.21.15.78:35498) 2024-05-21T15:23:21.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.800+0000 7f6e4562d640 1 -- 172.21.15.78:0/665688642 learned_addr learned my addr 172.21.15.78:0/665688642 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4562d640 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 0x563b166b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4562d640 1 -- 172.21.15.78:0/665688642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 msgr2=0x563b166b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4562d640 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4562d640 1 -- 172.21.15.78:0/665688642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 msgr2=0x563b166b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4562d640 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 0x563b166b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4562d640 1 -- 172.21.15.78:0/665688642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b163ec7d0 con 0x563b1630a390 2024-05-21T15:23:21.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4562d640 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b3300 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009230 tx=0x7f6e3002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88832d06963226d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:21.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e297fa640 1 -- 172.21.15.78:0/665688642 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30035040 con 0x563b1630a390 2024-05-21T15:23:21.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e297fa640 1 -- 172.21.15.78:0/665688642 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f6e30003c30 con 0x563b1630a390 2024-05-21T15:23:21.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4f018740 1 -- 172.21.15.78:0/665688642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e3c004c80 con 0x563b1630a390 2024-05-21T15:23:21.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e297fa640 1 -- 172.21.15.78:0/665688642 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e30003e30 con 0x563b1630a390 2024-05-21T15:23:21.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4f018740 10 client.17649 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:21.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e4f018740 1 -- 172.21.15.78:0/665688642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563b166c27a0 con 0x563b1630a390 2024-05-21T15:23:21.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.801+0000 7f6e297fa640 1 -- 172.21.15.78:0/665688642 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e3c004c80 con 0x563b1630a390 2024-05-21T15:23:21.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e297fa640 1 -- 172.21.15.78:0/665688642 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6e3003a040 con 0x563b1630a390 2024-05-21T15:23:21.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e4f018740 10 client.17649 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:21.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e4f018740 10 client.17649 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:23:21.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e4f018740 20 client.17649 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:21.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e4f018740 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x563b162c8f60 0x563b166c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e4f018740 4 client.17649 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:21.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e4f018740 1 -- 172.21.15.78:0/665688642 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b1661c370 con 0x563b162c8f60 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x563b162c8f60 0x563b166c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e297fa640 10 client.17649 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.802+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x563b162c8f60 0x563b166c5cf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d9934b974448a22 server_cookie=a0013b0561914fc2 in_seq=0 out_seq=0 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e297fa640 1 -- 172.21.15.78:0/665688642 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563b1661c370 con 0x563b162c8f60 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e297fa640 10 client.17649 handle_command_reply: tid=0 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 client.17649 shutdown 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 -- 172.21.15.78:0/665688642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 msgr2=0x563b166b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b3300 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009230 tx=0x7f6e3002a410 comp rx=0 tx=0).stop 2024-05-21T15:23:21.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 -- 172.21.15.78:0/665688642 shutdown_connections 2024-05-21T15:23:21.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x563b162c8f60 0x563b166c5cf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b3300 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 --2- 172.21.15.78:0/665688642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 0x563b166b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e297fa640 0 client.17649 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:23:21.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 -- 172.21.15.78:0/665688642 >> 172.21.15.78:0/665688642 conn(0x563b162fe4e0 msgr2=0x563b166aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 -- 172.21.15.78:0/665688642 shutdown_connections 2024-05-21T15:23:21.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.803+0000 7f6e4f018740 1 -- 172.21.15.78:0/665688642 wait complete. 2024-05-21T15:23:21.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.804+0000 7f6e4f018740 20 client.17649 trim_cache size 0 max 16384 2024-05-21T15:23:21.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4f018740 1 Processor -- start 2024-05-21T15:23:21.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4f018740 1 -- start start 2024-05-21T15:23:21.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 0x563b166cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4f018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401b0db0 con 0x563b1630a390 2024-05-21T15:23:21.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4f018740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401b28f0 con 0x563b162e14d0 2024-05-21T15:23:21.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4f018740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401b1480 con 0x563b162e96b0 2024-05-21T15:23:21.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35514/0 (socket says 172.21.15.78:35514) 2024-05-21T15:23:21.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 -- 172.21.15.78:0/518637964 learned_addr learned my addr 172.21.15.78:0/518637964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 --2- 172.21.15.78:0/518637964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 0x563b166cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 -- 172.21.15.78:0/518637964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 msgr2=0x563b166cb9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:23:21.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 --2- 172.21.15.78:0/518637964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166cb9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 -- 172.21.15.78:0/518637964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 msgr2=0x563b166cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 --2- 172.21.15.78:0/518637964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 0x563b166cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.817+0000 7f6e4562d640 1 -- 172.21.15.78:0/518637964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b166b1850 con 0x563b1630a390 2024-05-21T15:23:21.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4562d640 1 --2- 172.21.15.78:0/518637964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b2be0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6e300314a0 tx=0x7f6e30009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a0322351f75ef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e2b7fe640 1 -- 172.21.15.78:0/518637964 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30035070 con 0x563b1630a390 2024-05-21T15:23:21.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e2b7fe640 1 -- 172.21.15.78:0/518637964 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e30053040 con 0x563b1630a390 2024-05-21T15:23:21.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e2b7fe640 1 -- 172.21.15.78:0/518637964 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30052070 con 0x563b1630a390 2024-05-21T15:23:21.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4f018740 1 -- 172.21.15.78:0/518637964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 msgr2=0x563b166b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4f018740 1 --2- 172.21.15.78:0/518637964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b2be0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6e300314a0 tx=0x7f6e30009260 comp rx=0 tx=0).stop 2024-05-21T15:23:21.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4f018740 1 -- 172.21.15.78:0/518637964 shutdown_connections 2024-05-21T15:23:21.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4f018740 1 --2- 172.21.15.78:0/518637964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166b2be0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4f018740 1 --2- 172.21.15.78:0/518637964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e96b0 0x563b166cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4f018740 1 --2- 172.21.15.78:0/518637964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e14d0 0x563b166cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4f018740 1 -- 172.21.15.78:0/518637964 >> 172.21.15.78:0/518637964 conn(0x563b162fe4e0 msgr2=0x563b166a94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.818+0000 7f6e4f018740 1 -- 172.21.15.78:0/518637964 shutdown_connections 2024-05-21T15:23:21.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 -- 172.21.15.78:0/518637964 wait complete. 2024-05-21T15:23:21.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 -- start start 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 Processor -- start 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e14d0 0x563b166bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e96b0 0x563b166c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166bcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401b2220 con 0x563b1630a390 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401b1b50 con 0x563b162e96b0 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401aedb0 con 0x563b162e14d0 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4f018740 10 client.0 _ll_register_callbacks cb 0x7f6e3c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:21.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.819+0000 7f6e4562d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166bcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166bcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35530/0 (socket says 172.21.15.78:35530) 2024-05-21T15:23:21.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 -- 172.21.15.78:0/3221647053 learned_addr learned my addr 172.21.15.78:0/3221647053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e14d0 0x563b166bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 -- 172.21.15.78:0/3221647053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e14d0 msgr2=0x563b166bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e14d0 0x563b166bfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e96b0 0x563b166c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 -- 172.21.15.78:0/3221647053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e96b0 msgr2=0x563b166c0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e96b0 0x563b166c0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 -- 172.21.15.78:0/3221647053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b166bd720 con 0x563b1630a390 2024-05-21T15:23:21.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4562d640 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166bcfb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f6e3003da00 tx=0x7f6e3003d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33c2c7e9a2c5d1b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:21.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e297fa640 1 -- 172.21.15.78:0/3221647053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3005d060 con 0x563b1630a390 2024-05-21T15:23:21.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e297fa640 1 -- 172.21.15.78:0/3221647053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f6e3003a450 con 0x563b1630a390 2024-05-21T15:23:21.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4f018740 1 -- 172.21.15.78:0/3221647053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e30053040 con 0x563b1630a390 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e297fa640 1 -- 172.21.15.78:0/3221647053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e30035070 con 0x563b1630a390 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4f018740 10 client.17655 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.820+0000 7f6e4f018740 1 -- 172.21.15.78:0/3221647053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563b1672db70 con 0x563b1630a390 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e297fa640 1 -- 172.21.15.78:0/3221647053 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e30053040 con 0x563b1630a390 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e297fa640 1 -- 172.21.15.78:0/3221647053 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6e3004e5e0 con 0x563b1630a390 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e4f018740 10 client.17655 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e4f018740 10 client.17655 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e4f018740 20 client.17655 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:21.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e4f018740 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x563b1629e3a0 0x563b167301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e4f018740 4 client.17655 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:21.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e4f018740 1 -- 172.21.15.78:0/3221647053 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b166166c0 con 0x563b1629e3a0 2024-05-21T15:23:21.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x563b1629e3a0 0x563b167301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e297fa640 10 client.17655 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:23:21.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.821+0000 7f6e45e2e640 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x563b1629e3a0 0x563b167301e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9e0a364ee926e0 server_cookie=c10a9cb5bb458377 in_seq=0 out_seq=0 2024-05-21T15:23:21.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e297fa640 1 -- 172.21.15.78:0/3221647053 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b166166c0 con 0x563b1629e3a0 2024-05-21T15:23:21.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e297fa640 10 client.17655 handle_command_reply: tid=0 2024-05-21T15:23:21.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 client.17655 shutdown 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 -- 172.21.15.78:0/3221647053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 msgr2=0x563b166bcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166bcfb0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f6e3003da00 tx=0x7f6e3003d440 comp rx=0 tx=0).stop 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 -- 172.21.15.78:0/3221647053 shutdown_connections 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x563b1629e3a0 0x563b167301e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b162e96b0 0x563b166c0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e297fa640 0 client.17655 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b1630a390 0x563b166bcfb0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 --2- 172.21.15.78:0/3221647053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b162e14d0 0x563b166bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 -- 172.21.15.78:0/3221647053 >> 172.21.15.78:0/3221647053 conn(0x563b162fe4e0 msgr2=0x563b166a9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 -- 172.21.15.78:0/3221647053 shutdown_connections 2024-05-21T15:23:21.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 1 -- 172.21.15.78:0/3221647053 wait complete. 2024-05-21T15:23:21.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.822+0000 7f6e4f018740 20 client.17655 trim_cache size 0 max 16384 2024-05-21T15:23:21.837 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:21.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 -- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6e2c0a2d00 msgr2=0x7f6e2c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6e2c0a2d00 0x7f6e2c0a5210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e380cd280 tx=0x7f6e3806f010 comp rx=0 tx=0).stop 2024-05-21T15:23:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 -- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010f3d0 msgr2=0x7f6e4019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010f3d0 0x7f6e4019b810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800cce0 tx=0x7f6e3800a620 comp rx=0 tx=0).stop 2024-05-21T15:23:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 -- 172.21.15.78:0/3625913584 shutdown_connections 2024-05-21T15:23:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f6e2c0a2d00 0x7f6e2c0a5210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e4010f3d0 0x7f6e4019b810 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4010ca70 0x7f6e401a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 --2- 172.21.15.78:0/3625913584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e40101830 0x7f6e401a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 -- 172.21.15.78:0/3625913584 >> 172.21.15.78:0/3625913584 conn(0x7f6e400770c0 msgr2=0x7f6e40100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 -- 172.21.15.78:0/3625913584 shutdown_connections 2024-05-21T15:23:21.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.823+0000 7f6e48092640 1 -- 172.21.15.78:0/3625913584 wait complete. 2024-05-21T15:23:21.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:23:21.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 -- 172.21.15.78:0/3248760868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2890103c20 msgr2=0x7f2890104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 --2- 172.21.15.78:0/3248760868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2890103c20 0x7f2890104030 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2878009ca0 tx=0x7f287802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 -- 172.21.15.78:0/3248760868 shutdown_connections 2024-05-21T15:23:21.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 --2- 172.21.15.78:0/3248760868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f289010d070 0x7f289010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 --2- 172.21.15.78:0/3248760868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289010a680 0x7f289010caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 --2- 172.21.15.78:0/3248760868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2890103c20 0x7f2890104030 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 -- 172.21.15.78:0/3248760868 >> 172.21.15.78:0/3248760868 conn(0x7f28900770c0 msgr2=0x7f2890077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 -- 172.21.15.78:0/3248760868 shutdown_connections 2024-05-21T15:23:21.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.949+0000 7f2896aa8640 1 -- 172.21.15.78:0/3248760868 wait complete. 2024-05-21T15:23:21.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2896aa8640 1 Processor -- start 2024-05-21T15:23:21.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2896aa8640 1 -- start start 2024-05-21T15:23:21.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2896aa8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2890103c20 0x7f28901997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2896aa8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289010a680 0x7f2890199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2896aa8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f289010d070 0x7f289019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2896aa8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2890112e30 con 0x7f289010d070 2024-05-21T15:23:21.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2896aa8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2890112cb0 con 0x7f2890103c20 2024-05-21T15:23:21.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2896aa8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2890112fb0 con 0x7f289010a680 2024-05-21T15:23:21.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2895045640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f289010d070 0x7f289019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2895045640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f289010d070 0x7f289019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35552/0 (socket says 172.21.15.78:35552) 2024-05-21T15:23:21.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.950+0000 7f2895045640 1 -- 172.21.15.78:0/2288521741 learned_addr learned my addr 172.21.15.78:0/2288521741 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2894844640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2890103c20 0x7f28901997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f288ffff640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289010a680 0x7f2890199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2895045640 1 -- 172.21.15.78:0/2288521741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289010a680 msgr2=0x7f2890199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2895045640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289010a680 0x7f2890199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2895045640 1 -- 172.21.15.78:0/2288521741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2890103c20 msgr2=0x7f28901997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2895045640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2890103c20 0x7f28901997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2895045640 1 -- 172.21.15.78:0/2288521741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289019e650 con 0x7f289010d070 2024-05-21T15:23:21.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f288ffff640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289010a680 0x7f2890199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:21.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2895045640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f289010d070 0x7f289019e010 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f288800c970 tx=0x7f288800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f288dffb640 1 -- 172.21.15.78:0/2288521741 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888007bb0 con 0x7f289010d070 2024-05-21T15:23:21.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f288dffb640 1 -- 172.21.15.78:0/2288521741 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2888007d50 con 0x7f289010d070 2024-05-21T15:23:21.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f288dffb640 1 -- 172.21.15.78:0/2288521741 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288800f450 con 0x7f289010d070 2024-05-21T15:23:21.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2896aa8640 1 -- 172.21.15.78:0/2288521741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289019e940 con 0x7f289010d070 2024-05-21T15:23:21.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.951+0000 7f2896aa8640 1 -- 172.21.15.78:0/2288521741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289019ee40 con 0x7f289010d070 2024-05-21T15:23:21.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.954+0000 7f289da2c740 1 Processor -- start 2024-05-21T15:23:21.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.954+0000 7f288dffb640 1 -- 172.21.15.78:0/2288521741 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2888004f60 con 0x7f289010d070 2024-05-21T15:23:21.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.954+0000 7f289da2c740 1 -- start start 2024-05-21T15:23:21.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f289da2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c37bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f289da2c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f289da2c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 0x5632b7c38630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f289da2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5632b7b9dc40 con 0x5632b7899390 2024-05-21T15:23:21.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f289da2c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5632b7b45580 con 0x5632b78704d0 2024-05-21T15:23:21.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f289da2c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5632b7b99c00 con 0x5632b78786b0 2024-05-21T15:23:21.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c37bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c37bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35564/0 (socket says 172.21.15.78:35564) 2024-05-21T15:23:21.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 -- 172.21.15.78:0/2916086243 learned_addr learned my addr 172.21.15.78:0/2916086243 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f2894844640 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 0x5632b7c38630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288dffb640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f287c0a2d00 0x7f287c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 -- 172.21.15.78:0/2916086243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 msgr2=0x5632b7c38630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 0x5632b7c38630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 -- 172.21.15.78:0/2916086243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 msgr2=0x5632b7c380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 -- 172.21.15.78:0/2916086243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b78b13f0 con 0x5632b7899390 2024-05-21T15:23:21.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f2894844640 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:23:21.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f2895045640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f287c0a2d00 0x7f287c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288ffff640 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c37bb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2880015880 tx=0x7f2880009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1965265a1a074c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.955+0000 7f288dffb640 1 -- 172.21.15.78:0/2288521741 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f28880cf1a0 con 0x7f289010d070 2024-05-21T15:23:21.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.956+0000 7f2886ffd640 1 -- 172.21.15.78:0/2916086243 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880004030 con 0x5632b7899390 2024-05-21T15:23:21.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.956+0000 7f2886ffd640 1 -- 172.21.15.78:0/2916086243 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2880010bf0 con 0x5632b7899390 2024-05-21T15:23:21.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.956+0000 7f2886ffd640 1 -- 172.21.15.78:0/2916086243 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28800035f0 con 0x5632b7899390 2024-05-21T15:23:21.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.956+0000 7f289da2c740 1 -- 172.21.15.78:0/2916086243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 msgr2=0x5632b7c37bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.956+0000 7f2895045640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f287c0a2d00 0x7f287c0a5210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f28880cc9b0 tx=0x7f2888086040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.956+0000 7f289da2c740 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c37bb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2880015880 tx=0x7f2880009630 comp rx=0 tx=0).stop 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 -- 172.21.15.78:0/2916086243 shutdown_connections 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 0x5632b7c38630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 --2- 172.21.15.78:0/2916086243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c37bb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 -- 172.21.15.78:0/2916086243 >> 172.21.15.78:0/2916086243 conn(0x5632b788d4e0 msgr2=0x5632b7c344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 -- 172.21.15.78:0/2916086243 shutdown_connections 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 -- 172.21.15.78:0/2916086243 wait complete. 2024-05-21T15:23:21.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 -- start start 2024-05-21T15:23:21.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 Processor -- start 2024-05-21T15:23:21.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78704d0 0x5632b7c467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.957+0000 7f289da2c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78786b0 0x5632b7c46d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f289da2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c4c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f289da2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5632b7c49480 con 0x5632b7899390 2024-05-21T15:23:21.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f289da2c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5632b7b58c40 con 0x5632b78786b0 2024-05-21T15:23:21.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f289da2c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5632b7c49600 con 0x5632b78704d0 2024-05-21T15:23:21.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f289da2c740 10 client.0 _ll_register_callbacks cb 0x5632b78b0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:21.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c4c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c4c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35568/0 (socket says 172.21.15.78:35568) 2024-05-21T15:23:21.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 -- 172.21.15.78:0/3986672671 learned_addr learned my addr 172.21.15.78:0/3986672671 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78704d0 0x5632b7c467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f2894844640 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78786b0 0x5632b7c46d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 -- 172.21.15.78:0/3986672671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78704d0 msgr2=0x5632b7c467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78704d0 0x5632b7c467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 -- 172.21.15.78:0/3986672671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78786b0 msgr2=0x5632b7c46d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78786b0 0x5632b7c46d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 -- 172.21.15.78:0/3986672671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b797b7d0 con 0x5632b7899390 2024-05-21T15:23:21.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f288ffff640 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c4c970 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2880009e10 tx=0x7f2880004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b8a4a3cda24c06 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f286ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:21.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f286ffff640 1 -- 172.21.15.78:0/3986672671 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28800106e0 con 0x5632b7899390 2024-05-21T15:23:21.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f286ffff640 1 -- 172.21.15.78:0/3986672671 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f2880010880 con 0x5632b7899390 2024-05-21T15:23:21.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.958+0000 7f289da2c740 1 -- 172.21.15.78:0/3986672671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2880010bf0 con 0x5632b7899390 2024-05-21T15:23:21.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f286ffff640 1 -- 172.21.15.78:0/3986672671 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f28800037d0 con 0x5632b7899390 2024-05-21T15:23:21.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f289da2c740 10 client.17667 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:21.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f289da2c740 1 -- 172.21.15.78:0/3986672671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5632b7c55c20 con 0x5632b7899390 2024-05-21T15:23:21.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f286ffff640 1 -- 172.21.15.78:0/3986672671 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2880010bf0 con 0x5632b7899390 2024-05-21T15:23:21.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f286ffff640 1 -- 172.21.15.78:0/3986672671 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2880020420 con 0x5632b7899390 2024-05-21T15:23:21.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f289da2c740 10 client.17667 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:21.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f289da2c740 10 client.17667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:23:21.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f289da2c740 20 client.17667 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:21.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f289da2c740 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5632b7857f60 0x5632b7c59030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f289da2c740 4 client.17667 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:21.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f289da2c740 1 -- 172.21.15.78:0/3986672671 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632b7bab370 con 0x5632b7857f60 2024-05-21T15:23:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.959+0000 7f2894844640 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5632b7857f60 0x5632b7c59030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.960+0000 7f286ffff640 10 client.17667 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:23:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.960+0000 7f2894844640 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5632b7857f60 0x5632b7c59030 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbea8a5e969e0e88 server_cookie=a657d85e091f5cfd in_seq=0 out_seq=0 2024-05-21T15:23:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f286ffff640 1 -- 172.21.15.78:0/3986672671 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5632b7bab370 con 0x5632b7857f60 2024-05-21T15:23:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f286ffff640 10 client.17667 handle_command_reply: tid=0 2024-05-21T15:23:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 client.17667 shutdown 2024-05-21T15:23:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 -- 172.21.15.78:0/3986672671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 msgr2=0x5632b7c4c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c4c970 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2880009e10 tx=0x7f2880004270 comp rx=0 tx=0).stop 2024-05-21T15:23:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 -- 172.21.15.78:0/3986672671 shutdown_connections 2024-05-21T15:23:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5632b7857f60 0x5632b7c59030 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78786b0 0x5632b7c46d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b7899390 0x5632b7c4c970 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 --2- 172.21.15.78:0/3986672671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78704d0 0x5632b7c467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f286ffff640 0 client.17667 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:23:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 -- 172.21.15.78:0/3986672671 >> 172.21.15.78:0/3986672671 conn(0x5632b788d4e0 msgr2=0x5632b7c399d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 -- 172.21.15.78:0/3986672671 shutdown_connections 2024-05-21T15:23:21.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 1 -- 172.21.15.78:0/3986672671 wait complete. 2024-05-21T15:23:21.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.961+0000 7f289da2c740 20 client.17667 trim_cache size 0 max 16384 2024-05-21T15:23:21.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.974+0000 7f289da2c740 1 Processor -- start 2024-05-21T15:23:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.974+0000 7f289da2c740 1 -- start start 2024-05-21T15:23:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f289da2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78704d0 0x5632b7c37ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f289da2c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 0x5632b7c3ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f289da2c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b7899390 0x5632b7c404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f289da2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28901acb60 con 0x5632b78704d0 2024-05-21T15:23:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f289da2c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f28901ae6a0 con 0x5632b7899390 2024-05-21T15:23:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f289da2c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f28901ad230 con 0x5632b78786b0 2024-05-21T15:23:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78704d0 0x5632b7c37ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78704d0 0x5632b7c37ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35584/0 (socket says 172.21.15.78:35584) 2024-05-21T15:23:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 -- 172.21.15.78:0/1600799070 learned_addr learned my addr 172.21.15.78:0/1600799070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 -- 172.21.15.78:0/1600799070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 msgr2=0x5632b7c3ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 --2- 172.21.15.78:0/1600799070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 0x5632b7c3ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 -- 172.21.15.78:0/1600799070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b7899390 msgr2=0x5632b7c404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 --2- 172.21.15.78:0/1600799070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b7899390 0x5632b7c404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 -- 172.21.15.78:0/1600799070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b7c392c0 con 0x5632b78704d0 2024-05-21T15:23:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f288ffff640 1 --2- 172.21.15.78:0/1600799070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78704d0 0x5632b7c37ce0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2880009550 tx=0x7f2880009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6069e0398c59e843 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f28867fc640 1 -- 172.21.15.78:0/1600799070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880009b20 con 0x5632b78704d0 2024-05-21T15:23:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f28867fc640 1 -- 172.21.15.78:0/1600799070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2880010bc0 con 0x5632b78704d0 2024-05-21T15:23:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.975+0000 7f28867fc640 1 -- 172.21.15.78:0/1600799070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880020420 con 0x5632b78704d0 2024-05-21T15:23:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 -- 172.21.15.78:0/1600799070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78704d0 msgr2=0x5632b7c37ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 --2- 172.21.15.78:0/1600799070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78704d0 0x5632b7c37ce0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2880009550 tx=0x7f2880009580 comp rx=0 tx=0).stop 2024-05-21T15:23:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 -- 172.21.15.78:0/1600799070 shutdown_connections 2024-05-21T15:23:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 --2- 172.21.15.78:0/1600799070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b7899390 0x5632b7c404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 --2- 172.21.15.78:0/1600799070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b78786b0 0x5632b7c3ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 --2- 172.21.15.78:0/1600799070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78704d0 0x5632b7c37ce0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 -- 172.21.15.78:0/1600799070 >> 172.21.15.78:0/1600799070 conn(0x5632b788d4e0 msgr2=0x5632b7c3a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 -- 172.21.15.78:0/1600799070 shutdown_connections 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 -- 172.21.15.78:0/1600799070 wait complete. 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 -- start start 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.976+0000 7f289da2c740 1 Processor -- start 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f289da2c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c4dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f289da2c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78786b0 0x5632b7c4e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f289da2c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b7899390 0x5632b7c4e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f289da2c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28901adfd0 con 0x5632b78786b0 2024-05-21T15:23:21.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f289da2c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f28901ad900 con 0x5632b78704d0 2024-05-21T15:23:21.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f289da2c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f28901aab60 con 0x5632b7899390 2024-05-21T15:23:21.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f289da2c740 10 client.0 _ll_register_callbacks cb 0x7f2880005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:21.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78786b0 0x5632b7c4e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78786b0 0x5632b7c4e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35600/0 (socket says 172.21.15.78:35600) 2024-05-21T15:23:21.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 -- 172.21.15.78:0/3844089332 learned_addr learned my addr 172.21.15.78:0/3844089332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:21.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f288ffff640 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b7899390 0x5632b7c4e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f288ffff640 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c4dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 -- 172.21.15.78:0/3844089332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b7899390 msgr2=0x5632b7c4e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b7899390 0x5632b7c4e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 -- 172.21.15.78:0/3844089332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 msgr2=0x5632b7c4dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c4dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 -- 172.21.15.78:0/3844089332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b7c470e0 con 0x5632b78786b0 2024-05-21T15:23:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f2894844640 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78786b0 0x5632b7c4e400 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2878009ca0 tx=0x7f2878003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c153089d5f6c2b2c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.977+0000 7f286ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f286ffff640 1 -- 172.21.15.78:0/3844089332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878031650 con 0x5632b78786b0 2024-05-21T15:23:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f286ffff640 1 -- 172.21.15.78:0/3844089332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f28780317f0 con 0x5632b78786b0 2024-05-21T15:23:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 1 -- 172.21.15.78:0/3844089332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2880010bc0 con 0x5632b78786b0 2024-05-21T15:23:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f286ffff640 1 -- 172.21.15.78:0/3844089332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2878058050 con 0x5632b78786b0 2024-05-21T15:23:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 10 client.17673 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 1 -- 172.21.15.78:0/3844089332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5632b7cc0220 con 0x5632b78786b0 2024-05-21T15:23:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f286ffff640 1 -- 172.21.15.78:0/3844089332 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2880010bc0 con 0x5632b78786b0 2024-05-21T15:23:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f286ffff640 1 -- 172.21.15.78:0/3844089332 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2878045070 con 0x5632b78786b0 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 10 client.17673 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 10 client.17673 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 20 client.17673 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5632b782d3a0 0x5632b7cc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 4 client.17673 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f289da2c740 1 -- 172.21.15.78:0/3844089332 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632b7ba56c0 con 0x5632b782d3a0 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.978+0000 7f288ffff640 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5632b782d3a0 0x5632b7cc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f286ffff640 10 client.17673 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:23:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f288ffff640 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5632b782d3a0 0x5632b7cc2a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73e87455f4bb61ff server_cookie=7196d79085eafe07 in_seq=0 out_seq=0 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f286ffff640 1 -- 172.21.15.78:0/3844089332 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632b7ba56c0 con 0x5632b782d3a0 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f286ffff640 10 client.17673 handle_command_reply: tid=0 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 client.17673 shutdown 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 -- 172.21.15.78:0/3844089332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78786b0 msgr2=0x5632b7c4e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78786b0 0x5632b7c4e400 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2878009ca0 tx=0x7f2878003a60 comp rx=0 tx=0).stop 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 -- 172.21.15.78:0/3844089332 shutdown_connections 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x5632b782d3a0 0x5632b7cc2a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5632b7899390 0x5632b7c4e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f286ffff640 0 client.17673 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:23:21.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5632b78786b0 0x5632b7c4e400 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 --2- 172.21.15.78:0/3844089332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5632b78704d0 0x5632b7c4dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:21.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 -- 172.21.15.78:0/3844089332 >> 172.21.15.78:0/3844089332 conn(0x5632b788d4e0 msgr2=0x5632b7c4b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:21.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 -- 172.21.15.78:0/3844089332 shutdown_connections 2024-05-21T15:23:21.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.979+0000 7f289da2c740 1 -- 172.21.15.78:0/3844089332 wait complete. 2024-05-21T15:23:21.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f289da2c740 20 client.17673 trim_cache size 0 max 16384 2024-05-21T15:23:21.998 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 -- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f287c0a2d00 msgr2=0x7f287c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f287c0a2d00 0x7f287c0a5210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f28880cc9b0 tx=0x7f2888086040 comp rx=0 tx=0).stop 2024-05-21T15:23:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 -- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f289010d070 msgr2=0x7f289019e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f289010d070 0x7f289019e010 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f288800c970 tx=0x7f288800ce30 comp rx=0 tx=0).stop 2024-05-21T15:23:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 -- 172.21.15.78:0/2288521741 shutdown_connections 2024-05-21T15:23:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f287c0a2d00 0x7f287c0a5210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f289010d070 0x7f289019e010 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289010a680 0x7f2890199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 --2- 172.21.15.78:0/2288521741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2890103c20 0x7f28901997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 -- 172.21.15.78:0/2288521741 >> 172.21.15.78:0/2288521741 conn(0x7f28900770c0 msgr2=0x7f2890100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 -- 172.21.15.78:0/2288521741 shutdown_connections 2024-05-21T15:23:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:21.980+0000 7f2896aa8640 1 -- 172.21.15.78:0/2288521741 wait complete. 2024-05-21T15:23:22.001 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:23:22.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 -- 172.21.15.78:0/4176794760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd9010d000 msgr2=0x7fdd9010f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 --2- 172.21.15.78:0/4176794760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd9010d000 0x7fdd9010f510 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009ca0 tx=0x7fdd8802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 -- 172.21.15.78:0/4176794760 shutdown_connections 2024-05-21T15:23:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 --2- 172.21.15.78:0/4176794760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9010a610 0x7fdd9010ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 --2- 172.21.15.78:0/4176794760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd9010d000 0x7fdd9010f510 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 --2- 172.21.15.78:0/4176794760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd90103bb0 0x7fdd90103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 -- 172.21.15.78:0/4176794760 >> 172.21.15.78:0/4176794760 conn(0x7fdd900fd210 msgr2=0x7fdd900ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 -- 172.21.15.78:0/4176794760 shutdown_connections 2024-05-21T15:23:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.110+0000 7fdd94f38640 1 -- 172.21.15.78:0/4176794760 wait complete. 2024-05-21T15:23:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd94f38640 1 Processor -- start 2024-05-21T15:23:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd94f38640 1 -- start start 2024-05-21T15:23:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd94f38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90103bb0 0x7fdd90101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd94f38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9010a610 0x7fdd901019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd94f38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9010d000 0x7fdd90101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd94f38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90112df0 con 0x7fdd90103bb0 2024-05-21T15:23:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd94f38640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90112c70 con 0x7fdd9010a610 2024-05-21T15:23:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd94f38640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90112f70 con 0x7fdd9010d000 2024-05-21T15:23:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90103bb0 0x7fdd90101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90103bb0 0x7fdd90101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35610/0 (socket says 172.21.15.78:35610) 2024-05-21T15:23:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.111+0000 7fdd8e59c640 1 -- 172.21.15.78:0/2898899328 learned_addr learned my addr 172.21.15.78:0/2898899328 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9010d000 0x7fdd90101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd87fff640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9010a610 0x7fdd901019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd8e59c640 1 -- 172.21.15.78:0/2898899328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9010d000 msgr2=0x7fdd90101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9010d000 0x7fdd90101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd8e59c640 1 -- 172.21.15.78:0/2898899328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9010a610 msgr2=0x7fdd901019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9010a610 0x7fdd901019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd8e59c640 1 -- 172.21.15.78:0/2898899328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd901a7af0 con 0x7fdd90103bb0 2024-05-21T15:23:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90103bb0 0x7fdd90101470 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000d950 tx=0x7fdd8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd877fe640 1 -- 172.21.15.78:0/2898899328 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80017070 con 0x7fdd90103bb0 2024-05-21T15:23:22.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd877fe640 1 -- 172.21.15.78:0/2898899328 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd80009c30 con 0x7fdd90103bb0 2024-05-21T15:23:22.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd94f38640 1 -- 172.21.15.78:0/2898899328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd901a7de0 con 0x7fdd90103bb0 2024-05-21T15:23:22.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd877fe640 1 -- 172.21.15.78:0/2898899328 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8000f600 con 0x7fdd90103bb0 2024-05-21T15:23:22.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.112+0000 7fdd94f38640 1 -- 172.21.15.78:0/2898899328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd901a8250 con 0x7fdd90103bb0 2024-05-21T15:23:22.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd877fe640 1 -- 172.21.15.78:0/2898899328 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdd8001a070 con 0x7fdd90103bb0 2024-05-21T15:23:22.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd9bde8740 1 Processor -- start 2024-05-21T15:23:22.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd9bde8740 1 -- start start 2024-05-21T15:23:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0f9390 0x56454d497d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d498260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56454d3fdc40 con 0x56454d0f9390 2024-05-21T15:23:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56454d3a5580 con 0x56454d0d86b0 2024-05-21T15:23:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56454d3f9c00 con 0x56454d0d04d0 2024-05-21T15:23:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd8ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0f9390 0x56454d497d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd8ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0f9390 0x56454d497d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35612/0 (socket says 172.21.15.78:35612) 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd8ed9d640 1 -- 172.21.15.78:0/2910882319 learned_addr learned my addr 172.21.15.78:0/2910882319 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.115+0000 7fdd87fff640 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d498260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd877fe640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdd7c0a2d00 0x7fdd7c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8ed9d640 1 -- 172.21.15.78:0/2910882319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 msgr2=0x56454d498260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d498260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8ed9d640 1 -- 172.21.15.78:0/2910882319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 msgr2=0x56454d4987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8ed9d640 1 -- 172.21.15.78:0/2910882319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454d1113f0 con 0x56454d0f9390 2024-05-21T15:23:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d498260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd877fe640 1 -- 172.21.15.78:0/2898899328 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fdd800cf020 con 0x7fdd90103bb0 2024-05-21T15:23:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdd7c0a2d00 0x7fdd7c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0f9390 0x56454d497d20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005cc0 tx=0x7fdd88009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f4bf044480f572 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd84ff9640 1 -- 172.21.15.78:0/2910882319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88041040 con 0x56454d0f9390 2024-05-21T15:23:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd84ff9640 1 -- 172.21.15.78:0/2910882319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd88036070 con 0x56454d0f9390 2024-05-21T15:23:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.116+0000 7fdd84ff9640 1 -- 172.21.15.78:0/2910882319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8802f930 con 0x56454d0f9390 2024-05-21T15:23:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2910882319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0f9390 msgr2=0x56454d497d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdd7c0a2d00 0x7fdd7c0a5210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd8804c000 tx=0x7fdd8804d000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0f9390 0x56454d497d20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005cc0 tx=0x7fdd88009880 comp rx=0 tx=0).stop 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2910882319 shutdown_connections 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d498260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2910882319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0f9390 0x56454d497d20 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2910882319 >> 172.21.15.78:0/2910882319 conn(0x56454d0ed4e0 msgr2=0x56454d4944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2910882319 shutdown_connections 2024-05-21T15:23:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.117+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2910882319 wait complete. 2024-05-21T15:23:22.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 1 -- start start 2024-05-21T15:23:22.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 1 Processor -- start 2024-05-21T15:23:22.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0d04d0 0x56454d4a2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0f9390 0x56454d4a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56454d4a4fb0 con 0x56454d0d04d0 2024-05-21T15:23:22.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56454d3b8c40 con 0x56454d0d86b0 2024-05-21T15:23:22.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56454d4a5130 con 0x56454d0f9390 2024-05-21T15:23:22.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd9bde8740 10 client.0 _ll_register_callbacks cb 0x56454d110be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:22.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0d04d0 0x56454d4a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0d04d0 0x56454d4a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35622/0 (socket says 172.21.15.78:35622) 2024-05-21T15:23:22.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8e59c640 1 -- 172.21.15.78:0/2594508034 learned_addr learned my addr 172.21.15.78:0/2594508034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd87fff640 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0f9390 0x56454d4a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8e59c640 1 -- 172.21.15.78:0/2594508034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0f9390 msgr2=0x56454d4a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0f9390 0x56454d4a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8e59c640 1 -- 172.21.15.78:0/2594508034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 msgr2=0x56454d4a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd8e59c640 1 -- 172.21.15.78:0/2594508034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454d1db7d0 con 0x56454d0d04d0 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.118+0000 7fdd87fff640 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0d04d0 0x56454d4a2420 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdd8009b440 tx=0x7fdd8009b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d0f6b7eaa9e9b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd75ffb640 1 -- 172.21.15.78:0/2594508034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80021050 con 0x56454d0d04d0 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd75ffb640 1 -- 172.21.15.78:0/2594508034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fdd800d1110 con 0x56454d0d04d0 2024-05-21T15:23:22.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2594508034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd88036070 con 0x56454d0d04d0 2024-05-21T15:23:22.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd75ffb640 1 -- 172.21.15.78:0/2594508034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdd800b4050 con 0x56454d0d04d0 2024-05-21T15:23:22.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 10 client.17685 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:22.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2594508034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56454d4b1820 con 0x56454d0d04d0 2024-05-21T15:23:22.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd75ffb640 1 -- 172.21.15.78:0/2594508034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd88036070 con 0x56454d0d04d0 2024-05-21T15:23:22.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd75ffb640 1 -- 172.21.15.78:0/2594508034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fdd800b41f0 con 0x56454d0d04d0 2024-05-21T15:23:22.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 10 client.17685 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:22.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 10 client.17685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:23:22.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 20 client.17685 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:22.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x56454d0b7f60 0x56454d4b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 4 client.17685 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:22.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2594508034 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56454d40b370 con 0x56454d0b7f60 2024-05-21T15:23:22.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.119+0000 7fdd87fff640 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x56454d0b7f60 0x56454d4b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.120+0000 7fdd75ffb640 10 client.17685 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:23:22.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.120+0000 7fdd87fff640 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x56454d0b7f60 0x56454d4b4cf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7606ee0d29d379b7 server_cookie=d9c15760619d133f in_seq=0 out_seq=0 2024-05-21T15:23:22.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd75ffb640 1 -- 172.21.15.78:0/2594508034 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56454d40b370 con 0x56454d0b7f60 2024-05-21T15:23:22.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd75ffb640 10 client.17685 handle_command_reply: tid=0 2024-05-21T15:23:22.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 client.17685 shutdown 2024-05-21T15:23:22.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2594508034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0d04d0 msgr2=0x56454d4a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0d04d0 0x56454d4a2420 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdd8009b440 tx=0x7fdd8009b0b0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2594508034 shutdown_connections 2024-05-21T15:23:22.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x56454d0b7f60 0x56454d4b4cf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0f9390 0x56454d4a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd75ffb640 0 client.17685 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:23:22.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/2594508034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0d04d0 0x56454d4a2420 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2594508034 >> 172.21.15.78:0/2594508034 conn(0x56454d0ed4e0 msgr2=0x56454d499a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2594508034 shutdown_connections 2024-05-21T15:23:22.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 1 -- 172.21.15.78:0/2594508034 wait complete. 2024-05-21T15:23:22.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.121+0000 7fdd9bde8740 20 client.17685 trim_cache size 0 max 16384 2024-05-21T15:23:22.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.134+0000 7fdd9bde8740 1 Processor -- start 2024-05-21T15:23:22.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.134+0000 7fdd9bde8740 1 -- start start 2024-05-21T15:23:22.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.134+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d4a8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.134+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d04d0 0x56454d49a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.134+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d86b0 0x56454d49b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.134+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b0ea0 con 0x56454d0b7f60 2024-05-21T15:23:22.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.134+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b29e0 con 0x56454d0d04d0 2024-05-21T15:23:22.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.134+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b1570 con 0x56454d0d86b0 2024-05-21T15:23:22.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d4a8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d4a8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35636/0 (socket says 172.21.15.78:35636) 2024-05-21T15:23:22.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 -- 172.21.15.78:0/1409852392 learned_addr learned my addr 172.21.15.78:0/1409852392 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd87fff640 1 --2- 172.21.15.78:0/1409852392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d04d0 0x56454d49a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 -- 172.21.15.78:0/1409852392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d86b0 msgr2=0x56454d49b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/1409852392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d86b0 0x56454d49b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 -- 172.21.15.78:0/1409852392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d04d0 msgr2=0x56454d49a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/1409852392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d04d0 0x56454d49a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 -- 172.21.15.78:0/1409852392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454d4992c0 con 0x56454d0b7f60 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/1409852392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d4a8360 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fdd8009b1b0 tx=0x7fdd8009b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f7c019a2edec59 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd77fff640 1 -- 172.21.15.78:0/1409852392 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800b8030 con 0x56454d0b7f60 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd77fff640 1 -- 172.21.15.78:0/1409852392 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd8009f860 con 0x56454d0b7f60 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.135+0000 7fdd77fff640 1 -- 172.21.15.78:0/1409852392 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80021070 con 0x56454d0b7f60 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 -- 172.21.15.78:0/1409852392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 msgr2=0x56454d4a8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/1409852392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d4a8360 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fdd8009b1b0 tx=0x7fdd8009b120 comp rx=0 tx=0).stop 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 -- 172.21.15.78:0/1409852392 shutdown_connections 2024-05-21T15:23:22.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/1409852392 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d04d0 0x56454d49a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/1409852392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d86b0 0x56454d49b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/1409852392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d4a8360 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 -- 172.21.15.78:0/1409852392 >> 172.21.15.78:0/1409852392 conn(0x56454d0ed4e0 msgr2=0x56454d4999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 -- 172.21.15.78:0/1409852392 shutdown_connections 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 -- 172.21.15.78:0/1409852392 wait complete. 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 -- start start 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.136+0000 7fdd9bde8740 1 Processor -- start 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d515440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d515980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b2310 con 0x56454d0b7f60 2024-05-21T15:23:22.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b1c40 con 0x56454d0d86b0 2024-05-21T15:23:22.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901aeea0 con 0x56454d0d04d0 2024-05-21T15:23:22.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 10 client.0 _ll_register_callbacks cb 0x7fdd80097820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:22.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d515440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d515440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35648/0 (socket says 172.21.15.78:35648) 2024-05-21T15:23:22.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 -- 172.21.15.78:0/3517110763 learned_addr learned my addr 172.21.15.78:0/3517110763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd87fff640 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d515980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 -- 172.21.15.78:0/3517110763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 msgr2=0x56454d515980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d515980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 -- 172.21.15.78:0/3517110763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 msgr2=0x56454d4a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 -- 172.21.15.78:0/3517110763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454d4aa210 con 0x56454d0b7f60 2024-05-21T15:23:22.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8ed9d640 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd8e59c640 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d515440 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009610 tx=0x7fdd800b4500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8fb3b22a80632d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd75ffb640 1 -- 172.21.15.78:0/3517110763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800b8070 con 0x56454d0b7f60 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd75ffb640 1 -- 172.21.15.78:0/3517110763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fdd8009ab10 con 0x56454d0b7f60 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 1 -- 172.21.15.78:0/3517110763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd8009f860 con 0x56454d0b7f60 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd75ffb640 1 -- 172.21.15.78:0/3517110763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdd80021050 con 0x56454d0b7f60 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 10 client.17691 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.137+0000 7fdd9bde8740 1 -- 172.21.15.78:0/3517110763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56454d51ca70 con 0x56454d0b7f60 2024-05-21T15:23:22.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd75ffb640 1 -- 172.21.15.78:0/3517110763 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd8009f860 con 0x56454d0b7f60 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd75ffb640 1 -- 172.21.15.78:0/3517110763 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fdd80066030 con 0x56454d0b7f60 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd9bde8740 10 client.17691 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd9bde8740 10 client.17691 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd9bde8740 20 client.17691 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x56454d0f9390 0x56454d51ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd9bde8740 4 client.17691 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd9bde8740 1 -- 172.21.15.78:0/3517110763 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56454d4056c0 con 0x56454d0f9390 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd87fff640 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x56454d0f9390 0x56454d51ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd75ffb640 10 client.17691 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:23:22.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.138+0000 7fdd87fff640 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x56454d0f9390 0x56454d51ef50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73a3da7246277b76 server_cookie=69b3410bc81b647d in_seq=0 out_seq=0 2024-05-21T15:23:22.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd75ffb640 1 -- 172.21.15.78:0/3517110763 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56454d4056c0 con 0x56454d0f9390 2024-05-21T15:23:22.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd75ffb640 10 client.17691 handle_command_reply: tid=0 2024-05-21T15:23:22.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 client.17691 shutdown 2024-05-21T15:23:22.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 -- 172.21.15.78:0/3517110763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 msgr2=0x56454d515440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d515440 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009610 tx=0x7fdd800b4500 comp rx=0 tx=0).stop 2024-05-21T15:23:22.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 -- 172.21.15.78:0/3517110763 shutdown_connections 2024-05-21T15:23:22.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x56454d0f9390 0x56454d51ef50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56454d0d86b0 0x56454d4a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56454d0d04d0 0x56454d515980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd75ffb640 0 client.17691 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:23:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 --2- 172.21.15.78:0/3517110763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56454d0b7f60 0x56454d515440 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 -- 172.21.15.78:0/3517110763 >> 172.21.15.78:0/3517110763 conn(0x56454d0ed4e0 msgr2=0x56454d497ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 -- 172.21.15.78:0/3517110763 shutdown_connections 2024-05-21T15:23:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 1 -- 172.21.15.78:0/3517110763 wait complete. 2024-05-21T15:23:22.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd9bde8740 20 client.17691 trim_cache size 0 max 16384 2024-05-21T15:23:22.156 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:22.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 -- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdd7c0a2d00 msgr2=0x7fdd7c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdd7c0a2d00 0x7fdd7c0a5210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd8804c000 tx=0x7fdd8804d000 comp rx=0 tx=0).stop 2024-05-21T15:23:22.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 -- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90103bb0 msgr2=0x7fdd90101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90103bb0 0x7fdd90101470 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000d950 tx=0x7fdd8000de10 comp rx=0 tx=0).stop 2024-05-21T15:23:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 -- 172.21.15.78:0/2898899328 shutdown_connections 2024-05-21T15:23:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd9010a610 0x7fdd901019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdd7c0a2d00 0x7fdd7c0a5210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9010d000 0x7fdd90101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 --2- 172.21.15.78:0/2898899328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd90103bb0 0x7fdd90101470 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.139+0000 7fdd94f38640 1 -- 172.21.15.78:0/2898899328 >> 172.21.15.78:0/2898899328 conn(0x7fdd900fd210 msgr2=0x7fdd9010d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.140+0000 7fdd94f38640 1 -- 172.21.15.78:0/2898899328 shutdown_connections 2024-05-21T15:23:22.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.140+0000 7fdd94f38640 1 -- 172.21.15.78:0/2898899328 wait complete. 2024-05-21T15:23:22.159 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:23:22.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 -- 172.21.15.78:0/1759614117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c10c330 msgr2=0x7f2d9c10e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1759614117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c10c330 0x7f2d9c10e750 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009ca0 tx=0x7f2d9402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 -- 172.21.15.78:0/1759614117 shutdown_connections 2024-05-21T15:23:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1759614117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c101aa0 0x7f2d9c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1759614117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c10c330 0x7f2d9c10e750 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1759614117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d9c1079d0 0x7f2d9c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 -- 172.21.15.78:0/1759614117 >> 172.21.15.78:0/1759614117 conn(0x7f2d9c0770b0 msgr2=0x7f2d9c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 -- 172.21.15.78:0/1759614117 shutdown_connections 2024-05-21T15:23:22.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.268+0000 7f2da33ce640 1 -- 172.21.15.78:0/1759614117 wait complete. 2024-05-21T15:23:22.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da33ce640 1 Processor -- start 2024-05-21T15:23:22.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da33ce640 1 -- start start 2024-05-21T15:23:22.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da33ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c101aa0 0x7f2d9c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da33ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c1079d0 0x7f2d9c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da33ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d9c10c330 0x7f2d9c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da33ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c111ce0 con 0x7f2d9c101aa0 2024-05-21T15:23:22.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da33ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c111b60 con 0x7f2d9c1079d0 2024-05-21T15:23:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da33ce640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c111e60 con 0x7f2d9c10c330 2024-05-21T15:23:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da116a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c101aa0 0x7f2d9c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da116a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c101aa0 0x7f2d9c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35656/0 (socket says 172.21.15.78:35656) 2024-05-21T15:23:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da116a640 1 -- 172.21.15.78:0/1176601841 learned_addr learned my addr 172.21.15.78:0/1176601841 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da196b640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d9c10c330 0x7f2d9c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da0969640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c1079d0 0x7f2d9c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da116a640 1 -- 172.21.15.78:0/1176601841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d9c10c330 msgr2=0x7f2d9c19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da116a640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d9c10c330 0x7f2d9c19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da116a640 1 -- 172.21.15.78:0/1176601841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c1079d0 msgr2=0x7f2d9c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da116a640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c1079d0 0x7f2d9c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da116a640 1 -- 172.21.15.78:0/1176601841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c1a5850 con 0x7f2d9c101aa0 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da196b640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d9c10c330 0x7f2d9c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.269+0000 7f2da0969640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c1079d0 0x7f2d9c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.270+0000 7f2da116a640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c101aa0 0x7f2d9c19a540 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000d770 tx=0x7f2d9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.270+0000 7f2d8e7fc640 1 -- 172.21.15.78:0/1176601841 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90017070 con 0x7f2d9c101aa0 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.270+0000 7f2da33ce640 1 -- 172.21.15.78:0/1176601841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9c1a5b40 con 0x7f2d9c101aa0 2024-05-21T15:23:22.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.270+0000 7f2d8e7fc640 1 -- 172.21.15.78:0/1176601841 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d90009c30 con 0x7f2d9c101aa0 2024-05-21T15:23:22.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.270+0000 7f2d8e7fc640 1 -- 172.21.15.78:0/1176601841 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9000f620 con 0x7f2d9c101aa0 2024-05-21T15:23:22.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.270+0000 7f2da33ce640 1 -- 172.21.15.78:0/1176601841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9c1a6090 con 0x7f2d9c101aa0 2024-05-21T15:23:22.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.272+0000 7f2d8e7fc640 1 -- 172.21.15.78:0/1176601841 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2d9001a070 con 0x7f2d9c101aa0 2024-05-21T15:23:22.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2daa27a740 1 Processor -- start 2024-05-21T15:23:22.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2daa27a740 1 -- start start 2024-05-21T15:23:22.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253328c390 0x56253362ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x56253362b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2d8e7fc640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2d880a2d00 0x7f2d880a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253326b6b0 0x56253362b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2daa27a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562533590c40 con 0x56253328c390 2024-05-21T15:23:22.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2daa27a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562533538580 con 0x56253326b6b0 2024-05-21T15:23:22.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2daa27a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56253358cc00 con 0x5625332634d0 2024-05-21T15:23:22.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2da196b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253328c390 0x56253362ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2da116a640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2d880a2d00 0x7f2d880a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.273+0000 7f2da196b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253328c390 0x56253362ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35670/0 (socket says 172.21.15.78:35670) 2024-05-21T15:23:22.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da196b640 1 -- 172.21.15.78:0/1996172586 learned_addr learned my addr 172.21.15.78:0/1996172586 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da0969640 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x56253362b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2d8e7fc640 1 -- 172.21.15.78:0/1176601841 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f2d900cf070 con 0x7f2d9c101aa0 2024-05-21T15:23:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da196b640 1 -- 172.21.15.78:0/1996172586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 msgr2=0x56253362b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da196b640 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x56253362b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da196b640 1 -- 172.21.15.78:0/1996172586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253326b6b0 msgr2=0x56253362b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da196b640 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253326b6b0 0x56253362b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da196b640 1 -- 172.21.15.78:0/1996172586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625332a43f0 con 0x56253328c390 2024-05-21T15:23:22.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da0969640 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x56253362b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da116a640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2d880a2d00 0x7f2d880a5210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d9009a360 tx=0x7f2d90010cd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2da196b640 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253328c390 0x56253362ad20 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005cc0 tx=0x7f2d9402f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13575753492297e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2d877fe640 1 -- 172.21.15.78:0/1996172586 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94031bb0 con 0x56253328c390 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2d877fe640 1 -- 172.21.15.78:0/1996172586 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d94036050 con 0x56253328c390 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.274+0000 7f2d877fe640 1 -- 172.21.15.78:0/1996172586 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d940042a0 con 0x56253328c390 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 -- 172.21.15.78:0/1996172586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253328c390 msgr2=0x56253362ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253328c390 0x56253362ad20 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005cc0 tx=0x7f2d9402f3f0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 -- 172.21.15.78:0/1996172586 shutdown_connections 2024-05-21T15:23:22.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253326b6b0 0x56253362b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x56253362b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1996172586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253328c390 0x56253362ad20 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 -- 172.21.15.78:0/1996172586 >> 172.21.15.78:0/1996172586 conn(0x5625332804e0 msgr2=0x5625336274d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 -- 172.21.15.78:0/1996172586 shutdown_connections 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 -- 172.21.15.78:0/1996172586 wait complete. 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.275+0000 7f2daa27a740 1 -- start start 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2daa27a740 1 Processor -- start 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x5625336353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533635940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253328c390 0x56253363b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2daa27a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562533637fb0 con 0x56253326b6b0 2024-05-21T15:23:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2daa27a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56253354bc40 con 0x56253328c390 2024-05-21T15:23:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2daa27a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562533638130 con 0x5625332634d0 2024-05-21T15:23:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2daa27a740 10 client.0 _ll_register_callbacks cb 0x5625332a3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da0969640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533635940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da0969640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533635940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35676/0 (socket says 172.21.15.78:35676) 2024-05-21T15:23:22.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da0969640 1 -- 172.21.15.78:0/1250661112 learned_addr learned my addr 172.21.15.78:0/1250661112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da196b640 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x5625336353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da0969640 1 -- 172.21.15.78:0/1250661112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 msgr2=0x5625336353e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da0969640 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x5625336353e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da0969640 1 -- 172.21.15.78:0/1250661112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253328c390 msgr2=0x56253363b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da0969640 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253328c390 0x56253363b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.276+0000 7f2da0969640 1 -- 172.21.15.78:0/1250661112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253336e7d0 con 0x56253326b6b0 2024-05-21T15:23:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2da0969640 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533635940 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c074f80 tx=0x7f2d98012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e39c5e7ccf94db8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2d84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2d84ff9640 1 -- 172.21.15.78:0/1250661112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98005220 con 0x56253326b6b0 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2d84ff9640 1 -- 172.21.15.78:0/1250661112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f2d980053c0 con 0x56253326b6b0 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 1 -- 172.21.15.78:0/1250661112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d94036050 con 0x56253326b6b0 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2d84ff9640 1 -- 172.21.15.78:0/1250661112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2d980055c0 con 0x56253326b6b0 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 10 client.17703 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 1 -- 172.21.15.78:0/1250661112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5625336447e0 con 0x56253326b6b0 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2d84ff9640 1 -- 172.21.15.78:0/1250661112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d94036050 con 0x56253326b6b0 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2d84ff9640 1 -- 172.21.15.78:0/1250661112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2d9800f040 con 0x56253326b6b0 2024-05-21T15:23:22.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 10 client.17703 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 20 client.17703 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56253324af60 0x562533647d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 4 client.17703 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.277+0000 7f2daa27a740 1 -- 172.21.15.78:0/1250661112 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56253359e370 con 0x56253324af60 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.278+0000 7f2da196b640 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56253324af60 0x562533647d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.278+0000 7f2d84ff9640 10 client.17703 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.278+0000 7f2da196b640 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56253324af60 0x562533647d30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd71e3d40a856f09 server_cookie=2bbbef47cfc57153 in_seq=0 out_seq=0 2024-05-21T15:23:22.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2d84ff9640 1 -- 172.21.15.78:0/1250661112 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56253359e370 con 0x56253324af60 2024-05-21T15:23:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2d84ff9640 10 client.17703 handle_command_reply: tid=0 2024-05-21T15:23:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 client.17703 shutdown 2024-05-21T15:23:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 -- 172.21.15.78:0/1250661112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 msgr2=0x562533635940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533635940 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c074f80 tx=0x7f2d98012880 comp rx=0 tx=0).stop 2024-05-21T15:23:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 -- 172.21.15.78:0/1250661112 shutdown_connections 2024-05-21T15:23:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56253324af60 0x562533647d30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56253328c390 0x56253363b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533635940 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 --2- 172.21.15.78:0/1250661112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332634d0 0x5625336353e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2d84ff9640 0 client.17703 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 -- 172.21.15.78:0/1250661112 >> 172.21.15.78:0/1250661112 conn(0x5625332804e0 msgr2=0x56253362ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 -- 172.21.15.78:0/1250661112 shutdown_connections 2024-05-21T15:23:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 1 -- 172.21.15.78:0/1250661112 wait complete. 2024-05-21T15:23:22.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.279+0000 7f2daa27a740 20 client.17703 trim_cache size 0 max 16384 2024-05-21T15:23:22.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2daa27a740 1 Processor -- start 2024-05-21T15:23:22.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2daa27a740 1 -- start start 2024-05-21T15:23:22.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625332804e0 0x56253363b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 0x562533636e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56253326b6b0 0x5625336373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2daa27a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1aebe0 con 0x5625332804e0 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2daa27a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1b0720 con 0x5625332634d0 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2daa27a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1af2b0 con 0x56253326b6b0 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da196b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625332804e0 0x56253363b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da196b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625332804e0 0x56253363b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35692/0 (socket says 172.21.15.78:35692) 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da196b640 1 -- 172.21.15.78:0/4104021698 learned_addr learned my addr 172.21.15.78:0/4104021698 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da0969640 1 --2- 172.21.15.78:0/4104021698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 0x562533636e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da196b640 1 -- 172.21.15.78:0/4104021698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56253326b6b0 msgr2=0x5625336373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da196b640 1 --2- 172.21.15.78:0/4104021698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56253326b6b0 0x5625336373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da196b640 1 -- 172.21.15.78:0/4104021698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 msgr2=0x562533636e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da196b640 1 --2- 172.21.15.78:0/4104021698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 0x562533636e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.293+0000 7f2da196b640 1 -- 172.21.15.78:0/4104021698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625336217e0 con 0x5625332804e0 2024-05-21T15:23:22.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2da196b640 1 --2- 172.21.15.78:0/4104021698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625332804e0 0x56253363b390 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2d940317e0 tx=0x7f2d94031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39c22a9b0e27d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2d86ffd640 1 -- 172.21.15.78:0/4104021698 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94054020 con 0x5625332804e0 2024-05-21T15:23:22.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2d86ffd640 1 -- 172.21.15.78:0/4104021698 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d94036040 con 0x5625332804e0 2024-05-21T15:23:22.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2d86ffd640 1 -- 172.21.15.78:0/4104021698 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94050830 con 0x5625332804e0 2024-05-21T15:23:22.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2daa27a740 1 -- 172.21.15.78:0/4104021698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625332804e0 msgr2=0x56253363b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2daa27a740 1 --2- 172.21.15.78:0/4104021698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625332804e0 0x56253363b390 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2d940317e0 tx=0x7f2d94031810 comp rx=0 tx=0).stop 2024-05-21T15:23:22.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2daa27a740 1 -- 172.21.15.78:0/4104021698 shutdown_connections 2024-05-21T15:23:22.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2daa27a740 1 --2- 172.21.15.78:0/4104021698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 0x562533636e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2daa27a740 1 --2- 172.21.15.78:0/4104021698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56253326b6b0 0x5625336373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2daa27a740 1 --2- 172.21.15.78:0/4104021698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5625332804e0 0x56253363b390 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.294+0000 7f2daa27a740 1 -- 172.21.15.78:0/4104021698 >> 172.21.15.78:0/4104021698 conn(0x562533622740 msgr2=0x562533636750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 -- 172.21.15.78:0/4104021698 shutdown_connections 2024-05-21T15:23:22.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 -- 172.21.15.78:0/4104021698 wait complete. 2024-05-21T15:23:22.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 -- start start 2024-05-21T15:23:22.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 Processor -- start 2024-05-21T15:23:22.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 0x562533641910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533641e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332804e0 0x56253363ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1b0050 con 0x56253326b6b0 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1af980 con 0x5625332634d0 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.295+0000 7f2daa27a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1acbe0 con 0x5625332804e0 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2daa27a740 10 client.0 _ll_register_callbacks cb 0x7f2d98005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533641e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533641e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35696/0 (socket says 172.21.15.78:35696) 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 -- 172.21.15.78:0/2985595113 learned_addr learned my addr 172.21.15.78:0/2985595113 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da196b640 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 0x562533641910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da196b640 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332804e0 0x56253363ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 -- 172.21.15.78:0/2985595113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332804e0 msgr2=0x56253363ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332804e0 0x56253363ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 -- 172.21.15.78:0/2985595113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 msgr2=0x562533641910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 0x562533641910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 -- 172.21.15.78:0/2985595113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253363d460 con 0x56253326b6b0 2024-05-21T15:23:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da196b640 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332804e0 0x56253363ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2da0969640 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533641e50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2d98010d50 tx=0x7f2d98010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bbdc4a9e097faf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2d84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2d84ff9640 1 -- 172.21.15.78:0/2985595113 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98031060 con 0x56253326b6b0 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2d84ff9640 1 -- 172.21.15.78:0/2985595113 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f2d9802c680 con 0x56253326b6b0 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2daa27a740 1 -- 172.21.15.78:0/2985595113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d94036040 con 0x56253326b6b0 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2d84ff9640 1 -- 172.21.15.78:0/2985595113 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2d98038050 con 0x56253326b6b0 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2daa27a740 10 client.17709 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2daa27a740 1 -- 172.21.15.78:0/2985595113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5625336afac0 con 0x56253326b6b0 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.296+0000 7f2d84ff9640 1 -- 172.21.15.78:0/2985595113 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d94036040 con 0x56253326b6b0 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2d84ff9640 1 -- 172.21.15.78:0/2985595113 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2d98023ac0 con 0x56253326b6b0 2024-05-21T15:23:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2daa27a740 10 client.17709 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2daa27a740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2daa27a740 20 client.17709 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2daa27a740 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56253328c390 0x5625336b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2daa27a740 4 client.17709 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2daa27a740 1 -- 172.21.15.78:0/2985595113 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625335986c0 con 0x56253328c390 2024-05-21T15:23:22.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2da196b640 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56253328c390 0x5625336b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2d84ff9640 10 client.17709 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.297+0000 7f2da196b640 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56253328c390 0x5625336b22b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbbb0e9c3cc65fc server_cookie=74f1bfa682f559d4 in_seq=0 out_seq=0 2024-05-21T15:23:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2d84ff9640 1 -- 172.21.15.78:0/2985595113 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625335986c0 con 0x56253328c390 2024-05-21T15:23:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2d84ff9640 10 client.17709 handle_command_reply: tid=0 2024-05-21T15:23:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 client.17709 shutdown 2024-05-21T15:23:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 -- 172.21.15.78:0/2985595113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 msgr2=0x562533641e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533641e50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2d98010d50 tx=0x7f2d98010780 comp rx=0 tx=0).stop 2024-05-21T15:23:22.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 -- 172.21.15.78:0/2985595113 shutdown_connections 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x56253328c390 0x5625336b22b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625332804e0 0x56253363ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2d84ff9640 0 client.17709 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253326b6b0 0x562533641e50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 --2- 172.21.15.78:0/2985595113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5625332634d0 0x562533641910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 -- 172.21.15.78:0/2985595113 >> 172.21.15.78:0/2985595113 conn(0x562533622740 msgr2=0x56253362ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 -- 172.21.15.78:0/2985595113 shutdown_connections 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 1 -- 172.21.15.78:0/2985595113 wait complete. 2024-05-21T15:23:22.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2daa27a740 20 client.17709 trim_cache size 0 max 16384 2024-05-21T15:23:22.313 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:22.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2da33ce640 1 -- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2d880a2d00 msgr2=0x7f2d880a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2d880a2d00 0x7f2d880a5210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d9009a360 tx=0x7f2d90010cd0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2da33ce640 1 -- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c101aa0 msgr2=0x7f2d9c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.298+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c101aa0 0x7f2d9c19a540 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000d770 tx=0x7f2d9000dc30 comp rx=0 tx=0).stop 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.299+0000 7f2da33ce640 1 -- 172.21.15.78:0/1176601841 shutdown_connections 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.299+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d9c1079d0 0x7f2d9c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.299+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f2d880a2d00 0x7f2d880a5210 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.299+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d9c10c330 0x7f2d9c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.299+0000 7f2da33ce640 1 --2- 172.21.15.78:0/1176601841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d9c101aa0 0x7f2d9c19a540 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.299+0000 7f2da33ce640 1 -- 172.21.15.78:0/1176601841 >> 172.21.15.78:0/1176601841 conn(0x7f2d9c0770b0 msgr2=0x7f2d9c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.299+0000 7f2da33ce640 1 -- 172.21.15.78:0/1176601841 shutdown_connections 2024-05-21T15:23:22.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:22.299+0000 7f2da33ce640 1 -- 172.21.15.78:0/1176601841 wait complete. 2024-05-21T15:23:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:22 smithi078 ceph-mon[31997]: pgmap v223: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:22 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2495598011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:23:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:22 smithi084 ceph-mon[36334]: pgmap v223: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:22 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2495598011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:23:22.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:22 smithi172 ceph-mon[36402]: pgmap v223: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:23:22.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:22 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2495598011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:23:23.316 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:23:23.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:23:23.317 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:23:23.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 -- 172.21.15.78:0/3619628706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 msgr2=0x7f2138100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 --2- 172.21.15.78:0/3619628706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 0x7f2138100b20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2134009900 tx=0x7f213402a410 comp rx=0 tx=0).stop 2024-05-21T15:23:23.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 -- 172.21.15.78:0/3619628706 shutdown_connections 2024-05-21T15:23:23.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 --2- 172.21.15.78:0/3619628706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21381016d0 0x7f213810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 --2- 172.21.15.78:0/3619628706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2138108f40 0x7f2138101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 --2- 172.21.15.78:0/3619628706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 0x7f2138100b20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 -- 172.21.15.78:0/3619628706 >> 172.21.15.78:0/3619628706 conn(0x7f2138076830 msgr2=0x7f2138076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 -- 172.21.15.78:0/3619628706 shutdown_connections 2024-05-21T15:23:23.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.431+0000 7f213f937640 1 -- 172.21.15.78:0/3619628706 wait complete. 2024-05-21T15:23:23.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213f937640 1 Processor -- start 2024-05-21T15:23:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213f937640 1 -- start start 2024-05-21T15:23:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213f937640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21381016d0 0x7f21381997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213f937640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 0x7f2138199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213f937640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2138108f40 0x7f213819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213f937640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2138112e00 con 0x7f2138108aa0 2024-05-21T15:23:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213f937640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2138112c80 con 0x7f21381016d0 2024-05-21T15:23:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213f937640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2138112f80 con 0x7f2138108f40 2024-05-21T15:23:23.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 0x7f2138199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213d6d3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21381016d0 0x7f21381997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 0x7f2138199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35722/0 (socket says 172.21.15.78:35722) 2024-05-21T15:23:23.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 -- 172.21.15.78:0/1290852239 learned_addr learned my addr 172.21.15.78:0/1290852239 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ded4640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2138108f40 0x7f213819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 -- 172.21.15.78:0/1290852239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2138108f40 msgr2=0x7f213819dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2138108f40 0x7f213819dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 -- 172.21.15.78:0/1290852239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21381016d0 msgr2=0x7f21381997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21381016d0 0x7f21381997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 -- 172.21.15.78:0/1290852239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213819e580 con 0x7f2138108aa0 2024-05-21T15:23:23.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.432+0000 7f213ced2640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 0x7f2138199d00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f212800d770 tx=0x7f212800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.433+0000 7f212e7fc640 1 -- 172.21.15.78:0/1290852239 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128017070 con 0x7f2138108aa0 2024-05-21T15:23:23.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.433+0000 7f212e7fc640 1 -- 172.21.15.78:0/1290852239 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f212800de80 con 0x7f2138108aa0 2024-05-21T15:23:23.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.433+0000 7f213f937640 1 -- 172.21.15.78:0/1290852239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213819e870 con 0x7f2138108aa0 2024-05-21T15:23:23.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.433+0000 7f212e7fc640 1 -- 172.21.15.78:0/1290852239 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128004e00 con 0x7f2138108aa0 2024-05-21T15:23:23.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.433+0000 7f213f937640 1 -- 172.21.15.78:0/1290852239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213819ed70 con 0x7f2138108aa0 2024-05-21T15:23:23.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.435+0000 7f212e7fc640 1 -- 172.21.15.78:0/1290852239 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2128004fa0 con 0x7f2138108aa0 2024-05-21T15:23:23.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f21467da740 1 Processor -- start 2024-05-21T15:23:23.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f21467da740 1 -- start start 2024-05-21T15:23:23.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f21467da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5a6410 0x55cbed945620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f21467da740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed57d550 0x55cbed945b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f21467da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed585730 0x55cbed9460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f21467da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cbed8aa990 con 0x55cbed585730 2024-05-21T15:23:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f21467da740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cbed8525b0 con 0x55cbed57d550 2024-05-21T15:23:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f21467da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cbed8a6950 con 0x55cbed5a6410 2024-05-21T15:23:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ced2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed585730 0x55cbed9460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ded4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5a6410 0x55cbed945620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ced2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed585730 0x55cbed9460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35734/0 (socket says 172.21.15.78:35734) 2024-05-21T15:23:23.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ced2640 1 -- 172.21.15.78:0/567011335 learned_addr learned my addr 172.21.15.78:0/567011335 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213d6d3640 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed57d550 0x55cbed945b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f212e7fc640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f21240a2d00 0x7f21240a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ced2640 1 -- 172.21.15.78:0/567011335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5a6410 msgr2=0x55cbed945620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ced2640 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5a6410 0x55cbed945620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ced2640 1 -- 172.21.15.78:0/567011335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed57d550 msgr2=0x55cbed945b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ced2640 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed57d550 0x55cbed945b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ced2640 1 -- 172.21.15.78:0/567011335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbed6867e0 con 0x55cbed585730 2024-05-21T15:23:23.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213d6d3640 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed57d550 0x55cbed945b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ded4640 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5a6410 0x55cbed945620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:23:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.436+0000 7f213ded4640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f21240a2d00 0x7f21240a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.437+0000 7f212e7fc640 1 -- 172.21.15.78:0/1290852239 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f21280cee20 con 0x7f2138108aa0 2024-05-21T15:23:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.437+0000 7f213ced2640 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed585730 0x55cbed9460a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f21280cc450 tx=0x7f212805b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf532c447db53e74 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.437+0000 7f213ded4640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f21240a2d00 0x7f21240a5210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55cbed942790 tx=0x7f2130004ab0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.437+0000 7f21237fe640 1 -- 172.21.15.78:0/567011335 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128020030 con 0x55cbed585730 2024-05-21T15:23:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.437+0000 7f21237fe640 1 -- 172.21.15.78:0/567011335 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21280046c0 con 0x55cbed585730 2024-05-21T15:23:23.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.437+0000 7f21237fe640 1 -- 172.21.15.78:0/567011335 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212809f660 con 0x55cbed585730 2024-05-21T15:23:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 -- 172.21.15.78:0/567011335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed585730 msgr2=0x55cbed9460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed585730 0x55cbed9460a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f21280cc450 tx=0x7f212805b430 comp rx=0 tx=0).stop 2024-05-21T15:23:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 -- 172.21.15.78:0/567011335 shutdown_connections 2024-05-21T15:23:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed57d550 0x55cbed945b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed585730 0x55cbed9460a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 --2- 172.21.15.78:0/567011335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5a6410 0x55cbed945620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 -- 172.21.15.78:0/567011335 >> 172.21.15.78:0/567011335 conn(0x55cbed59a560 msgr2=0x55cbed941dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 -- 172.21.15.78:0/567011335 shutdown_connections 2024-05-21T15:23:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 -- 172.21.15.78:0/567011335 wait complete. 2024-05-21T15:23:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 -- start start 2024-05-21T15:23:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.438+0000 7f21467da740 1 Processor -- start 2024-05-21T15:23:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f21467da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed94fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f21467da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed585730 0x55cbed950240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f21467da740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed5a6410 0x55cbed955da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f21467da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cbed9528b0 con 0x55cbed57d550 2024-05-21T15:23:23.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f21467da740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cbed865b40 con 0x55cbed5a6410 2024-05-21T15:23:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f21467da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cbed952a30 con 0x55cbed585730 2024-05-21T15:23:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f21467da740 10 client.0 _ll_register_callbacks cb 0x55cbed85a0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed94fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed94fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35748/0 (socket says 172.21.15.78:35748) 2024-05-21T15:23:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 -- 172.21.15.78:0/3865396172 learned_addr learned my addr 172.21.15.78:0/3865396172 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213ced2640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed585730 0x55cbed950240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213ded4640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed5a6410 0x55cbed955da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 -- 172.21.15.78:0/3865396172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed585730 msgr2=0x55cbed950240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed585730 0x55cbed950240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 -- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed5a6410 msgr2=0x55cbed955da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed5a6410 0x55cbed955da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 -- 172.21.15.78:0/3865396172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbed6e6f70 con 0x55cbed57d550 2024-05-21T15:23:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213ded4640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed5a6410 0x55cbed955da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:23:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213ced2640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed585730 0x55cbed950240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:23:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f213d6d3640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed94fce0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f213402a930 tx=0x7f21340318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7ef7214b2aa693 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.439+0000 7f2120ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f2120ff9640 1 -- 172.21.15.78:0/3865396172 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134035040 con 0x55cbed57d550 2024-05-21T15:23:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f2120ff9640 1 -- 172.21.15.78:0/3865396172 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f2134004240 con 0x55cbed57d550 2024-05-21T15:23:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 1 -- 172.21.15.78:0/3865396172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbed8b80c0 con 0x55cbed57d550 2024-05-21T15:23:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f2120ff9640 1 -- 172.21.15.78:0/3865396172 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2134004440 con 0x55cbed57d550 2024-05-21T15:23:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 10 client.17721 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 1 -- 172.21.15.78:0/3865396172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cbed57e170 con 0x55cbed57d550 2024-05-21T15:23:23.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f2120ff9640 1 -- 172.21.15.78:0/3865396172 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cbed8b80c0 con 0x55cbed57d550 2024-05-21T15:23:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f2120ff9640 1 -- 172.21.15.78:0/3865396172 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2134004c30 con 0x55cbed57d550 2024-05-21T15:23:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 10 client.17721 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 20 client.17721 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cbed5651f0 0x55cbed9620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 4 client.17721 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.440+0000 7f21467da740 1 -- 172.21.15.78:0/3865396172 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbed8b2410 con 0x55cbed5651f0 2024-05-21T15:23:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.441+0000 7f213ced2640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cbed5651f0 0x55cbed9620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.441+0000 7f2120ff9640 10 client.17721 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.441+0000 7f213ced2640 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cbed5651f0 0x55cbed9620d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741474ce05b0ecdc server_cookie=46bfcd78999a2fb in_seq=0 out_seq=0 2024-05-21T15:23:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f2120ff9640 1 -- 172.21.15.78:0/3865396172 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f2128080c70 con 0x55cbed5651f0 2024-05-21T15:23:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f2120ff9640 10 client.17721 handle_command_reply: tid=0 2024-05-21T15:23:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 client.17721 shutdown 2024-05-21T15:23:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 -- 172.21.15.78:0/3865396172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 msgr2=0x55cbed94fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed94fce0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f213402a930 tx=0x7f21340318d0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 -- 172.21.15.78:0/3865396172 shutdown_connections 2024-05-21T15:23:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cbed5651f0 0x55cbed9620d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed5a6410 0x55cbed955da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed585730 0x55cbed950240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f2120ff9640 0 client.17721 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 --2- 172.21.15.78:0/3865396172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed94fce0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 -- 172.21.15.78:0/3865396172 >> 172.21.15.78:0/3865396172 conn(0x55cbed59a560 msgr2=0x55cbed947340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 -- 172.21.15.78:0/3865396172 shutdown_connections 2024-05-21T15:23:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 1 -- 172.21.15.78:0/3865396172 wait complete. 2024-05-21T15:23:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.442+0000 7f21467da740 20 client.17721 trim_cache size 0 max 16384 2024-05-21T15:23:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f21467da740 1 Processor -- start 2024-05-21T15:23:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f21467da740 1 -- start start 2024-05-21T15:23:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f21467da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed967890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f21467da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed967dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f21467da740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 0x55cbed955ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f21467da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21381aece0 con 0x55cbed57d550 2024-05-21T15:23:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f21467da740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f21381b0820 con 0x55cbed585730 2024-05-21T15:23:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f21467da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21381af3b0 con 0x55cbed5651f0 2024-05-21T15:23:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f213ced2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed967dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f213ced2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed967dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35752/0 (socket says 172.21.15.78:35752) 2024-05-21T15:23:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.460+0000 7f213ced2640 1 -- 172.21.15.78:0/3530139994 learned_addr learned my addr 172.21.15.78:0/3530139994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213ded4640 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 0x55cbed955ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213d6d3640 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed967890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213ced2640 1 -- 172.21.15.78:0/3530139994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 msgr2=0x55cbed967890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213ced2640 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed967890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213ced2640 1 -- 172.21.15.78:0/3530139994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 msgr2=0x55cbed955ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213ced2640 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 0x55cbed955ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213ced2640 1 -- 172.21.15.78:0/3530139994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbed950a20 con 0x55cbed57d550 2024-05-21T15:23:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213d6d3640 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed967890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f213ced2640 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed967dd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f21280cc540 tx=0x7f21280bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc280cbf6236a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f2122ffd640 1 -- 172.21.15.78:0/3530139994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21280b4070 con 0x55cbed57d550 2024-05-21T15:23:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f2122ffd640 1 -- 172.21.15.78:0/3530139994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cbed8b2410 con 0x55cbed57d550 2024-05-21T15:23:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.461+0000 7f2122ffd640 1 -- 172.21.15.78:0/3530139994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128020040 con 0x55cbed57d550 2024-05-21T15:23:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 -- 172.21.15.78:0/3530139994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 msgr2=0x55cbed967dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed967dd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f21280cc540 tx=0x7f21280bb040 comp rx=0 tx=0).stop 2024-05-21T15:23:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 -- 172.21.15.78:0/3530139994 shutdown_connections 2024-05-21T15:23:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 0x55cbed955ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed967dd0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 --2- 172.21.15.78:0/3530139994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed967890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 -- 172.21.15.78:0/3530139994 >> 172.21.15.78:0/3530139994 conn(0x55cbed59a560 msgr2=0x55cbed947940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 -- 172.21.15.78:0/3530139994 shutdown_connections 2024-05-21T15:23:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 -- 172.21.15.78:0/3530139994 wait complete. 2024-05-21T15:23:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 -- start start 2024-05-21T15:23:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.462+0000 7f21467da740 1 Processor -- start 2024-05-21T15:23:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f21467da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed957760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f21467da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed957ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f21467da740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 0x55cbed9c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f21467da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21381b0150 con 0x55cbed57d550 2024-05-21T15:23:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f21467da740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f21381afa80 con 0x55cbed585730 2024-05-21T15:23:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f21467da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21381acce0 con 0x55cbed5651f0 2024-05-21T15:23:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f21467da740 10 client.0 _ll_register_callbacks cb 0x7f2134009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213ced2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed957ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213d6d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed957760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213d6d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed957760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:35636/0 (socket says 172.21.15.78:35636) 2024-05-21T15:23:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213d6d3640 1 -- 172.21.15.78:0/2680669741 learned_addr learned my addr 172.21.15.78:0/2680669741 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213ded4640 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 0x55cbed9c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213d6d3640 1 -- 172.21.15.78:0/2680669741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 msgr2=0x55cbed9c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213d6d3640 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 0x55cbed9c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213d6d3640 1 -- 172.21.15.78:0/2680669741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 msgr2=0x55cbed957ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213d6d3640 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed957ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213d6d3640 1 -- 172.21.15.78:0/2680669741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbed9c8970 con 0x55cbed5651f0 2024-05-21T15:23:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.463+0000 7f213ced2640 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed957ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f213d6d3640 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed957760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f213402a930 tx=0x7f2134000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88feb260eee98b1b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f2120ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f2120ff9640 1 -- 172.21.15.78:0/2680669741 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134035040 con 0x55cbed5651f0 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 1 -- 172.21.15.78:0/2680669741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbed8b2410 con 0x55cbed5651f0 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f2120ff9640 1 -- 172.21.15.78:0/2680669741 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f21340051d0 con 0x55cbed5651f0 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f2120ff9640 1 -- 172.21.15.78:0/2680669741 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f21340053d0 con 0x55cbed5651f0 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 10 client.26914 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 1 -- 172.21.15.78:0/2680669741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cbed9c9c00 con 0x55cbed5651f0 2024-05-21T15:23:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f2120ff9640 1 -- 172.21.15.78:0/2680669741 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cbed8b2410 con 0x55cbed5651f0 2024-05-21T15:23:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f2120ff9640 1 -- 172.21.15.78:0/2680669741 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2134004880 con 0x55cbed5651f0 2024-05-21T15:23:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 10 client.26914 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 10 client.26914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 20 client.26914 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cbed5a6410 0x55cbed9cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 4 client.26914 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:23:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.464+0000 7f21467da740 1 -- 172.21.15.78:0/2680669741 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cbed82fda0 con 0x55cbed5a6410 2024-05-21T15:23:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.465+0000 7f213ced2640 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cbed5a6410 0x55cbed9cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.465+0000 7f2120ff9640 10 client.26914 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.465+0000 7f213ced2640 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cbed5a6410 0x55cbed9cc3c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f6d9df9ed9a985 server_cookie=3847e3af0e4b1a9d in_seq=0 out_seq=0 2024-05-21T15:23:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f2120ff9640 1 -- 172.21.15.78:0/2680669741 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cbed82fda0 con 0x55cbed5a6410 2024-05-21T15:23:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f2120ff9640 10 client.26914 handle_command_reply: tid=0 2024-05-21T15:23:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 client.26914 shutdown 2024-05-21T15:23:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 -- 172.21.15.78:0/2680669741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 msgr2=0x55cbed957760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed957760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f213402a930 tx=0x7f2134000bc0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 -- 172.21.15.78:0/2680669741 shutdown_connections 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cbed5a6410 0x55cbed9cc3c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cbed585730 0x55cbed9c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f2120ff9640 0 client.26914 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cbed57d550 0x55cbed957ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 --2- 172.21.15.78:0/2680669741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbed5651f0 0x55cbed957760 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.466+0000 7f21467da740 1 -- 172.21.15.78:0/2680669741 >> 172.21.15.78:0/2680669741 conn(0x55cbed59a560 msgr2=0x55cbed94fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f21467da740 1 -- 172.21.15.78:0/2680669741 shutdown_connections 2024-05-21T15:23:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f21467da740 1 -- 172.21.15.78:0/2680669741 wait complete. 2024-05-21T15:23:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f21467da740 20 client.26914 trim_cache size 0 max 16384 2024-05-21T15:23:23.481 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:23:23.481 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:23:23.481 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "8076c33e-155e-4db1-831f-f777b74ce51c", 2024-05-21T15:23:23.481 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:23:23.482 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:23:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 -- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f21240a2d00 msgr2=0x7f21240a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f21240a2d00 0x7f21240a5210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55cbed942790 tx=0x7f2130004ab0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 -- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 msgr2=0x7f2138199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 0x7f2138199d00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f212800d770 tx=0x7f212800dc30 comp rx=0 tx=0).stop 2024-05-21T15:23:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 -- 172.21.15.78:0/1290852239 shutdown_connections 2024-05-21T15:23:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f21240a2d00 0x7f21240a5210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2138108f40 0x7f213819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2138108aa0 0x7f2138199d00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 --2- 172.21.15.78:0/1290852239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21381016d0 0x7f21381997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 -- 172.21.15.78:0/1290852239 >> 172.21.15.78:0/1290852239 conn(0x7f2138076830 msgr2=0x7f21381041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 -- 172.21.15.78:0/1290852239 shutdown_connections 2024-05-21T15:23:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.467+0000 7f213f937640 1 -- 172.21.15.78:0/1290852239 wait complete. 2024-05-21T15:23:23.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:23:23.485 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:23:23.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:23 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:23:23.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:23 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:23:23.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:23 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:23:23.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:23 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:23:23.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 -- 172.21.15.78:0/3952734567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098101770 msgr2=0x7f309810f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 --2- 172.21.15.78:0/3952734567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098101770 0x7f309810f5f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3090009ca0 tx=0x7f309002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 -- 172.21.15.78:0/3952734567 shutdown_connections 2024-05-21T15:23:23.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 --2- 172.21.15.78:0/3952734567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098101770 0x7f309810f5f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 --2- 172.21.15.78:0/3952734567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3098108f70 0x7f3098101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 --2- 172.21.15.78:0/3952734567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3098108ad0 0x7f3098100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 -- 172.21.15.78:0/3952734567 >> 172.21.15.78:0/3952734567 conn(0x7f3098076830 msgr2=0x7f3098076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 -- 172.21.15.78:0/3952734567 shutdown_connections 2024-05-21T15:23:23.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.599+0000 7f309e3b7640 1 -- 172.21.15.78:0/3952734567 wait complete. 2024-05-21T15:23:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f309e3b7640 1 Processor -- start 2024-05-21T15:23:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f309e3b7640 1 -- start start 2024-05-21T15:23:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f309e3b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3098101770 0x7f30981a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f309e3b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098108ad0 0x7f30981a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f309e3b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3098108f70 0x7f309819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f309e3b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3098112e10 con 0x7f3098108ad0 2024-05-21T15:23:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f309e3b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3098112c90 con 0x7f3098101770 2024-05-21T15:23:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f309e3b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3098112f90 con 0x7f3098108f70 2024-05-21T15:23:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f30977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098108ad0 0x7f30981a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f30977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098108ad0 0x7f30981a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35780/0 (socket says 172.21.15.78:35780) 2024-05-21T15:23:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f30977fe640 1 -- 172.21.15.78:0/58108470 learned_addr learned my addr 172.21.15.78:0/58108470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f3097fff640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3098101770 0x7f30981a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f30977fe640 1 -- 172.21.15.78:0/58108470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3098108f70 msgr2=0x7f309819b7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T15:23:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f30977fe640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3098108f70 0x7f309819b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f30977fe640 1 -- 172.21.15.78:0/58108470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3098101770 msgr2=0x7f30981a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f30977fe640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3098101770 0x7f30981a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.600+0000 7f30977fe640 1 -- 172.21.15.78:0/58108470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309819bed0 con 0x7f3098108ad0 2024-05-21T15:23:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.601+0000 7f30977fe640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098108ad0 0x7f30981a1b00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f308c00e9f0 tx=0x7f308c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.601+0000 7f30957fa640 1 -- 172.21.15.78:0/58108470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00ccc0 con 0x7f3098108ad0 2024-05-21T15:23:23.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.601+0000 7f309e3b7640 1 -- 172.21.15.78:0/58108470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309819c1c0 con 0x7f3098108ad0 2024-05-21T15:23:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.601+0000 7f30957fa640 1 -- 172.21.15.78:0/58108470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308c0035f0 con 0x7f3098108ad0 2024-05-21T15:23:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.601+0000 7f30957fa640 1 -- 172.21.15.78:0/58108470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c00ce60 con 0x7f3098108ad0 2024-05-21T15:23:23.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.601+0000 7f309e3b7640 1 -- 172.21.15.78:0/58108470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30981a8410 con 0x7f3098108ad0 2024-05-21T15:23:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.603+0000 7f30a5256740 1 Processor -- start 2024-05-21T15:23:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.603+0000 7f30a5256740 1 -- start start 2024-05-21T15:23:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.603+0000 7f30957fa640 1 -- 172.21.15.78:0/58108470 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f308c010750 con 0x7f3098108ad0 2024-05-21T15:23:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30a5256740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aadbf390 0x5607ab15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30a5256740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab15e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30a5256740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab15e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30a5256740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607ab0c3d00 con 0x5607aad964d0 2024-05-21T15:23:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30a5256740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5607ab06b4c0 con 0x5607aad9e6b0 2024-05-21T15:23:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30a5256740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5607ab0bfcc0 con 0x5607aadbf390 2024-05-21T15:23:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35794/0 (socket says 172.21.15.78:35794) 2024-05-21T15:23:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 -- 172.21.15.78:0/1230567957 learned_addr learned my addr 172.21.15.78:0/1230567957 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f309c954640 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aadbf390 0x5607ab15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30977fe640 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab15e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 -- 172.21.15.78:0/1230567957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aadbf390 msgr2=0x5607ab15dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aadbf390 0x5607ab15dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 -- 172.21.15.78:0/1230567957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 msgr2=0x5607ab15e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab15e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 -- 172.21.15.78:0/1230567957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607aaea17d0 con 0x5607aad964d0 2024-05-21T15:23:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30977fe640 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab15e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f3097fff640 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab15e270 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3088009c90 tx=0x7f3088011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decbbdbda90087f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30957fa640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30800a2d00 0x7f30800a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f309c954640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30800a2d00 0x7f30800a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f307e7fc640 1 -- 172.21.15.78:0/1230567957 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088005220 con 0x5607aad964d0 2024-05-21T15:23:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f30957fa640 1 -- 172.21.15.78:0/58108470 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f308c0cfa50 con 0x7f3098108ad0 2024-05-21T15:23:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f307e7fc640 1 -- 172.21.15.78:0/1230567957 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3088004c80 con 0x5607aad964d0 2024-05-21T15:23:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.604+0000 7f307e7fc640 1 -- 172.21.15.78:0/1230567957 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088019710 con 0x5607aad964d0 2024-05-21T15:23:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f309c954640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30800a2d00 0x7f30800a5210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3090006010 tx=0x7f30900093f0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 -- 172.21.15.78:0/1230567957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 msgr2=0x5607ab15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab15e270 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3088009c90 tx=0x7f3088011810 comp rx=0 tx=0).stop 2024-05-21T15:23:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 -- 172.21.15.78:0/1230567957 shutdown_connections 2024-05-21T15:23:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab15e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab15e270 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 --2- 172.21.15.78:0/1230567957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aadbf390 0x5607ab15dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 -- 172.21.15.78:0/1230567957 >> 172.21.15.78:0/1230567957 conn(0x5607aadb34e0 msgr2=0x5607ab15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 -- 172.21.15.78:0/1230567957 shutdown_connections 2024-05-21T15:23:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.605+0000 7f30a5256740 1 -- 172.21.15.78:0/1230567957 wait complete. 2024-05-21T15:23:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 1 -- start start 2024-05-21T15:23:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 1 Processor -- start 2024-05-21T15:23:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab169e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad9e6b0 0x5607ab16a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aadbf390 0x5607ab168370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607ab16afc0 con 0x5607aad964d0 2024-05-21T15:23:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5607ab07eb90 con 0x5607aadbf390 2024-05-21T15:23:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5607ab16b140 con 0x5607aad9e6b0 2024-05-21T15:23:23.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f30a5256740 10 client.0 _ll_register_callbacks cb 0x5607ab0bada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f3097fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab169e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f3097fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab169e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35796/0 (socket says 172.21.15.78:35796) 2024-05-21T15:23:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.606+0000 7f3097fff640 1 -- 172.21.15.78:0/1833612140 learned_addr learned my addr 172.21.15.78:0/1833612140 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f309c954640 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aadbf390 0x5607ab168370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30977fe640 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad9e6b0 0x5607ab16a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f3097fff640 1 -- 172.21.15.78:0/1833612140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad9e6b0 msgr2=0x5607ab16a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f3097fff640 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad9e6b0 0x5607ab16a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f3097fff640 1 -- 172.21.15.78:0/1833612140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aadbf390 msgr2=0x5607ab168370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f3097fff640 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aadbf390 0x5607ab168370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f3097fff640 1 -- 172.21.15.78:0/1833612140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607aadd73f0 con 0x5607aad964d0 2024-05-21T15:23:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f3097fff640 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab169e60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3088000c00 tx=0x7f3088004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861ece32886dc737 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30777fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30777fe640 1 -- 172.21.15.78:0/1833612140 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088029070 con 0x5607aad964d0 2024-05-21T15:23:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30777fe640 1 -- 172.21.15.78:0/1833612140 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f3088023840 con 0x5607aad964d0 2024-05-21T15:23:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30a5256740 1 -- 172.21.15.78:0/1833612140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3088004c80 con 0x5607aad964d0 2024-05-21T15:23:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30777fe640 1 -- 172.21.15.78:0/1833612140 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3088035050 con 0x5607aad964d0 2024-05-21T15:23:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30a5256740 10 client.17739 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30a5256740 1 -- 172.21.15.78:0/1833612140 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5607aad970f0 con 0x5607aad964d0 2024-05-21T15:23:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.607+0000 7f30777fe640 1 -- 172.21.15.78:0/1833612140 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3088004c80 con 0x5607aad964d0 2024-05-21T15:23:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30777fe640 1 -- 172.21.15.78:0/1833612140 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f30880195d0 con 0x5607aad964d0 2024-05-21T15:23:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30a5256740 10 client.17739 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30a5256740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30a5256740 20 client.17739 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30a5256740 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5607aad7df60 0x5607ab17a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30a5256740 4 client.17739 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30a5256740 1 -- 172.21.15.78:0/1833612140 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607ab0d1430 con 0x5607aad7df60 2024-05-21T15:23:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30977fe640 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5607aad7df60 0x5607ab17a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30777fe640 10 client.17739 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.608+0000 7f30977fe640 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5607aad7df60 0x5607ab17a9d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a4cbf22e00802c server_cookie=af40edb6626bdb36 in_seq=0 out_seq=0 2024-05-21T15:23:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30777fe640 1 -- 172.21.15.78:0/1833612140 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607ab0d1430 con 0x5607aad7df60 2024-05-21T15:23:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30777fe640 10 client.17739 handle_command_reply: tid=0 2024-05-21T15:23:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 client.17739 shutdown 2024-05-21T15:23:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 -- 172.21.15.78:0/1833612140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 msgr2=0x5607ab169e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab169e60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3088000c00 tx=0x7f3088004970 comp rx=0 tx=0).stop 2024-05-21T15:23:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 -- 172.21.15.78:0/1833612140 shutdown_connections 2024-05-21T15:23:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5607aad7df60 0x5607ab17a9d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aadbf390 0x5607ab168370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad9e6b0 0x5607ab16a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30777fe640 0 client.17739 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 --2- 172.21.15.78:0/1833612140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab169e60 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 -- 172.21.15.78:0/1833612140 >> 172.21.15.78:0/1833612140 conn(0x5607aadb34e0 msgr2=0x5607ab15fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 -- 172.21.15.78:0/1833612140 shutdown_connections 2024-05-21T15:23:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 1 -- 172.21.15.78:0/1833612140 wait complete. 2024-05-21T15:23:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.609+0000 7f30a5256740 20 client.17739 trim_cache size 0 max 16384 2024-05-21T15:23:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.627+0000 7f30a5256740 1 Processor -- start 2024-05-21T15:23:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.627+0000 7f30a5256740 1 -- start start 2024-05-21T15:23:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.627+0000 7f30a5256740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad7df60 0x5607ab167e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.627+0000 7f30a5256740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab161a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.627+0000 7f30a5256740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.627+0000 7f30a5256740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30981b0fe0 con 0x5607aad964d0 2024-05-21T15:23:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.627+0000 7f30a5256740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f30981b2b20 con 0x5607aad9e6b0 2024-05-21T15:23:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.627+0000 7f30a5256740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30981b16b0 con 0x5607aad7df60 2024-05-21T15:23:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab161a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35812/0 (socket says 172.21.15.78:35812) 2024-05-21T15:23:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 -- 172.21.15.78:0/2838344402 learned_addr learned my addr 172.21.15.78:0/2838344402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f309c954640 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f3097fff640 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad7df60 0x5607ab167e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 -- 172.21.15.78:0/2838344402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad7df60 msgr2=0x5607ab167e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad7df60 0x5607ab167e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 -- 172.21.15.78:0/2838344402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 msgr2=0x5607ab161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab161fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 -- 172.21.15.78:0/2838344402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ab14b990 con 0x5607aad964d0 2024-05-21T15:23:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f30977fe640 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab161a90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f308c098340 tx=0x7f308c0d1f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd3f67f2c6a99b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f307dffb640 1 -- 172.21.15.78:0/2838344402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c0a0290 con 0x5607aad964d0 2024-05-21T15:23:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f307dffb640 1 -- 172.21.15.78:0/2838344402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308c0a4b50 con 0x5607aad964d0 2024-05-21T15:23:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.628+0000 7f307dffb640 1 -- 172.21.15.78:0/2838344402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c0037a0 con 0x5607aad964d0 2024-05-21T15:23:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 -- 172.21.15.78:0/2838344402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 msgr2=0x5607ab161a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab161a90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f308c098340 tx=0x7f308c0d1f60 comp rx=0 tx=0).stop 2024-05-21T15:23:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 -- 172.21.15.78:0/2838344402 shutdown_connections 2024-05-21T15:23:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad964d0 0x5607ab161a90 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 --2- 172.21.15.78:0/2838344402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad7df60 0x5607ab167e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 -- 172.21.15.78:0/2838344402 >> 172.21.15.78:0/2838344402 conn(0x5607aadb34e0 msgr2=0x5607ab15ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 -- 172.21.15.78:0/2838344402 shutdown_connections 2024-05-21T15:23:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 -- 172.21.15.78:0/2838344402 wait complete. 2024-05-21T15:23:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.629+0000 7f30a5256740 1 -- start start 2024-05-21T15:23:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30a5256740 1 Processor -- start 2024-05-21T15:23:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30a5256740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad7df60 0x5607ab1da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30a5256740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad964d0 0x5607ab1da600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30a5256740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab16d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30a5256740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f30981b2450 con 0x5607aad7df60 2024-05-21T15:23:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30a5256740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f30981b1d80 con 0x5607aad9e6b0 2024-05-21T15:23:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30a5256740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30981aefe0 con 0x5607aad964d0 2024-05-21T15:23:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30a5256740 10 client.0 _ll_register_callbacks cb 0x7f3088009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad7df60 0x5607ab1da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad7df60 0x5607ab1da0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35828/0 (socket says 172.21.15.78:35828) 2024-05-21T15:23:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 -- 172.21.15.78:0/3752325696 learned_addr learned my addr 172.21.15.78:0/3752325696 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f309c954640 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab16d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30977fe640 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad964d0 0x5607ab1da600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 -- 172.21.15.78:0/3752325696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad964d0 msgr2=0x5607ab1da600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad964d0 0x5607ab1da600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 -- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 msgr2=0x5607ab16d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab16d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 -- 172.21.15.78:0/3752325696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ab16df40 con 0x5607aad7df60 2024-05-21T15:23:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f309c954640 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab16d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f3097fff640 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad7df60 0x5607ab1da0c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3088014040 tx=0x7f3088005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2b2fb387e60a8d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.630+0000 7f30777fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30777fe640 1 -- 172.21.15.78:0/3752325696 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088029040 con 0x5607aad7df60 2024-05-21T15:23:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30777fe640 1 -- 172.21.15.78:0/3752325696 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f3088019430 con 0x5607aad7df60 2024-05-21T15:23:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 1 -- 172.21.15.78:0/3752325696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f308c0a4b50 con 0x5607aad7df60 2024-05-21T15:23:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30777fe640 1 -- 172.21.15.78:0/3752325696 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3088019630 con 0x5607aad7df60 2024-05-21T15:23:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 10 client.17745 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 1 -- 172.21.15.78:0/3752325696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5607ab1e26e0 con 0x5607aad7df60 2024-05-21T15:23:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30777fe640 1 -- 172.21.15.78:0/3752325696 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308c0a4b50 con 0x5607aad7df60 2024-05-21T15:23:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30777fe640 1 -- 172.21.15.78:0/3752325696 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3088023640 con 0x5607aad7df60 2024-05-21T15:23:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 10 client.17745 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 20 client.17745 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5607aadbf390 0x5607ab1e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 4 client.17745 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:23:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30a5256740 1 -- 172.21.15.78:0/3752325696 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607ab0cb780 con 0x5607aadbf390 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.631+0000 7f30977fe640 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5607aadbf390 0x5607ab1e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30977fe640 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5607aadbf390 0x5607ab1e4d80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1689f7dd1df5e2f server_cookie=d2807ed91a111e23 in_seq=0 out_seq=0 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30777fe640 10 client.17745 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30777fe640 1 -- 172.21.15.78:0/3752325696 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5607ab0cb780 con 0x5607aadbf390 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30777fe640 10 client.17745 handle_command_reply: tid=0 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 client.17745 shutdown 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 -- 172.21.15.78:0/3752325696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad7df60 msgr2=0x5607ab1da0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad7df60 0x5607ab1da0c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3088014040 tx=0x7f3088005540 comp rx=0 tx=0).stop 2024-05-21T15:23:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 -- 172.21.15.78:0/3752325696 shutdown_connections 2024-05-21T15:23:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5607aadbf390 0x5607ab1e4d80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5607aad9e6b0 0x5607ab16d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607aad964d0 0x5607ab1da600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30777fe640 0 client.17745 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 --2- 172.21.15.78:0/3752325696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607aad7df60 0x5607ab1da0c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 -- 172.21.15.78:0/3752325696 >> 172.21.15.78:0/3752325696 conn(0x5607aadb34e0 msgr2=0x5607ab166c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 -- 172.21.15.78:0/3752325696 shutdown_connections 2024-05-21T15:23:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 1 -- 172.21.15.78:0/3752325696 wait complete. 2024-05-21T15:23:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.632+0000 7f30a5256740 20 client.17745 trim_cache size 0 max 16384 2024-05-21T15:23:23.647 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:23:23.647 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:23:23.647 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:23:23.647 INFO:teuthology.orchestra.run.smithi078.stdout: "8076c33e-155e-4db1-831f-f777b74ce51c": { 2024-05-21T15:23:23.647 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:23:23.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "8076c33e-155e-4db1-831f-f777b74ce51c", 2024-05-21T15:23:23.648 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:23:23.648 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:23:23.648 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:23:23.648 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:23:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 -- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30800a2d00 msgr2=0x7f30800a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30800a2d00 0x7f30800a5210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3090006010 tx=0x7f30900093f0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 -- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098108ad0 msgr2=0x7f30981a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098108ad0 0x7f30981a1b00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f308c00e9f0 tx=0x7f308c00eeb0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 -- 172.21.15.78:0/58108470 shutdown_connections 2024-05-21T15:23:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f30800a2d00 0x7f30800a5210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3098108f70 0x7f309819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3098108ad0 0x7f30981a1b00 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 --2- 172.21.15.78:0/58108470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3098101770 0x7f30981a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 -- 172.21.15.78:0/58108470 >> 172.21.15.78:0/58108470 conn(0x7f3098076830 msgr2=0x7f309810d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 -- 172.21.15.78:0/58108470 shutdown_connections 2024-05-21T15:23:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:23.633+0000 7f309e3b7640 1 -- 172.21.15.78:0/58108470 wait complete. 2024-05-21T15:23:23.651 INFO:tasks.cephfs.filesystem:scrub status for tag:8076c33e-155e-4db1-831f-f777b74ce51c - {'path': '/', 'tag': '8076c33e-155e-4db1-831f-f777b74ce51c', 'options': 'recursive,force'} 2024-05-21T15:23:23.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:23 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:23:23.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:23 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:23:23.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:23 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:23:23.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:23 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:23:23.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:23 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:23:23.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:23 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:23:23.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:23 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:23:23.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:23 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:23:24.835 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: pgmap v224: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:23:24.835 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:23:24.835 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.835 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:24.835 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:24.836 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.836 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:24.836 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:24.836 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:24.836 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:24.836 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.836 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.837 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.837 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.837 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:24 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: pgmap v224: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:23:24.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:23:24.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:24.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:24.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:24.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:24.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:24.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:24.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:24 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: pgmap v224: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:23:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:23:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:24 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:26.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:26 smithi078 ceph-mon[31997]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:23:26.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:26 smithi084 ceph-mon[36334]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:23:26.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:26 smithi172 ceph-mon[36402]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:23:28.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:28 smithi078 ceph-mon[31997]: pgmap v226: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 97 B/s wr, 41 op/s 2024-05-21T15:23:28.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:28 smithi084 ceph-mon[36334]: pgmap v226: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 97 B/s wr, 41 op/s 2024-05-21T15:23:28.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:28 smithi172 ceph-mon[36402]: pgmap v226: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 35 KiB/s rd, 97 B/s wr, 41 op/s 2024-05-21T15:23:30.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:30 smithi078 ceph-mon[31997]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:30.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:30 smithi084 ceph-mon[36334]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:30.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:30 smithi172 ceph-mon[36402]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:32.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:32 smithi078 ceph-mon[31997]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:32.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:32 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:23:32.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:32 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:23:32.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:32 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:23:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:32 smithi084 ceph-mon[36334]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:32 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:23:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:32 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:23:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:32 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:23:32.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:32 smithi172 ceph-mon[36402]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:32.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:32 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:23:32.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:32 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:23:32.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:32 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:23:33.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:33 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:23:33.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:33 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:23:33.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:33 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:23:33.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:33 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:23:33.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:33 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:23:33.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:33 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:23:34.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:34 smithi078 ceph-mon[31997]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:23:34.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:34 smithi084 ceph-mon[36334]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:23:34.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:34 smithi172 ceph-mon[36402]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:23:36.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:36 smithi078 ceph-mon[31997]: pgmap v230: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:36 smithi084 ceph-mon[36334]: pgmap v230: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:36.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:36 smithi172 ceph-mon[36402]: pgmap v230: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:23:37.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:37 smithi078 ceph-mon[31997]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:23:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:37 smithi084 ceph-mon[36334]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:23:37.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:37 smithi172 ceph-mon[36402]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:23:40.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:39 smithi172 ceph-mon[36402]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 13 KiB/s rd, 17 op/s 2024-05-21T15:23:40.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:39 smithi078 ceph-mon[31997]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 13 KiB/s rd, 17 op/s 2024-05-21T15:23:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:39 smithi084 ceph-mon[36334]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 13 KiB/s rd, 17 op/s 2024-05-21T15:23:42.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:41 smithi172 ceph-mon[36402]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:42.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:41 smithi078 ceph-mon[31997]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:42.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:41 smithi084 ceph-mon[36334]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:44.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:43 smithi172 ceph-mon[36402]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:44.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:43 smithi078 ceph-mon[31997]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:44.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:43 smithi084 ceph-mon[36334]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:46.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:45 smithi172 ceph-mon[36402]: pgmap v235: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:46.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:45 smithi078 ceph-mon[31997]: pgmap v235: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:46.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:45 smithi084 ceph-mon[36334]: pgmap v235: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:48.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:47 smithi172 ceph-mon[36402]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:48.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:47 smithi078 ceph-mon[31997]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:48.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:47 smithi084 ceph-mon[36334]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:50.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:49 smithi078 ceph-mon[31997]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:50.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:49 smithi084 ceph-mon[36334]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:50.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:49 smithi172 ceph-mon[36402]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:52.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:51 smithi078 ceph-mon[31997]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:51 smithi084 ceph-mon[36334]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:52.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:51 smithi172 ceph-mon[36402]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:53.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:23:53.652 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:23:53.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 -- 172.21.15.78:0/1210086913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c103220 msgr2=0x7f550c10f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 --2- 172.21.15.78:0/1210086913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c103220 0x7f550c10f5f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f54fc009ca0 tx=0x7f54fc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 -- 172.21.15.78:0/1210086913 shutdown_connections 2024-05-21T15:23:53.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 --2- 172.21.15.78:0/1210086913 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f550c108f70 0x7f550c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 --2- 172.21.15.78:0/1210086913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c103220 0x7f550c10f5f0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 --2- 172.21.15.78:0/1210086913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c108ad0 0x7f550c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 -- 172.21.15.78:0/1210086913 >> 172.21.15.78:0/1210086913 conn(0x7f550c0fe190 msgr2=0x7f550c1005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 -- 172.21.15.78:0/1210086913 shutdown_connections 2024-05-21T15:23:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.778+0000 7f55140e9640 1 -- 172.21.15.78:0/1210086913 wait complete. 2024-05-21T15:23:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f55140e9640 1 Processor -- start 2024-05-21T15:23:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f55140e9640 1 -- start start 2024-05-21T15:23:53.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f55140e9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c103220 0x7f550c074f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f55140e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f550c108ad0 0x7f550c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f55140e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c108f70 0x7f550c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f55140e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f550c112e10 con 0x7f550c108f70 2024-05-21T15:23:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f55140e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f550c112c90 con 0x7f550c108ad0 2024-05-21T15:23:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f55140e9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f550c112f90 con 0x7f550c103220 2024-05-21T15:23:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5512686640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c108f70 0x7f550c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5512686640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c108f70 0x7f550c0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50044/0 (socket says 172.21.15.78:50044) 2024-05-21T15:23:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5512686640 1 -- 172.21.15.78:0/2727620994 learned_addr learned my addr 172.21.15.78:0/2727620994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:53.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5511684640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f550c108ad0 0x7f550c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5511e85640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c103220 0x7f550c074f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5512686640 1 -- 172.21.15.78:0/2727620994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c103220 msgr2=0x7f550c074f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5512686640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c103220 0x7f550c074f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5512686640 1 -- 172.21.15.78:0/2727620994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f550c108ad0 msgr2=0x7f550c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5512686640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f550c108ad0 0x7f550c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5512686640 1 -- 172.21.15.78:0/2727620994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c1a7b30 con 0x7f550c108f70 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.779+0000 7f5511684640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f550c108ad0 0x7f550c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.780+0000 7f5512686640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c108f70 0x7f550c0759e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f54fc031a60 tx=0x7f54fc031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:53.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.780+0000 7f550affd640 1 -- 172.21.15.78:0/2727620994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc003c30 con 0x7f550c108f70 2024-05-21T15:23:53.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.780+0000 7f550affd640 1 -- 172.21.15.78:0/2727620994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54fc003dd0 con 0x7f550c108f70 2024-05-21T15:23:53.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.780+0000 7f55140e9640 1 -- 172.21.15.78:0/2727620994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c1a7dc0 con 0x7f550c108f70 2024-05-21T15:23:53.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.780+0000 7f550affd640 1 -- 172.21.15.78:0/2727620994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc0047e0 con 0x7f550c108f70 2024-05-21T15:23:53.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.780+0000 7f55140e9640 1 -- 172.21.15.78:0/2727620994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c1a8280 con 0x7f550c108f70 2024-05-21T15:23:53.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.782+0000 7f550affd640 1 -- 172.21.15.78:0/2727620994 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f54fc02f790 con 0x7f550c108f70 2024-05-21T15:23:53.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f551af8f740 1 Processor -- start 2024-05-21T15:23:53.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f551af8f740 1 -- start start 2024-05-21T15:23:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f550affd640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f54f40a2d00 0x7f54f40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f551af8f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c995c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f551af8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f551af8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f551af8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556c99528d00 con 0x556c991fb4d0 2024-05-21T15:23:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f551af8f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556c994d04c0 con 0x556c99224390 2024-05-21T15:23:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f551af8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556c99524cc0 con 0x556c992036b0 2024-05-21T15:23:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f550affd640 1 -- 172.21.15.78:0/2727620994 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f54fc0f0910 con 0x7f550c108f70 2024-05-21T15:23:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f5511684640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f54f40a2d00 0x7f54f40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f5511684640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f5511684640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995c3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50046/0 (socket says 172.21.15.78:50046) 2024-05-21T15:23:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.783+0000 7f5511684640 1 -- 172.21.15.78:0/470374456 learned_addr learned my addr 172.21.15.78:0/470374456 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5511e85640 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c995c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5512686640 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5511684640 1 -- 172.21.15.78:0/470374456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 msgr2=0x556c995c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5511684640 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5511684640 1 -- 172.21.15.78:0/470374456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 msgr2=0x556c995c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5511684640 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c995c2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5511684640 1 -- 172.21.15.78:0/470374456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c993067d0 con 0x556c991fb4d0 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5511684640 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995c3270 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5500010c10 tx=0x7f550001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ba2593730d15f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f5511684640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f54f40a2d00 0x7f54f40a5210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f55000138f0 tx=0x7f550000bac0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f54f3fff640 1 -- 172.21.15.78:0/470374456 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500021070 con 0x556c991fb4d0 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f54f3fff640 1 -- 172.21.15.78:0/470374456 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f550000be10 con 0x556c991fb4d0 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.784+0000 7f54f3fff640 1 -- 172.21.15.78:0/470374456 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500003810 con 0x556c991fb4d0 2024-05-21T15:23:53.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 -- 172.21.15.78:0/470374456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 msgr2=0x556c995c3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995c3270 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5500010c10 tx=0x7f550001a000 comp rx=0 tx=0).stop 2024-05-21T15:23:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 -- 172.21.15.78:0/470374456 shutdown_connections 2024-05-21T15:23:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995c3270 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 --2- 172.21.15.78:0/470374456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c995c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 -- 172.21.15.78:0/470374456 >> 172.21.15.78:0/470374456 conn(0x556c992184e0 msgr2=0x556c995bf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 -- 172.21.15.78:0/470374456 shutdown_connections 2024-05-21T15:23:53.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.785+0000 7f551af8f740 1 -- 172.21.15.78:0/470374456 wait complete. 2024-05-21T15:23:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 1 -- start start 2024-05-21T15:23:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 1 Processor -- start 2024-05-21T15:23:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c992036b0 0x556c995cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c99224390 0x556c995cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556c995cffc0 con 0x556c991fb4d0 2024-05-21T15:23:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556c994e3b90 con 0x556c992036b0 2024-05-21T15:23:53.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556c995d0140 con 0x556c99224390 2024-05-21T15:23:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f551af8f740 10 client.0 _ll_register_callbacks cb 0x556c9951fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f5511e85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f5511e85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50058/0 (socket says 172.21.15.78:50058) 2024-05-21T15:23:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f5511e85640 1 -- 172.21.15.78:0/4009315115 learned_addr learned my addr 172.21.15.78:0/4009315115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.786+0000 7f5512686640 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c99224390 0x556c995cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f5511e85640 1 -- 172.21.15.78:0/4009315115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c99224390 msgr2=0x556c995cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f5511684640 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c992036b0 0x556c995cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f5511e85640 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c99224390 0x556c995cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f5511e85640 1 -- 172.21.15.78:0/4009315115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c992036b0 msgr2=0x556c995cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f5511e85640 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c992036b0 0x556c995cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f5511e85640 1 -- 172.21.15.78:0/4009315115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9923c3f0 con 0x556c991fb4d0 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f5512686640 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c99224390 0x556c995cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f5511e85640 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995cee90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f55040117c0 tx=0x7f5504011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428a593987a781f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f54f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f54f17fa640 1 -- 172.21.15.78:0/4009315115 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550401e060 con 0x556c991fb4d0 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f54f17fa640 1 -- 172.21.15.78:0/4009315115 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f550400fdb0 con 0x556c991fb4d0 2024-05-21T15:23:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f551af8f740 1 -- 172.21.15.78:0/4009315115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c99536430 con 0x556c991fb4d0 2024-05-21T15:23:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f54f17fa640 1 -- 172.21.15.78:0/4009315115 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5504025050 con 0x556c991fb4d0 2024-05-21T15:23:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f551af8f740 10 client.17757 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f551af8f740 1 -- 172.21.15.78:0/4009315115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556c991fc0f0 con 0x556c991fb4d0 2024-05-21T15:23:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.787+0000 7f54f17fa640 1 -- 172.21.15.78:0/4009315115 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556c99536430 con 0x556c991fb4d0 2024-05-21T15:23:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f54f17fa640 1 -- 172.21.15.78:0/4009315115 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5504018030 con 0x556c991fb4d0 2024-05-21T15:23:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f551af8f740 10 client.17757 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f551af8f740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f551af8f740 20 client.17757 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f551af8f740 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556c991e2f60 0x556c995dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f551af8f740 4 client.17757 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f551af8f740 1 -- 172.21.15.78:0/4009315115 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c99530780 con 0x556c991e2f60 2024-05-21T15:23:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f5511684640 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556c991e2f60 0x556c995dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f5511684640 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556c991e2f60 0x556c995dfdd0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac2559a0dfbdfcc server_cookie=53e9214a0d6dda0f in_seq=0 out_seq=0 2024-05-21T15:23:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.788+0000 7f54f17fa640 10 client.17757 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f54f17fa640 1 -- 172.21.15.78:0/4009315115 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c99530780 con 0x556c991e2f60 2024-05-21T15:23:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f54f17fa640 10 client.17757 handle_command_reply: tid=0 2024-05-21T15:23:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 client.17757 shutdown 2024-05-21T15:23:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 -- 172.21.15.78:0/4009315115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 msgr2=0x556c995cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995cee90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f55040117c0 tx=0x7f5504011c80 comp rx=0 tx=0).stop 2024-05-21T15:23:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 -- 172.21.15.78:0/4009315115 shutdown_connections 2024-05-21T15:23:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556c991e2f60 0x556c995dfdd0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c992036b0 0x556c995cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c99224390 0x556c995cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 --2- 172.21.15.78:0/4009315115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995cee90 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f54f17fa640 0 client.17757 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:53.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 -- 172.21.15.78:0/4009315115 >> 172.21.15.78:0/4009315115 conn(0x556c992184e0 msgr2=0x556c995c4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:53.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 -- 172.21.15.78:0/4009315115 shutdown_connections 2024-05-21T15:23:53.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 1 -- 172.21.15.78:0/4009315115 wait complete. 2024-05-21T15:23:53.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.789+0000 7f551af8f740 20 client.17757 trim_cache size 0 max 16384 2024-05-21T15:23:53.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.807+0000 7f551af8f740 1 Processor -- start 2024-05-21T15:23:53.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.807+0000 7f551af8f740 1 -- start start 2024-05-21T15:23:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.807+0000 7f551af8f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c991fb4d0 0x556c995cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.807+0000 7f551af8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.807+0000 7f551af8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c99224390 0x556c995c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.807+0000 7f551af8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b0dc0 con 0x556c99224390 2024-05-21T15:23:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.807+0000 7f551af8f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b2900 con 0x556c991fb4d0 2024-05-21T15:23:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.807+0000 7f551af8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b1490 con 0x556c992036b0 2024-05-21T15:23:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c99224390 0x556c995c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c99224390 0x556c995c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50070/0 (socket says 172.21.15.78:50070) 2024-05-21T15:23:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 -- 172.21.15.78:0/3688002014 learned_addr learned my addr 172.21.15.78:0/3688002014 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5511e85640 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c991fb4d0 0x556c995cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5511684640 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 -- 172.21.15.78:0/3688002014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 msgr2=0x556c995c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 -- 172.21.15.78:0/3688002014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c991fb4d0 msgr2=0x556c995cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c991fb4d0 0x556c995cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 -- 172.21.15.78:0/3688002014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c995cf6f0 con 0x556c99224390 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f5512686640 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c99224390 0x556c995c6f20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0ee0e0 tx=0x7f54fc0dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536b318f775a1e25 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f54f37fe640 1 -- 172.21.15.78:0/3688002014 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc0bce50 con 0x556c99224390 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f54f37fe640 1 -- 172.21.15.78:0/3688002014 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54fc0bf070 con 0x556c99224390 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.808+0000 7f54f37fe640 1 -- 172.21.15.78:0/3688002014 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc0c5630 con 0x556c99224390 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 -- 172.21.15.78:0/3688002014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c99224390 msgr2=0x556c995c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c99224390 0x556c995c6f20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0ee0e0 tx=0x7f54fc0dd040 comp rx=0 tx=0).stop 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 -- 172.21.15.78:0/3688002014 shutdown_connections 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c99224390 0x556c995c6f20 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 --2- 172.21.15.78:0/3688002014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c991fb4d0 0x556c995cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 -- 172.21.15.78:0/3688002014 >> 172.21.15.78:0/3688002014 conn(0x556c992184e0 msgr2=0x556c995ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:53.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 -- 172.21.15.78:0/3688002014 shutdown_connections 2024-05-21T15:23:53.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 -- 172.21.15.78:0/3688002014 wait complete. 2024-05-21T15:23:53.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 -- start start 2024-05-21T15:23:53.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.809+0000 7f551af8f740 1 Processor -- start 2024-05-21T15:23:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f551af8f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f551af8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995d72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f551af8f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c99646140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f551af8f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b2230 con 0x556c991fb4d0 2024-05-21T15:23:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f551af8f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1b1b60 con 0x556c99224390 2024-05-21T15:23:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f551af8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f550c1aedc0 con 0x556c992036b0 2024-05-21T15:23:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f551af8f740 10 client.0 _ll_register_callbacks cb 0x7f550400c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511e85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511e85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995d6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50072/0 (socket says 172.21.15.78:50072) 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511e85640 1 -- 172.21.15.78:0/2146536332 learned_addr learned my addr 172.21.15.78:0/2146536332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511684640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995d72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5512686640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c99646140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511e85640 1 -- 172.21.15.78:0/2146536332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 msgr2=0x556c995d72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511e85640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995d72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511e85640 1 -- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 msgr2=0x556c99646140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511e85640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c99646140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511e85640 1 -- 172.21.15.78:0/2146536332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c99646680 con 0x556c991fb4d0 2024-05-21T15:23:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5511684640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995d72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.810+0000 7f5512686640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c99646140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f5511e85640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995d6d70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f550400d010 tx=0x7f55040056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f228d9b5e73adb24 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f54f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f54f17fa640 1 -- 172.21.15.78:0/2146536332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550402d070 con 0x556c991fb4d0 2024-05-21T15:23:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f54f17fa640 1 -- 172.21.15.78:0/2146536332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f5504011040 con 0x556c991fb4d0 2024-05-21T15:23:53.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 1 -- 172.21.15.78:0/2146536332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54fc0bf070 con 0x556c991fb4d0 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f54f17fa640 1 -- 172.21.15.78:0/2146536332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5504038050 con 0x556c991fb4d0 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 10 client.17763 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 1 -- 172.21.15.78:0/2146536332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556c996477e0 con 0x556c991fb4d0 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f54f17fa640 1 -- 172.21.15.78:0/2146536332 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54fc0bf070 con 0x556c991fb4d0 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f54f17fa640 1 -- 172.21.15.78:0/2146536332 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5504018040 con 0x556c991fb4d0 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 10 client.17763 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 10 client.17763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 20 client.17763 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:53.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556c991b83a0 0x556c99649f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 4 client.17763 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:23:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f551af8f740 1 -- 172.21.15.78:0/2146536332 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c994ad6f0 con 0x556c991b83a0 2024-05-21T15:23:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.811+0000 7f5511684640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556c991b83a0 0x556c99649f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f54f17fa640 10 client.17763 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f5511684640 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556c991b83a0 0x556c99649f40 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b40683ad5f4afa9 server_cookie=cfb78afbf7308666 in_seq=0 out_seq=0 2024-05-21T15:23:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f54f17fa640 1 -- 172.21.15.78:0/2146536332 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c994ad6f0 con 0x556c991b83a0 2024-05-21T15:23:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f54f17fa640 10 client.17763 handle_command_reply: tid=0 2024-05-21T15:23:53.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 client.17763 shutdown 2024-05-21T15:23:53.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 -- 172.21.15.78:0/2146536332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 msgr2=0x556c995d6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995d6d70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f550400d010 tx=0x7f55040056e0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 -- 172.21.15.78:0/2146536332 shutdown_connections 2024-05-21T15:23:53.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x556c991b83a0 0x556c99649f40 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c99224390 0x556c99646140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556c992036b0 0x556c995d72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 --2- 172.21.15.78:0/2146536332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556c991fb4d0 0x556c995d6d70 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f54f17fa640 0 client.17763 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:53.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 -- 172.21.15.78:0/2146536332 >> 172.21.15.78:0/2146536332 conn(0x556c992184e0 msgr2=0x556c995cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:53.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 -- 172.21.15.78:0/2146536332 shutdown_connections 2024-05-21T15:23:53.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 1 -- 172.21.15.78:0/2146536332 wait complete. 2024-05-21T15:23:53.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.812+0000 7f551af8f740 20 client.17763 trim_cache size 0 max 16384 2024-05-21T15:23:53.826 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:23:53.827 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:23:53.827 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:23:53.827 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:23:53.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 -- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f54f40a2d00 msgr2=0x7f54f40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f54f40a2d00 0x7f54f40a5210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f55000138f0 tx=0x7f550000bac0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 -- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c108f70 msgr2=0x7f550c0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c108f70 0x7f550c0759e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f54fc031a60 tx=0x7f54fc031a90 comp rx=0 tx=0).stop 2024-05-21T15:23:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 -- 172.21.15.78:0/2727620994 shutdown_connections 2024-05-21T15:23:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f550c108ad0 0x7f550c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f54f40a2d00 0x7f54f40a5210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f550c108f70 0x7f550c0759e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 --2- 172.21.15.78:0/2727620994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c103220 0x7f550c074f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 -- 172.21.15.78:0/2727620994 >> 172.21.15.78:0/2727620994 conn(0x7f550c0fe190 msgr2=0x7f550c105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 -- 172.21.15.78:0/2727620994 shutdown_connections 2024-05-21T15:23:53.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.813+0000 7f55140e9640 1 -- 172.21.15.78:0/2727620994 wait complete. 2024-05-21T15:23:53.830 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:23:53.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:23:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 -- 172.21.15.78:0/2361449855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188101b30 msgr2=0x7ff188101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 --2- 172.21.15.78:0/2361449855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188101b30 0x7ff188101f40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff170009900 tx=0x7ff17002a410 comp rx=0 tx=0).stop 2024-05-21T15:23:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 -- 172.21.15.78:0/2361449855 shutdown_connections 2024-05-21T15:23:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 --2- 172.21.15.78:0/2361449855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff18810c370 0x7ff18810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 --2- 172.21.15.78:0/2361449855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188101b30 0x7ff188101f40 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 --2- 172.21.15.78:0/2361449855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff188107a10 0x7ff1881015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 -- 172.21.15.78:0/2361449855 >> 172.21.15.78:0/2361449855 conn(0x7ff188077130 msgr2=0x7ff188077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:53.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 -- 172.21.15.78:0/2361449855 shutdown_connections 2024-05-21T15:23:53.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.941+0000 7ff18cb96640 1 -- 172.21.15.78:0/2361449855 wait complete. 2024-05-21T15:23:53.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.942+0000 7ff18cb96640 1 Processor -- start 2024-05-21T15:23:53.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.942+0000 7ff18cb96640 1 -- start start 2024-05-21T15:23:53.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff18cb96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff188101b30 0x7ff188198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff18cb96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188107a10 0x7ff188198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff18cb96640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff18810c370 0x7ff18819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff18cb96640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff188111d70 con 0x7ff188107a10 2024-05-21T15:23:53.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff18cb96640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff188111bf0 con 0x7ff18810c370 2024-05-21T15:23:53.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff18cb96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff188111ef0 con 0x7ff188101b30 2024-05-21T15:23:53.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188107a10 0x7ff188198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188107a10 0x7ff188198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50092/0 (socket says 172.21.15.78:50092) 2024-05-21T15:23:53.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 -- 172.21.15.78:0/728242117 learned_addr learned my addr 172.21.15.78:0/728242117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:53.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff18659c640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff188101b30 0x7ff188198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff186d9d640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff18810c370 0x7ff18819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 -- 172.21.15.78:0/728242117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff188101b30 msgr2=0x7ff188198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff188101b30 0x7ff188198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 -- 172.21.15.78:0/728242117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff18810c370 msgr2=0x7ff18819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:53.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff18810c370 0x7ff18819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:53.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 -- 172.21.15.78:0/728242117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18819d570 con 0x7ff188107a10 2024-05-21T15:23:53.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.943+0000 7ff17dd9b640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188107a10 0x7ff188198c70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff17002a930 tx=0x7ff170003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:53.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.944+0000 7ff17ffff640 1 -- 172.21.15.78:0/728242117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17003f070 con 0x7ff188107a10 2024-05-21T15:23:53.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.944+0000 7ff17ffff640 1 -- 172.21.15.78:0/728242117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff170031c90 con 0x7ff188107a10 2024-05-21T15:23:53.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.944+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18819d800 con 0x7ff188107a10 2024-05-21T15:23:53.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.944+0000 7ff17ffff640 1 -- 172.21.15.78:0/728242117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170035030 con 0x7ff188107a10 2024-05-21T15:23:53.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.944+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18819dcf0 con 0x7ff188107a10 2024-05-21T15:23:53.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.945+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff188069140 con 0x7ff188107a10 2024-05-21T15:23:53.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.946+0000 7ff17ffff640 1 -- 172.21.15.78:0/728242117 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff170042070 con 0x7ff188107a10 2024-05-21T15:23:53.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.946+0000 7ff17ffff640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1740a2d00 0x7ff1740a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:53.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.947+0000 7ff18659c640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1740a2d00 0x7ff1740a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:53.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.947+0000 7ff17ffff640 1 -- 172.21.15.78:0/728242117 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7ff1700efea0 con 0x7ff188107a10 2024-05-21T15:23:53.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.947+0000 7ff18659c640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1740a2d00 0x7ff1740a5210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff188102a10 tx=0x7ff178005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:53.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:53.951+0000 7ff17ffff640 1 -- 172.21.15.78:0/728242117 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1700bc3e0 con 0x7ff188107a10 2024-05-21T15:23:54.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.163+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff188105ed0 con 0x7ff188107a10 2024-05-21T15:23:54.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.163+0000 7ff17ffff640 1 -- 172.21.15.78:0/728242117 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7ff1700c06e0 con 0x7ff188107a10 2024-05-21T15:23:54.167 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:23:54.167 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:23:54.167 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:23:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1740a2d00 msgr2=0x7ff1740a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1740a2d00 0x7ff1740a5210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff188102a10 tx=0x7ff178005f90 comp rx=0 tx=0).stop 2024-05-21T15:23:54.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188107a10 msgr2=0x7ff188198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188107a10 0x7ff188198c70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff17002a930 tx=0x7ff170003b20 comp rx=0 tx=0).stop 2024-05-21T15:23:54.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 shutdown_connections 2024-05-21T15:23:54.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff18810c370 0x7ff18819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1740a2d00 0x7ff1740a5210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff188107a10 0x7ff188198c70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 --2- 172.21.15.78:0/728242117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff188101b30 0x7ff188198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 >> 172.21.15.78:0/728242117 conn(0x7ff188077130 msgr2=0x7ff1881030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 shutdown_connections 2024-05-21T15:23:54.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.166+0000 7ff18cb96640 1 -- 172.21.15.78:0/728242117 wait complete. 2024-05-21T15:23:54.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:23:54.201 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:53 smithi078 ceph-mon[31997]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:54.236 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:53 smithi172 ceph-mon[36402]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:54.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.290+0000 7f433a81b640 1 -- 172.21.15.78:0/1259767481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 msgr2=0x7f4334101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.290+0000 7f433a81b640 1 --2- 172.21.15.78:0/1259767481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 0x7f4334101eb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4328009860 tx=0x7f432802a340 comp rx=0 tx=0).stop 2024-05-21T15:23:54.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 -- 172.21.15.78:0/1259767481 shutdown_connections 2024-05-21T15:23:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 --2- 172.21.15.78:0/1259767481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f433410c330 0x7f433410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 --2- 172.21.15.78:0/1259767481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 0x7f4334101eb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 --2- 172.21.15.78:0/1259767481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43341079d0 0x7f4334101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 -- 172.21.15.78:0/1259767481 >> 172.21.15.78:0/1259767481 conn(0x7f43340770b0 msgr2=0x7f43340774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 -- 172.21.15.78:0/1259767481 shutdown_connections 2024-05-21T15:23:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 -- 172.21.15.78:0/1259767481 wait complete. 2024-05-21T15:23:54.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 Processor -- start 2024-05-21T15:23:54.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.291+0000 7f433a81b640 1 -- start start 2024-05-21T15:23:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f433a81b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 0x7f43341997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f433a81b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43341079d0 0x7f4334199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f433a81b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f433410c330 0x7f433419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f433a81b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4334112de0 con 0x7f4334101aa0 2024-05-21T15:23:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f433a81b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4334112c60 con 0x7f43341079d0 2024-05-21T15:23:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f433a81b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4334112f60 con 0x7f433410c330 2024-05-21T15:23:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 0x7f43341997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 0x7f43341997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50114/0 (socket says 172.21.15.78:50114) 2024-05-21T15:23:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 -- 172.21.15.78:0/1323333278 learned_addr learned my addr 172.21.15.78:0/1323333278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f43337fe640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43341079d0 0x7f4334199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 -- 172.21.15.78:0/1323333278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f433410c330 msgr2=0x7f433419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f433410c330 0x7f433419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 -- 172.21.15.78:0/1323333278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43341079d0 msgr2=0x7f4334199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43341079d0 0x7f4334199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 -- 172.21.15.78:0/1323333278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433419e600 con 0x7f4334101aa0 2024-05-21T15:23:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f43337fe640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43341079d0 0x7f4334199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f4333fff640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 0x7f43341997a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f432400e9f0 tx=0x7f432400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f43317fa640 1 -- 172.21.15.78:0/1323333278 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432400ccc0 con 0x7f4334101aa0 2024-05-21T15:23:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f43317fa640 1 -- 172.21.15.78:0/1323333278 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4324010910 con 0x7f4334101aa0 2024-05-21T15:23:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f433a81b640 1 -- 172.21.15.78:0/1323333278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433419e8f0 con 0x7f4334101aa0 2024-05-21T15:23:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f43317fa640 1 -- 172.21.15.78:0/1323333278 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324018960 con 0x7f4334101aa0 2024-05-21T15:23:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.292+0000 7f433a81b640 1 -- 172.21.15.78:0/1323333278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341a6270 con 0x7f4334101aa0 2024-05-21T15:23:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43317fa640 1 -- 172.21.15.78:0/1323333278 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4324010ac0 con 0x7f4334101aa0 2024-05-21T15:23:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43417a6740 1 Processor -- start 2024-05-21T15:23:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43417a6740 1 -- start start 2024-05-21T15:23:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43417a6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 0x55b894013bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43317fa640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f431c0a2d00 0x7f431c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43417a6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c4c4d0 0x55b894014130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43417a6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c546b0 0x55b894014670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43417a6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b893f79c40 con 0x55b893c546b0 2024-05-21T15:23:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43417a6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b893f21580 con 0x55b893c4c4d0 2024-05-21T15:23:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.295+0000 7f43417a6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b893f75c00 con 0x55b893c75390 2024-05-21T15:23:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4333fff640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f431c0a2d00 0x7f431c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c546b0 0x55b894014670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c546b0 0x55b894014670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50120/0 (socket says 172.21.15.78:50120) 2024-05-21T15:23:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f43337fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c4c4d0 0x55b894014130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 -- 172.21.15.78:0/1870605208 learned_addr learned my addr 172.21.15.78:0/1870605208 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f43317fa640 1 -- 172.21.15.78:0/1323333278 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f43240cf8e0 con 0x7f4334101aa0 2024-05-21T15:23:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 --2- 172.21.15.78:0/1870605208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 0x55b894013bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 -- 172.21.15.78:0/1870605208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 msgr2=0x55b894013bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 --2- 172.21.15.78:0/1870605208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 0x55b894013bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 -- 172.21.15.78:0/1870605208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c4c4d0 msgr2=0x55b894014130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 --2- 172.21.15.78:0/1870605208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c4c4d0 0x55b894014130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 -- 172.21.15.78:0/1870605208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b893c8d3f0 con 0x55b893c546b0 2024-05-21T15:23:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4333fff640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f431c0a2d00 0x7f431c0a5210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f432409ba70 tx=0x7f432406ad60 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f4338db8640 1 --2- 172.21.15.78:0/1870605208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c546b0 0x55b894014670 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f432c009ee0 tx=0x7f432c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732fae5fb2050480 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f431a7fc640 1 -- 172.21.15.78:0/1870605208 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c0140f0 con 0x55b893c546b0 2024-05-21T15:23:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f431a7fc640 1 -- 172.21.15.78:0/1870605208 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432c014290 con 0x55b893c546b0 2024-05-21T15:23:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.296+0000 7f431a7fc640 1 -- 172.21.15.78:0/1870605208 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c005310 con 0x55b893c546b0 2024-05-21T15:23:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 -- 172.21.15.78:0/1870605208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c546b0 msgr2=0x55b894014670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 --2- 172.21.15.78:0/1870605208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c546b0 0x55b894014670 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f432c009ee0 tx=0x7f432c0097a0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 -- 172.21.15.78:0/1870605208 shutdown_connections 2024-05-21T15:23:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 --2- 172.21.15.78:0/1870605208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c4c4d0 0x55b894014130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 --2- 172.21.15.78:0/1870605208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c546b0 0x55b894014670 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 --2- 172.21.15.78:0/1870605208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 0x55b894013bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 -- 172.21.15.78:0/1870605208 >> 172.21.15.78:0/1870605208 conn(0x55b893c694e0 msgr2=0x55b8940104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 -- 172.21.15.78:0/1870605208 shutdown_connections 2024-05-21T15:23:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.297+0000 7f43417a6740 1 -- 172.21.15.78:0/1870605208 wait complete. 2024-05-21T15:23:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 1 -- start start 2024-05-21T15:23:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 1 Processor -- start 2024-05-21T15:23:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b894022730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 0x55b894022c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 0x55b89402e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b894025500 con 0x55b893c4c4d0 2024-05-21T15:23:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b893f34c40 con 0x55b893c546b0 2024-05-21T15:23:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b894025680 con 0x55b893c75390 2024-05-21T15:23:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f4338db8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b894022730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f43417a6740 10 client.0 _ll_register_callbacks cb 0x55b893c8cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f4338db8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b894022730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50128/0 (socket says 172.21.15.78:50128) 2024-05-21T15:23:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.298+0000 7f4338db8640 1 -- 172.21.15.78:0/3486936171 learned_addr learned my addr 172.21.15.78:0/3486936171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f4338db8640 1 -- 172.21.15.78:0/3486936171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 msgr2=0x55b89402e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f4338db8640 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 0x55b89402e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f4338db8640 1 -- 172.21.15.78:0/3486936171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 msgr2=0x55b894022c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43337fe640 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 0x55b894022c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f4338db8640 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 0x55b894022c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f4338db8640 1 -- 172.21.15.78:0/3486936171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b893d577d0 con 0x55b893c4c4d0 2024-05-21T15:23:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f4338db8640 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b894022730 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f432c009660 tx=0x7f432c014500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4c9cad165490fc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43137fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43137fe640 1 -- 172.21.15.78:0/3486936171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c00f830 con 0x55b893c4c4d0 2024-05-21T15:23:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43137fe640 1 -- 172.21.15.78:0/3486936171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f432c00f9d0 con 0x55b893c4c4d0 2024-05-21T15:23:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43417a6740 1 -- 172.21.15.78:0/3486936171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f432c014290 con 0x55b893c4c4d0 2024-05-21T15:23:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43137fe640 1 -- 172.21.15.78:0/3486936171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f432c00fbd0 con 0x55b893c4c4d0 2024-05-21T15:23:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43417a6740 10 client.17781 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43417a6740 1 -- 172.21.15.78:0/3486936171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b894031b80 con 0x55b893c4c4d0 2024-05-21T15:23:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43137fe640 1 -- 172.21.15.78:0/3486936171 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432c014290 con 0x55b893c4c4d0 2024-05-21T15:23:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.299+0000 7f43137fe640 1 -- 172.21.15.78:0/3486936171 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f432c02f030 con 0x55b893c4c4d0 2024-05-21T15:23:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43417a6740 10 client.17781 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43417a6740 10 client.17781 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:23:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43417a6740 20 client.17781 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43417a6740 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55b893c33f60 0x55b894034f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43417a6740 4 client.17781 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43417a6740 1 -- 172.21.15.78:0/3486936171 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b893f87370 con 0x55b893c33f60 2024-05-21T15:23:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43337fe640 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55b893c33f60 0x55b894034f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43137fe640 10 client.17781 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:23:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.300+0000 7f43337fe640 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55b893c33f60 0x55b894034f90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cc3578b3e43c61fd server_cookie=fc539c853947640d in_seq=0 out_seq=0 2024-05-21T15:23:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.301+0000 7f43137fe640 1 -- 172.21.15.78:0/3486936171 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b893f87370 con 0x55b893c33f60 2024-05-21T15:23:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.301+0000 7f43137fe640 10 client.17781 handle_command_reply: tid=0 2024-05-21T15:23:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.301+0000 7f43417a6740 1 client.17781 shutdown 2024-05-21T15:23:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 -- 172.21.15.78:0/3486936171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 msgr2=0x55b894022730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b894022730 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f432c009660 tx=0x7f432c014500 comp rx=0 tx=0).stop 2024-05-21T15:23:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 -- 172.21.15.78:0/3486936171 shutdown_connections 2024-05-21T15:23:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55b893c33f60 0x55b894034f90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 0x55b894022c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c75390 0x55b89402e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43137fe640 0 client.17781 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:23:54.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 --2- 172.21.15.78:0/3486936171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b894022730 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 -- 172.21.15.78:0/3486936171 >> 172.21.15.78:0/3486936171 conn(0x55b893c694e0 msgr2=0x55b8940144e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 -- 172.21.15.78:0/3486936171 shutdown_connections 2024-05-21T15:23:54.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 1 -- 172.21.15.78:0/3486936171 wait complete. 2024-05-21T15:23:54.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.302+0000 7f43417a6740 20 client.17781 trim_cache size 0 max 16384 2024-05-21T15:23:54.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.315+0000 7f43417a6740 1 Processor -- start 2024-05-21T15:23:54.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.315+0000 7f43417a6740 1 -- start start 2024-05-21T15:23:54.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.315+0000 7f43417a6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c33f60 0x55b894010560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.315+0000 7f43417a6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b89402e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.315+0000 7f43417a6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c546b0 0x55b89403a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.315+0000 7f43417a6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43341acbb0 con 0x55b893c4c4d0 2024-05-21T15:23:54.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.315+0000 7f43417a6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ae6f0 con 0x55b893c33f60 2024-05-21T15:23:54.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.315+0000 7f43417a6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ad280 con 0x55b893c546b0 2024-05-21T15:23:54.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b89402e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b89402e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50134/0 (socket says 172.21.15.78:50134) 2024-05-21T15:23:54.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 -- 172.21.15.78:0/1856990808 learned_addr learned my addr 172.21.15.78:0/1856990808 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f4338db8640 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c33f60 0x55b894010560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 -- 172.21.15.78:0/1856990808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c546b0 msgr2=0x55b89403a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f4338db8640 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c546b0 0x55b89403a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c546b0 0x55b89403a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 -- 172.21.15.78:0/1856990808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c33f60 msgr2=0x55b894010560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c33f60 0x55b894010560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 -- 172.21.15.78:0/1856990808 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8940253e0 con 0x55b893c4c4d0 2024-05-21T15:23:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f43337fe640 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b89402e590 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f43280095f0 tx=0x7f4328005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa07c7a2417c18a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f4319ffb640 1 -- 172.21.15.78:0/1856990808 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328044020 con 0x55b893c4c4d0 2024-05-21T15:23:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f4319ffb640 1 -- 172.21.15.78:0/1856990808 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4328038040 con 0x55b893c4c4d0 2024-05-21T15:23:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.316+0000 7f4319ffb640 1 -- 172.21.15.78:0/1856990808 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328037040 con 0x55b893c4c4d0 2024-05-21T15:23:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 -- 172.21.15.78:0/1856990808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 msgr2=0x55b89402e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b89402e590 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f43280095f0 tx=0x7f4328005f60 comp rx=0 tx=0).stop 2024-05-21T15:23:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 -- 172.21.15.78:0/1856990808 shutdown_connections 2024-05-21T15:23:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c546b0 0x55b89403a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c4c4d0 0x55b89402e590 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 --2- 172.21.15.78:0/1856990808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c33f60 0x55b894010560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 -- 172.21.15.78:0/1856990808 >> 172.21.15.78:0/1856990808 conn(0x55b893c694e0 msgr2=0x55b89400fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 -- 172.21.15.78:0/1856990808 shutdown_connections 2024-05-21T15:23:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 -- 172.21.15.78:0/1856990808 wait complete. 2024-05-21T15:23:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.317+0000 7f43417a6740 1 -- start start 2024-05-21T15:23:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43417a6740 1 Processor -- start 2024-05-21T15:23:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43417a6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c33f60 0x55b89402aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43417a6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c4c4d0 0x55b89402afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43417a6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 0x55b894025fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43417a6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ae020 con 0x55b893c33f60 2024-05-21T15:23:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43417a6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f43341ad950 con 0x55b893c546b0 2024-05-21T15:23:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43417a6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43341aabb0 con 0x55b893c4c4d0 2024-05-21T15:23:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43417a6740 10 client.0 _ll_register_callbacks cb 0x7f4328035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c33f60 0x55b89402aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c33f60 0x55b89402aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50144/0 (socket says 172.21.15.78:50144) 2024-05-21T15:23:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 -- 172.21.15.78:0/603108853 learned_addr learned my addr 172.21.15.78:0/603108853 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43337fe640 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c4c4d0 0x55b89402afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 0x55b894025fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 -- 172.21.15.78:0/603108853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c4c4d0 msgr2=0x55b89402afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c4c4d0 0x55b89402afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 -- 172.21.15.78:0/603108853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 msgr2=0x55b894025fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 0x55b894025fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 -- 172.21.15.78:0/603108853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b894026720 con 0x55b893c33f60 2024-05-21T15:23:54.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f4338db8640 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c33f60 0x55b89402aa70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f432c01eb20 tx=0x7f432c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97993c0c3c082829 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.318+0000 7f43137fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43137fe640 1 -- 172.21.15.78:0/603108853 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c014690 con 0x55b893c33f60 2024-05-21T15:23:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43137fe640 1 -- 172.21.15.78:0/603108853 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f432c00f7b0 con 0x55b893c33f60 2024-05-21T15:23:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 1 -- 172.21.15.78:0/603108853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4328038040 con 0x55b893c33f60 2024-05-21T15:23:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43137fe640 1 -- 172.21.15.78:0/603108853 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f432c01e450 con 0x55b893c33f60 2024-05-21T15:23:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 10 client.17787 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 1 -- 172.21.15.78:0/603108853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b89409cbd0 con 0x55b893c33f60 2024-05-21T15:23:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43137fe640 1 -- 172.21.15.78:0/603108853 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4328038040 con 0x55b893c33f60 2024-05-21T15:23:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43137fe640 1 -- 172.21.15.78:0/603108853 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f432c042030 con 0x55b893c33f60 2024-05-21T15:23:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 10 client.17787 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 10 client.17787 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:23:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 20 client.17787 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55b893c75390 0x55b89409f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 4 client.17787 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43417a6740 1 -- 172.21.15.78:0/603108853 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b893f816c0 con 0x55b893c75390 2024-05-21T15:23:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.319+0000 7f43337fe640 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55b893c75390 0x55b89409f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43137fe640 10 client.17787 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:23:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43337fe640 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55b893c75390 0x55b89409f240 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb7b756b7ec954c9 server_cookie=d612df7c09d479eb in_seq=0 out_seq=0 2024-05-21T15:23:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43137fe640 1 -- 172.21.15.78:0/603108853 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b893f816c0 con 0x55b893c75390 2024-05-21T15:23:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43137fe640 10 client.17787 handle_command_reply: tid=0 2024-05-21T15:23:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 client.17787 shutdown 2024-05-21T15:23:54.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 -- 172.21.15.78:0/603108853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c33f60 msgr2=0x55b89402aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c33f60 0x55b89402aa70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f432c01eb20 tx=0x7f432c000f40 comp rx=0 tx=0).stop 2024-05-21T15:23:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 -- 172.21.15.78:0/603108853 shutdown_connections 2024-05-21T15:23:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55b893c75390 0x55b89409f240 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b893c546b0 0x55b894025fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b893c4c4d0 0x55b89402afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43137fe640 0 client.17787 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:23:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 --2- 172.21.15.78:0/603108853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b893c33f60 0x55b89402aa70 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 -- 172.21.15.78:0/603108853 >> 172.21.15.78:0/603108853 conn(0x55b893c694e0 msgr2=0x55b894015880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 -- 172.21.15.78:0/603108853 shutdown_connections 2024-05-21T15:23:54.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 1 -- 172.21.15.78:0/603108853 wait complete. 2024-05-21T15:23:54.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.320+0000 7f43417a6740 20 client.17787 trim_cache size 0 max 16384 2024-05-21T15:23:54.338 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 -- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f431c0a2d00 msgr2=0x7f431c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f431c0a2d00 0x7f431c0a5210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f432409ba70 tx=0x7f432406ad60 comp rx=0 tx=0).stop 2024-05-21T15:23:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 -- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 msgr2=0x7f43341997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 0x7f43341997a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f432400e9f0 tx=0x7f432400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 -- 172.21.15.78:0/1323333278 shutdown_connections 2024-05-21T15:23:54.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43341079d0 0x7f4334199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f431c0a2d00 0x7f431c0a5210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f433410c330 0x7f433419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 --2- 172.21.15.78:0/1323333278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4334101aa0 0x7f43341997a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 -- 172.21.15.78:0/1323333278 >> 172.21.15.78:0/1323333278 conn(0x7f43340770b0 msgr2=0x7f4334103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 -- 172.21.15.78:0/1323333278 shutdown_connections 2024-05-21T15:23:54.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.321+0000 7f433a81b640 1 -- 172.21.15.78:0/1323333278 wait complete. 2024-05-21T15:23:54.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:23:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:53 smithi084 ceph-mon[36334]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:23:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1817227052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781079e0 msgr2=0x7fe878101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1817227052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781079e0 0x7fe878101570 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe874009900 tx=0x7fe87402a410 comp rx=0 tx=0).stop 2024-05-21T15:23:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1817227052 shutdown_connections 2024-05-21T15:23:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1817227052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878101ab0 0x7fe878101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1817227052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe87810c340 0x7fe87810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1817227052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781079e0 0x7fe878101570 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1817227052 >> 172.21.15.78:0/1817227052 conn(0x7fe8780fd210 msgr2=0x7fe8780ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1817227052 shutdown_connections 2024-05-21T15:23:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.490+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1817227052 wait complete. 2024-05-21T15:23:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87f4eb640 1 Processor -- start 2024-05-21T15:23:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87f4eb640 1 -- start start 2024-05-21T15:23:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87f4eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878101ab0 0x7fe87819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87f4eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8781079e0 0x7fe87819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87f4eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe87810c340 0x7fe87819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87f4eb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe878111cf0 con 0x7fe87810c340 2024-05-21T15:23:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87f4eb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe878111b70 con 0x7fe878101ab0 2024-05-21T15:23:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87f4eb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe878111e70 con 0x7fe8781079e0 2024-05-21T15:23:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.491+0000 7fe87da88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe87810c340 0x7fe87819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87da88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe87810c340 0x7fe87819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50152/0 (socket says 172.21.15.78:50152) 2024-05-21T15:23:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87da88640 1 -- 172.21.15.78:0/1753059327 learned_addr learned my addr 172.21.15.78:0/1753059327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87d287640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878101ab0 0x7fe87819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8781079e0 0x7fe87819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87da88640 1 -- 172.21.15.78:0/1753059327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8781079e0 msgr2=0x7fe87819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87da88640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8781079e0 0x7fe87819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87da88640 1 -- 172.21.15.78:0/1753059327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878101ab0 msgr2=0x7fe87819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87da88640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878101ab0 0x7fe87819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87da88640 1 -- 172.21.15.78:0/1753059327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8781a58f0 con 0x7fe87810c340 2024-05-21T15:23:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8781079e0 0x7fe87819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87da88640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe87810c340 0x7fe87819b0f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cce0 tx=0x7fe87000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe86e7fc640 1 -- 172.21.15.78:0/1753059327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87000f450 con 0x7fe87810c340 2024-05-21T15:23:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe86e7fc640 1 -- 172.21.15.78:0/1753059327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87000fad0 con 0x7fe87810c340 2024-05-21T15:23:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe86e7fc640 1 -- 172.21.15.78:0/1753059327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870018c10 con 0x7fe87810c340 2024-05-21T15:23:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1753059327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8781a5be0 con 0x7fe87810c340 2024-05-21T15:23:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.492+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1753059327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781a6130 con 0x7fe87810c340 2024-05-21T15:23:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe86e7fc640 1 -- 172.21.15.78:0/1753059327 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe87000f5f0 con 0x7fe87810c340 2024-05-21T15:23:54.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe886383740 1 Processor -- start 2024-05-21T15:23:54.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe886383740 1 -- start start 2024-05-21T15:23:54.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe886383740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af61dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe886383740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af61e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe886383740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af61e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe886383740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1af583c40 con 0x55d1af27f390 2024-05-21T15:23:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe886383740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d1af52b580 con 0x55d1af2564d0 2024-05-21T15:23:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe886383740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1af57fc00 con 0x55d1af25e6b0 2024-05-21T15:23:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe86e7fc640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe8640a2d80 0x7fe8640a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.495+0000 7fe87ca86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af61dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af61dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50154/0 (socket says 172.21.15.78:50154) 2024-05-21T15:23:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 -- 172.21.15.78:0/1425192742 learned_addr learned my addr 172.21.15.78:0/1425192742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87da88640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe8640a2d80 0x7fe8640a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe86e7fc640 1 -- 172.21.15.78:0/1753059327 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fe8700cfb10 con 0x7fe87810c340 2024-05-21T15:23:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87d287640 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af61e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af61e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 -- 172.21.15.78:0/1425192742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 msgr2=0x55d1af61e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af61e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 -- 172.21.15.78:0/1425192742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 msgr2=0x55d1af61e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af61e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 -- 172.21.15.78:0/1425192742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1af2973f0 con 0x55d1af27f390 2024-05-21T15:23:54.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87d287640 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af61e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:54.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af61dd20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe868014490 tx=0x7fe868009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cf72db97c0eead server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe87da88640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe8640a2d80 0x7fe8640a5290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe8700cd280 tx=0x7fe870066fc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe8637fe640 1 -- 172.21.15.78:0/1425192742 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86801a030 con 0x55d1af27f390 2024-05-21T15:23:54.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe8637fe640 1 -- 172.21.15.78:0/1425192742 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe868005500 con 0x55d1af27f390 2024-05-21T15:23:54.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.496+0000 7fe8637fe640 1 -- 172.21.15.78:0/1425192742 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8680126b0 con 0x55d1af27f390 2024-05-21T15:23:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 -- 172.21.15.78:0/1425192742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 msgr2=0x55d1af61dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af61dd20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe868014490 tx=0x7fe868009680 comp rx=0 tx=0).stop 2024-05-21T15:23:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 -- 172.21.15.78:0/1425192742 shutdown_connections 2024-05-21T15:23:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af61e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af61e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 --2- 172.21.15.78:0/1425192742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af61dd20 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 -- 172.21.15.78:0/1425192742 >> 172.21.15.78:0/1425192742 conn(0x55d1af2734e0 msgr2=0x55d1af61a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 -- 172.21.15.78:0/1425192742 shutdown_connections 2024-05-21T15:23:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.497+0000 7fe886383740 1 -- 172.21.15.78:0/1425192742 wait complete. 2024-05-21T15:23:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 1 -- start start 2024-05-21T15:23:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 1 Processor -- start 2024-05-21T15:23:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af629d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af62a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af6281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1af62afb0 con 0x55d1af27f390 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d1af53ec40 con 0x55d1af2564d0 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1af62b130 con 0x55d1af25e6b0 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe886383740 10 client.0 _ll_register_callbacks cb 0x55d1af296be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe87ca86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af6281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe87ca86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af6281e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50168/0 (socket says 172.21.15.78:50168) 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe87ca86640 1 -- 172.21.15.78:0/544016766 learned_addr learned my addr 172.21.15.78:0/544016766 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.498+0000 7fe87ca86640 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af629d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe87d287640 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af62a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe87ca86640 1 -- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 msgr2=0x55d1af62a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe87ca86640 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af62a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe87ca86640 1 -- 172.21.15.78:0/544016766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 msgr2=0x55d1af629d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe87ca86640 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af629d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe87ca86640 1 -- 172.21.15.78:0/544016766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1af3617d0 con 0x55d1af27f390 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe87d287640 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af62a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe87ca86640 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af6281e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe868004bf0 tx=0x7fe868004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c750978b5adb4c1d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe860ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe860ff9640 1 -- 172.21.15.78:0/544016766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86801a070 con 0x55d1af27f390 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe860ff9640 1 -- 172.21.15.78:0/544016766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fe868005220 con 0x55d1af27f390 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe886383740 1 -- 172.21.15.78:0/544016766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe868005500 con 0x55d1af27f390 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe860ff9640 1 -- 172.21.15.78:0/544016766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe868012890 con 0x55d1af27f390 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe886383740 10 client.17799 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe886383740 1 -- 172.21.15.78:0/544016766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d1af637790 con 0x55d1af27f390 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.499+0000 7fe860ff9640 1 -- 172.21.15.78:0/544016766 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe868005500 con 0x55d1af27f390 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe860ff9640 1 -- 172.21.15.78:0/544016766 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe86800f040 con 0x55d1af27f390 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe886383740 10 client.17799 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe886383740 10 client.17799 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe886383740 20 client.17799 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe886383740 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d1af23df60 0x55d1af63aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe886383740 4 client.17799 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe886383740 1 -- 172.21.15.78:0/544016766 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1af591370 con 0x55d1af23df60 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe87d287640 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d1af23df60 0x55d1af63aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe860ff9640 10 client.17799 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.500+0000 7fe87d287640 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d1af23df60 0x55d1af63aaf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c676661df49862e server_cookie=ce7a9f766f0f200a in_seq=0 out_seq=0 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe860ff9640 1 -- 172.21.15.78:0/544016766 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1af591370 con 0x55d1af23df60 2024-05-21T15:23:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe860ff9640 10 client.17799 handle_command_reply: tid=0 2024-05-21T15:23:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 client.17799 shutdown 2024-05-21T15:23:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 -- 172.21.15.78:0/544016766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 msgr2=0x55d1af6281e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af6281e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe868004bf0 tx=0x7fe868004c50 comp rx=0 tx=0).stop 2024-05-21T15:23:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 -- 172.21.15.78:0/544016766 shutdown_connections 2024-05-21T15:23:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d1af23df60 0x55d1af63aaf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af6281e0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af62a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 --2- 172.21.15.78:0/544016766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af629d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe860ff9640 0 client.17799 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:23:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.501+0000 7fe886383740 1 -- 172.21.15.78:0/544016766 >> 172.21.15.78:0/544016766 conn(0x55d1af2734e0 msgr2=0x55d1af61fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.502+0000 7fe886383740 1 -- 172.21.15.78:0/544016766 shutdown_connections 2024-05-21T15:23:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.502+0000 7fe886383740 1 -- 172.21.15.78:0/544016766 wait complete. 2024-05-21T15:23:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.502+0000 7fe886383740 20 client.17799 trim_cache size 0 max 16384 2024-05-21T15:23:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe886383740 1 Processor -- start 2024-05-21T15:23:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe886383740 1 -- start start 2024-05-21T15:23:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe886383740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af2564d0 0x55d1af63db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe886383740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af25e6b0 0x55d1af63e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe886383740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af27f390 0x55d1af619e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe886383740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781aec80 con 0x55d1af2564d0 2024-05-21T15:23:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe886383740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b07c0 con 0x55d1af25e6b0 2024-05-21T15:23:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe886383740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781af350 con 0x55d1af27f390 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe87ca86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af2564d0 0x55d1af63db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe87ca86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af2564d0 0x55d1af63db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50178/0 (socket says 172.21.15.78:50178) 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.515+0000 7fe87ca86640 1 -- 172.21.15.78:0/1292244902 learned_addr learned my addr 172.21.15.78:0/1292244902 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe87ca86640 1 -- 172.21.15.78:0/1292244902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af27f390 msgr2=0x55d1af619e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1292244902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af27f390 0x55d1af619e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe87ca86640 1 -- 172.21.15.78:0/1292244902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af25e6b0 msgr2=0x55d1af63e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1292244902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af25e6b0 0x55d1af63e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe87ca86640 1 -- 172.21.15.78:0/1292244902 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1af63e5a0 con 0x55d1af2564d0 2024-05-21T15:23:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe87ca86640 1 --2- 172.21.15.78:0/1292244902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af2564d0 0x55d1af63db20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe868006010 tx=0x7fe868009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110b561b3a5707f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe862ffd640 1 -- 172.21.15.78:0/1292244902 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86801a040 con 0x55d1af2564d0 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe862ffd640 1 -- 172.21.15.78:0/1292244902 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe868029040 con 0x55d1af2564d0 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.516+0000 7fe862ffd640 1 -- 172.21.15.78:0/1292244902 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86801fd30 con 0x55d1af2564d0 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 -- 172.21.15.78:0/1292244902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af2564d0 msgr2=0x55d1af63db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 --2- 172.21.15.78:0/1292244902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af2564d0 0x55d1af63db20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe868006010 tx=0x7fe868009370 comp rx=0 tx=0).stop 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 -- 172.21.15.78:0/1292244902 shutdown_connections 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 --2- 172.21.15.78:0/1292244902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af25e6b0 0x55d1af63e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 --2- 172.21.15.78:0/1292244902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af27f390 0x55d1af619e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 --2- 172.21.15.78:0/1292244902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af2564d0 0x55d1af63db20 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 -- 172.21.15.78:0/1292244902 >> 172.21.15.78:0/1292244902 conn(0x55d1af2734e0 msgr2=0x55d1af626ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 -- 172.21.15.78:0/1292244902 shutdown_connections 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 -- 172.21.15.78:0/1292244902 wait complete. 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 -- start start 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.517+0000 7fe886383740 1 Processor -- start 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe886383740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af63c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe886383740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af63cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe886383740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af63d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe886383740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b00f0 con 0x55d1af27f390 2024-05-21T15:23:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe886383740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781afa20 con 0x55d1af2564d0 2024-05-21T15:23:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe886383740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781acc80 con 0x55d1af25e6b0 2024-05-21T15:23:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe886383740 10 client.0 _ll_register_callbacks cb 0x7fe868009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af63d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af63d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50180/0 (socket says 172.21.15.78:50180) 2024-05-21T15:23:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 -- 172.21.15.78:0/2795800221 learned_addr learned my addr 172.21.15.78:0/2795800221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af63c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 -- 172.21.15.78:0/2795800221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 msgr2=0x55d1af63cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af63cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 -- 172.21.15.78:0/2795800221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 msgr2=0x55d1af63c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af63c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 -- 172.21.15.78:0/2795800221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1af6a0d80 con 0x55d1af27f390 2024-05-21T15:23:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe87ca86640 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af63d190 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x55d1af63bf70 tx=0x7fe868016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428f14f9f519077d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe860ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe860ff9640 1 -- 172.21.15.78:0/2795800221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86804a060 con 0x55d1af27f390 2024-05-21T15:23:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe860ff9640 1 -- 172.21.15.78:0/2795800221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7fe86801fce0 con 0x55d1af27f390 2024-05-21T15:23:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.518+0000 7fe886383740 1 -- 172.21.15.78:0/2795800221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe868029040 con 0x55d1af27f390 2024-05-21T15:23:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe860ff9640 1 -- 172.21.15.78:0/2795800221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe868014080 con 0x55d1af27f390 2024-05-21T15:23:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe886383740 10 client.17805 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe886383740 1 -- 172.21.15.78:0/2795800221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d1af6a1fc0 con 0x55d1af27f390 2024-05-21T15:23:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe860ff9640 1 -- 172.21.15.78:0/2795800221 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe868029040 con 0x55d1af27f390 2024-05-21T15:23:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe860ff9640 1 -- 172.21.15.78:0/2795800221 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe86801f460 con 0x55d1af27f390 2024-05-21T15:23:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe886383740 10 client.17805 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe886383740 10 client.17805 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:23:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe886383740 20 client.17805 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe886383740 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d1af2133a0 0x55d1af6a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe886383740 4 client.17805 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe886383740 1 -- 172.21.15.78:0/2795800221 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1af58b6c0 con 0x55d1af2133a0 2024-05-21T15:23:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.519+0000 7fe87d287640 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d1af2133a0 0x55d1af6a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe860ff9640 10 client.17805 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:23:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe87d287640 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d1af2133a0 0x55d1af6a4610 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a43796d96a455b5c server_cookie=5664eb6d9da6755e in_seq=0 out_seq=0 2024-05-21T15:23:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe860ff9640 1 -- 172.21.15.78:0/2795800221 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1af58b6c0 con 0x55d1af2133a0 2024-05-21T15:23:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe860ff9640 10 client.17805 handle_command_reply: tid=0 2024-05-21T15:23:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 client.17805 shutdown 2024-05-21T15:23:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 -- 172.21.15.78:0/2795800221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 msgr2=0x55d1af63d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af63d190 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x55d1af63bf70 tx=0x7fe868016040 comp rx=0 tx=0).stop 2024-05-21T15:23:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 -- 172.21.15.78:0/2795800221 shutdown_connections 2024-05-21T15:23:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d1af2133a0 0x55d1af6a4610 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1af27f390 0x55d1af63d190 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1af25e6b0 0x55d1af63cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 --2- 172.21.15.78:0/2795800221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d1af2564d0 0x55d1af63c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe860ff9640 0 client.17805 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:23:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 -- 172.21.15.78:0/2795800221 >> 172.21.15.78:0/2795800221 conn(0x55d1af2734e0 msgr2=0x55d1af626670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 -- 172.21.15.78:0/2795800221 shutdown_connections 2024-05-21T15:23:54.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 1 -- 172.21.15.78:0/2795800221 wait complete. 2024-05-21T15:23:54.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.520+0000 7fe886383740 20 client.17805 trim_cache size 0 max 16384 2024-05-21T15:23:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:54.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe8640a2d80 msgr2=0x7fe8640a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe8640a2d80 0x7fe8640a5290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe8700cd280 tx=0x7fe870066fc0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe87810c340 msgr2=0x7fe87819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe87810c340 0x7fe87819b0f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cce0 tx=0x7fe87000a620 comp rx=0 tx=0).stop 2024-05-21T15:23:54.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1753059327 shutdown_connections 2024-05-21T15:23:54.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe8640a2d80 0x7fe8640a5290 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe87810c340 0x7fe87819b0f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8781079e0 0x7fe87819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 --2- 172.21.15.78:0/1753059327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe878101ab0 0x7fe87819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1753059327 >> 172.21.15.78:0/1753059327 conn(0x7fe8780fd210 msgr2=0x7fe8780fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1753059327 shutdown_connections 2024-05-21T15:23:54.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.521+0000 7fe87f4eb640 1 -- 172.21.15.78:0/1753059327 wait complete. 2024-05-21T15:23:54.542 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:23:54.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 -- 172.21.15.78:0/1709019067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410d030 msgr2=0x7f70c410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 --2- 172.21.15.78:0/1709019067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410d030 0x7f70c410f520 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009ca0 tx=0x7f70bc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 -- 172.21.15.78:0/1709019067 shutdown_connections 2024-05-21T15:23:54.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 --2- 172.21.15.78:0/1709019067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70c410a640 0x7f70c410ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 --2- 172.21.15.78:0/1709019067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410d030 0x7f70c410f520 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 --2- 172.21.15.78:0/1709019067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70c4103be0 0x7f70c4103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 -- 172.21.15.78:0/1709019067 >> 172.21.15.78:0/1709019067 conn(0x7f70c40770c0 msgr2=0x7f70c4077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 -- 172.21.15.78:0/1709019067 shutdown_connections 2024-05-21T15:23:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 -- 172.21.15.78:0/1709019067 wait complete. 2024-05-21T15:23:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.654+0000 7f70cb8de640 1 Processor -- start 2024-05-21T15:23:54.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70cb8de640 1 -- start start 2024-05-21T15:23:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70cb8de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70c4103be0 0x7f70c4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70cb8de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410a640 0x7f70c4199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70cb8de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70c410d030 0x7f70c419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70cb8de640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4112e30 con 0x7f70c410a640 2024-05-21T15:23:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70cb8de640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4112cb0 con 0x7f70c4103be0 2024-05-21T15:23:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70cb8de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4112fb0 con 0x7f70c410d030 2024-05-21T15:23:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410a640 0x7f70c4199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410a640 0x7f70c4199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50194/0 (socket says 172.21.15.78:50194) 2024-05-21T15:23:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 -- 172.21.15.78:0/2629289096 learned_addr learned my addr 172.21.15.78:0/2629289096 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c967a640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70c4103be0 0x7f70c4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70c410d030 0x7f70c419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 -- 172.21.15.78:0/2629289096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70c410d030 msgr2=0x7f70c419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70c410d030 0x7f70c419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 -- 172.21.15.78:0/2629289096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70c4103be0 msgr2=0x7f70c4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70c4103be0 0x7f70c4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 -- 172.21.15.78:0/2629289096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c419e510 con 0x7f70c410a640 2024-05-21T15:23:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.655+0000 7f70c8e79640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410a640 0x7f70c4199c90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f70b400d980 tx=0x7f70b400dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.656+0000 7f70ba7fc640 1 -- 172.21.15.78:0/2629289096 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4017070 con 0x7f70c410a640 2024-05-21T15:23:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.656+0000 7f70ba7fc640 1 -- 172.21.15.78:0/2629289096 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b4009c30 con 0x7f70c410a640 2024-05-21T15:23:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.656+0000 7f70cb8de640 1 -- 172.21.15.78:0/2629289096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c419e800 con 0x7f70c410a640 2024-05-21T15:23:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.656+0000 7f70ba7fc640 1 -- 172.21.15.78:0/2629289096 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b400f620 con 0x7f70c410a640 2024-05-21T15:23:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.656+0000 7f70cb8de640 1 -- 172.21.15.78:0/2629289096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c419ed50 con 0x7f70c410a640 2024-05-21T15:23:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.658+0000 7f70ba7fc640 1 -- 172.21.15.78:0/2629289096 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f70b4010460 con 0x7f70c410a640 2024-05-21T15:23:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70d277e740 1 Processor -- start 2024-05-21T15:23:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70d277e740 1 -- start start 2024-05-21T15:23:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70d277e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d893d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70d277e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 0x56314d894260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70d277e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70ba7fc640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70b00a2d00 0x7f70b00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70d277e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56314d7f9c40 con 0x56314d4d46b0 2024-05-21T15:23:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70d277e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56314d7a1580 con 0x56314d4f5390 2024-05-21T15:23:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70d277e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56314d7f5c00 con 0x56314d4cc4d0 2024-05-21T15:23:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c8e79640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70b00a2d00 0x7f70b00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d893d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c967a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 0x56314d894260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d893d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:53350/0 (socket says 172.21.15.78:53350) 2024-05-21T15:23:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 -- 172.21.15.78:0/3669383293 learned_addr learned my addr 172.21.15.78:0/3669383293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70ba7fc640 1 -- 172.21.15.78:0/2629289096 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f70b40cefa0 con 0x7f70c410a640 2024-05-21T15:23:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 -- 172.21.15.78:0/3669383293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 msgr2=0x56314d894260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/3669383293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 0x56314d894260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 -- 172.21.15.78:0/3669383293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 msgr2=0x56314d8947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/3669383293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.659+0000 7f70c9e7b640 1 -- 172.21.15.78:0/3669383293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56314d50d3f0 con 0x56314d4f5390 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.660+0000 7f70c8e79640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70b00a2d00 0x7f70b00a5210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70b409b130 tx=0x7f70b406d3f0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.660+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/3669383293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d893d20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f70bc02aeb0 tx=0x7f70bc005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaec7bffebd39b1c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.660+0000 7f70af7fe640 1 -- 172.21.15.78:0/3669383293 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc045040 con 0x56314d4f5390 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.660+0000 7f70af7fe640 1 -- 172.21.15.78:0/3669383293 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70bc036050 con 0x56314d4f5390 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.660+0000 7f70af7fe640 1 -- 172.21.15.78:0/3669383293 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0305c0 con 0x56314d4f5390 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 -- 172.21.15.78:0/3669383293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 msgr2=0x56314d893d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 --2- 172.21.15.78:0/3669383293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d893d20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f70bc02aeb0 tx=0x7f70bc005d80 comp rx=0 tx=0).stop 2024-05-21T15:23:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 -- 172.21.15.78:0/3669383293 shutdown_connections 2024-05-21T15:23:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 --2- 172.21.15.78:0/3669383293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 --2- 172.21.15.78:0/3669383293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 0x56314d894260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 --2- 172.21.15.78:0/3669383293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d893d20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 -- 172.21.15.78:0/3669383293 >> 172.21.15.78:0/3669383293 conn(0x56314d4e94e0 msgr2=0x56314d8904d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 -- 172.21.15.78:0/3669383293 shutdown_connections 2024-05-21T15:23:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 -- 172.21.15.78:0/3669383293 wait complete. 2024-05-21T15:23:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 -- start start 2024-05-21T15:23:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.661+0000 7f70d277e740 1 Processor -- start 2024-05-21T15:23:54.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70d277e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4cc4d0 0x56314d89fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70d277e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4d46b0 0x56314d8a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70d277e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d89e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70d277e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56314d8a0fb0 con 0x56314d4cc4d0 2024-05-21T15:23:54.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70d277e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56314d7b4c40 con 0x56314d4f5390 2024-05-21T15:23:54.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70d277e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56314d8a1130 con 0x56314d4d46b0 2024-05-21T15:23:54.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70d277e740 10 client.0 _ll_register_callbacks cb 0x56314d50cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4cc4d0 0x56314d89fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4cc4d0 0x56314d89fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50206/0 (socket says 172.21.15.78:50206) 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 -- 172.21.15.78:0/1832074936 learned_addr learned my addr 172.21.15.78:0/1832074936 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c967a640 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4d46b0 0x56314d8a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 -- 172.21.15.78:0/1832074936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4d46b0 msgr2=0x56314d8a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4d46b0 0x56314d8a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 -- 172.21.15.78:0/1832074936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 msgr2=0x56314d89e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d89e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 -- 172.21.15.78:0/1832074936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56314d5d77d0 con 0x56314d4cc4d0 2024-05-21T15:23:54.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4cc4d0 0x56314d89fe20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0096a0 tx=0x7f70bc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1feee30c793d6985 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.662+0000 7f70acff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70acff9640 1 -- 172.21.15.78:0/1832074936 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc02af40 con 0x56314d4cc4d0 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70acff9640 1 -- 172.21.15.78:0/1832074936 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f70bc004b40 con 0x56314d4cc4d0 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 1 -- 172.21.15.78:0/1832074936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70bc036050 con 0x56314d4cc4d0 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70acff9640 1 -- 172.21.15.78:0/1832074936 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f70bc004d60 con 0x56314d4cc4d0 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 10 client.17817 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 1 -- 172.21.15.78:0/1832074936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56314d8a0fb0 con 0x56314d4cc4d0 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70acff9640 1 -- 172.21.15.78:0/1832074936 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70bc036050 con 0x56314d4cc4d0 2024-05-21T15:23:54.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70acff9640 1 -- 172.21.15.78:0/1832074936 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f70bc0308d0 con 0x56314d4cc4d0 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 10 client.17817 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 10 client.17817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 20 client.17817 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x56314d4b3f60 0x56314d8b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 4 client.17817 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.663+0000 7f70d277e740 1 -- 172.21.15.78:0/1832074936 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56314d807370 con 0x56314d4b3f60 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.664+0000 7f70c967a640 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x56314d4b3f60 0x56314d8b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.664+0000 7f70acff9640 10 client.17817 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:23:54.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.664+0000 7f70c967a640 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x56314d4b3f60 0x56314d8b08e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4f103568263f77a server_cookie=719ca051fd520a7c in_seq=0 out_seq=0 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70acff9640 1 -- 172.21.15.78:0/1832074936 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56314d807370 con 0x56314d4b3f60 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70acff9640 10 client.17817 handle_command_reply: tid=0 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 client.17817 shutdown 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 -- 172.21.15.78:0/1832074936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4cc4d0 msgr2=0x56314d89fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4cc4d0 0x56314d89fe20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0096a0 tx=0x7f70bc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 -- 172.21.15.78:0/1832074936 shutdown_connections 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x56314d4b3f60 0x56314d8b08e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4f5390 0x56314d89e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70acff9640 0 client.17817 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:23:54.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4d46b0 0x56314d8a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 --2- 172.21.15.78:0/1832074936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4cc4d0 0x56314d89fe20 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 -- 172.21.15.78:0/1832074936 >> 172.21.15.78:0/1832074936 conn(0x56314d4e94e0 msgr2=0x56314d895a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 -- 172.21.15.78:0/1832074936 shutdown_connections 2024-05-21T15:23:54.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 1 -- 172.21.15.78:0/1832074936 wait complete. 2024-05-21T15:23:54.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.665+0000 7f70d277e740 20 client.17817 trim_cache size 0 max 16384 2024-05-21T15:23:54.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70d277e740 1 Processor -- start 2024-05-21T15:23:54.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70d277e740 1 -- start start 2024-05-21T15:23:54.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70d277e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 0x56314d8943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70d277e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4d46b0 0x56314d8b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70d277e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4f5390 0x56314d8b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70d277e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41aec20 con 0x56314d4f5390 2024-05-21T15:23:54.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70d277e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41b0760 con 0x56314d4d46b0 2024-05-21T15:23:54.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70d277e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41af2f0 con 0x56314d4cc4d0 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4f5390 0x56314d8b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4f5390 0x56314d8b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50208/0 (socket says 172.21.15.78:50208) 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 -- 172.21.15.78:0/1553168353 learned_addr learned my addr 172.21.15.78:0/1553168353 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 0x56314d8943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c967a640 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4d46b0 0x56314d8b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 -- 172.21.15.78:0/1553168353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 msgr2=0x56314d8943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 0x56314d8943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 -- 172.21.15.78:0/1553168353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4d46b0 msgr2=0x56314d8b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4d46b0 0x56314d8b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 -- 172.21.15.78:0/1553168353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56314d8ab230 con 0x56314d4f5390 2024-05-21T15:23:54.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.679+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4f5390 0x56314d8b63c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f70bc030d20 tx=0x7f70bc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34f604c533c7544 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70aeffd640 1 -- 172.21.15.78:0/1553168353 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc02af40 con 0x56314d4f5390 2024-05-21T15:23:54.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70aeffd640 1 -- 172.21.15.78:0/1553168353 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70bc036040 con 0x56314d4f5390 2024-05-21T15:23:54.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70aeffd640 1 -- 172.21.15.78:0/1553168353 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0319e0 con 0x56314d4f5390 2024-05-21T15:23:54.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 -- 172.21.15.78:0/1553168353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4f5390 msgr2=0x56314d8b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4f5390 0x56314d8b63c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f70bc030d20 tx=0x7f70bc009a20 comp rx=0 tx=0).stop 2024-05-21T15:23:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 -- 172.21.15.78:0/1553168353 shutdown_connections 2024-05-21T15:23:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4d46b0 0x56314d8b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4f5390 0x56314d8b63c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 --2- 172.21.15.78:0/1553168353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4cc4d0 0x56314d8943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 -- 172.21.15.78:0/1553168353 >> 172.21.15.78:0/1553168353 conn(0x56314d4e94e0 msgr2=0x56314d893cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 -- 172.21.15.78:0/1553168353 shutdown_connections 2024-05-21T15:23:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.680+0000 7f70d277e740 1 -- 172.21.15.78:0/1553168353 wait complete. 2024-05-21T15:23:54.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 1 -- start start 2024-05-21T15:23:54.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 1 Processor -- start 2024-05-21T15:23:54.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4cc4d0 0x56314d8a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4f5390 0x56314d8a4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41b0090 con 0x56314d4d46b0 2024-05-21T15:23:54.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41af9c0 con 0x56314d4cc4d0 2024-05-21T15:23:54.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41acc20 con 0x56314d4f5390 2024-05-21T15:23:54.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70d277e740 10 client.0 _ll_register_callbacks cb 0x7f70bc032be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70c967a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70c967a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8a3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50222/0 (socket says 172.21.15.78:50222) 2024-05-21T15:23:54.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.681+0000 7f70c967a640 1 -- 172.21.15.78:0/457885587 learned_addr learned my addr 172.21.15.78:0/457885587 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4cc4d0 0x56314d8a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4f5390 0x56314d8a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c967a640 1 -- 172.21.15.78:0/457885587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4f5390 msgr2=0x56314d8a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c967a640 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4f5390 0x56314d8a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c967a640 1 -- 172.21.15.78:0/457885587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4cc4d0 msgr2=0x56314d8a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c967a640 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4cc4d0 0x56314d8a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c967a640 1 -- 172.21.15.78:0/457885587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56314d8a45a0 con 0x56314d4d46b0 2024-05-21T15:23:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4cc4d0 0x56314d8a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70c967a640 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8a3ad0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f70c0000c00 tx=0x7f70c0012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6ff3337fd26888 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70acff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70acff9640 1 -- 172.21.15.78:0/457885587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0005220 con 0x56314d4d46b0 2024-05-21T15:23:54.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70acff9640 1 -- 172.21.15.78:0/457885587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f70c00053c0 con 0x56314d4d46b0 2024-05-21T15:23:54.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70d277e740 1 -- 172.21.15.78:0/457885587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70bc036040 con 0x56314d4d46b0 2024-05-21T15:23:54.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70acff9640 1 -- 172.21.15.78:0/457885587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f70c0012e60 con 0x56314d4d46b0 2024-05-21T15:23:54.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70d277e740 10 client.17823 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70d277e740 1 -- 172.21.15.78:0/457885587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56314d9187e0 con 0x56314d4d46b0 2024-05-21T15:23:54.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.682+0000 7f70acff9640 1 -- 172.21.15.78:0/457885587 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70bc036040 con 0x56314d4d46b0 2024-05-21T15:23:54.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.683+0000 7f70acff9640 1 -- 172.21.15.78:0/457885587 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f70c002d030 con 0x56314d4d46b0 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.683+0000 7f70d277e740 10 client.17823 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.683+0000 7f70d277e740 10 client.17823 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.683+0000 7f70d277e740 20 client.17823 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.683+0000 7f70d277e740 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x56314d4893a0 0x56314d91af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.683+0000 7f70d277e740 4 client.17823 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.683+0000 7f70d277e740 1 -- 172.21.15.78:0/457885587 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56314d8016c0 con 0x56314d4893a0 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.683+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x56314d4893a0 0x56314d91af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70acff9640 10 client.17823 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:23:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70c9e7b640 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x56314d4893a0 0x56314d91af50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27af910c4ff9bd79 server_cookie=6bbcad35b12e28b8 in_seq=0 out_seq=0 2024-05-21T15:23:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70acff9640 1 -- 172.21.15.78:0/457885587 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56314d8016c0 con 0x56314d4893a0 2024-05-21T15:23:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70acff9640 10 client.17823 handle_command_reply: tid=0 2024-05-21T15:23:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 client.17823 shutdown 2024-05-21T15:23:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 -- 172.21.15.78:0/457885587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 msgr2=0x56314d8a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8a3ad0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f70c0000c00 tx=0x7f70c0012680 comp rx=0 tx=0).stop 2024-05-21T15:23:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 -- 172.21.15.78:0/457885587 shutdown_connections 2024-05-21T15:23:54.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x56314d4893a0 0x56314d91af50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56314d4f5390 0x56314d8a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56314d4d46b0 0x56314d8a3ad0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70acff9640 0 client.17823 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:23:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 --2- 172.21.15.78:0/457885587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56314d4cc4d0 0x56314d8a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 -- 172.21.15.78:0/457885587 >> 172.21.15.78:0/457885587 conn(0x56314d4e94e0 msgr2=0x56314d8946f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 -- 172.21.15.78:0/457885587 shutdown_connections 2024-05-21T15:23:54.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 1 -- 172.21.15.78:0/457885587 wait complete. 2024-05-21T15:23:54.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.684+0000 7f70d277e740 20 client.17823 trim_cache size 0 max 16384 2024-05-21T15:23:54.699 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 -- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70b00a2d00 msgr2=0x7f70b00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70b00a2d00 0x7f70b00a5210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70b409b130 tx=0x7f70b406d3f0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 -- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410a640 msgr2=0x7f70c4199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410a640 0x7f70c4199c90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f70b400d980 tx=0x7f70b400dd80 comp rx=0 tx=0).stop 2024-05-21T15:23:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 -- 172.21.15.78:0/2629289096 shutdown_connections 2024-05-21T15:23:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70b00a2d00 0x7f70b00a5210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70c410d030 0x7f70c419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70c410a640 0x7f70c4199c90 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 --2- 172.21.15.78:0/2629289096 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70c4103be0 0x7f70c4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 -- 172.21.15.78:0/2629289096 >> 172.21.15.78:0/2629289096 conn(0x7f70c40770c0 msgr2=0x7f70c4100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 -- 172.21.15.78:0/2629289096 shutdown_connections 2024-05-21T15:23:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.685+0000 7f70cb8de640 1 -- 172.21.15.78:0/2629289096 wait complete. 2024-05-21T15:23:54.702 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:23:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 -- 172.21.15.78:0/1029744670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781063f0 msgr2=0x7efe7810db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/1029744670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781063f0 0x7efe7810db90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7efe70009ca0 tx=0x7efe7002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 -- 172.21.15.78:0/1029744670 shutdown_connections 2024-05-21T15:23:54.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/1029744670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe78105930 0x7efe78105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/1029744670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781063f0 0x7efe7810db90 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/1029744670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe781013f0 0x7efe78101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 -- 172.21.15.78:0/1029744670 >> 172.21.15.78:0/1029744670 conn(0x7efe78076830 msgr2=0x7efe78076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 -- 172.21.15.78:0/1029744670 shutdown_connections 2024-05-21T15:23:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 -- 172.21.15.78:0/1029744670 wait complete. 2024-05-21T15:23:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.813+0000 7efe7e4ab640 1 Processor -- start 2024-05-21T15:23:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe7e4ab640 1 -- start start 2024-05-21T15:23:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe7e4ab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781013f0 0x7efe781973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe7e4ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe78105930 0x7efe781978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe7e4ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe781063f0 0x7efe78197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe7e4ab640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe78110bd0 con 0x7efe781013f0 2024-05-21T15:23:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe7e4ab640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe78110a50 con 0x7efe78105930 2024-05-21T15:23:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe7e4ab640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe78110d50 con 0x7efe781063f0 2024-05-21T15:23:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781013f0 0x7efe781973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781013f0 0x7efe781973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50232/0 (socket says 172.21.15.78:50232) 2024-05-21T15:23:54.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 -- 172.21.15.78:0/3479645566 learned_addr learned my addr 172.21.15.78:0/3479645566 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe6ffff640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe78105930 0x7efe781978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe7ca48640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe781063f0 0x7efe78197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 -- 172.21.15.78:0/3479645566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe781063f0 msgr2=0x7efe78197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe781063f0 0x7efe78197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 -- 172.21.15.78:0/3479645566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe78105930 msgr2=0x7efe781978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe78105930 0x7efe781978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 -- 172.21.15.78:0/3479645566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe781a7b00 con 0x7efe781013f0 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.814+0000 7efe77fff640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781013f0 0x7efe781973b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efe6800d770 tx=0x7efe6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.815+0000 7efe75ffb640 1 -- 172.21.15.78:0/3479645566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68017070 con 0x7efe781013f0 2024-05-21T15:23:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.815+0000 7efe75ffb640 1 -- 172.21.15.78:0/3479645566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe68005500 con 0x7efe781013f0 2024-05-21T15:23:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.815+0000 7efe7e4ab640 1 -- 172.21.15.78:0/3479645566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe781a7df0 con 0x7efe781013f0 2024-05-21T15:23:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.815+0000 7efe75ffb640 1 -- 172.21.15.78:0/3479645566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68004e00 con 0x7efe781013f0 2024-05-21T15:23:54.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.815+0000 7efe7e4ab640 1 -- 172.21.15.78:0/3479645566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe781a8250 con 0x7efe781013f0 2024-05-21T15:23:54.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.817+0000 7efe8542a740 1 Processor -- start 2024-05-21T15:23:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.817+0000 7efe8542a740 1 -- start start 2024-05-21T15:23:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe75ffb640 1 -- 172.21.15.78:0/3479645566 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efe68004fa0 con 0x7efe781013f0 2024-05-21T15:23:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe8542a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040ee1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe8542a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b1a4d0 0x557040ee2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe8542a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b226b0 0x557040ee27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe8542a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557040e47c40 con 0x557040b43390 2024-05-21T15:23:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe8542a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557040def580 con 0x557040b1a4d0 2024-05-21T15:23:54.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe8542a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557040e43c00 con 0x557040b226b0 2024-05-21T15:23:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040ee1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040ee1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50242/0 (socket says 172.21.15.78:50242) 2024-05-21T15:23:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 -- 172.21.15.78:0/2947828208 learned_addr learned my addr 172.21.15.78:0/2947828208 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe77fff640 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b1a4d0 0x557040ee2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe6ffff640 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b226b0 0x557040ee27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 -- 172.21.15.78:0/2947828208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b226b0 msgr2=0x557040ee27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b226b0 0x557040ee27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 -- 172.21.15.78:0/2947828208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b1a4d0 msgr2=0x557040ee2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b1a4d0 0x557040ee2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 -- 172.21.15.78:0/2947828208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557040b5b3f0 con 0x557040b43390 2024-05-21T15:23:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe77fff640 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b1a4d0 0x557040ee2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040ee1d20 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7efe70009ca0 tx=0x7efe700098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bab6164526144a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.818+0000 7efe6e7fc640 1 -- 172.21.15.78:0/2947828208 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70003970 con 0x557040b43390 2024-05-21T15:23:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.819+0000 7efe6e7fc640 1 -- 172.21.15.78:0/2947828208 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70036040 con 0x557040b43390 2024-05-21T15:23:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.819+0000 7efe6e7fc640 1 -- 172.21.15.78:0/2947828208 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe700043c0 con 0x557040b43390 2024-05-21T15:23:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.819+0000 7efe75ffb640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe640a2d80 0x7efe640a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.819+0000 7efe77fff640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe640a2d80 0x7efe640a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.819+0000 7efe75ffb640 1 -- 172.21.15.78:0/3479645566 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7efe680cede0 con 0x7efe781013f0 2024-05-21T15:23:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.819+0000 7efe8542a740 1 -- 172.21.15.78:0/2947828208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 msgr2=0x557040ee1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.819+0000 7efe77fff640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe640a2d80 0x7efe640a5290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe680cc550 tx=0x7efe680bb040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.819+0000 7efe8542a740 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040ee1d20 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7efe70009ca0 tx=0x7efe700098b0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 -- 172.21.15.78:0/2947828208 shutdown_connections 2024-05-21T15:23:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b1a4d0 0x557040ee2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b226b0 0x557040ee27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 --2- 172.21.15.78:0/2947828208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040ee1d20 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 -- 172.21.15.78:0/2947828208 >> 172.21.15.78:0/2947828208 conn(0x557040b374e0 msgr2=0x557040ede4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 -- 172.21.15.78:0/2947828208 shutdown_connections 2024-05-21T15:23:54.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 -- 172.21.15.78:0/2947828208 wait complete. 2024-05-21T15:23:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 -- start start 2024-05-21T15:23:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 Processor -- start 2024-05-21T15:23:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040eec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040ef8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557040eeee90 con 0x557040b226b0 2024-05-21T15:23:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557040e02c40 con 0x557040b43390 2024-05-21T15:23:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.820+0000 7efe8542a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557040eef010 con 0x557040b1a4d0 2024-05-21T15:23:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe8542a740 10 client.0 _ll_register_callbacks cb 0x557040b5abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50258/0 (socket says 172.21.15.78:50258) 2024-05-21T15:23:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 -- 172.21.15.78:0/3419245309 learned_addr learned my addr 172.21.15.78:0/3419245309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe7ca48640 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040ef8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 -- 172.21.15.78:0/3419245309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 msgr2=0x557040eec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040eec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 -- 172.21.15.78:0/3419245309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 msgr2=0x557040ef8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040ef8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 -- 172.21.15.78:0/3419245309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557040c257d0 con 0x557040b226b0 2024-05-21T15:23:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe7ca48640 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040ef8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe6ffff640 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eec870 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7efe6000ea70 tx=0x7efe6000ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3879d284adc22c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe577fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe577fe640 1 -- 172.21.15.78:0/3419245309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6001d060 con 0x557040b226b0 2024-05-21T15:23:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe577fe640 1 -- 172.21.15.78:0/3419245309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7efe6000de10 con 0x557040b226b0 2024-05-21T15:23:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe8542a740 1 -- 172.21.15.78:0/3419245309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe70036040 con 0x557040b226b0 2024-05-21T15:23:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe577fe640 1 -- 172.21.15.78:0/3419245309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efe60024050 con 0x557040b226b0 2024-05-21T15:23:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe8542a740 10 client.17835 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.821+0000 7efe8542a740 1 -- 172.21.15.78:0/3419245309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557040efb580 con 0x557040b226b0 2024-05-21T15:23:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe577fe640 1 -- 172.21.15.78:0/3419245309 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70036040 con 0x557040b226b0 2024-05-21T15:23:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe577fe640 1 -- 172.21.15.78:0/3419245309 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7efe60017070 con 0x557040b226b0 2024-05-21T15:23:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe8542a740 10 client.17835 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe8542a740 10 client.17835 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe8542a740 20 client.17835 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe8542a740 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x557040b01f60 0x557040efeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe8542a740 4 client.17835 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe8542a740 1 -- 172.21.15.78:0/3419245309 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557040e55370 con 0x557040b01f60 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.822+0000 7efe7ca48640 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x557040b01f60 0x557040efeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.823+0000 7efe7ca48640 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x557040b01f60 0x557040efeb70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a15885faf0289c99 server_cookie=d987aaa54d8d73f6 in_seq=0 out_seq=0 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.823+0000 7efe577fe640 10 client.17835 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.823+0000 7efe577fe640 1 -- 172.21.15.78:0/3419245309 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557040e55370 con 0x557040b01f60 2024-05-21T15:23:54.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.823+0000 7efe577fe640 10 client.17835 handle_command_reply: tid=0 2024-05-21T15:23:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.823+0000 7efe8542a740 1 client.17835 shutdown 2024-05-21T15:23:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 -- 172.21.15.78:0/3419245309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 msgr2=0x557040eec870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eec870 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7efe6000ea70 tx=0x7efe6000ef30 comp rx=0 tx=0).stop 2024-05-21T15:23:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 -- 172.21.15.78:0/3419245309 shutdown_connections 2024-05-21T15:23:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x557040b01f60 0x557040efeb70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040ef8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eec870 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe577fe640 0 client.17835 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:23:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 --2- 172.21.15.78:0/3419245309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040eec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 -- 172.21.15.78:0/3419245309 >> 172.21.15.78:0/3419245309 conn(0x557040b374e0 msgr2=0x557040ee3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 -- 172.21.15.78:0/3419245309 shutdown_connections 2024-05-21T15:23:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 1 -- 172.21.15.78:0/3419245309 wait complete. 2024-05-21T15:23:54.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.824+0000 7efe8542a740 20 client.17835 trim_cache size 0 max 16384 2024-05-21T15:23:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.837+0000 7efe8542a740 1 Processor -- start 2024-05-21T15:23:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.837+0000 7efe8542a740 1 -- start start 2024-05-21T15:23:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe8542a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040ee41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe8542a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b226b0 0x557040ef7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe8542a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040eedf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe8542a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe781b0df0 con 0x557040b43390 2024-05-21T15:23:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe8542a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe781b2930 con 0x557040b226b0 2024-05-21T15:23:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe8542a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe781b14c0 con 0x557040b1a4d0 2024-05-21T15:23:54.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040eedf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040eedf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50260/0 (socket says 172.21.15.78:50260) 2024-05-21T15:23:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 -- 172.21.15.78:0/2847473918 learned_addr learned my addr 172.21.15.78:0/2847473918 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe6ffff640 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b226b0 0x557040ef7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040ee41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 -- 172.21.15.78:0/2847473918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 msgr2=0x557040ee41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040ee41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 -- 172.21.15.78:0/2847473918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b226b0 msgr2=0x557040ef7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b226b0 0x557040ef7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 -- 172.21.15.78:0/2847473918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557040ee2cb0 con 0x557040b43390 2024-05-21T15:23:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040eedf50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7efe70031a80 tx=0x7efe70031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e691080ee1ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe6dffb640 1 -- 172.21.15.78:0/2847473918 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70031450 con 0x557040b43390 2024-05-21T15:23:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe6dffb640 1 -- 172.21.15.78:0/2847473918 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70036030 con 0x557040b43390 2024-05-21T15:23:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.838+0000 7efe6dffb640 1 -- 172.21.15.78:0/2847473918 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe700040b0 con 0x557040b43390 2024-05-21T15:23:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 -- 172.21.15.78:0/2847473918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 msgr2=0x557040eedf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040eedf50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7efe70031a80 tx=0x7efe70031ab0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 -- 172.21.15.78:0/2847473918 shutdown_connections 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b226b0 0x557040ef7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b43390 0x557040eedf50 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 --2- 172.21.15.78:0/2847473918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040ee41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 -- 172.21.15.78:0/2847473918 >> 172.21.15.78:0/2847473918 conn(0x557040b374e0 msgr2=0x557040eed840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 -- 172.21.15.78:0/2847473918 shutdown_connections 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.839+0000 7efe8542a740 1 -- 172.21.15.78:0/2847473918 wait complete. 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 1 -- start start 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 1 Processor -- start 2024-05-21T15:23:54.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040ef46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eefa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040eeffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe781b2260 con 0x557040b226b0 2024-05-21T15:23:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe781b1b90 con 0x557040b43390 2024-05-21T15:23:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe781aedf0 con 0x557040b1a4d0 2024-05-21T15:23:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe8542a740 10 client.0 _ll_register_callbacks cb 0x7efe6000b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eefa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eefa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50262/0 (socket says 172.21.15.78:50262) 2024-05-21T15:23:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe6ffff640 1 -- 172.21.15.78:0/2068255186 learned_addr learned my addr 172.21.15.78:0/2068255186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040ef46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040eeffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe6ffff640 1 -- 172.21.15.78:0/2068255186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 msgr2=0x557040ef46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe6ffff640 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040ef46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe6ffff640 1 -- 172.21.15.78:0/2068255186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 msgr2=0x557040eeffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.840+0000 7efe6ffff640 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040eeffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe6ffff640 1 -- 172.21.15.78:0/2068255186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557040ef05f0 con 0x557040b226b0 2024-05-21T15:23:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040eeffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe6ffff640 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eefa70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7efe60001eb0 tx=0x7efe6000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c89014afb5d0ff server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe577fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe577fe640 1 -- 172.21.15.78:0/2068255186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe600052c0 con 0x557040b226b0 2024-05-21T15:23:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe577fe640 1 -- 172.21.15.78:0/2068255186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7efe60005460 con 0x557040b226b0 2024-05-21T15:23:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 1 -- 172.21.15.78:0/2068255186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe70036030 con 0x557040b226b0 2024-05-21T15:23:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe577fe640 1 -- 172.21.15.78:0/2068255186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efe60037050 con 0x557040b226b0 2024-05-21T15:23:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 10 client.17841 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 1 -- 172.21.15.78:0/2068255186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557040f66cb0 con 0x557040b226b0 2024-05-21T15:23:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe577fe640 1 -- 172.21.15.78:0/2068255186 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70036030 con 0x557040b226b0 2024-05-21T15:23:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe577fe640 1 -- 172.21.15.78:0/2068255186 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7efe60017040 con 0x557040b226b0 2024-05-21T15:23:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 10 client.17841 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 10 client.17841 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:23:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 20 client.17841 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x557040ad73a0 0x557040f69490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 4 client.17841 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.841+0000 7efe8542a740 1 -- 172.21.15.78:0/2068255186 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557040e4f6c0 con 0x557040ad73a0 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.842+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x557040ad73a0 0x557040f69490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.842+0000 7efe577fe640 10 client.17841 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.842+0000 7efe7ca48640 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x557040ad73a0 0x557040f69490 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=945e09f302489cf9 server_cookie=69b2e0e716ccec70 in_seq=0 out_seq=0 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.842+0000 7efe577fe640 1 -- 172.21.15.78:0/2068255186 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557040e4f6c0 con 0x557040ad73a0 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.842+0000 7efe577fe640 10 client.17841 handle_command_reply: tid=0 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.842+0000 7efe8542a740 1 client.17841 shutdown 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.842+0000 7efe8542a740 1 -- 172.21.15.78:0/2068255186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 msgr2=0x557040eefa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.842+0000 7efe8542a740 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eefa70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7efe60001eb0 tx=0x7efe6000dc10 comp rx=0 tx=0).stop 2024-05-21T15:23:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 1 -- 172.21.15.78:0/2068255186 shutdown_connections 2024-05-21T15:23:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x557040ad73a0 0x557040f69490 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557040b43390 0x557040eeffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557040b226b0 0x557040eefa70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe577fe640 0 client.17841 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:23:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 1 --2- 172.21.15.78:0/2068255186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557040b1a4d0 0x557040ef46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 1 -- 172.21.15.78:0/2068255186 >> 172.21.15.78:0/2068255186 conn(0x557040b374e0 msgr2=0x557040ee3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 1 -- 172.21.15.78:0/2068255186 shutdown_connections 2024-05-21T15:23:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 1 -- 172.21.15.78:0/2068255186 wait complete. 2024-05-21T15:23:54.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe8542a740 20 client.17841 trim_cache size 0 max 16384 2024-05-21T15:23:54.858 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:54.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 -- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe640a2d80 msgr2=0x7efe640a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe640a2d80 0x7efe640a5290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe680cc550 tx=0x7efe680bb040 comp rx=0 tx=0).stop 2024-05-21T15:23:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 -- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781013f0 msgr2=0x7efe781973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781013f0 0x7efe781973b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efe6800d770 tx=0x7efe6800dc30 comp rx=0 tx=0).stop 2024-05-21T15:23:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 -- 172.21.15.78:0/3479645566 shutdown_connections 2024-05-21T15:23:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe78105930 0x7efe781978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efe640a2d80 0x7efe640a5290 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe781063f0 0x7efe78197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 --2- 172.21.15.78:0/3479645566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe781013f0 0x7efe781973b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 -- 172.21.15.78:0/3479645566 >> 172.21.15.78:0/3479645566 conn(0x7efe78076830 msgr2=0x7efe780fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 -- 172.21.15.78:0/3479645566 shutdown_connections 2024-05-21T15:23:54.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.843+0000 7efe7e4ab640 1 -- 172.21.15.78:0/3479645566 wait complete. 2024-05-21T15:23:54.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:23:54.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.971+0000 7f7029b9d640 1 -- 172.21.15.78:0/3577271874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7024100ea0 msgr2=0x7f70241012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.971+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3577271874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7024100ea0 0x7f70241012b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7010009900 tx=0x7f701002a410 comp rx=0 tx=0).stop 2024-05-21T15:23:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.971+0000 7f7029b9d640 1 -- 172.21.15.78:0/3577271874 shutdown_connections 2024-05-21T15:23:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.971+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3577271874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f702410d260 0x7f702410f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.971+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3577271874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7024100ea0 0x7f70241012b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.971+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3577271874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7024108960 0x7f7024100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.971+0000 7f7029b9d640 1 -- 172.21.15.78:0/3577271874 >> 172.21.15.78:0/3577271874 conn(0x7f7024076730 msgr2=0x7f7024076b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 -- 172.21.15.78:0/3577271874 shutdown_connections 2024-05-21T15:23:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 -- 172.21.15.78:0/3577271874 wait complete. 2024-05-21T15:23:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 Processor -- start 2024-05-21T15:23:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 -- start start 2024-05-21T15:23:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7024100ea0 0x7f7024199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7024108960 0x7f7024199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f702410d260 0x7f702419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7024112c60 con 0x7f702410d260 2024-05-21T15:23:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7024112ae0 con 0x7f7024100ea0 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.972+0000 7f7029b9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7024112de0 con 0x7f7024108960 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f702410d260 0x7f702419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f702410d260 0x7f702419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50280/0 (socket says 172.21.15.78:50280) 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 -- 172.21.15.78:0/3688730574 learned_addr learned my addr 172.21.15.78:0/3688730574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f7023fff640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7024108960 0x7f7024199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f7028b9b640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7024100ea0 0x7f7024199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 -- 172.21.15.78:0/3688730574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7024108960 msgr2=0x7f7024199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7024108960 0x7f7024199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 -- 172.21.15.78:0/3688730574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7024100ea0 msgr2=0x7f7024199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7024100ea0 0x7f7024199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 -- 172.21.15.78:0/3688730574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702419e4a0 con 0x7f702410d260 2024-05-21T15:23:54.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f7023fff640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7024108960 0x7f7024199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:54.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f702939c640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f702410d260 0x7f702419de60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f701c00eaa0 tx=0x7f701c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f7021ffb640 1 -- 172.21.15.78:0/3688730574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c00cdb0 con 0x7f702410d260 2024-05-21T15:23:54.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f7021ffb640 1 -- 172.21.15.78:0/3688730574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701c005020 con 0x7f702410d260 2024-05-21T15:23:54.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f7029b9d640 1 -- 172.21.15.78:0/3688730574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702419e790 con 0x7f702410d260 2024-05-21T15:23:54.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f7021ffb640 1 -- 172.21.15.78:0/3688730574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0116b0 con 0x7f702410d260 2024-05-21T15:23:54.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.973+0000 7f7029b9d640 1 -- 172.21.15.78:0/3688730574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70241a6110 con 0x7f702410d260 2024-05-21T15:23:54.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7021ffb640 1 -- 172.21.15.78:0/3688730574 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f701c005560 con 0x7f702410d260 2024-05-21T15:23:54.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7031d7c740 1 Processor -- start 2024-05-21T15:23:54.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7031d7c740 1 -- start start 2024-05-21T15:23:54.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6eab390 0x555af7249f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af724a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af724a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7031d7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555af71afc40 con 0x555af6e824d0 2024-05-21T15:23:54.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7031d7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555af7157580 con 0x555af6eab390 2024-05-21T15:23:54.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7031d7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555af71abc00 con 0x555af6e8a6b0 2024-05-21T15:23:54.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.976+0000 7f7021ffb640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70140a2d70 0x7f70140a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af724a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af724a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50284/0 (socket says 172.21.15.78:50284) 2024-05-21T15:23:54.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 -- 172.21.15.78:0/3280217037 learned_addr learned my addr 172.21.15.78:0/3280217037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f702939c640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70140a2d70 0x7f70140a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7021ffb640 1 -- 172.21.15.78:0/3688730574 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f701c0cf710 con 0x7f702410d260 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7023fff640 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6eab390 0x555af7249f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7023fff640 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af724a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 -- 172.21.15.78:0/3280217037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 msgr2=0x555af724a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af724a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 -- 172.21.15.78:0/3280217037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6eab390 msgr2=0x555af7249f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6eab390 0x555af7249f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 -- 172.21.15.78:0/3280217037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af6ec33f0 con 0x555af6e824d0 2024-05-21T15:23:54.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7023fff640 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af724a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f702939c640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70140a2d70 0x7f70140a5280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f701c098570 tx=0x7f701c04b040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f7028b9b640 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af724a4a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7018009c30 tx=0x7f70180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937ea61a719df5ff server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f700effd640 1 -- 172.21.15.78:0/3280217037 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018012450 con 0x555af6e824d0 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f700effd640 1 -- 172.21.15.78:0/3280217037 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7018012ad0 con 0x555af6e824d0 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.977+0000 7f700effd640 1 -- 172.21.15.78:0/3280217037 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701800fbf0 con 0x555af6e824d0 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 -- 172.21.15.78:0/3280217037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 msgr2=0x555af724a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af724a4a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7018009c30 tx=0x7f70180117b0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 -- 172.21.15.78:0/3280217037 shutdown_connections 2024-05-21T15:23:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af724a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af724a4a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3280217037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6eab390 0x555af7249f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 -- 172.21.15.78:0/3280217037 >> 172.21.15.78:0/3280217037 conn(0x555af6e9f4e0 msgr2=0x555af7246710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 -- 172.21.15.78:0/3280217037 shutdown_connections 2024-05-21T15:23:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.978+0000 7f7031d7c740 1 -- 172.21.15.78:0/3280217037 wait complete. 2024-05-21T15:23:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 1 -- start start 2024-05-21T15:23:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 1 Processor -- start 2024-05-21T15:23:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af72545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e8a6b0 0x555af7254b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 0x555af725a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555af72572b0 con 0x555af6e8a6b0 2024-05-21T15:23:54.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555af716ac40 con 0x555af6e824d0 2024-05-21T15:23:54.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555af7257430 con 0x555af6eab390 2024-05-21T15:23:54.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7031d7c740 10 client.0 _ll_register_callbacks cb 0x555af6ec2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:54.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7028b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e8a6b0 0x555af7254b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7028b9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e8a6b0 0x555af7254b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50296/0 (socket says 172.21.15.78:50296) 2024-05-21T15:23:54.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7028b9b640 1 -- 172.21.15.78:0/3818102235 learned_addr learned my addr 172.21.15.78:0/3818102235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:54.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7023fff640 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af72545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.979+0000 7f7023fff640 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 0x555af725a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7028b9b640 1 -- 172.21.15.78:0/3818102235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 msgr2=0x555af725a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7028b9b640 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 0x555af725a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7028b9b640 1 -- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 msgr2=0x555af72545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:54.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7028b9b640 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af72545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7028b9b640 1 -- 172.21.15.78:0/3818102235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af6f8d7d0 con 0x555af6e8a6b0 2024-05-21T15:23:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7023fff640 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af72545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:23:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7028b9b640 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e8a6b0 0x555af7254b00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f701801c040 tx=0x7f7018012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2166edcfd372920a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7003fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7003fff640 1 -- 172.21.15.78:0/3818102235 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018028070 con 0x555af6e8a6b0 2024-05-21T15:23:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7003fff640 1 -- 172.21.15.78:0/3818102235 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f701800f930 con 0x555af6e8a6b0 2024-05-21T15:23:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7031d7c740 1 -- 172.21.15.78:0/3818102235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7018012ad0 con 0x555af6e8a6b0 2024-05-21T15:23:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7003fff640 1 -- 172.21.15.78:0/3818102235 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7018034050 con 0x555af6e8a6b0 2024-05-21T15:23:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7031d7c740 10 client.17853 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7031d7c740 1 -- 172.21.15.78:0/3818102235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555af7263970 con 0x555af6e8a6b0 2024-05-21T15:23:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7003fff640 1 -- 172.21.15.78:0/3818102235 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7018012ad0 con 0x555af6e8a6b0 2024-05-21T15:23:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7003fff640 1 -- 172.21.15.78:0/3818102235 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f70180046c0 con 0x555af6e8a6b0 2024-05-21T15:23:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.980+0000 7f7031d7c740 10 client.17853 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.981+0000 7f7031d7c740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.981+0000 7f7031d7c740 20 client.17853 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.981+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555af6e69f60 0x555af7266d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:54.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.981+0000 7f7031d7c740 4 client.17853 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:54.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.981+0000 7f7031d7c740 1 -- 172.21.15.78:0/3818102235 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555af71bd370 con 0x555af6e69f60 2024-05-21T15:23:54.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.981+0000 7f7023fff640 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555af6e69f60 0x555af7266d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:54.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.981+0000 7f7003fff640 10 client.17853 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:54.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.981+0000 7f7023fff640 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555af6e69f60 0x555af7266d30 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802737271c913545 server_cookie=7d6d7d6648d37c08 in_seq=0 out_seq=0 2024-05-21T15:23:54.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7003fff640 1 -- 172.21.15.78:0/3818102235 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555af71bd370 con 0x555af6e69f60 2024-05-21T15:23:54.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7003fff640 10 client.17853 handle_command_reply: tid=0 2024-05-21T15:23:54.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 client.17853 shutdown 2024-05-21T15:23:54.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 -- 172.21.15.78:0/3818102235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e8a6b0 msgr2=0x555af7254b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:54.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e8a6b0 0x555af7254b00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f701801c040 tx=0x7f7018012f30 comp rx=0 tx=0).stop 2024-05-21T15:23:54.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 -- 172.21.15.78:0/3818102235 shutdown_connections 2024-05-21T15:23:54.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555af6e69f60 0x555af7266d30 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 0x555af725a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e8a6b0 0x555af7254b00 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7003fff640 0 client.17853 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:54.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 --2- 172.21.15.78:0/3818102235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af72545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:54.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.982+0000 7f7031d7c740 1 -- 172.21.15.78:0/3818102235 >> 172.21.15.78:0/3818102235 conn(0x555af6e9f4e0 msgr2=0x555af724bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:54.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.983+0000 7f7031d7c740 1 -- 172.21.15.78:0/3818102235 shutdown_connections 2024-05-21T15:23:54.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.983+0000 7f7031d7c740 1 -- 172.21.15.78:0/3818102235 wait complete. 2024-05-21T15:23:54.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.983+0000 7f7031d7c740 20 client.17853 trim_cache size 0 max 16384 2024-05-21T15:23:55.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7031d7c740 1 Processor -- start 2024-05-21T15:23:55.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7031d7c740 1 -- start start 2024-05-21T15:23:55.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af725a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:55.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e8a6b0 0x555af7253390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:55.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 0x555af724dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:55.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7031d7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70241aeb60 con 0x555af6e824d0 2024-05-21T15:23:55.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7031d7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70241b06a0 con 0x555af6e8a6b0 2024-05-21T15:23:55.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7031d7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70241af230 con 0x555af6eab390 2024-05-21T15:23:55.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7023fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af725a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:55.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7023fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af725a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50298/0 (socket says 172.21.15.78:50298) 2024-05-21T15:23:55.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.996+0000 7f7023fff640 1 -- 172.21.15.78:0/250550439 learned_addr learned my addr 172.21.15.78:0/250550439 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:55.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f7023fff640 1 -- 172.21.15.78:0/250550439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 msgr2=0x555af724dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:55.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f7023fff640 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 0x555af724dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f7023fff640 1 -- 172.21.15.78:0/250550439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e8a6b0 msgr2=0x555af7253390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:55.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f7028b9b640 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e8a6b0 0x555af7253390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:55.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f7023fff640 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e8a6b0 0x555af7253390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f7023fff640 1 -- 172.21.15.78:0/250550439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af7252900 con 0x555af6e824d0 2024-05-21T15:23:55.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f7028b9b640 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e8a6b0 0x555af7253390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:55.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f7023fff640 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af725a6a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7010005f70 tx=0x7f7010031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee19e1ec921b1a3b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:55.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f700e7fc640 1 -- 172.21.15.78:0/250550439 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701004e020 con 0x555af6e824d0 2024-05-21T15:23:55.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f700e7fc640 1 -- 172.21.15.78:0/250550439 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7010031990 con 0x555af6e824d0 2024-05-21T15:23:55.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.997+0000 7f700e7fc640 1 -- 172.21.15.78:0/250550439 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010035040 con 0x555af6e824d0 2024-05-21T15:23:55.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 -- 172.21.15.78:0/250550439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 msgr2=0x555af725a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:55.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af725a6a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7010005f70 tx=0x7f7010031500 comp rx=0 tx=0).stop 2024-05-21T15:23:55.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 -- 172.21.15.78:0/250550439 shutdown_connections 2024-05-21T15:23:55.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e8a6b0 0x555af7253390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6eab390 0x555af724dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 --2- 172.21.15.78:0/250550439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6e824d0 0x555af725a6a0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 -- 172.21.15.78:0/250550439 >> 172.21.15.78:0/250550439 conn(0x555af6e9f4e0 msgr2=0x555af7252c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 -- 172.21.15.78:0/250550439 shutdown_connections 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 -- 172.21.15.78:0/250550439 wait complete. 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 -- start start 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.998+0000 7f7031d7c740 1 Processor -- start 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af7260950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:55.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af7260e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6eab390 0x555af725de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70241affd0 con 0x555af6eab390 2024-05-21T15:23:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70241af900 con 0x555af6e824d0 2024-05-21T15:23:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70241acb60 con 0x555af6e8a6b0 2024-05-21T15:23:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 10 client.0 _ll_register_callbacks cb 0x7f7018009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6eab390 0x555af725de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6eab390 0x555af725de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50314/0 (socket says 172.21.15.78:50314) 2024-05-21T15:23:55.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 -- 172.21.15.78:0/877623764 learned_addr learned my addr 172.21.15.78:0/877623764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af7260950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7028b9b640 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af7260e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 -- 172.21.15.78:0/877623764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 msgr2=0x555af7260e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af7260e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 -- 172.21.15.78:0/877623764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 msgr2=0x555af7260950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af7260950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 -- 172.21.15.78:0/877623764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af725e590 con 0x555af6eab390 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7023fff640 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6eab390 0x555af725de20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7010006010 tx=0x7f7010005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66f02a5aa74d05f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7003fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:55.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7003fff640 1 -- 172.21.15.78:0/877623764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010035070 con 0x555af6eab390 2024-05-21T15:23:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7003fff640 1 -- 172.21.15.78:0/877623764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f7010004380 con 0x555af6eab390 2024-05-21T15:23:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 1 -- 172.21.15.78:0/877623764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7010031990 con 0x555af6eab390 2024-05-21T15:23:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7003fff640 1 -- 172.21.15.78:0/877623764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7010064050 con 0x555af6eab390 2024-05-21T15:23:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 10 client.17859 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:54.999+0000 7f7031d7c740 1 -- 172.21.15.78:0/877623764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555af72ce910 con 0x555af6eab390 2024-05-21T15:23:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7003fff640 1 -- 172.21.15.78:0/877623764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7010031990 con 0x555af6eab390 2024-05-21T15:23:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7003fff640 1 -- 172.21.15.78:0/877623764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7018009d20 con 0x555af6eab390 2024-05-21T15:23:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7031d7c740 10 client.17859 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7031d7c740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7031d7c740 20 client.17859 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7031d7c740 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555af6e3f3a0 0x555af72d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7031d7c740 4 client.17859 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7031d7c740 1 -- 172.21.15.78:0/877623764 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555af71b76c0 con 0x555af6e3f3a0 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.000+0000 7f7028b9b640 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555af6e3f3a0 0x555af72d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7003fff640 10 client.17859 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7028b9b640 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555af6e3f3a0 0x555af72d0fc0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81504c57f84927fa server_cookie=46e6f3f72d9b39b9 in_seq=0 out_seq=0 2024-05-21T15:23:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7003fff640 1 -- 172.21.15.78:0/877623764 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555af71b76c0 con 0x555af6e3f3a0 2024-05-21T15:23:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7003fff640 10 client.17859 handle_command_reply: tid=0 2024-05-21T15:23:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 client.17859 shutdown 2024-05-21T15:23:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 -- 172.21.15.78:0/877623764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6eab390 msgr2=0x555af725de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6eab390 0x555af725de20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7010006010 tx=0x7f7010005500 comp rx=0 tx=0).stop 2024-05-21T15:23:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 -- 172.21.15.78:0/877623764 shutdown_connections 2024-05-21T15:23:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x555af6e3f3a0 0x555af72d0fc0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555af6eab390 0x555af725de20 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555af6e8a6b0 0x555af7260e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7003fff640 0 client.17859 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 --2- 172.21.15.78:0/877623764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555af6e824d0 0x555af7260950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 -- 172.21.15.78:0/877623764 >> 172.21.15.78:0/877623764 conn(0x555af6e9f4e0 msgr2=0x555af7249cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 -- 172.21.15.78:0/877623764 shutdown_connections 2024-05-21T15:23:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 1 -- 172.21.15.78:0/877623764 wait complete. 2024-05-21T15:23:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.001+0000 7f7031d7c740 20 client.17859 trim_cache size 0 max 16384 2024-05-21T15:23:55.017 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:23:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 -- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70140a2d70 msgr2=0x7f70140a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70140a2d70 0x7f70140a5280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f701c098570 tx=0x7f701c04b040 comp rx=0 tx=0).stop 2024-05-21T15:23:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 -- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f702410d260 msgr2=0x7f702419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f702410d260 0x7f702419de60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f701c00eaa0 tx=0x7f701c00ef60 comp rx=0 tx=0).stop 2024-05-21T15:23:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 -- 172.21.15.78:0/3688730574 shutdown_connections 2024-05-21T15:23:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f70140a2d70 0x7f70140a5280 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f702410d260 0x7f702419de60 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7024108960 0x7f7024199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 --2- 172.21.15.78:0/3688730574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7024100ea0 0x7f7024199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 -- 172.21.15.78:0/3688730574 >> 172.21.15.78:0/3688730574 conn(0x7f7024076730 msgr2=0x7f70241091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 -- 172.21.15.78:0/3688730574 shutdown_connections 2024-05-21T15:23:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:55.002+0000 7f7029b9d640 1 -- 172.21.15.78:0/3688730574 wait complete. 2024-05-21T15:23:55.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:54 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/728242117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:23:55.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:54 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/728242117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:23:55.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:54 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/728242117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:23:56.021 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:23:56.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:23:56.022 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:23:56.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.146+0000 7f5395a7a640 1 -- 172.21.15.78:0/566932632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53901016d0 msgr2=0x7f539010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.146+0000 7f5395a7a640 1 --2- 172.21.15.78:0/566932632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53901016d0 0x7f539010f520 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5388009ca0 tx=0x7f538802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 -- 172.21.15.78:0/566932632 shutdown_connections 2024-05-21T15:23:56.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 --2- 172.21.15.78:0/566932632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53901016d0 0x7f539010f520 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 --2- 172.21.15.78:0/566932632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5390108f40 0x7f5390101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 --2- 172.21.15.78:0/566932632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5390108aa0 0x7f5390100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 -- 172.21.15.78:0/566932632 >> 172.21.15.78:0/566932632 conn(0x7f5390076830 msgr2=0x7f5390076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 -- 172.21.15.78:0/566932632 shutdown_connections 2024-05-21T15:23:56.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 -- 172.21.15.78:0/566932632 wait complete. 2024-05-21T15:23:56.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 Processor -- start 2024-05-21T15:23:56.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.147+0000 7f5395a7a640 1 -- start start 2024-05-21T15:23:56.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f5395a7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53901016d0 0x7f5390199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f5395a7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5390108aa0 0x7f5390199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f5395a7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5390108f40 0x7f539019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f5395a7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5390112de0 con 0x7f5390108aa0 2024-05-21T15:23:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f5395a7a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5390112c60 con 0x7f5390108f40 2024-05-21T15:23:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f5395a7a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5390112f60 con 0x7f53901016d0 2024-05-21T15:23:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5390108aa0 0x7f5390199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5390108aa0 0x7f5390199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50338/0 (socket says 172.21.15.78:50338) 2024-05-21T15:23:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 -- 172.21.15.78:0/2667967158 learned_addr learned my addr 172.21.15.78:0/2667967158 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538ffff640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5390108f40 0x7f539019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538f7fe640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53901016d0 0x7f5390199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 -- 172.21.15.78:0/2667967158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53901016d0 msgr2=0x7f5390199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53901016d0 0x7f5390199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 -- 172.21.15.78:0/2667967158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5390108f40 msgr2=0x7f539019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5390108f40 0x7f539019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 -- 172.21.15.78:0/2667967158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539019e5a0 con 0x7f5390108aa0 2024-05-21T15:23:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538effd640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5390108aa0 0x7f5390199ca0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f538400e9f0 tx=0x7f538400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538cff9640 1 -- 172.21.15.78:0/2667967158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538400ccc0 con 0x7f5390108aa0 2024-05-21T15:23:56.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f538cff9640 1 -- 172.21.15.78:0/2667967158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5384005500 con 0x7f5390108aa0 2024-05-21T15:23:56.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.148+0000 7f5395a7a640 1 -- 172.21.15.78:0/2667967158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539019e890 con 0x7f5390108aa0 2024-05-21T15:23:56.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.149+0000 7f538cff9640 1 -- 172.21.15.78:0/2667967158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384010690 con 0x7f5390108aa0 2024-05-21T15:23:56.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.149+0000 7f5395a7a640 1 -- 172.21.15.78:0/2667967158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a6210 con 0x7f5390108aa0 2024-05-21T15:23:56.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.151+0000 7f538cff9640 1 -- 172.21.15.78:0/2667967158 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f53840046c0 con 0x7f5390108aa0 2024-05-21T15:23:56.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.151+0000 7f539ca04740 1 Processor -- start 2024-05-21T15:23:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.151+0000 7f539ca04740 1 -- start start 2024-05-21T15:23:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.151+0000 7f538cff9640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f53780a2d80 0x7f53780a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f539ca04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89769410 0x55be89b08620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f539ca04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538ffff640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f53780a2d80 0x7f53780a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f539ca04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f539ca04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be89a6d990 con 0x55be89740550 2024-05-21T15:23:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f539ca04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55be89a155b0 con 0x55be89769410 2024-05-21T15:23:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f539ca04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55be89a69950 con 0x55be89748730 2024-05-21T15:23:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538cff9640 1 -- 172.21.15.78:0/2667967158 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f53840cf860 con 0x7f5390108aa0 2024-05-21T15:23:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50354/0 (socket says 172.21.15.78:50354) 2024-05-21T15:23:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 -- 172.21.15.78:0/3144556345 learned_addr learned my addr 172.21.15.78:0/3144556345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538f7fe640 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89769410 0x55be89b08620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538ffff640 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 -- 172.21.15.78:0/3144556345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 msgr2=0x55be89b090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 -- 172.21.15.78:0/3144556345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89769410 msgr2=0x55be89b08620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89769410 0x55be89b08620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 -- 172.21.15.78:0/3144556345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be898497e0 con 0x55be89740550 2024-05-21T15:23:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538ffff640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f53780a2d80 0x7f53780a5290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5388005e30 tx=0x7f5388009850 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f538effd640 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08b60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f53840986c0 tx=0x7f538409b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b252e5137731f63e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f5375ffb640 1 -- 172.21.15.78:0/3144556345 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53840a00a0 con 0x55be89740550 2024-05-21T15:23:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f5375ffb640 1 -- 172.21.15.78:0/3144556345 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53840a4960 con 0x55be89740550 2024-05-21T15:23:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.152+0000 7f5375ffb640 1 -- 172.21.15.78:0/3144556345 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53840a4760 con 0x55be89740550 2024-05-21T15:23:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 -- 172.21.15.78:0/3144556345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 msgr2=0x55be89b08b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08b60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f53840986c0 tx=0x7f538409b8f0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 -- 172.21.15.78:0/3144556345 shutdown_connections 2024-05-21T15:23:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08b60 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 --2- 172.21.15.78:0/3144556345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89769410 0x55be89b08620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 -- 172.21.15.78:0/3144556345 >> 172.21.15.78:0/3144556345 conn(0x55be8975d560 msgr2=0x55be89b04dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 -- 172.21.15.78:0/3144556345 shutdown_connections 2024-05-21T15:23:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.153+0000 7f539ca04740 1 -- 172.21.15.78:0/3144556345 wait complete. 2024-05-21T15:23:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 1 -- start start 2024-05-21T15:23:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 1 Processor -- start 2024-05-21T15:23:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89740550 0x55be89b14680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b14be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89769410 0x55be89b12b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be89b15910 con 0x55be89769410 2024-05-21T15:23:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55be89a28b40 con 0x55be89740550 2024-05-21T15:23:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55be89b15a90 con 0x55be89748730 2024-05-21T15:23:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f539ca04740 10 client.0 _ll_register_callbacks cb 0x55be89a1d0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f538ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89769410 0x55be89b12b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f538ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89769410 0x55be89b12b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50358/0 (socket says 172.21.15.78:50358) 2024-05-21T15:23:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.154+0000 7f538ffff640 1 -- 172.21.15.78:0/2436268468 learned_addr learned my addr 172.21.15.78:0/2436268468 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538f7fe640 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89740550 0x55be89b14680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538effd640 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b14be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538ffff640 1 -- 172.21.15.78:0/2436268468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 msgr2=0x55be89b14be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538ffff640 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b14be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538ffff640 1 -- 172.21.15.78:0/2436268468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89740550 msgr2=0x55be89b14680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538ffff640 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89740550 0x55be89b14680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538ffff640 1 -- 172.21.15.78:0/2436268468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be898a9f70 con 0x55be89769410 2024-05-21T15:23:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538effd640 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b14be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f538ffff640 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89769410 0x55be89b12b40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5388004af0 tx=0x7f538802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd066366e8b93fdb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f536effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f536effd640 1 -- 172.21.15.78:0/2436268468 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388042040 con 0x55be89769410 2024-05-21T15:23:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f536effd640 1 -- 172.21.15.78:0/2436268468 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f5388004450 con 0x55be89769410 2024-05-21T15:23:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f539ca04740 1 -- 172.21.15.78:0/2436268468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53840a4960 con 0x55be89769410 2024-05-21T15:23:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f536effd640 1 -- 172.21.15.78:0/2436268468 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5388004650 con 0x55be89769410 2024-05-21T15:23:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f539ca04740 10 client.17871 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:56.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f539ca04740 1 -- 172.21.15.78:0/2436268468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55be89741170 con 0x55be89769410 2024-05-21T15:23:56.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.155+0000 7f536effd640 1 -- 172.21.15.78:0/2436268468 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53840a4960 con 0x55be89769410 2024-05-21T15:23:56.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f536effd640 1 -- 172.21.15.78:0/2436268468 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f53880306e0 con 0x55be89769410 2024-05-21T15:23:56.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f539ca04740 10 client.17871 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:56.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f539ca04740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:56.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f539ca04740 20 client.17871 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:56.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f539ca04740 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be897281f0 0x55be89b25110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f539ca04740 4 client.17871 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:56.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f539ca04740 1 -- 172.21.15.78:0/2436268468 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be89a7b0c0 con 0x55be897281f0 2024-05-21T15:23:56.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f538f7fe640 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be897281f0 0x55be89b25110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f536effd640 10 client.17871 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:56.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.156+0000 7f538f7fe640 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be897281f0 0x55be89b25110 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91425c4a726b2d94 server_cookie=82420ef774c3098d in_seq=0 out_seq=0 2024-05-21T15:23:56.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.157+0000 7f536effd640 1 -- 172.21.15.78:0/2436268468 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be89a7b0c0 con 0x55be897281f0 2024-05-21T15:23:56.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.157+0000 7f536effd640 10 client.17871 handle_command_reply: tid=0 2024-05-21T15:23:56.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.157+0000 7f539ca04740 1 client.17871 shutdown 2024-05-21T15:23:56.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 -- 172.21.15.78:0/2436268468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89769410 msgr2=0x55be89b12b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89769410 0x55be89b12b40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5388004af0 tx=0x7f538802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 -- 172.21.15.78:0/2436268468 shutdown_connections 2024-05-21T15:23:56.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be897281f0 0x55be89b25110 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89769410 0x55be89b12b40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b14be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f536effd640 0 client.17871 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:56.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 --2- 172.21.15.78:0/2436268468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be89740550 0x55be89b14680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 -- 172.21.15.78:0/2436268468 >> 172.21.15.78:0/2436268468 conn(0x55be8975d560 msgr2=0x55be89b0a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 -- 172.21.15.78:0/2436268468 shutdown_connections 2024-05-21T15:23:56.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 1 -- 172.21.15.78:0/2436268468 wait complete. 2024-05-21T15:23:56.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.158+0000 7f539ca04740 20 client.17871 trim_cache size 0 max 16384 2024-05-21T15:23:56.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f539ca04740 1 Processor -- start 2024-05-21T15:23:56.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f539ca04740 1 -- start start 2024-05-21T15:23:56.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f539ca04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 0x55be89b1b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f539ca04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f539ca04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b0c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f539ca04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53901aec40 con 0x55be89740550 2024-05-21T15:23:56.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f539ca04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b0780 con 0x55be897281f0 2024-05-21T15:23:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f539ca04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53901af310 con 0x55be89748730 2024-05-21T15:23:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50372/0 (socket says 172.21.15.78:50372) 2024-05-21T15:23:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538effd640 1 -- 172.21.15.78:0/2068167241 learned_addr learned my addr 172.21.15.78:0/2068167241 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538ffff640 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b0c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538effd640 1 -- 172.21.15.78:0/2068167241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 msgr2=0x55be89b0c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538effd640 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b0c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538effd640 1 -- 172.21.15.78:0/2068167241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 msgr2=0x55be89b1b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538effd640 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 0x55be89b1b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538effd640 1 -- 172.21.15.78:0/2068167241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be89b083b0 con 0x55be89740550 2024-05-21T15:23:56.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.176+0000 7f538ffff640 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b0c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:56.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f538effd640 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08cb0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5384098150 tx=0x7f538409bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463965472b6bea27 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f53757fa640 1 -- 172.21.15.78:0/2068167241 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538407f070 con 0x55be89740550 2024-05-21T15:23:56.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f53757fa640 1 -- 172.21.15.78:0/2068167241 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53840a4960 con 0x55be89740550 2024-05-21T15:23:56.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f53757fa640 1 -- 172.21.15.78:0/2068167241 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538409b140 con 0x55be89740550 2024-05-21T15:23:56.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f539ca04740 1 -- 172.21.15.78:0/2068167241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 msgr2=0x55be89b08cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f539ca04740 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08cb0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5384098150 tx=0x7f538409bb70 comp rx=0 tx=0).stop 2024-05-21T15:23:56.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f539ca04740 1 -- 172.21.15.78:0/2068167241 shutdown_connections 2024-05-21T15:23:56.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f539ca04740 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89748730 0x55be89b0c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f539ca04740 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89740550 0x55be89b08cb0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f539ca04740 1 --2- 172.21.15.78:0/2068167241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 0x55be89b1b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.177+0000 7f539ca04740 1 -- 172.21.15.78:0/2068167241 >> 172.21.15.78:0/2068167241 conn(0x55be8975d560 msgr2=0x55be89b085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 -- 172.21.15.78:0/2068167241 shutdown_connections 2024-05-21T15:23:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 -- 172.21.15.78:0/2068167241 wait complete. 2024-05-21T15:23:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 -- start start 2024-05-21T15:23:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 Processor -- start 2024-05-21T15:23:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 0x55be89b85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89740550 0x55be89b85980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89748730 0x55be89b904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53901b00b0 con 0x55be89748730 2024-05-21T15:23:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f53901af9e0 con 0x55be897281f0 2024-05-21T15:23:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53901acc40 con 0x55be89740550 2024-05-21T15:23:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.178+0000 7f539ca04740 10 client.0 _ll_register_callbacks cb 0x7f5384005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89748730 0x55be89b904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89748730 0x55be89b904c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50382/0 (socket says 172.21.15.78:50382) 2024-05-21T15:23:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 -- 172.21.15.78:0/2051675823 learned_addr learned my addr 172.21.15.78:0/2051675823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538effd640 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89740550 0x55be89b85980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 -- 172.21.15.78:0/2051675823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89740550 msgr2=0x55be89b85980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89740550 0x55be89b85980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538f7fe640 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 0x55be89b85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 -- 172.21.15.78:0/2051675823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 msgr2=0x55be89b85440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 0x55be89b85440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 -- 172.21.15.78:0/2051675823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be89b86250 con 0x55be89748730 2024-05-21T15:23:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f538ffff640 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89748730 0x55be89b904c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f538802ade0 tx=0x7f5388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8c4eb3cad8416a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f536effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f536effd640 1 -- 172.21.15.78:0/2051675823 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388042070 con 0x55be89748730 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f536effd640 1 -- 172.21.15.78:0/2051675823 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f53880307c0 con 0x55be89748730 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f539ca04740 1 -- 172.21.15.78:0/2051675823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53840a4960 con 0x55be89748730 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f536effd640 1 -- 172.21.15.78:0/2051675823 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5388004460 con 0x55be89748730 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f539ca04740 10 client.17877 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f539ca04740 1 -- 172.21.15.78:0/2051675823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55be89b91a40 con 0x55be89748730 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.179+0000 7f536effd640 1 -- 172.21.15.78:0/2051675823 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53840a4960 con 0x55be89748730 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f536effd640 1 -- 172.21.15.78:0/2051675823 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5388053670 con 0x55be89748730 2024-05-21T15:23:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f539ca04740 10 client.17877 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f539ca04740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f539ca04740 20 client.17877 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f539ca04740 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be89769410 0x55be89b94280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f539ca04740 4 client.17877 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:23:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f539ca04740 1 -- 172.21.15.78:0/2051675823 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55be89a75410 con 0x55be89769410 2024-05-21T15:23:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f538f7fe640 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be89769410 0x55be89b94280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f536effd640 10 client.17877 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.180+0000 7f538f7fe640 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be89769410 0x55be89b94280 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c8dc5bc2d53710 server_cookie=538cf5e74895e7b8 in_seq=0 out_seq=0 2024-05-21T15:23:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f536effd640 1 -- 172.21.15.78:0/2051675823 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55be89a75410 con 0x55be89769410 2024-05-21T15:23:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f536effd640 10 client.17877 handle_command_reply: tid=0 2024-05-21T15:23:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 client.17877 shutdown 2024-05-21T15:23:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 -- 172.21.15.78:0/2051675823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89748730 msgr2=0x55be89b904c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89748730 0x55be89b904c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f538802ade0 tx=0x7f5388009bb0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 -- 172.21.15.78:0/2051675823 shutdown_connections 2024-05-21T15:23:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55be89769410 0x55be89b94280 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f536effd640 0 client.17877 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be89748730 0x55be89b904c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89740550 0x55be89b85980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 --2- 172.21.15.78:0/2051675823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55be897281f0 0x55be89b85440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 -- 172.21.15.78:0/2051675823 >> 172.21.15.78:0/2051675823 conn(0x55be8975d560 msgr2=0x55be89b0a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 -- 172.21.15.78:0/2051675823 shutdown_connections 2024-05-21T15:23:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 1 -- 172.21.15.78:0/2051675823 wait complete. 2024-05-21T15:23:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.182+0000 7f539ca04740 20 client.17877 trim_cache size 0 max 16384 2024-05-21T15:23:56.195 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:23:56.195 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:23:56.195 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "0042ace1-0264-456f-8dad-a9a2e973c501", 2024-05-21T15:23:56.195 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:23:56.195 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:23:56.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 -- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f53780a2d80 msgr2=0x7f53780a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f53780a2d80 0x7f53780a5290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5388005e30 tx=0x7f5388009850 comp rx=0 tx=0).stop 2024-05-21T15:23:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 -- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5390108aa0 msgr2=0x7f5390199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5390108aa0 0x7f5390199ca0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f538400e9f0 tx=0x7f538400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 -- 172.21.15.78:0/2667967158 shutdown_connections 2024-05-21T15:23:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5390108f40 0x7f539019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f53780a2d80 0x7f53780a5290 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5390108aa0 0x7f5390199ca0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 --2- 172.21.15.78:0/2667967158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53901016d0 0x7f5390199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 -- 172.21.15.78:0/2667967158 >> 172.21.15.78:0/2667967158 conn(0x7f5390076830 msgr2=0x7f53900ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 -- 172.21.15.78:0/2667967158 shutdown_connections 2024-05-21T15:23:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.183+0000 7f5395a7a640 1 -- 172.21.15.78:0/2667967158 wait complete. 2024-05-21T15:23:56.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:23:56.200 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:23:56.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:55 smithi078 ceph-mon[31997]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:56.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 -- 172.21.15.78:0/1591710067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b181079d0 msgr2=0x7f5b18101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 --2- 172.21.15.78:0/1591710067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b181079d0 0x7f5b18101560 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009ca0 tx=0x7f5b1002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 -- 172.21.15.78:0/1591710067 shutdown_connections 2024-05-21T15:23:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 --2- 172.21.15.78:0/1591710067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b1810c330 0x7f5b1810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 --2- 172.21.15.78:0/1591710067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b18101aa0 0x7f5b18101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 --2- 172.21.15.78:0/1591710067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b181079d0 0x7f5b18101560 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 -- 172.21.15.78:0/1591710067 >> 172.21.15.78:0/1591710067 conn(0x7f5b180770b0 msgr2=0x7f5b180774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 -- 172.21.15.78:0/1591710067 shutdown_connections 2024-05-21T15:23:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.313+0000 7f5b20338640 1 -- 172.21.15.78:0/1591710067 wait complete. 2024-05-21T15:23:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b20338640 1 Processor -- start 2024-05-21T15:23:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b20338640 1 -- start start 2024-05-21T15:23:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b20338640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b18101aa0 0x7f5b18198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b20338640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b181079d0 0x7f5b18198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b20338640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1810c330 0x7f5b1819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b20338640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18111ce0 con 0x7f5b1810c330 2024-05-21T15:23:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b20338640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18111b60 con 0x7f5b18101aa0 2024-05-21T15:23:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b20338640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18111e60 con 0x7f5b181079d0 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e8d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1810c330 0x7f5b1819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e8d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1810c330 0x7f5b1819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50406/0 (socket says 172.21.15.78:50406) 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e8d5640 1 -- 172.21.15.78:0/3661361667 learned_addr learned my addr 172.21.15.78:0/3661361667 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b18101aa0 0x7f5b18198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b181079d0 0x7f5b18198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e8d5640 1 -- 172.21.15.78:0/3661361667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b181079d0 msgr2=0x7f5b18198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e8d5640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b181079d0 0x7f5b18198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e8d5640 1 -- 172.21.15.78:0/3661361667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b18101aa0 msgr2=0x7f5b18198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e8d5640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b18101aa0 0x7f5b18198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.314+0000 7f5b1e8d5640 1 -- 172.21.15.78:0/3661361667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1819d580 con 0x7f5b1810c330 2024-05-21T15:23:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.315+0000 7f5b1e8d5640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1810c330 0x7f5b1819cf40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800eb00 tx=0x7f5b0800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.315+0000 7f5b177fe640 1 -- 172.21.15.78:0/3661361667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800bbb0 con 0x7f5b1810c330 2024-05-21T15:23:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.315+0000 7f5b177fe640 1 -- 172.21.15.78:0/3661361667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b0800bd50 con 0x7f5b1810c330 2024-05-21T15:23:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.315+0000 7f5b20338640 1 -- 172.21.15.78:0/3661361667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1819d870 con 0x7f5b1810c330 2024-05-21T15:23:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.315+0000 7f5b177fe640 1 -- 172.21.15.78:0/3661361667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08017410 con 0x7f5b1810c330 2024-05-21T15:23:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.315+0000 7f5b20338640 1 -- 172.21.15.78:0/3661361667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1819dd70 con 0x7f5b1810c330 2024-05-21T15:23:56.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.317+0000 7f5b271e2740 1 Processor -- start 2024-05-21T15:23:56.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b177fe640 1 -- 172.21.15.78:0/3661361667 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5b080175b0 con 0x7f5b1810c330 2024-05-21T15:23:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b271e2740 1 -- start start 2024-05-21T15:23:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0f02390 0x5584c12a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12a1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b271e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584c1206d00 con 0x5584c0ee16b0 2024-05-21T15:23:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b271e2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5584c11ae4c0 con 0x5584c0f02390 2024-05-21T15:23:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b271e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5584c1202cc0 con 0x5584c0ed94d0 2024-05-21T15:23:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b1d8d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b1d8d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50410/0 (socket says 172.21.15.78:50410) 2024-05-21T15:23:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/4252912701 learned_addr learned my addr 172.21.15.78:0/4252912701 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.318+0000 7f5b177fe640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b000a2d00 0x7f5b000a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0f02390 0x5584c12a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/4252912701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 msgr2=0x5584c12a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1e8d5640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b000a2d00 0x7f5b000a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/4252912701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0f02390 msgr2=0x5584c12a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0f02390 0x5584c12a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/4252912701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c0fe47d0 con 0x5584c0ee16b0 2024-05-21T15:23:56.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b177fe640 1 -- 172.21.15.78:0/3661361667 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f5b080d0c20 con 0x7f5b1810c330 2024-05-21T15:23:56.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12a17b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c005d00 tx=0x7f5b0c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b46f281b0d0542 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b14bfe640 1 -- 172.21.15.78:0/4252912701 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c005220 con 0x5584c0ee16b0 2024-05-21T15:23:56.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b14bfe640 1 -- 172.21.15.78:0/4252912701 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b0c004c80 con 0x5584c0ee16b0 2024-05-21T15:23:56.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.319+0000 7f5b14bfe640 1 -- 172.21.15.78:0/4252912701 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c010970 con 0x5584c0ee16b0 2024-05-21T15:23:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 -- 172.21.15.78:0/4252912701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 msgr2=0x5584c12a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b1e8d5640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b000a2d00 0x7f5b000a5210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b080ce390 tx=0x7f5b08054040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12a17b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c005d00 tx=0x7f5b0c005c90 comp rx=0 tx=0).stop 2024-05-21T15:23:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 -- 172.21.15.78:0/4252912701 shutdown_connections 2024-05-21T15:23:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12a17b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 --2- 172.21.15.78:0/4252912701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0f02390 0x5584c12a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 -- 172.21.15.78:0/4252912701 >> 172.21.15.78:0/4252912701 conn(0x5584c0ef64e0 msgr2=0x5584c129d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 -- 172.21.15.78:0/4252912701 shutdown_connections 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 -- 172.21.15.78:0/4252912701 wait complete. 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.320+0000 7f5b271e2740 1 -- start start 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b271e2740 1 Processor -- start 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ed94d0 0x5584c12ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0f02390 0x5584c12ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b271e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584c12adfc0 con 0x5584c0ee16b0 2024-05-21T15:23:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b271e2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5584c11c1b90 con 0x5584c0ed94d0 2024-05-21T15:23:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b271e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5584c12ae140 con 0x5584c0f02390 2024-05-21T15:23:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b271e2740 10 client.0 _ll_register_callbacks cb 0x5584c11fdda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1e0d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1e0d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50414/0 (socket says 172.21.15.78:50414) 2024-05-21T15:23:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1e0d4640 1 -- 172.21.15.78:0/1420799740 learned_addr learned my addr 172.21.15.78:0/1420799740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ed94d0 0x5584c12ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0f02390 0x5584c12ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1e0d4640 1 -- 172.21.15.78:0/1420799740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0f02390 msgr2=0x5584c12ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0f02390 0x5584c12ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1e0d4640 1 -- 172.21.15.78:0/1420799740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ed94d0 msgr2=0x5584c12ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ed94d0 0x5584c12ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.321+0000 7f5b1e0d4640 1 -- 172.21.15.78:0/1420799740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c0f1a3f0 con 0x5584c0ee16b0 2024-05-21T15:23:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12ad3b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5b181029d0 tx=0x7f5b1002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d504fc9ba0b80271 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5afdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5afdffb640 1 -- 172.21.15.78:0/1420799740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10031860 con 0x5584c0ee16b0 2024-05-21T15:23:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5afdffb640 1 -- 172.21.15.78:0/1420799740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f5b1002f610 con 0x5584c0ee16b0 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 1 -- 172.21.15.78:0/1420799740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b0c004c80 con 0x5584c0ee16b0 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5afdffb640 1 -- 172.21.15.78:0/1420799740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5b10053050 con 0x5584c0ee16b0 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 10 client.17889 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 1 -- 172.21.15.78:0/1420799740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5584c0eda0f0 con 0x5584c0ee16b0 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5afdffb640 1 -- 172.21.15.78:0/1420799740 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b0c004c80 con 0x5584c0ee16b0 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5afdffb640 1 -- 172.21.15.78:0/1420799740 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5b1004e030 con 0x5584c0ee16b0 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 10 client.17889 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 20 client.17889 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5584c0ec0f60 0x5584c12bdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 4 client.17889 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:23:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.322+0000 7f5b271e2740 1 -- 172.21.15.78:0/1420799740 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584c1214430 con 0x5584c0ec0f60 2024-05-21T15:23:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.323+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5584c0ec0f60 0x5584c12bdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.323+0000 7f5afdffb640 10 client.17889 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.323+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5584c0ec0f60 0x5584c12bdc10 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58499ecf3190cd2 server_cookie=20c171b6f51be726 in_seq=0 out_seq=0 2024-05-21T15:23:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5afdffb640 1 -- 172.21.15.78:0/1420799740 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584c1214430 con 0x5584c0ec0f60 2024-05-21T15:23:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5afdffb640 10 client.17889 handle_command_reply: tid=0 2024-05-21T15:23:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 client.17889 shutdown 2024-05-21T15:23:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 -- 172.21.15.78:0/1420799740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 msgr2=0x5584c12ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12ad3b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5b181029d0 tx=0x7f5b1002f450 comp rx=0 tx=0).stop 2024-05-21T15:23:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 -- 172.21.15.78:0/1420799740 shutdown_connections 2024-05-21T15:23:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5584c0ec0f60 0x5584c12bdc10 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0f02390 0x5584c12ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5afdffb640 0 client.17889 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12ad3b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 --2- 172.21.15.78:0/1420799740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ed94d0 0x5584c12ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 -- 172.21.15.78:0/1420799740 >> 172.21.15.78:0/1420799740 conn(0x5584c0ef64e0 msgr2=0x5584c12a2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 -- 172.21.15.78:0/1420799740 shutdown_connections 2024-05-21T15:23:56.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 1 -- 172.21.15.78:0/1420799740 wait complete. 2024-05-21T15:23:56.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.324+0000 7f5b271e2740 20 client.17889 trim_cache size 0 max 16384 2024-05-21T15:23:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.341+0000 7f5b271e2740 1 Processor -- start 2024-05-21T15:23:56.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b271e2740 1 -- start start 2024-05-21T15:23:56.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ec0f60 0x5584c12a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ed94d0 0x5584c12a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ee16b0 0x5584c12a2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b271e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181acae0 con 0x5584c0ed94d0 2024-05-21T15:23:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b271e2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181ae620 con 0x5584c0ee16b0 2024-05-21T15:23:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b271e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181ad1b0 con 0x5584c0ec0f60 2024-05-21T15:23:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1d8d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ec0f60 0x5584c12a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1e0d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ed94d0 0x5584c12a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1d8d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ec0f60 0x5584c12a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:60780/0 (socket says 172.21.15.78:60780) 2024-05-21T15:23:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/3830439753 learned_addr learned my addr 172.21.15.78:0/3830439753 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/3830439753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ee16b0 0x5584c12a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1e0d4640 1 -- 172.21.15.78:0/3830439753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ec0f60 msgr2=0x5584c12a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/3830439753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ec0f60 0x5584c12a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1e0d4640 1 -- 172.21.15.78:0/3830439753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ee16b0 msgr2=0x5584c12a2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/3830439753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ee16b0 0x5584c12a2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.342+0000 7f5b1e0d4640 1 -- 172.21.15.78:0/3830439753 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c12978c0 con 0x5584c0ed94d0 2024-05-21T15:23:56.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.343+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/3830439753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ed94d0 0x5584c12a4fd0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5b10030c00 tx=0x7f5b100021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe016aa626342ea server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.343+0000 7f5afffff640 1 -- 172.21.15.78:0/3830439753 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1005c070 con 0x5584c0ed94d0 2024-05-21T15:23:56.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.343+0000 7f5afffff640 1 -- 172.21.15.78:0/3830439753 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b10036040 con 0x5584c0ed94d0 2024-05-21T15:23:56.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.343+0000 7f5afffff640 1 -- 172.21.15.78:0/3830439753 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10003cf0 con 0x5584c0ed94d0 2024-05-21T15:23:56.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.343+0000 7f5b271e2740 1 -- 172.21.15.78:0/3830439753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ed94d0 msgr2=0x5584c12a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.343+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3830439753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ed94d0 0x5584c12a4fd0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5b10030c00 tx=0x7f5b100021f0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 -- 172.21.15.78:0/3830439753 shutdown_connections 2024-05-21T15:23:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3830439753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ee16b0 0x5584c12a2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3830439753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ed94d0 0x5584c12a4fd0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3830439753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ec0f60 0x5584c12a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 -- 172.21.15.78:0/3830439753 >> 172.21.15.78:0/3830439753 conn(0x5584c0ef64e0 msgr2=0x5584c12a9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 -- 172.21.15.78:0/3830439753 shutdown_connections 2024-05-21T15:23:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 -- 172.21.15.78:0/3830439753 wait complete. 2024-05-21T15:23:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 -- start start 2024-05-21T15:23:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 Processor -- start 2024-05-21T15:23:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ec0f60 0x5584c12b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12b8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181adf50 con 0x5584c0ee16b0 2024-05-21T15:23:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181ad880 con 0x5584c0ec0f60 2024-05-21T15:23:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b181aaae0 con 0x5584c0ed94d0 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b271e2740 10 client.0 _ll_register_callbacks cb 0x7f5b10009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b1d8d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b1d8d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50438/0 (socket says 172.21.15.78:50438) 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.344+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/3633128311 learned_addr learned my addr 172.21.15.78:0/3633128311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ec0f60 0x5584c12b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/3633128311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 msgr2=0x5584c12b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12b8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/3633128311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ec0f60 msgr2=0x5584c12b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:23:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ec0f60 0x5584c12b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1d8d3640 1 -- 172.21.15.78:0/3633128311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c12b57d0 con 0x5584c0ee16b0 2024-05-21T15:23:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:23:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b1d8d3640 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12b5060 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c000c00 tx=0x7f5b0c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b520e65131610f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:23:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5afdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:23:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5afdffb640 1 -- 172.21.15.78:0/3633128311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c026040 con 0x5584c0ee16b0 2024-05-21T15:23:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5afdffb640 1 -- 172.21.15.78:0/3633128311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8658+0+0 (secure 0 0 0) 0x7f5b0c005130 con 0x5584c0ee16b0 2024-05-21T15:23:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b271e2740 1 -- 172.21.15.78:0/3633128311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b10036040 con 0x5584c0ee16b0 2024-05-21T15:23:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5afdffb640 1 -- 172.21.15.78:0/3633128311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5b0c041050 con 0x5584c0ee16b0 2024-05-21T15:23:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b271e2740 10 client.17895 fetch_fsmap learned FSMap version 37 2024-05-21T15:23:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5b271e2740 1 -- 172.21.15.78:0/3633128311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5584c1325a80 con 0x5584c0ee16b0 2024-05-21T15:23:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.345+0000 7f5afdffb640 1 -- 172.21.15.78:0/3633128311 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b10036040 con 0x5584c0ee16b0 2024-05-21T15:23:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5afdffb640 1 -- 172.21.15.78:0/3633128311 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5b0c01c030 con 0x5584c0ee16b0 2024-05-21T15:23:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5b271e2740 10 client.17895 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:23:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5b271e2740 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:23:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5b271e2740 20 client.17895 populate_metadata read hostname 'smithi078' 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5584c0f02390 0x5584c1328010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5b271e2740 4 client.17895 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5b271e2740 1 -- 172.21.15.78:0/3633128311 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584c120e780 con 0x5584c0f02390 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5584c0f02390 0x5584c1328010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5afdffb640 10 client.17895 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.346+0000 7f5b1e0d4640 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5584c0f02390 0x5584c1328010 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44cef6774bf37093 server_cookie=776bb94124f1d42d in_seq=0 out_seq=0 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5afdffb640 1 -- 172.21.15.78:0/3633128311 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5584c120e780 con 0x5584c0f02390 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5afdffb640 10 client.17895 handle_command_reply: tid=0 2024-05-21T15:23:56.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 client.17895 shutdown 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 -- 172.21.15.78:0/3633128311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 msgr2=0x5584c12b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12b5060 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c000c00 tx=0x7f5b0c015040 comp rx=0 tx=0).stop 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 -- 172.21.15.78:0/3633128311 shutdown_connections 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5584c0f02390 0x5584c1328010 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584c0ee16b0 0x5584c12b5060 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584c0ed94d0 0x5584c12b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5afdffb640 0 client.17895 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 --2- 172.21.15.78:0/3633128311 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584c0ec0f60 0x5584c12b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 -- 172.21.15.78:0/3633128311 >> 172.21.15.78:0/3633128311 conn(0x5584c0ef64e0 msgr2=0x5584c12a2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 -- 172.21.15.78:0/3633128311 shutdown_connections 2024-05-21T15:23:56.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 1 -- 172.21.15.78:0/3633128311 wait complete. 2024-05-21T15:23:56.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b271e2740 20 client.17895 trim_cache size 0 max 16384 2024-05-21T15:23:56.362 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:23:56.362 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:23:56.363 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:23:56.363 INFO:teuthology.orchestra.run.smithi078.stdout: "0042ace1-0264-456f-8dad-a9a2e973c501": { 2024-05-21T15:23:56.363 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:23:56.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "0042ace1-0264-456f-8dad-a9a2e973c501", 2024-05-21T15:23:56.363 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:23:56.363 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:23:56.363 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:23:56.363 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:23:56.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 -- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b000a2d00 msgr2=0x7f5b000a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b000a2d00 0x7f5b000a5210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b080ce390 tx=0x7f5b08054040 comp rx=0 tx=0).stop 2024-05-21T15:23:56.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 -- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1810c330 msgr2=0x7f5b1819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:23:56.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1810c330 0x7f5b1819cf40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800eb00 tx=0x7f5b0800ef00 comp rx=0 tx=0).stop 2024-05-21T15:23:56.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 -- 172.21.15.78:0/3661361667 shutdown_connections 2024-05-21T15:23:56.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5b000a2d00 0x7f5b000a5210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1810c330 0x7f5b1819cf40 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b181079d0 0x7f5b18198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 --2- 172.21.15.78:0/3661361667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b18101aa0 0x7f5b18198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:23:56.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.347+0000 7f5b20338640 1 -- 172.21.15.78:0/3661361667 >> 172.21.15.78:0/3661361667 conn(0x7f5b180770b0 msgr2=0x7f5b18103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:23:56.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.348+0000 7f5b20338640 1 -- 172.21.15.78:0/3661361667 shutdown_connections 2024-05-21T15:23:56.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:23:56.348+0000 7f5b20338640 1 -- 172.21.15.78:0/3661361667 wait complete. 2024-05-21T15:23:56.366 INFO:tasks.cephfs.filesystem:scrub status for tag:0042ace1-0264-456f-8dad-a9a2e973c501 - {'path': '/', 'tag': '0042ace1-0264-456f-8dad-a9a2e973c501', 'options': 'recursive,force'} 2024-05-21T15:23:56.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:55 smithi084 ceph-mon[36334]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:56.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:55 smithi172 ceph-mon[36402]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:23:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:56 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:56 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:56 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:57.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:56 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:57.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:56 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:57.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:56 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:56 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:56 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:56 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:57.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:56 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:56 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:57.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:56 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:57.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:56 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:57.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:56 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:56 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:56 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:56 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:57.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:56 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:56 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:57.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:56 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:57.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:56 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:57.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:56 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:56 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:57.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:56 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:57 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:23:58.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:57 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:57 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:58.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:57 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:23:58.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:57 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:57 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.299 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:57 smithi084 ceph-mon[36334]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 37 op/s 2024-05-21T15:23:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:57 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:23:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:57 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:57 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:57 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:23:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:57 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:57 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:57 smithi078 ceph-mon[31997]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 37 op/s 2024-05-21T15:23:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:57 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:23:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:57 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:57 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:57 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:23:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:57 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:57 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:23:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:57 smithi172 ceph-mon[36402]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 31 KiB/s rd, 37 op/s 2024-05-21T15:24:00.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:23:59 smithi078 ceph-mon[31997]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 40 op/s 2024-05-21T15:24:00.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:23:59 smithi084 ceph-mon[36334]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 40 op/s 2024-05-21T15:24:00.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:23:59 smithi172 ceph-mon[36402]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 36 KiB/s rd, 40 op/s 2024-05-21T15:24:02.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:01 smithi078 ceph-mon[31997]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:02.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:01 smithi084 ceph-mon[36334]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:02.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:01 smithi172 ceph-mon[36402]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:03.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:02 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:03.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:02 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:03.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:02 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:03.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:02 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:03.351 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:02 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:02 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:02 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:02 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:02 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:03.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:02 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:03.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:02 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:03.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:02 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:03.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:02 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:03.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:02 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:03.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:02 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:04.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:03 smithi078 ceph-mon[31997]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:24:04.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:03 smithi084 ceph-mon[36334]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:24:04.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:03 smithi172 ceph-mon[36402]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:24:06.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:05 smithi078 ceph-mon[31997]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 58 op/s 2024-05-21T15:24:06.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:05 smithi084 ceph-mon[36334]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 58 op/s 2024-05-21T15:24:06.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:05 smithi172 ceph-mon[36402]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 58 op/s 2024-05-21T15:24:08.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:07 smithi078 ceph-mon[31997]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:24:08.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:07 smithi084 ceph-mon[36334]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:24:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:07 smithi172 ceph-mon[36402]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 62 op/s 2024-05-21T15:24:09.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:08 smithi078 ceph-mon[31997]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T15:24:09.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:08 smithi084 ceph-mon[36334]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T15:24:09.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:08 smithi172 ceph-mon[36402]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T15:24:10.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:09 smithi078 ceph-mon[31997]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-05-21T15:24:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:09 smithi084 ceph-mon[36334]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-05-21T15:24:10.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:09 smithi172 ceph-mon[36402]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-05-21T15:24:12.350 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:11 smithi078 ceph-mon[31997]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:24:12.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:11 smithi084 ceph-mon[36334]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:24:12.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:11 smithi172 ceph-mon[36402]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-21T15:24:14.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:14 smithi078 ceph-mon[31997]: pgmap v250: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:14.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:14 smithi084 ceph-mon[36334]: pgmap v250: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:14.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:14 smithi172 ceph-mon[36402]: pgmap v250: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:16.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:16 smithi078 ceph-mon[31997]: pgmap v251: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:16 smithi084 ceph-mon[36334]: pgmap v251: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:16.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:16 smithi172 ceph-mon[36402]: pgmap v251: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:17.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:17 smithi172 ceph-mon[36402]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T15:24:17.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:17 smithi078 ceph-mon[31997]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T15:24:17.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:17 smithi084 ceph-mon[36334]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T15:24:18.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:18 smithi172 ceph-mon[36402]: pgmap v253: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T15:24:18.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:18 smithi078 ceph-mon[31997]: pgmap v253: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T15:24:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:18 smithi084 ceph-mon[36334]: pgmap v253: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T15:24:20.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:20 smithi172 ceph-mon[36402]: pgmap v254: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:20.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:20 smithi078 ceph-mon[31997]: pgmap v254: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:20.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:20 smithi084 ceph-mon[36334]: pgmap v254: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:22.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:22 smithi078 ceph-mon[31997]: pgmap v255: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:22 smithi084 ceph-mon[36334]: pgmap v255: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:22.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:22 smithi172 ceph-mon[36402]: pgmap v255: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:24:24.736 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:24 smithi172 ceph-mon[36402]: pgmap v256: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:24:24.736 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:24 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:24:24.736 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:24 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:24:24.737 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:24 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:24:24.737 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:24 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:24:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:24 smithi078 ceph-mon[31997]: pgmap v256: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:24:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:24 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:24:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:24 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:24:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:24 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:24:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:24 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:24:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:24 smithi084 ceph-mon[36334]: pgmap v256: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:24:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:24 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:24:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:24 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:24:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:24 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:24:24.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:24 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:24:26.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:24:26.367 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:24:26.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 -- 172.21.15.78:0/1253621950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40481013b0 msgr2=0x7f40481017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 --2- 172.21.15.78:0/1253621950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40481013b0 0x7f40481017c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f403800aa10 tx=0x7f403802b720 comp rx=0 tx=0).stop 2024-05-21T15:24:26.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 -- 172.21.15.78:0/1253621950 shutdown_connections 2024-05-21T15:24:26.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 --2- 172.21.15.78:0/1253621950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4048105940 0x7f4048105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 --2- 172.21.15.78:0/1253621950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4048106400 0x7f404810dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 --2- 172.21.15.78:0/1253621950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40481013b0 0x7f40481017c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 -- 172.21.15.78:0/1253621950 >> 172.21.15.78:0/1253621950 conn(0x7f4048076830 msgr2=0x7f4048076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 -- 172.21.15.78:0/1253621950 shutdown_connections 2024-05-21T15:24:26.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.484+0000 7f404ddce640 1 -- 172.21.15.78:0/1253621950 wait complete. 2024-05-21T15:24:26.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.485+0000 7f404ddce640 1 Processor -- start 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.485+0000 7f404ddce640 1 -- start start 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f404ddce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40481013b0 0x7f40481997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f404ddce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4048105940 0x7f4048199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f404ddce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4048106400 0x7f404819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f404ddce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4048110c30 con 0x7f4048105940 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f404ddce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4048110ab0 con 0x7f4048106400 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f404ddce640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4048110db0 con 0x7f40481013b0 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4048105940 0x7f4048199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4048105940 0x7f4048199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47644/0 (socket says 172.21.15.78:47644) 2024-05-21T15:24:26.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 -- 172.21.15.78:0/3998703300 learned_addr learned my addr 172.21.15.78:0/3998703300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:26.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4047fff640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4048106400 0x7f404819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f40477fe640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40481013b0 0x7f40481997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 -- 172.21.15.78:0/3998703300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40481013b0 msgr2=0x7f40481997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40481013b0 0x7f40481997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 -- 172.21.15.78:0/3998703300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4048106400 msgr2=0x7f404819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4048106400 0x7f404819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 -- 172.21.15.78:0/3998703300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404819e620 con 0x7f4048105940 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f40477fe640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40481013b0 0x7f40481997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4047fff640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4048106400 0x7f404819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4046ffd640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4048105940 0x7f4048199d20 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f403000e9c0 tx=0x7f403000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4044ff9640 1 -- 172.21.15.78:0/3998703300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403000cd80 con 0x7f4048105940 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4044ff9640 1 -- 172.21.15.78:0/3998703300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4030005500 con 0x7f4048105940 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f4044ff9640 1 -- 172.21.15.78:0/3998703300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030010690 con 0x7f4048105940 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.486+0000 7f404ddce640 1 -- 172.21.15.78:0/3998703300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404819e910 con 0x7f4048105940 2024-05-21T15:24:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.487+0000 7f404ddce640 1 -- 172.21.15.78:0/3998703300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40481a6290 con 0x7f4048105940 2024-05-21T15:24:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4044ff9640 1 -- 172.21.15.78:0/3998703300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f40300046c0 con 0x7f4048105940 2024-05-21T15:24:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4054c7c740 1 Processor -- start 2024-05-21T15:24:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4054c7c740 1 -- start start 2024-05-21T15:24:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799ab390 0x55da79d49d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799824d0 0x55da79d4a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da7998a6b0 0x55da79d4a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4054c7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da79cafd00 con 0x55da799ab390 2024-05-21T15:24:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4054c7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55da79c574c0 con 0x55da7998a6b0 2024-05-21T15:24:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4054c7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da79cabcc0 con 0x55da799824d0 2024-05-21T15:24:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.489+0000 7f4047fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799ab390 0x55da79d49d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799ab390 0x55da79d49d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47658/0 (socket says 172.21.15.78:47658) 2024-05-21T15:24:26.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 -- 172.21.15.78:0/2443642498 learned_addr learned my addr 172.21.15.78:0/2443642498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:26.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4046ffd640 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da7998a6b0 0x55da79d4a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f40477fe640 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799824d0 0x55da79d4a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4044ff9640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40340a2d00 0x7f40340a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 -- 172.21.15.78:0/2443642498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799824d0 msgr2=0x55da79d4a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:26.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799824d0 0x55da79d4a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 -- 172.21.15.78:0/2443642498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da7998a6b0 msgr2=0x55da79d4a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:26.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da7998a6b0 0x55da79d4a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 -- 172.21.15.78:0/2443642498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da79a8d7d0 con 0x55da799ab390 2024-05-21T15:24:26.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f40477fe640 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799824d0 0x55da79d4a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:24:26.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4046ffd640 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da7998a6b0 0x55da79d4a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:26.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40340a2d00 0x7f40340a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4044ff9640 1 -- 172.21.15.78:0/3998703300 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f40300cf930 con 0x7f4048105940 2024-05-21T15:24:26.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799ab390 0x55da79d49d30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f40400118d0 tx=0x7f4040011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935970955c796e67 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f403dffb640 1 -- 172.21.15.78:0/2443642498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040017050 con 0x55da799ab390 2024-05-21T15:24:26.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f403dffb640 1 -- 172.21.15.78:0/2443642498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4040026440 con 0x55da799ab390 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f403dffb640 1 -- 172.21.15.78:0/2443642498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404000e610 con 0x55da799ab390 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.490+0000 7f4047fff640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40340a2d00 0x7f40340a5210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4040023000 tx=0x7f4040024000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 -- 172.21.15.78:0/2443642498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799ab390 msgr2=0x55da79d49d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799ab390 0x55da79d49d30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f40400118d0 tx=0x7f4040011d90 comp rx=0 tx=0).stop 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 -- 172.21.15.78:0/2443642498 shutdown_connections 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da7998a6b0 0x55da79d4a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799824d0 0x55da79d4a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 --2- 172.21.15.78:0/2443642498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799ab390 0x55da79d49d30 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 -- 172.21.15.78:0/2443642498 >> 172.21.15.78:0/2443642498 conn(0x55da7999f4e0 msgr2=0x55da79d464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 -- 172.21.15.78:0/2443642498 shutdown_connections 2024-05-21T15:24:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.491+0000 7f4054c7c740 1 -- 172.21.15.78:0/2443642498 wait complete. 2024-05-21T15:24:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 1 -- start start 2024-05-21T15:24:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 1 Processor -- start 2024-05-21T15:24:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da7998a6b0 0x55da79d563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799ab390 0x55da79d54380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da79d56fc0 con 0x55da7998a6b0 2024-05-21T15:24:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55da79c6ab90 con 0x55da799824d0 2024-05-21T15:24:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da79d57140 con 0x55da799ab390 2024-05-21T15:24:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4054c7c740 10 client.0 _ll_register_callbacks cb 0x55da79ca6da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da7998a6b0 0x55da79d563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f40477fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da7998a6b0 0x55da79d563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47672/0 (socket says 172.21.15.78:47672) 2024-05-21T15:24:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.492+0000 7f4046ffd640 1 -- 172.21.15.78:0/3788053519 learned_addr learned my addr 172.21.15.78:0/3788053519 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4047fff640 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799ab390 0x55da79d54380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4046ffd640 1 -- 172.21.15.78:0/3788053519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799ab390 msgr2=0x55da79d54380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4046ffd640 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799ab390 0x55da79d54380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4046ffd640 1 -- 172.21.15.78:0/3788053519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 msgr2=0x55da79d55e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4046ffd640 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4046ffd640 1 -- 172.21.15.78:0/3788053519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da799c33f0 con 0x55da7998a6b0 2024-05-21T15:24:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f40477fe640 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4046ffd640 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da7998a6b0 0x55da79d563b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4030098790 tx=0x7f40300d3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c7fdbda4a89c60 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4026ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4026ffd640 1 -- 172.21.15.78:0/3788053519 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403009bb90 con 0x55da7998a6b0 2024-05-21T15:24:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4026ffd640 1 -- 172.21.15.78:0/3788053519 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f40300a47a0 con 0x55da7998a6b0 2024-05-21T15:24:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4054c7c740 1 -- 172.21.15.78:0/3788053519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4040026440 con 0x55da7998a6b0 2024-05-21T15:24:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4026ffd640 1 -- 172.21.15.78:0/3788053519 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f403009c4c0 con 0x55da7998a6b0 2024-05-21T15:24:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4054c7c740 10 client.17907 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4054c7c740 1 -- 172.21.15.78:0/3788053519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55da799830f0 con 0x55da7998a6b0 2024-05-21T15:24:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.493+0000 7f4026ffd640 1 -- 172.21.15.78:0/3788053519 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4040026440 con 0x55da7998a6b0 2024-05-21T15:24:26.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f4026ffd640 1 -- 172.21.15.78:0/3788053519 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f403009ac80 con 0x55da7998a6b0 2024-05-21T15:24:26.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f4054c7c740 10 client.17907 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:26.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f4054c7c740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:26.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f4054c7c740 20 client.17907 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:26.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f4054c7c740 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da79969f60 0x55da79d66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f4054c7c740 4 client.17907 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:26.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f4054c7c740 1 -- 172.21.15.78:0/3788053519 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da79cbd430 con 0x55da79969f60 2024-05-21T15:24:26.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f40477fe640 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da79969f60 0x55da79d66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f4026ffd640 10 client.17907 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.494+0000 7f40477fe640 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da79969f60 0x55da79d66b40 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ba51dd67c2bdd1 server_cookie=24a1e2dc6de3121d in_seq=0 out_seq=0 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4026ffd640 1 -- 172.21.15.78:0/3788053519 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da79cbd430 con 0x55da79969f60 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4026ffd640 10 client.17907 handle_command_reply: tid=0 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 client.17907 shutdown 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 -- 172.21.15.78:0/3788053519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da7998a6b0 msgr2=0x55da79d563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da7998a6b0 0x55da79d563b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4030098790 tx=0x7f40300d3040 comp rx=0 tx=0).stop 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 -- 172.21.15.78:0/3788053519 shutdown_connections 2024-05-21T15:24:26.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da79969f60 0x55da79d66b40 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da799ab390 0x55da79d54380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da7998a6b0 0x55da79d563b0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 --2- 172.21.15.78:0/3788053519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4026ffd640 0 client.17907 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:26.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 -- 172.21.15.78:0/3788053519 >> 172.21.15.78:0/3788053519 conn(0x55da7999f4e0 msgr2=0x55da79d4ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 -- 172.21.15.78:0/3788053519 shutdown_connections 2024-05-21T15:24:26.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.495+0000 7f4054c7c740 1 -- 172.21.15.78:0/3788053519 wait complete. 2024-05-21T15:24:26.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.496+0000 7f4054c7c740 20 client.17907 trim_cache size 0 max 16384 2024-05-21T15:24:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.513+0000 7f4054c7c740 1 Processor -- start 2024-05-21T15:24:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.513+0000 7f4054c7c740 1 -- start start 2024-05-21T15:24:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.513+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da79969f60 0x55da79d4fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.513+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 0x55da79d4e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4054c7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40481aed60 con 0x55da799824d0 2024-05-21T15:24:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4054c7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40481b08a0 con 0x55da79969f60 2024-05-21T15:24:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4054c7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40481af430 con 0x55da7998a6b0 2024-05-21T15:24:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47678/0 (socket says 172.21.15.78:47678) 2024-05-21T15:24:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 -- 172.21.15.78:0/2967703413 learned_addr learned my addr 172.21.15.78:0/2967703413 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f40477fe640 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da79969f60 0x55da79d4fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4047fff640 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 0x55da79d4e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 -- 172.21.15.78:0/2967703413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 msgr2=0x55da79d4e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 0x55da79d4e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 -- 172.21.15.78:0/2967703413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da79969f60 msgr2=0x55da79d4fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da79969f60 0x55da79d4fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 -- 172.21.15.78:0/2967703413 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da79d46ed0 con 0x55da799824d0 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f4046ffd640 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799824d0 0x55da79d55e70 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4030098220 tx=0x7f4030009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6fa39bc301a322 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f403d7fa640 1 -- 172.21.15.78:0/2967703413 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403007f070 con 0x55da799824d0 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f403d7fa640 1 -- 172.21.15.78:0/2967703413 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40300a4a30 con 0x55da799824d0 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.514+0000 7f403d7fa640 1 -- 172.21.15.78:0/2967703413 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40300a45a0 con 0x55da799824d0 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 -- 172.21.15.78:0/2967703413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799824d0 msgr2=0x55da79d55e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799824d0 0x55da79d55e70 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4030098220 tx=0x7f4030009700 comp rx=0 tx=0).stop 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 -- 172.21.15.78:0/2967703413 shutdown_connections 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 0x55da79d4e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da799824d0 0x55da79d55e70 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 --2- 172.21.15.78:0/2967703413 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da79969f60 0x55da79d4fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 -- 172.21.15.78:0/2967703413 >> 172.21.15.78:0/2967703413 conn(0x55da7999f4e0 msgr2=0x55da79d4d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 -- 172.21.15.78:0/2967703413 shutdown_connections 2024-05-21T15:24:26.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 -- 172.21.15.78:0/2967703413 wait complete. 2024-05-21T15:24:26.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.515+0000 7f4054c7c740 1 -- start start 2024-05-21T15:24:26.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4054c7c740 1 Processor -- start 2024-05-21T15:24:26.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da79969f60 0x55da79d61190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d5e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4054c7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 0x55da79d5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4054c7c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40481b01d0 con 0x55da79969f60 2024-05-21T15:24:26.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4054c7c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40481afb00 con 0x55da799824d0 2024-05-21T15:24:26.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4054c7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40481acd60 con 0x55da7998a6b0 2024-05-21T15:24:26.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4054c7c740 10 client.0 _ll_register_callbacks cb 0x7f4030009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:26.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f40477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da79969f60 0x55da79d61190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f40477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da79969f60 0x55da79d61190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47694/0 (socket says 172.21.15.78:47694) 2024-05-21T15:24:26.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f40477fe640 1 -- 172.21.15.78:0/6681979 learned_addr learned my addr 172.21.15.78:0/6681979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:26.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4046ffd640 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d5e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4047fff640 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 0x55da79d5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f40477fe640 1 -- 172.21.15.78:0/6681979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 msgr2=0x55da79d5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:26.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f40477fe640 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 0x55da79d5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f40477fe640 1 -- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 msgr2=0x55da79d5e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:26.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f40477fe640 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d5e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f40477fe640 1 -- 172.21.15.78:0/6681979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da79d5f260 con 0x55da79969f60 2024-05-21T15:24:26.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.516+0000 7f4046ffd640 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d5e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:26.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f40477fe640 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da79969f60 0x55da79d61190 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4038005ab0 tx=0x7f40380052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce23f9e4ff49cf22 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4026ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4026ffd640 1 -- 172.21.15.78:0/6681979 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038036040 con 0x55da79969f60 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4026ffd640 1 -- 172.21.15.78:0/6681979 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f4038046500 con 0x55da79969f60 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 1 -- 172.21.15.78:0/6681979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40300a4a30 con 0x55da79969f60 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4026ffd640 1 -- 172.21.15.78:0/6681979 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4038046700 con 0x55da79969f60 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 10 client.17913 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 1 -- 172.21.15.78:0/6681979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55da79dceea0 con 0x55da79969f60 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4026ffd640 1 -- 172.21.15.78:0/6681979 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40300a4a30 con 0x55da79969f60 2024-05-21T15:24:26.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4026ffd640 1 -- 172.21.15.78:0/6681979 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f4038055030 con 0x55da79969f60 2024-05-21T15:24:26.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 10 client.17913 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:26.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 10 client.17913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:26.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 20 client.17913 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:26.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da799ab390 0x55da79dd1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 4 client.17913 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:24:26.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.517+0000 7f4054c7c740 1 -- 172.21.15.78:0/6681979 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da79cb7780 con 0x55da799ab390 2024-05-21T15:24:26.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4046ffd640 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da799ab390 0x55da79dd1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4026ffd640 10 client.17913 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:26.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4046ffd640 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da799ab390 0x55da79dd1690 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b9e0f279d77796 server_cookie=e9aa9acad88a1fda in_seq=0 out_seq=0 2024-05-21T15:24:26.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4026ffd640 1 -- 172.21.15.78:0/6681979 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55da79cb7780 con 0x55da799ab390 2024-05-21T15:24:26.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4026ffd640 10 client.17913 handle_command_reply: tid=0 2024-05-21T15:24:26.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 client.17913 shutdown 2024-05-21T15:24:26.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 -- 172.21.15.78:0/6681979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da79969f60 msgr2=0x55da79d61190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da79969f60 0x55da79d61190 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4038005ab0 tx=0x7f40380052a0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 -- 172.21.15.78:0/6681979 shutdown_connections 2024-05-21T15:24:26.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55da799ab390 0x55da79dd1690 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55da799824d0 0x55da79d5e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4026ffd640 0 client.17913 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:26.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da7998a6b0 0x55da79d5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 --2- 172.21.15.78:0/6681979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da79969f60 0x55da79d61190 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 -- 172.21.15.78:0/6681979 >> 172.21.15.78:0/6681979 conn(0x55da7999f4e0 msgr2=0x55da79d4d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 -- 172.21.15.78:0/6681979 shutdown_connections 2024-05-21T15:24:26.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.518+0000 7f4054c7c740 1 -- 172.21.15.78:0/6681979 wait complete. 2024-05-21T15:24:26.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f4054c7c740 20 client.17913 trim_cache size 0 max 16384 2024-05-21T15:24:26.535 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:24:26.535 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:24:26.535 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:24:26.535 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:24:26.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 -- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40340a2d00 msgr2=0x7f40340a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40340a2d00 0x7f40340a5210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4040023000 tx=0x7f4040024000 comp rx=0 tx=0).stop 2024-05-21T15:24:26.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 -- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4048105940 msgr2=0x7f4048199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4048105940 0x7f4048199d20 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f403000e9c0 tx=0x7f403000ee80 comp rx=0 tx=0).stop 2024-05-21T15:24:26.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 -- 172.21.15.78:0/3998703300 shutdown_connections 2024-05-21T15:24:26.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4048106400 0x7f404819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f40340a2d00 0x7f40340a5210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4048105940 0x7f4048199d20 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 --2- 172.21.15.78:0/3998703300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40481013b0 0x7f40481997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 -- 172.21.15.78:0/3998703300 >> 172.21.15.78:0/3998703300 conn(0x7f4048076830 msgr2=0x7f40480ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 -- 172.21.15.78:0/3998703300 shutdown_connections 2024-05-21T15:24:26.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.519+0000 7f404ddce640 1 -- 172.21.15.78:0/3998703300 wait complete. 2024-05-21T15:24:26.538 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:24:26.539 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:24:26.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 -- 172.21.15.78:0/1814964037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 msgr2=0x7f792c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 --2- 172.21.15.78:0/1814964037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 0x7f792c101eb0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f791c009ca0 tx=0x7f791c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 -- 172.21.15.78:0/1814964037 shutdown_connections 2024-05-21T15:24:26.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 --2- 172.21.15.78:0/1814964037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f792c10c330 0x7f792c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 --2- 172.21.15.78:0/1814964037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 0x7f792c101eb0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 --2- 172.21.15.78:0/1814964037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792c1079d0 0x7f792c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 -- 172.21.15.78:0/1814964037 >> 172.21.15.78:0/1814964037 conn(0x7f792c0770c0 msgr2=0x7f792c077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 -- 172.21.15.78:0/1814964037 shutdown_connections 2024-05-21T15:24:26.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.647+0000 7f7932b38640 1 -- 172.21.15.78:0/1814964037 wait complete. 2024-05-21T15:24:26.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f7932b38640 1 Processor -- start 2024-05-21T15:24:26.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f7932b38640 1 -- start start 2024-05-21T15:24:26.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f7932b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 0x7f792c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f7932b38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f792c1079d0 0x7f792c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f7932b38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792c10c330 0x7f792c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f7932b38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f792c111ce0 con 0x7f792c101aa0 2024-05-21T15:24:26.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f7932b38640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f792c111b60 con 0x7f792c1079d0 2024-05-21T15:24:26.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f7932b38640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f792c111e60 con 0x7f792c10c330 2024-05-21T15:24:26.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f79308d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 0x7f792c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f79308d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 0x7f792c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47724/0 (socket says 172.21.15.78:47724) 2024-05-21T15:24:26.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f79308d4640 1 -- 172.21.15.78:0/1370362282 learned_addr learned my addr 172.21.15.78:0/1370362282 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:26.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f79310d5640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792c10c330 0x7f792c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.648+0000 7f792bfff640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f792c1079d0 0x7f792c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f79308d4640 1 -- 172.21.15.78:0/1370362282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792c10c330 msgr2=0x7f792c19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f79308d4640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792c10c330 0x7f792c19df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f79308d4640 1 -- 172.21.15.78:0/1370362282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f792c1079d0 msgr2=0x7f792c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f79308d4640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f792c1079d0 0x7f792c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f79308d4640 1 -- 172.21.15.78:0/1370362282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c19e5b0 con 0x7f792c101aa0 2024-05-21T15:24:26.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f79308d4640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 0x7f792c199750 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f792400e9c0 tx=0x7f792400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f7929ffb640 1 -- 172.21.15.78:0/1370362282 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792400ccc0 con 0x7f792c101aa0 2024-05-21T15:24:26.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f7929ffb640 1 -- 172.21.15.78:0/1370362282 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7924005500 con 0x7f792c101aa0 2024-05-21T15:24:26.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c19e8a0 con 0x7f792c101aa0 2024-05-21T15:24:26.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f7929ffb640 1 -- 172.21.15.78:0/1370362282 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924010430 con 0x7f792c101aa0 2024-05-21T15:24:26.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.649+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c1a6220 con 0x7f792c101aa0 2024-05-21T15:24:26.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.650+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c102c80 con 0x7f792c101aa0 2024-05-21T15:24:26.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.651+0000 7f7929ffb640 1 -- 172.21.15.78:0/1370362282 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7924005000 con 0x7f792c101aa0 2024-05-21T15:24:26.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.652+0000 7f7929ffb640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f79180a2d00 0x7f79180a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:26.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.652+0000 7f792bfff640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f79180a2d00 0x7f79180a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:26.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.652+0000 7f7929ffb640 1 -- 172.21.15.78:0/1370362282 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f79240cf930 con 0x7f792c101aa0 2024-05-21T15:24:26.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.653+0000 7f792bfff640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f79180a2d00 0x7f79180a5210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f791c000f80 tx=0x7f791c005fd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:26.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.657+0000 7f7929ffb640 1 -- 172.21.15.78:0/1370362282 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f792409c4c0 con 0x7f792c101aa0 2024-05-21T15:24:26.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:26 smithi078 ceph-mon[31997]: pgmap v257: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:24:26.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.886+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f792c069f50 con 0x7f792c101aa0 2024-05-21T15:24:26.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.886+0000 7f7929ffb640 1 -- 172.21.15.78:0/1370362282 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7f792409bdc0 con 0x7f792c101aa0 2024-05-21T15:24:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:24:26.891 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:24:26.891 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:24:26.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.888+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f79180a2d00 msgr2=0x7f79180a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.888+0000 7f7932b38640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f79180a2d00 0x7f79180a5210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f791c000f80 tx=0x7f791c005fd0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.888+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 msgr2=0x7f792c199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:26.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.888+0000 7f7932b38640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 0x7f792c199750 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f792400e9c0 tx=0x7f792400edc0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.889+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 shutdown_connections 2024-05-21T15:24:26.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.889+0000 7f7932b38640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f792c1079d0 0x7f792c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.889+0000 7f7932b38640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f79180a2d00 0x7f79180a5210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.889+0000 7f7932b38640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792c10c330 0x7f792c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.889+0000 7f7932b38640 1 --2- 172.21.15.78:0/1370362282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f792c101aa0 0x7f792c199750 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:26.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.889+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 >> 172.21.15.78:0/1370362282 conn(0x7f792c0770c0 msgr2=0x7f792c068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:26.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.889+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 shutdown_connections 2024-05-21T15:24:26.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:26.889+0000 7f7932b38640 1 -- 172.21.15.78:0/1370362282 wait complete. 2024-05-21T15:24:26.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:26 smithi084 ceph-mon[36334]: pgmap v257: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:24:26.905 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:24:26.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:26 smithi172 ceph-mon[36402]: pgmap v257: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:24:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 -- 172.21.15.78:0/313679623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 msgr2=0x7feda0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 --2- 172.21.15.78:0/313679623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 0x7feda0101560 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fed90009900 tx=0x7fed9002a410 comp rx=0 tx=0).stop 2024-05-21T15:24:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 -- 172.21.15.78:0/313679623 shutdown_connections 2024-05-21T15:24:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 --2- 172.21.15.78:0/313679623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda010c330 0x7feda010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 --2- 172.21.15.78:0/313679623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feda0101aa0 0x7feda0101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 --2- 172.21.15.78:0/313679623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 0x7feda0101560 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 -- 172.21.15.78:0/313679623 >> 172.21.15.78:0/313679623 conn(0x7feda00770b0 msgr2=0x7feda00774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 -- 172.21.15.78:0/313679623 shutdown_connections 2024-05-21T15:24:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.016+0000 7feda5a32640 1 -- 172.21.15.78:0/313679623 wait complete. 2024-05-21T15:24:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.017+0000 7feda5a32640 1 Processor -- start 2024-05-21T15:24:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.017+0000 7feda5a32640 1 -- start start 2024-05-21T15:24:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.017+0000 7feda5a32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda0101aa0 0x7feda00fec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.017+0000 7feda5a32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 0x7feda00ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.017+0000 7feda5a32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feda010c330 0x7feda00ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.017+0000 7feda5a32640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feda0111ce0 con 0x7feda01079d0 2024-05-21T15:24:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.017+0000 7feda5a32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feda0111b60 con 0x7feda0101aa0 2024-05-21T15:24:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.017+0000 7feda5a32640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feda0111e60 con 0x7feda010c330 2024-05-21T15:24:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 0x7feda00ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 0x7feda00ff170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47744/0 (socket says 172.21.15.78:47744) 2024-05-21T15:24:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 -- 172.21.15.78:0/1819619242 learned_addr learned my addr 172.21.15.78:0/1819619242 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feda010c330 0x7feda00ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed9effd640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda0101aa0 0x7feda00fec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 -- 172.21.15.78:0/1819619242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feda010c330 msgr2=0x7feda00ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feda010c330 0x7feda00ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 -- 172.21.15.78:0/1819619242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda0101aa0 msgr2=0x7feda00fec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda0101aa0 0x7feda00fec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 -- 172.21.15.78:0/1819619242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda01a7b10 con 0x7feda01079d0 2024-05-21T15:24:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7fed967fc640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 0x7feda00ff170 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fed8800e9f0 tx=0x7fed8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7feda4a30640 1 -- 172.21.15.78:0/1819619242 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8800ccc0 con 0x7feda01079d0 2024-05-21T15:24:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7feda4a30640 1 -- 172.21.15.78:0/1819619242 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed880035f0 con 0x7feda01079d0 2024-05-21T15:24:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7feda5a32640 1 -- 172.21.15.78:0/1819619242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda01a7e00 con 0x7feda01079d0 2024-05-21T15:24:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7feda4a30640 1 -- 172.21.15.78:0/1819619242 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8800ce60 con 0x7feda01079d0 2024-05-21T15:24:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.018+0000 7feda5a32640 1 -- 172.21.15.78:0/1819619242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda01a8270 con 0x7feda01079d0 2024-05-21T15:24:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.021+0000 7feda4a30640 1 -- 172.21.15.78:0/1819619242 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fed880037a0 con 0x7feda01079d0 2024-05-21T15:24:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.021+0000 7fedac9bc740 1 Processor -- start 2024-05-21T15:24:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.021+0000 7fedac9bc740 1 -- start start 2024-05-21T15:24:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.021+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a61b390 0x55939a9b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.021+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9ba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.021+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.021+0000 7fedac9bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55939a91fc40 con 0x55939a61b390 2024-05-21T15:24:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fedac9bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55939a8c7580 con 0x55939a5fa6b0 2024-05-21T15:24:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fedac9bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55939a91bc00 con 0x55939a5f24d0 2024-05-21T15:24:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a61b390 0x55939a9b9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a61b390 0x55939a9b9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47750/0 (socket says 172.21.15.78:47750) 2024-05-21T15:24:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 -- 172.21.15.78:0/1535817646 learned_addr learned my addr 172.21.15.78:0/1535817646 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed967fc640 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9effd640 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 -- 172.21.15.78:0/1535817646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 msgr2=0x55939a9ba260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9ba260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7feda4a30640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fed8c0a2d00 0x7fed8c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 -- 172.21.15.78:0/1535817646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 msgr2=0x55939a9ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 -- 172.21.15.78:0/1535817646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55939a6333f0 con 0x55939a61b390 2024-05-21T15:24:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fed8c0a2d00 0x7fed8c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7feda4a30640 1 -- 172.21.15.78:0/1819619242 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fed880cf9e0 con 0x7feda01079d0 2024-05-21T15:24:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a61b390 0x55939a9b9d20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fed98009f50 tx=0x7fed980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2529688eeeab4c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.022+0000 7fed95ffb640 1 -- 172.21.15.78:0/1535817646 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98003bf0 con 0x55939a61b390 2024-05-21T15:24:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.023+0000 7fed95ffb640 1 -- 172.21.15.78:0/1535817646 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed980047e0 con 0x55939a61b390 2024-05-21T15:24:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.023+0000 7fed95ffb640 1 -- 172.21.15.78:0/1535817646 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98019640 con 0x55939a61b390 2024-05-21T15:24:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.023+0000 7fedac9bc740 1 -- 172.21.15.78:0/1535817646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a61b390 msgr2=0x55939a9b9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.023+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a61b390 0x55939a9b9d20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fed98009f50 tx=0x7fed980118d0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.023+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fed8c0a2d00 0x7fed8c0a5210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fed98012fd0 tx=0x7fed9801e000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 -- 172.21.15.78:0/1535817646 shutdown_connections 2024-05-21T15:24:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9ba260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1535817646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a61b390 0x55939a9b9d20 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 -- 172.21.15.78:0/1535817646 >> 172.21.15.78:0/1535817646 conn(0x55939a60f4e0 msgr2=0x55939a9b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 -- 172.21.15.78:0/1535817646 shutdown_connections 2024-05-21T15:24:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 -- 172.21.15.78:0/1535817646 wait complete. 2024-05-21T15:24:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 -- start start 2024-05-21T15:24:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.024+0000 7fedac9bc740 1 Processor -- start 2024-05-21T15:24:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5f24d0 0x55939a9c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a61b390 0x55939a9ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fedac9bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55939a9c6fb0 con 0x55939a5f24d0 2024-05-21T15:24:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fedac9bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55939a8dac40 con 0x55939a5fa6b0 2024-05-21T15:24:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fedac9bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55939a9c7130 con 0x55939a61b390 2024-05-21T15:24:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fedac9bc740 10 client.0 _ll_register_callbacks cb 0x55939a632be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed9effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5f24d0 0x55939a9c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed9effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5f24d0 0x55939a9c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47758/0 (socket says 172.21.15.78:47758) 2024-05-21T15:24:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed9effd640 1 -- 172.21.15.78:0/3088510103 learned_addr learned my addr 172.21.15.78:0/3088510103 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed967fc640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a61b390 0x55939a9ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed967fc640 1 -- 172.21.15.78:0/3088510103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a61b390 msgr2=0x55939a9ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed967fc640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a61b390 0x55939a9ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed967fc640 1 -- 172.21.15.78:0/3088510103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5f24d0 msgr2=0x55939a9c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed967fc640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5f24d0 0x55939a9c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed967fc640 1 -- 172.21.15.78:0/3088510103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55939a6fd7d0 con 0x55939a5fa6b0 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed9effd640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5f24d0 0x55939a9c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a61b390 0x55939a9ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed967fc640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c4920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fed880cd0a0 tx=0x7fed880d3120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a58dd1af6d60daa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed7effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:24:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed7effd640 1 -- 172.21.15.78:0/3088510103 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8809b9b0 con 0x55939a5fa6b0 2024-05-21T15:24:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fed7effd640 1 -- 172.21.15.78:0/3088510103 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fed8809b2c0 con 0x55939a5fa6b0 2024-05-21T15:24:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.025+0000 7fedac9bc740 1 -- 172.21.15.78:0/3088510103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed980047e0 con 0x55939a5fa6b0 2024-05-21T15:24:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fed7effd640 1 -- 172.21.15.78:0/3088510103 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fed880b8020 con 0x55939a5fa6b0 2024-05-21T15:24:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fedac9bc740 10 client.27119 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fedac9bc740 1 -- 172.21.15.78:0/3088510103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55939a9d3750 con 0x55939a5fa6b0 2024-05-21T15:24:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fed7effd640 1 -- 172.21.15.78:0/3088510103 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed980047e0 con 0x55939a5fa6b0 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fed7effd640 1 -- 172.21.15.78:0/3088510103 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fed880baab0 con 0x55939a5fa6b0 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fedac9bc740 10 client.27119 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fedac9bc740 10 client.27119 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fedac9bc740 20 client.27119 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fedac9bc740 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55939a5d9f60 0x55939a9d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fedac9bc740 4 client.27119 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fedac9bc740 1 -- 172.21.15.78:0/3088510103 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55939a92d370 con 0x55939a5d9f60 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.026+0000 7fed9effd640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55939a5d9f60 0x55939a9d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.027+0000 7fed7effd640 10 client.27119 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:24:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.027+0000 7fed9effd640 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55939a5d9f60 0x55939a9d6bd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e250f660d97e53ac server_cookie=6b2278079b643ece in_seq=0 out_seq=0 2024-05-21T15:24:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.027+0000 7fed7effd640 1 -- 172.21.15.78:0/3088510103 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55939a92d370 con 0x55939a5d9f60 2024-05-21T15:24:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.027+0000 7fed7effd640 10 client.27119 handle_command_reply: tid=0 2024-05-21T15:24:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.027+0000 7fedac9bc740 1 client.27119 shutdown 2024-05-21T15:24:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 -- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 msgr2=0x55939a9c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c4920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fed880cd0a0 tx=0x7fed880d3120 comp rx=0 tx=0).stop 2024-05-21T15:24:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 -- 172.21.15.78:0/3088510103 shutdown_connections 2024-05-21T15:24:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55939a5d9f60 0x55939a9d6bd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c4920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a61b390 0x55939a9ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fed7effd640 0 client.27119 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:24:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 --2- 172.21.15.78:0/3088510103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5f24d0 0x55939a9c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 -- 172.21.15.78:0/3088510103 >> 172.21.15.78:0/3088510103 conn(0x55939a60f4e0 msgr2=0x55939a9bba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 -- 172.21.15.78:0/3088510103 shutdown_connections 2024-05-21T15:24:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 1 -- 172.21.15.78:0/3088510103 wait complete. 2024-05-21T15:24:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.028+0000 7fedac9bc740 20 client.27119 trim_cache size 0 max 16384 2024-05-21T15:24:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.042+0000 7fedac9bc740 1 Processor -- start 2024-05-21T15:24:27.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.042+0000 7fedac9bc740 1 -- start start 2024-05-21T15:24:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.042+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5d9f60 0x55939a9dc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.042+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9dc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.042+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.042+0000 7fedac9bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feda01b0e90 con 0x55939a5d9f60 2024-05-21T15:24:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.042+0000 7fedac9bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feda01b29d0 con 0x55939a5fa6b0 2024-05-21T15:24:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.042+0000 7fedac9bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feda01b1560 con 0x55939a5f24d0 2024-05-21T15:24:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5d9f60 0x55939a9dc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5d9f60 0x55939a9dc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47766/0 (socket says 172.21.15.78:47766) 2024-05-21T15:24:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 -- 172.21.15.78:0/2460695054 learned_addr learned my addr 172.21.15.78:0/2460695054 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed967fc640 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9dc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9f7fe640 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 -- 172.21.15.78:0/2460695054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 msgr2=0x55939a9dc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9dc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 -- 172.21.15.78:0/2460695054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 msgr2=0x55939a9c1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 -- 172.21.15.78:0/2460695054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55939a9bfd70 con 0x55939a5d9f60 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed9effd640 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5d9f60 0x55939a9dc360 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fed9002a930 tx=0x7fed90031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21053034b32b71d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed957fa640 1 -- 172.21.15.78:0/2460695054 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90035040 con 0x55939a5d9f60 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed957fa640 1 -- 172.21.15.78:0/2460695054 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9004d070 con 0x55939a5d9f60 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.043+0000 7fed957fa640 1 -- 172.21.15.78:0/2460695054 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed900486d0 con 0x55939a5d9f60 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 -- 172.21.15.78:0/2460695054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5d9f60 msgr2=0x55939a9dc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5d9f60 0x55939a9dc360 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fed9002a930 tx=0x7fed90031850 comp rx=0 tx=0).stop 2024-05-21T15:24:27.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 -- 172.21.15.78:0/2460695054 shutdown_connections 2024-05-21T15:24:27.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5fa6b0 0x55939a9c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9dc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 --2- 172.21.15.78:0/2460695054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5d9f60 0x55939a9dc360 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 -- 172.21.15.78:0/2460695054 >> 172.21.15.78:0/2460695054 conn(0x55939a60f4e0 msgr2=0x55939a9c2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 -- 172.21.15.78:0/2460695054 shutdown_connections 2024-05-21T15:24:27.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 -- 172.21.15.78:0/2460695054 wait complete. 2024-05-21T15:24:27.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 -- start start 2024-05-21T15:24:27.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.044+0000 7fedac9bc740 1 Processor -- start 2024-05-21T15:24:27.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5d9f60 0x55939a9cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fedac9bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5fa6b0 0x55939aa3cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fedac9bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feda01b2300 con 0x55939a5fa6b0 2024-05-21T15:24:27.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fedac9bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feda01b1c30 con 0x55939a5d9f60 2024-05-21T15:24:27.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fedac9bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feda01aee90 con 0x55939a5f24d0 2024-05-21T15:24:27.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fedac9bc740 10 client.0 _ll_register_callbacks cb 0x7fed88003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5fa6b0 0x55939aa3cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5d9f60 0x55939a9cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5d9f60 0x55939a9cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:41306/0 (socket says 172.21.15.78:41306) 2024-05-21T15:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 -- 172.21.15.78:0/1792244158 learned_addr learned my addr 172.21.15.78:0/1792244158 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed967fc640 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 -- 172.21.15.78:0/1792244158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 msgr2=0x55939a9cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 -- 172.21.15.78:0/1792244158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5fa6b0 msgr2=0x55939aa3cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5fa6b0 0x55939aa3cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 -- 172.21.15.78:0/1792244158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55939a9ce660 con 0x55939a5d9f60 2024-05-21T15:24:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed9effd640 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5d9f60 0x55939a9cd850 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fed90005940 tx=0x7fed9002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e11dc8ce50612e98 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.045+0000 7fed7effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fed7effd640 1 -- 172.21.15.78:0/1792244158 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90055070 con 0x55939a5d9f60 2024-05-21T15:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fed7effd640 1 -- 172.21.15.78:0/1792244158 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fed90005450 con 0x55939a5d9f60 2024-05-21T15:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 1 -- 172.21.15.78:0/1792244158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed9004d070 con 0x55939a5d9f60 2024-05-21T15:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fed7effd640 1 -- 172.21.15.78:0/1792244158 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fed90035040 con 0x55939a5d9f60 2024-05-21T15:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 10 client.27125 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 1 -- 172.21.15.78:0/1792244158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55939aa3e4d0 con 0x55939a5d9f60 2024-05-21T15:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fed7effd640 1 -- 172.21.15.78:0/1792244158 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9004d070 con 0x55939a5d9f60 2024-05-21T15:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fed7effd640 1 -- 172.21.15.78:0/1792244158 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fed90004c40 con 0x55939a5d9f60 2024-05-21T15:24:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 10 client.27125 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 10 client.27125 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:24:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 20 client.27125 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55939a61b390 0x55939aa40c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 4 client.27125 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:24:27.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fedac9bc740 1 -- 172.21.15.78:0/1792244158 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55939a9276c0 con 0x55939a61b390 2024-05-21T15:24:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.046+0000 7fed967fc640 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55939a61b390 0x55939aa40c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fed7effd640 10 client.27125 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:24:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fed967fc640 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55939a61b390 0x55939aa40c40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f5b251e3457b822 server_cookie=d8f80e6ea71a6822 in_seq=0 out_seq=0 2024-05-21T15:24:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fed7effd640 1 -- 172.21.15.78:0/1792244158 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55939a9276c0 con 0x55939a61b390 2024-05-21T15:24:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fed7effd640 10 client.27125 handle_command_reply: tid=0 2024-05-21T15:24:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 client.27125 shutdown 2024-05-21T15:24:27.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 -- 172.21.15.78:0/1792244158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5d9f60 msgr2=0x55939a9cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5d9f60 0x55939a9cd850 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fed90005940 tx=0x7fed9002a410 comp rx=0 tx=0).stop 2024-05-21T15:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 -- 172.21.15.78:0/1792244158 shutdown_connections 2024-05-21T15:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x55939a61b390 0x55939aa40c40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55939a5fa6b0 0x55939aa3cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55939a5f24d0 0x55939a9cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 --2- 172.21.15.78:0/1792244158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55939a5d9f60 0x55939a9cd850 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fed7effd640 0 client.27125 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 -- 172.21.15.78:0/1792244158 >> 172.21.15.78:0/1792244158 conn(0x55939a60f4e0 msgr2=0x55939a9c26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 -- 172.21.15.78:0/1792244158 shutdown_connections 2024-05-21T15:24:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 1 -- 172.21.15.78:0/1792244158 wait complete. 2024-05-21T15:24:27.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.047+0000 7fedac9bc740 20 client.27125 trim_cache size 0 max 16384 2024-05-21T15:24:27.064 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:24:27.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 -- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fed8c0a2d00 msgr2=0x7fed8c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fed8c0a2d00 0x7fed8c0a5210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fed98012fd0 tx=0x7fed9801e000 comp rx=0 tx=0).stop 2024-05-21T15:24:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 -- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 msgr2=0x7feda00ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 0x7feda00ff170 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fed8800e9f0 tx=0x7fed8800eeb0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 -- 172.21.15.78:0/1819619242 shutdown_connections 2024-05-21T15:24:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fed8c0a2d00 0x7fed8c0a5210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feda010c330 0x7feda00ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feda01079d0 0x7feda00ff170 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 --2- 172.21.15.78:0/1819619242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda0101aa0 0x7feda00fec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 -- 172.21.15.78:0/1819619242 >> 172.21.15.78:0/1819619242 conn(0x7feda00770b0 msgr2=0x7feda0068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 -- 172.21.15.78:0/1819619242 shutdown_connections 2024-05-21T15:24:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.048+0000 7feda5a32640 1 -- 172.21.15.78:0/1819619242 wait complete. 2024-05-21T15:24:27.067 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:24:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 -- 172.21.15.78:0/1668366640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10d010 msgr2=0x7ff13c10f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 --2- 172.21.15.78:0/1668366640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10d010 0x7ff13c10f500 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff134009ca0 tx=0x7ff13402a9e0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 -- 172.21.15.78:0/1668366640 shutdown_connections 2024-05-21T15:24:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 --2- 172.21.15.78:0/1668366640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10d010 0x7ff13c10f500 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 --2- 172.21.15.78:0/1668366640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff13c10a660 0x7ff13c10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 --2- 172.21.15.78:0/1668366640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff13c103c00 0x7ff13c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 -- 172.21.15.78:0/1668366640 >> 172.21.15.78:0/1668366640 conn(0x7ff13c0770b0 msgr2=0x7ff13c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 -- 172.21.15.78:0/1668366640 shutdown_connections 2024-05-21T15:24:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.179+0000 7ff141b2c640 1 -- 172.21.15.78:0/1668366640 wait complete. 2024-05-21T15:24:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.180+0000 7ff141b2c640 1 Processor -- start 2024-05-21T15:24:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.180+0000 7ff141b2c640 1 -- start start 2024-05-21T15:24:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.180+0000 7ff141b2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff13c103c00 0x7ff13c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.180+0000 7ff141b2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10a660 0x7ff13c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.180+0000 7ff141b2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff13c10d010 0x7ff13c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.180+0000 7ff141b2c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112dc0 con 0x7ff13c10a660 2024-05-21T15:24:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.180+0000 7ff141b2c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112c40 con 0x7ff13c10d010 2024-05-21T15:24:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.180+0000 7ff141b2c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112f40 con 0x7ff13c103c00 2024-05-21T15:24:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10a660 0x7ff13c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10a660 0x7ff13c199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47794/0 (socket says 172.21.15.78:47794) 2024-05-21T15:24:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 -- 172.21.15.78:0/2673588086 learned_addr learned my addr 172.21.15.78:0/2673588086 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13bfff640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff13c10d010 0x7ff13c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff13c103c00 0x7ff13c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 -- 172.21.15.78:0/2673588086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff13c103c00 msgr2=0x7ff13c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff13c103c00 0x7ff13c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 -- 172.21.15.78:0/2673588086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff13c10d010 msgr2=0x7ff13c19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff13c10d010 0x7ff13c19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 -- 172.21.15.78:0/2673588086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c19e5c0 con 0x7ff13c10a660 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff13affd640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10a660 0x7ff13c199cc0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff12400e9f0 tx=0x7ff12400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff138ff9640 1 -- 172.21.15.78:0/2673588086 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12400cd80 con 0x7ff13c10a660 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff138ff9640 1 -- 172.21.15.78:0/2673588086 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff124005500 con 0x7ff13c10a660 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff138ff9640 1 -- 172.21.15.78:0/2673588086 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124010690 con 0x7ff13c10a660 2024-05-21T15:24:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff141b2c640 1 -- 172.21.15.78:0/2673588086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c19e8b0 con 0x7ff13c10a660 2024-05-21T15:24:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.181+0000 7ff141b2c640 1 -- 172.21.15.78:0/2673588086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c19edb0 con 0x7ff13c10a660 2024-05-21T15:24:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff138ff9640 1 -- 172.21.15.78:0/2673588086 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff1240046c0 con 0x7ff13c10a660 2024-05-21T15:24:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff1489ce740 1 Processor -- start 2024-05-21T15:24:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff1489ce740 1 -- start start 2024-05-21T15:24:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606e5390 0x558460a83bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a84630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff1489ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584609e9c40 con 0x5584606bc4d0 2024-05-21T15:24:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff1489ce740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558460991580 con 0x5584606e5390 2024-05-21T15:24:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff1489ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5584609e5c00 con 0x5584606c46b0 2024-05-21T15:24:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47804/0 (socket says 172.21.15.78:47804) 2024-05-21T15:24:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff13b7fe640 1 -- 172.21.15.78:0/1981530779 learned_addr learned my addr 172.21.15.78:0/1981530779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff13affd640 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a84630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff13bfff640 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606e5390 0x558460a83bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.184+0000 7ff138ff9640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1280a2d00 0x7ff1280a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff13b7fe640 1 -- 172.21.15.78:0/1981530779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 msgr2=0x558460a84630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a84630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff13bfff640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1280a2d00 0x7ff1280a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff13b7fe640 1 -- 172.21.15.78:0/1981530779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606e5390 msgr2=0x558460a83bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606e5390 0x558460a83bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff13b7fe640 1 -- 172.21.15.78:0/1981530779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584606fd3f0 con 0x5584606bc4d0 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff138ff9640 1 -- 172.21.15.78:0/2673588086 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7ff1240cf930 con 0x7ff13c10a660 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a840f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009c80 tx=0x7ff12c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4c557971d1a831 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff131ffb640 1 -- 172.21.15.78:0/1981530779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c012450 con 0x5584606bc4d0 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff131ffb640 1 -- 172.21.15.78:0/1981530779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff12c012ad0 con 0x5584606bc4d0 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff13bfff640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1280a2d00 0x7ff1280a5210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff134036550 tx=0x7ff13402a9e0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.185+0000 7ff131ffb640 1 -- 172.21.15.78:0/1981530779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00fc50 con 0x5584606bc4d0 2024-05-21T15:24:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 -- 172.21.15.78:0/1981530779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 msgr2=0x558460a840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a840f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009c80 tx=0x7ff12c0117b0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 -- 172.21.15.78:0/1981530779 shutdown_connections 2024-05-21T15:24:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a84630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a840f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 --2- 172.21.15.78:0/1981530779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606e5390 0x558460a83bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 -- 172.21.15.78:0/1981530779 >> 172.21.15.78:0/1981530779 conn(0x5584606d94e0 msgr2=0x558460a804d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 -- 172.21.15.78:0/1981530779 shutdown_connections 2024-05-21T15:24:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 -- 172.21.15.78:0/1981530779 wait complete. 2024-05-21T15:24:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.186+0000 7ff1489ce740 1 -- start start 2024-05-21T15:24:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff1489ce740 1 Processor -- start 2024-05-21T15:24:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606bc4d0 0x558460a92750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a92c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a9e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff1489ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558460a95480 con 0x5584606e5390 2024-05-21T15:24:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff1489ce740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5584609a4c40 con 0x5584606c46b0 2024-05-21T15:24:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff1489ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558460a95600 con 0x5584606bc4d0 2024-05-21T15:24:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff1489ce740 10 client.0 _ll_register_callbacks cb 0x5584606fcbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a9e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a92c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a9e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47808/0 (socket says 172.21.15.78:47808) 2024-05-21T15:24:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13bfff640 1 -- 172.21.15.78:0/2516932985 learned_addr learned my addr 172.21.15.78:0/2516932985 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606bc4d0 0x558460a92750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13affd640 1 -- 172.21.15.78:0/2516932985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606bc4d0 msgr2=0x558460a92750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13affd640 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606bc4d0 0x558460a92750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13affd640 1 -- 172.21.15.78:0/2516932985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 msgr2=0x558460a9e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13affd640 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a9e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13affd640 1 -- 172.21.15.78:0/2516932985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584607c77d0 con 0x5584606c46b0 2024-05-21T15:24:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.187+0000 7ff13bfff640 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a9e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff13affd640 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a92c90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff1240ccff0 tx=0x7ff124009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87310cda2dc39b3b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff11affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:24:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff11affd640 1 -- 172.21.15.78:0/2516932985 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12409b720 con 0x5584606c46b0 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff11affd640 1 -- 172.21.15.78:0/2516932985 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7ff1240a47a0 con 0x5584606c46b0 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 1 -- 172.21.15.78:0/2516932985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff12c012ad0 con 0x5584606c46b0 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff11affd640 1 -- 172.21.15.78:0/2516932985 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff12409c4c0 con 0x5584606c46b0 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 10 client.27137 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 1 -- 172.21.15.78:0/2516932985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558460aa1be0 con 0x5584606c46b0 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff11affd640 1 -- 172.21.15.78:0/2516932985 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff12c012ad0 con 0x5584606c46b0 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff11affd640 1 -- 172.21.15.78:0/2516932985 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff12409ac80 con 0x5584606c46b0 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 10 client.27137 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 10 client.27137 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:24:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 20 client.27137 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5584606a3f60 0x558460aa50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 4 client.27137 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff1489ce740 1 -- 172.21.15.78:0/2516932985 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584609f7370 con 0x5584606a3f60 2024-05-21T15:24:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.188+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5584606a3f60 0x558460aa50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.189+0000 7ff11affd640 10 client.27137 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:24:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.189+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5584606a3f60 0x558460aa50f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c59866c81aa968fc server_cookie=27b6a508f13ffba3 in_seq=0 out_seq=0 2024-05-21T15:24:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.189+0000 7ff11affd640 1 -- 172.21.15.78:0/2516932985 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584609f7370 con 0x5584606a3f60 2024-05-21T15:24:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.189+0000 7ff11affd640 10 client.27137 handle_command_reply: tid=0 2024-05-21T15:24:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 client.27137 shutdown 2024-05-21T15:24:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 -- 172.21.15.78:0/2516932985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 msgr2=0x558460a92c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a92c90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff1240ccff0 tx=0x7ff124009310 comp rx=0 tx=0).stop 2024-05-21T15:24:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 -- 172.21.15.78:0/2516932985 shutdown_connections 2024-05-21T15:24:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5584606a3f60 0x558460aa50f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a92c90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a9e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff11affd640 0 client.27137 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:24:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2516932985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606bc4d0 0x558460a92750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 -- 172.21.15.78:0/2516932985 >> 172.21.15.78:0/2516932985 conn(0x5584606d94e0 msgr2=0x558460a85a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 -- 172.21.15.78:0/2516932985 shutdown_connections 2024-05-21T15:24:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 1 -- 172.21.15.78:0/2516932985 wait complete. 2024-05-21T15:24:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.190+0000 7ff1489ce740 20 client.27137 trim_cache size 0 max 16384 2024-05-21T15:24:27.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.203+0000 7ff1489ce740 1 Processor -- start 2024-05-21T15:24:27.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.203+0000 7ff1489ce740 1 -- start start 2024-05-21T15:24:27.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.203+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a91520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.203+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a9e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.203+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606e5390 0x558460a8bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.203+0000 7ff1489ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1acae0 con 0x5584606bc4d0 2024-05-21T15:24:27.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.203+0000 7ff1489ce740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1ae620 con 0x5584606c46b0 2024-05-21T15:24:27.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.203+0000 7ff1489ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1ad1b0 con 0x5584606e5390 2024-05-21T15:24:27.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a91520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a91520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47824/0 (socket says 172.21.15.78:47824) 2024-05-21T15:24:27.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 -- 172.21.15.78:0/3736947750 learned_addr learned my addr 172.21.15.78:0/3736947750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13affd640 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a9e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13bfff640 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606e5390 0x558460a8bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 -- 172.21.15.78:0/3736947750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606e5390 msgr2=0x558460a8bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606e5390 0x558460a8bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 -- 172.21.15.78:0/3736947750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 msgr2=0x558460a9e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a9e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 -- 172.21.15.78:0/3736947750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558460a934b0 con 0x5584606bc4d0 2024-05-21T15:24:27.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13affd640 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a9e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a91520 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00f840 tx=0x7ff12c00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66accd831b1f226e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff1317fa640 1 -- 172.21.15.78:0/3736947750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0098a0 con 0x5584606bc4d0 2024-05-21T15:24:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff1317fa640 1 -- 172.21.15.78:0/3736947750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff12c005590 con 0x5584606bc4d0 2024-05-21T15:24:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.204+0000 7ff1317fa640 1 -- 172.21.15.78:0/3736947750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00fe80 con 0x5584606bc4d0 2024-05-21T15:24:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 -- 172.21.15.78:0/3736947750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 msgr2=0x558460a91520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a91520 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00f840 tx=0x7ff12c00f7b0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 -- 172.21.15.78:0/3736947750 shutdown_connections 2024-05-21T15:24:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606c46b0 0x558460a9e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606e5390 0x558460a8bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 --2- 172.21.15.78:0/3736947750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606bc4d0 0x558460a91520 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 -- 172.21.15.78:0/3736947750 >> 172.21.15.78:0/3736947750 conn(0x5584606d94e0 msgr2=0x558460a90e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 -- 172.21.15.78:0/3736947750 shutdown_connections 2024-05-21T15:24:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 -- 172.21.15.78:0/3736947750 wait complete. 2024-05-21T15:24:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.205+0000 7ff1489ce740 1 -- start start 2024-05-21T15:24:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff1489ce740 1 Processor -- start 2024-05-21T15:24:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606bc4d0 0x558460a95850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a95d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff1489ce740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a962d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff1489ce740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1adf50 con 0x5584606e5390 2024-05-21T15:24:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff1489ce740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1ad880 con 0x5584606bc4d0 2024-05-21T15:24:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff1489ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1aaae0 con 0x5584606c46b0 2024-05-21T15:24:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff1489ce740 10 client.0 _ll_register_callbacks cb 0x7ff12409bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a95d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47826/0 (socket says 172.21.15.78:47826) 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13bfff640 1 -- 172.21.15.78:0/2477216532 learned_addr learned my addr 172.21.15.78:0/2477216532 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606bc4d0 0x558460a95850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13bfff640 1 -- 172.21.15.78:0/2477216532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 msgr2=0x558460a95d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13bfff640 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a95d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13bfff640 1 -- 172.21.15.78:0/2477216532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606bc4d0 msgr2=0x558460a95850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13bfff640 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606bc4d0 0x558460a95850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13bfff640 1 -- 172.21.15.78:0/2477216532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558460b0b550 con 0x5584606e5390 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.206+0000 7ff13affd640 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a95d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff13bfff640 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a962d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff134009ca0 tx=0x7ff1340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae85e958f920beef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff11affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff11affd640 1 -- 172.21.15.78:0/2477216532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134059060 con 0x5584606e5390 2024-05-21T15:24:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff11affd640 1 -- 172.21.15.78:0/2477216532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7ff13402f640 con 0x5584606e5390 2024-05-21T15:24:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 1 -- 172.21.15.78:0/2477216532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff12c005590 con 0x5584606e5390 2024-05-21T15:24:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff11affd640 1 -- 172.21.15.78:0/2477216532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff134060050 con 0x5584606e5390 2024-05-21T15:24:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 10 client.17955 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 1 -- 172.21.15.78:0/2477216532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558460b0c7d0 con 0x5584606e5390 2024-05-21T15:24:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff11affd640 1 -- 172.21.15.78:0/2477216532 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff12c005590 con 0x5584606e5390 2024-05-21T15:24:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff11affd640 1 -- 172.21.15.78:0/2477216532 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff134036af0 con 0x5584606e5390 2024-05-21T15:24:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 10 client.17955 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 10 client.17955 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:24:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 20 client.17955 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5584606793a0 0x558460b0ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 4 client.17955 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:24:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.207+0000 7ff1489ce740 1 -- 172.21.15.78:0/2477216532 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584609f16c0 con 0x5584606793a0 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5584606793a0 0x558460b0ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff11affd640 10 client.17955 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff13b7fe640 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5584606793a0 0x558460b0ef60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfa08bd2cbf8d9de server_cookie=b8dc95fcd0869cf2 in_seq=0 out_seq=0 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff11affd640 1 -- 172.21.15.78:0/2477216532 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584609f16c0 con 0x5584606793a0 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff11affd640 10 client.17955 handle_command_reply: tid=0 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 client.17955 shutdown 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 -- 172.21.15.78:0/2477216532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 msgr2=0x558460a962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a962d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff134009ca0 tx=0x7ff1340095c0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 -- 172.21.15.78:0/2477216532 shutdown_connections 2024-05-21T15:24:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5584606793a0 0x558460b0ef60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584606e5390 0x558460a962d0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584606c46b0 0x558460a95d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff11affd640 0 client.17955 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:24:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 --2- 172.21.15.78:0/2477216532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5584606bc4d0 0x558460a95850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 -- 172.21.15.78:0/2477216532 >> 172.21.15.78:0/2477216532 conn(0x5584606d94e0 msgr2=0x558460a83ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 -- 172.21.15.78:0/2477216532 shutdown_connections 2024-05-21T15:24:27.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.208+0000 7ff1489ce740 1 -- 172.21.15.78:0/2477216532 wait complete. 2024-05-21T15:24:27.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff1489ce740 20 client.17955 trim_cache size 0 max 16384 2024-05-21T15:24:27.225 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:24:27.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 -- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1280a2d00 msgr2=0x7ff1280a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1280a2d00 0x7ff1280a5210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff134036550 tx=0x7ff13402a9e0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 -- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10a660 msgr2=0x7ff13c199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10a660 0x7ff13c199cc0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff12400e9f0 tx=0x7ff12400edf0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 -- 172.21.15.78:0/2673588086 shutdown_connections 2024-05-21T15:24:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff13c10d010 0x7ff13c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff1280a2d00 0x7ff1280a5210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c10a660 0x7ff13c199cc0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 --2- 172.21.15.78:0/2673588086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff13c103c00 0x7ff13c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 -- 172.21.15.78:0/2673588086 >> 172.21.15.78:0/2673588086 conn(0x7ff13c0770b0 msgr2=0x7ff13c100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 -- 172.21.15.78:0/2673588086 shutdown_connections 2024-05-21T15:24:27.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.209+0000 7ff141b2c640 1 -- 172.21.15.78:0/2673588086 wait complete. 2024-05-21T15:24:27.228 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:24:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 -- 172.21.15.78:0/23189938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c10c330 msgr2=0x7f157c10e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 --2- 172.21.15.78:0/23189938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c10c330 0x7f157c10e750 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1574009ca0 tx=0x7f157402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 -- 172.21.15.78:0/23189938 shutdown_connections 2024-05-21T15:24:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 --2- 172.21.15.78:0/23189938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c10c330 0x7f157c10e750 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 --2- 172.21.15.78:0/23189938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c101aa0 0x7f157c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 --2- 172.21.15.78:0/23189938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f157c1079d0 0x7f157c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 -- 172.21.15.78:0/23189938 >> 172.21.15.78:0/23189938 conn(0x7f157c0770b0 msgr2=0x7f157c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 -- 172.21.15.78:0/23189938 shutdown_connections 2024-05-21T15:24:27.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 -- 172.21.15.78:0/23189938 wait complete. 2024-05-21T15:24:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 Processor -- start 2024-05-21T15:24:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.338+0000 7f1583d72640 1 -- start start 2024-05-21T15:24:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1583d72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c101aa0 0x7f157c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1583d72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c1079d0 0x7f157c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1583d72640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f157c10c330 0x7f157c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1583d72640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f157c111ce0 con 0x7f157c101aa0 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1583d72640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f157c111b60 con 0x7f157c10c330 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1583d72640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f157c111e60 con 0x7f157c1079d0 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c101aa0 0x7f157c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c101aa0 0x7f157c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47848/0 (socket says 172.21.15.78:47848) 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 -- 172.21.15.78:0/38911603 learned_addr learned my addr 172.21.15.78:0/38911603 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f158230f640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f157c10c330 0x7f157c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f158130d640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c1079d0 0x7f157c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 -- 172.21.15.78:0/38911603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c1079d0 msgr2=0x7f157c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c1079d0 0x7f157c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 -- 172.21.15.78:0/38911603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f157c10c330 msgr2=0x7f157c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f157c10c330 0x7f157c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 -- 172.21.15.78:0/38911603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c19d440 con 0x7f157c101aa0 2024-05-21T15:24:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f1581b0e640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c101aa0 0x7f157c1985e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f157800e9a0 tx=0x7f157800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f156effd640 1 -- 172.21.15.78:0/38911603 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157800cd80 con 0x7f157c101aa0 2024-05-21T15:24:27.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.339+0000 7f156effd640 1 -- 172.21.15.78:0/38911603 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15780035f0 con 0x7f157c101aa0 2024-05-21T15:24:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.340+0000 7f1583d72640 1 -- 172.21.15.78:0/38911603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c19d730 con 0x7f157c101aa0 2024-05-21T15:24:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.340+0000 7f156effd640 1 -- 172.21.15.78:0/38911603 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578010670 con 0x7f157c101aa0 2024-05-21T15:24:27.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.340+0000 7f1583d72640 1 -- 172.21.15.78:0/38911603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1a50b0 con 0x7f157c101aa0 2024-05-21T15:24:27.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.342+0000 7f156effd640 1 -- 172.21.15.78:0/38911603 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f15780037a0 con 0x7f157c101aa0 2024-05-21T15:24:27.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.342+0000 7f158ac16740 1 Processor -- start 2024-05-21T15:24:27.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158ac16740 1 -- start start 2024-05-21T15:24:27.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158ac16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b603d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158ac16740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b604260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158ac16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158ac16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b39b569c40 con 0x55b39b2446b0 2024-05-21T15:24:27.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158ac16740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b39b511580 con 0x55b39b23c4d0 2024-05-21T15:24:27.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158ac16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b39b565c00 con 0x55b39b265390 2024-05-21T15:24:27.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158130d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f1581b0e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b604260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158130d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47860/0 (socket says 172.21.15.78:47860) 2024-05-21T15:24:27.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158130d640 1 -- 172.21.15.78:0/968530485 learned_addr learned my addr 172.21.15.78:0/968530485 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158230f640 1 --2- 172.21.15.78:0/968530485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b603d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f156effd640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15680a2d80 0x7f15680a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158130d640 1 -- 172.21.15.78:0/968530485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 msgr2=0x55b39b603d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158130d640 1 --2- 172.21.15.78:0/968530485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b603d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158130d640 1 -- 172.21.15.78:0/968530485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 msgr2=0x55b39b604260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158130d640 1 --2- 172.21.15.78:0/968530485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b604260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.343+0000 7f158230f640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15680a2d80 0x7f15680a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.344+0000 7f158130d640 1 -- 172.21.15.78:0/968530485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39b27d3f0 con 0x55b39b2446b0 2024-05-21T15:24:27.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.344+0000 7f156effd640 1 -- 172.21.15.78:0/38911603 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f15780cf6b0 con 0x7f157c101aa0 2024-05-21T15:24:27.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.344+0000 7f158130d640 1 --2- 172.21.15.78:0/968530485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6047a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1570009c20 tx=0x7f1570011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128bbe55decb5b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.344+0000 7f1567fff640 1 -- 172.21.15.78:0/968530485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570004030 con 0x55b39b2446b0 2024-05-21T15:24:27.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.344+0000 7f1567fff640 1 -- 172.21.15.78:0/968530485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15700041d0 con 0x55b39b2446b0 2024-05-21T15:24:27.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.344+0000 7f1567fff640 1 -- 172.21.15.78:0/968530485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570019740 con 0x55b39b2446b0 2024-05-21T15:24:27.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 -- 172.21.15.78:0/968530485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 msgr2=0x55b39b6047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 --2- 172.21.15.78:0/968530485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6047a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1570009c20 tx=0x7f1570011810 comp rx=0 tx=0).stop 2024-05-21T15:24:27.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158230f640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15680a2d80 0x7f15680a5290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1574004af0 tx=0x7f157402a8c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 -- 172.21.15.78:0/968530485 shutdown_connections 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 --2- 172.21.15.78:0/968530485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b604260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 --2- 172.21.15.78:0/968530485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6047a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 --2- 172.21.15.78:0/968530485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b603d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 -- 172.21.15.78:0/968530485 >> 172.21.15.78:0/968530485 conn(0x55b39b2594e0 msgr2=0x55b39b6004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 -- 172.21.15.78:0/968530485 shutdown_connections 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 -- 172.21.15.78:0/968530485 wait complete. 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.345+0000 7f158ac16740 1 -- start start 2024-05-21T15:24:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158ac16740 1 Processor -- start 2024-05-21T15:24:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158ac16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b23c4d0 0x55b39b60e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158ac16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b60e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158ac16740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b265390 0x55b39b6144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158ac16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b39b610fb0 con 0x55b39b2446b0 2024-05-21T15:24:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158ac16740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b39b524c40 con 0x55b39b265390 2024-05-21T15:24:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158ac16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b39b611130 con 0x55b39b23c4d0 2024-05-21T15:24:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158ac16740 10 client.0 _ll_register_callbacks cb 0x55b39b27cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158130d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b60e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158130d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b60e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47862/0 (socket says 172.21.15.78:47862) 2024-05-21T15:24:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158130d640 1 -- 172.21.15.78:0/1452960418 learned_addr learned my addr 172.21.15.78:0/1452960418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f1581b0e640 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b23c4d0 0x55b39b60e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158230f640 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b265390 0x55b39b6144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158130d640 1 -- 172.21.15.78:0/1452960418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b23c4d0 msgr2=0x55b39b60e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.346+0000 7f158130d640 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b23c4d0 0x55b39b60e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158130d640 1 -- 172.21.15.78:0/1452960418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b265390 msgr2=0x55b39b6144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158130d640 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b265390 0x55b39b6144a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158130d640 1 -- 172.21.15.78:0/1452960418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39b3477d0 con 0x55b39b2446b0 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158230f640 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b265390 0x55b39b6144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158130d640 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b60e920 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1570005bc0 tx=0x7f1570009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d58ec759cb52c47 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f15657fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f15657fa640 1 -- 172.21.15.78:0/1452960418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570028020 con 0x55b39b2446b0 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f15657fa640 1 -- 172.21.15.78:0/1452960418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f1570004380 con 0x55b39b2446b0 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158ac16740 1 -- 172.21.15.78:0/1452960418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15700041d0 con 0x55b39b2446b0 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f15657fa640 1 -- 172.21.15.78:0/1452960418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1570033050 con 0x55b39b2446b0 2024-05-21T15:24:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158ac16740 10 client.17967 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158ac16740 1 -- 172.21.15.78:0/1452960418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b39b61d750 con 0x55b39b2446b0 2024-05-21T15:24:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f15657fa640 1 -- 172.21.15.78:0/1452960418 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15700041d0 con 0x55b39b2446b0 2024-05-21T15:24:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f15657fa640 1 -- 172.21.15.78:0/1452960418 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1570018070 con 0x55b39b2446b0 2024-05-21T15:24:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158ac16740 10 client.17967 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.347+0000 7f158ac16740 10 client.17967 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:24:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.348+0000 7f158ac16740 20 client.17967 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.348+0000 7f158ac16740 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b39b223f60 0x55b39b620ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.348+0000 7f158ac16740 4 client.17967 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.348+0000 7f158ac16740 1 -- 172.21.15.78:0/1452960418 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39b577370 con 0x55b39b223f60 2024-05-21T15:24:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.348+0000 7f1581b0e640 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b39b223f60 0x55b39b620ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.348+0000 7f1581b0e640 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b39b223f60 0x55b39b620ab0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea6a0f1b0866f164 server_cookie=3bc08f9ccf279af9 in_seq=0 out_seq=0 2024-05-21T15:24:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.348+0000 7f15657fa640 10 client.17967 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:24:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f15657fa640 1 -- 172.21.15.78:0/1452960418 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b39b577370 con 0x55b39b223f60 2024-05-21T15:24:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f15657fa640 10 client.17967 handle_command_reply: tid=0 2024-05-21T15:24:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 client.17967 shutdown 2024-05-21T15:24:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 -- 172.21.15.78:0/1452960418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 msgr2=0x55b39b60e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b60e920 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1570005bc0 tx=0x7f1570009dd0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 -- 172.21.15.78:0/1452960418 shutdown_connections 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b39b223f60 0x55b39b620ab0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b265390 0x55b39b6144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b60e920 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 --2- 172.21.15.78:0/1452960418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b23c4d0 0x55b39b60e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f15657fa640 0 client.17967 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 -- 172.21.15.78:0/1452960418 >> 172.21.15.78:0/1452960418 conn(0x55b39b2594e0 msgr2=0x55b39b605a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 -- 172.21.15.78:0/1452960418 shutdown_connections 2024-05-21T15:24:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 1 -- 172.21.15.78:0/1452960418 wait complete. 2024-05-21T15:24:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.349+0000 7f158ac16740 20 client.17967 trim_cache size 0 max 16384 2024-05-21T15:24:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.363+0000 7f158ac16740 1 Processor -- start 2024-05-21T15:24:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.363+0000 7f158ac16740 1 -- start start 2024-05-21T15:24:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.363+0000 7f158ac16740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b604730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.363+0000 7f158ac16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.363+0000 7f158ac16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b60ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.363+0000 7f158ac16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1acaa0 con 0x55b39b2446b0 2024-05-21T15:24:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.363+0000 7f158ac16740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1ae5e0 con 0x55b39b23c4d0 2024-05-21T15:24:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.363+0000 7f158ac16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1ad170 con 0x55b39b265390 2024-05-21T15:24:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47874/0 (socket says 172.21.15.78:47874) 2024-05-21T15:24:27.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 -- 172.21.15.78:0/479891587 learned_addr learned my addr 172.21.15.78:0/479891587 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f1581b0e640 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b604730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158230f640 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b60ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 -- 172.21.15.78:0/479891587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 msgr2=0x55b39b60ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b60ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 -- 172.21.15.78:0/479891587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 msgr2=0x55b39b604730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b604730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 -- 172.21.15.78:0/479891587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39b603ab0 con 0x55b39b2446b0 2024-05-21T15:24:27.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f158130d640 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6087e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1570000c00 tx=0x7f15700095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18a86f4351da86d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f15677fe640 1 -- 172.21.15.78:0/479891587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570028070 con 0x55b39b2446b0 2024-05-21T15:24:27.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f15677fe640 1 -- 172.21.15.78:0/479891587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1570011d10 con 0x55b39b2446b0 2024-05-21T15:24:27.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.364+0000 7f15677fe640 1 -- 172.21.15.78:0/479891587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157001f040 con 0x55b39b2446b0 2024-05-21T15:24:27.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 -- 172.21.15.78:0/479891587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 msgr2=0x55b39b6087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6087e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1570000c00 tx=0x7f15700095b0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 -- 172.21.15.78:0/479891587 shutdown_connections 2024-05-21T15:24:27.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b60ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b6087e0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 --2- 172.21.15.78:0/479891587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b604730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 -- 172.21.15.78:0/479891587 >> 172.21.15.78:0/479891587 conn(0x55b39b2594e0 msgr2=0x55b39b60f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 -- 172.21.15.78:0/479891587 shutdown_connections 2024-05-21T15:24:27.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 -- 172.21.15.78:0/479891587 wait complete. 2024-05-21T15:24:27.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 -- start start 2024-05-21T15:24:27.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.365+0000 7f158ac16740 1 Processor -- start 2024-05-21T15:24:27.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158ac16740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b615830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158ac16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b615d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158ac16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b6162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158ac16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1adf10 con 0x55b39b2446b0 2024-05-21T15:24:27.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158ac16740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1ad840 con 0x55b39b23c4d0 2024-05-21T15:24:27.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158ac16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1aaaa0 con 0x55b39b265390 2024-05-21T15:24:27.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158ac16740 10 client.0 _ll_register_callbacks cb 0x7f1578097c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158130d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b615d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158130d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b615d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47890/0 (socket says 172.21.15.78:47890) 2024-05-21T15:24:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158130d640 1 -- 172.21.15.78:0/3656327514 learned_addr learned my addr 172.21.15.78:0/3656327514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158230f640 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b6162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f1581b0e640 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b615830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158130d640 1 -- 172.21.15.78:0/3656327514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 msgr2=0x55b39b6162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158130d640 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b6162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158130d640 1 -- 172.21.15.78:0/3656327514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 msgr2=0x55b39b615830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158130d640 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b615830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.366+0000 7f158130d640 1 -- 172.21.15.78:0/3656327514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39b686f50 con 0x55b39b2446b0 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158130d640 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b615d70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f1570014040 tx=0x7f1570009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e626ca0660ea74 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f15657fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f15657fa640 1 -- 172.21.15.78:0/3656327514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570028040 con 0x55b39b2446b0 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 1 -- 172.21.15.78:0/3656327514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1570011d10 con 0x55b39b2446b0 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f15657fa640 1 -- 172.21.15.78:0/3656327514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f1570003770 con 0x55b39b2446b0 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f15657fa640 1 -- 172.21.15.78:0/3656327514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1570048050 con 0x55b39b2446b0 2024-05-21T15:24:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 10 client.17973 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 1 -- 172.21.15.78:0/3656327514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b39b688190 con 0x55b39b2446b0 2024-05-21T15:24:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f15657fa640 1 -- 172.21.15.78:0/3656327514 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1570011d10 con 0x55b39b2446b0 2024-05-21T15:24:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f15657fa640 1 -- 172.21.15.78:0/3656327514 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1570018070 con 0x55b39b2446b0 2024-05-21T15:24:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 10 client.17973 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 10 client.17973 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:24:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 20 client.17973 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b39b1f93a0 0x55b39b68a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 4 client.17973 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:24:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.367+0000 7f158ac16740 1 -- 172.21.15.78:0/3656327514 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b39b5716c0 con 0x55b39b1f93a0 2024-05-21T15:24:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f1581b0e640 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b39b1f93a0 0x55b39b68a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f15657fa640 10 client.17973 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:24:27.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f1581b0e640 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b39b1f93a0 0x55b39b68a840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b89f0b0d40a3e4f8 server_cookie=5dd00e9f15a13f6f in_seq=0 out_seq=0 2024-05-21T15:24:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f15657fa640 1 -- 172.21.15.78:0/3656327514 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b39b5716c0 con 0x55b39b1f93a0 2024-05-21T15:24:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f15657fa640 10 client.17973 handle_command_reply: tid=0 2024-05-21T15:24:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 client.17973 shutdown 2024-05-21T15:24:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 -- 172.21.15.78:0/3656327514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 msgr2=0x55b39b615d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b615d70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f1570014040 tx=0x7f1570009940 comp rx=0 tx=0).stop 2024-05-21T15:24:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 -- 172.21.15.78:0/3656327514 shutdown_connections 2024-05-21T15:24:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55b39b1f93a0 0x55b39b68a840 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b39b265390 0x55b39b6162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b39b2446b0 0x55b39b615d70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 --2- 172.21.15.78:0/3656327514 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b39b23c4d0 0x55b39b615830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 -- 172.21.15.78:0/3656327514 >> 172.21.15.78:0/3656327514 conn(0x55b39b2594e0 msgr2=0x55b39b604450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f15657fa640 0 client.17973 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:24:27.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 -- 172.21.15.78:0/3656327514 shutdown_connections 2024-05-21T15:24:27.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.368+0000 7f158ac16740 1 -- 172.21.15.78:0/3656327514 wait complete. 2024-05-21T15:24:27.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f158ac16740 20 client.17973 trim_cache size 0 max 16384 2024-05-21T15:24:27.384 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:24:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 -- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15680a2d80 msgr2=0x7f15680a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15680a2d80 0x7f15680a5290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1574004af0 tx=0x7f157402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 -- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c101aa0 msgr2=0x7f157c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c101aa0 0x7f157c1985e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f157800e9a0 tx=0x7f157800ee60 comp rx=0 tx=0).stop 2024-05-21T15:24:27.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 -- 172.21.15.78:0/38911603 shutdown_connections 2024-05-21T15:24:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f157c10c330 0x7f157c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f15680a2d80 0x7f15680a5290 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f157c1079d0 0x7f157c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 --2- 172.21.15.78:0/38911603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f157c101aa0 0x7f157c1985e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 -- 172.21.15.78:0/38911603 >> 172.21.15.78:0/38911603 conn(0x7f157c0770b0 msgr2=0x7f157c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 -- 172.21.15.78:0/38911603 shutdown_connections 2024-05-21T15:24:27.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.369+0000 7f1583d72640 1 -- 172.21.15.78:0/38911603 wait complete. 2024-05-21T15:24:27.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:24:27.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 -- 172.21.15.78:0/218890285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0107a10 msgr2=0x7fdca01015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 --2- 172.21.15.78:0/218890285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0107a10 0x7fdca01015a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009900 tx=0x7fdc9c02a410 comp rx=0 tx=0).stop 2024-05-21T15:24:27.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 -- 172.21.15.78:0/218890285 shutdown_connections 2024-05-21T15:24:27.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 --2- 172.21.15.78:0/218890285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0101ae0 0x7fdca0101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 --2- 172.21.15.78:0/218890285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca010c370 0x7fdca010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 --2- 172.21.15.78:0/218890285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0107a10 0x7fdca01015a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 -- 172.21.15.78:0/218890285 >> 172.21.15.78:0/218890285 conn(0x7fdca00770c0 msgr2=0x7fdca0077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 -- 172.21.15.78:0/218890285 shutdown_connections 2024-05-21T15:24:27.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 -- 172.21.15.78:0/218890285 wait complete. 2024-05-21T15:24:27.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.497+0000 7fdca80f6640 1 Processor -- start 2024-05-21T15:24:27.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca80f6640 1 -- start start 2024-05-21T15:24:27.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca80f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0101ae0 0x7fdca01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca80f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107a10 0x7fdca0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca80f6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca010c370 0x7fdca019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca80f6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111d20 con 0x7fdca010c370 2024-05-21T15:24:27.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca80f6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111ba0 con 0x7fdca0107a10 2024-05-21T15:24:27.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca80f6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111ea0 con 0x7fdca0101ae0 2024-05-21T15:24:27.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca010c370 0x7fdca019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca010c370 0x7fdca019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47910/0 (socket says 172.21.15.78:47910) 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 -- 172.21.15.78:0/3890331996 learned_addr learned my addr 172.21.15.78:0/3890331996 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca5691640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107a10 0x7fdca0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca5e92640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0101ae0 0x7fdca01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 -- 172.21.15.78:0/3890331996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0101ae0 msgr2=0x7fdca01997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0101ae0 0x7fdca01997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 -- 172.21.15.78:0/3890331996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107a10 msgr2=0x7fdca0199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107a10 0x7fdca0199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 -- 172.21.15.78:0/3890331996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca019e610 con 0x7fdca010c370 2024-05-21T15:24:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca5691640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107a10 0x7fdca0199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.498+0000 7fdca6693640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca010c370 0x7fdca019dfd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800eaa0 tx=0x7fdc9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.499+0000 7fdc96ffd640 1 -- 172.21.15.78:0/3890331996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800cdb0 con 0x7fdca010c370 2024-05-21T15:24:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.499+0000 7fdca80f6640 1 -- 172.21.15.78:0/3890331996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca019e900 con 0x7fdca010c370 2024-05-21T15:24:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.499+0000 7fdc96ffd640 1 -- 172.21.15.78:0/3890331996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc98005020 con 0x7fdca010c370 2024-05-21T15:24:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.499+0000 7fdc96ffd640 1 -- 172.21.15.78:0/3890331996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980116d0 con 0x7fdca010c370 2024-05-21T15:24:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.499+0000 7fdca80f6640 1 -- 172.21.15.78:0/3890331996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a6280 con 0x7fdca010c370 2024-05-21T15:24:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.501+0000 7fdcaef95740 1 Processor -- start 2024-05-21T15:24:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.501+0000 7fdc96ffd640 1 -- 172.21.15.78:0/3890331996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdc980054d0 con 0x7fdca010c370 2024-05-21T15:24:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.501+0000 7fdcaef95740 1 -- start start 2024-05-21T15:24:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767618d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 0x555767619260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x5557676197a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdcaef95740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55576757ec40 con 0x5557672596b0 2024-05-21T15:24:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdcaef95740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555767526580 con 0x5557672514d0 2024-05-21T15:24:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdcaef95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55576757ac00 con 0x55576727a390 2024-05-21T15:24:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767618d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767618d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44790/0 (socket says 172.21.15.78:44790) 2024-05-21T15:24:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 -- 172.21.15.78:0/1677064321 learned_addr learned my addr 172.21.15.78:0/1677064321 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x5557676197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5e92640 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 0x555767619260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdc96ffd640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc8c0a2d00 0x7fdc8c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 -- 172.21.15.78:0/1677064321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 msgr2=0x555767619260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 0x555767619260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 -- 172.21.15.78:0/1677064321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 msgr2=0x5557676197a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x5557676197a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 -- 172.21.15.78:0/1677064321 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557672923f0 con 0x55576727a390 2024-05-21T15:24:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca6693640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc8c0a2d00 0x7fdc8c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdca5691640 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767618d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdc90011b80 tx=0x7fdc90005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc23b59aa84f69d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.502+0000 7fdc96ffd640 1 -- 172.21.15.78:0/3890331996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fdc980cf6b0 con 0x7fdca010c370 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.503+0000 7fdc8bfff640 1 -- 172.21.15.78:0/1677064321 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000fc50 con 0x55576727a390 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.503+0000 7fdc8bfff640 1 -- 172.21.15.78:0/1677064321 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9000fdf0 con 0x55576727a390 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.503+0000 7fdc8bfff640 1 -- 172.21.15.78:0/1677064321 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90003a90 con 0x55576727a390 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 -- 172.21.15.78:0/1677064321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 msgr2=0x555767618d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdca6693640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc8c0a2d00 0x7fdc8c0a5210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdc98098560 tx=0x7fdc980866c0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767618d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdc90011b80 tx=0x7fdc90005850 comp rx=0 tx=0).stop 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 -- 172.21.15.78:0/1677064321 shutdown_connections 2024-05-21T15:24:27.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 0x555767619260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x5557676197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 --2- 172.21.15.78:0/1677064321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767618d20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 -- 172.21.15.78:0/1677064321 >> 172.21.15.78:0/1677064321 conn(0x55576726e4e0 msgr2=0x5557676154d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 -- 172.21.15.78:0/1677064321 shutdown_connections 2024-05-21T15:24:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 -- 172.21.15.78:0/1677064321 wait complete. 2024-05-21T15:24:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 -- start start 2024-05-21T15:24:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.504+0000 7fdcaef95740 1 Processor -- start 2024-05-21T15:24:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767624e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672596b0 0x5557676253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767623310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdcaef95740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555767625fb0 con 0x5557672514d0 2024-05-21T15:24:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdcaef95740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555767539c40 con 0x5557672596b0 2024-05-21T15:24:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdcaef95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555767626130 con 0x55576727a390 2024-05-21T15:24:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdcaef95740 10 client.0 _ll_register_callbacks cb 0x555767291be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767624e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767624e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47918/0 (socket says 172.21.15.78:47918) 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 -- 172.21.15.78:0/166145871 learned_addr learned my addr 172.21.15.78:0/166145871 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5e92640 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672596b0 0x5557676253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767623310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 -- 172.21.15.78:0/166145871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 msgr2=0x555767623310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767623310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 -- 172.21.15.78:0/166145871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672596b0 msgr2=0x5557676253b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672596b0 0x5557676253b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 -- 172.21.15.78:0/166145871 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55576735c7d0 con 0x5557672514d0 2024-05-21T15:24:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdca5691640 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767624e70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009bb0 tx=0x7fdc90003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b818d4f7ec44e69 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdc897fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdc897fa640 1 -- 172.21.15.78:0/166145871 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90005450 con 0x5557672514d0 2024-05-21T15:24:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdc897fa640 1 -- 172.21.15.78:0/166145871 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fdc900055f0 con 0x5557672514d0 2024-05-21T15:24:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.505+0000 7fdcaef95740 1 -- 172.21.15.78:0/166145871 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9000fdf0 con 0x5557672514d0 2024-05-21T15:24:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdc897fa640 1 -- 172.21.15.78:0/166145871 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdc90003830 con 0x5557672514d0 2024-05-21T15:24:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdcaef95740 10 client.17985 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdcaef95740 1 -- 172.21.15.78:0/166145871 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555767632810 con 0x5557672514d0 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdc897fa640 1 -- 172.21.15.78:0/166145871 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9000fdf0 con 0x5557672514d0 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdc897fa640 1 -- 172.21.15.78:0/166145871 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fdc90004c50 con 0x5557672514d0 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdcaef95740 10 client.17985 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdcaef95740 10 client.17985 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdcaef95740 20 client.17985 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdcaef95740 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x555767238f60 0x555767635b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdcaef95740 4 client.17985 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdcaef95740 1 -- 172.21.15.78:0/166145871 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55576758c370 con 0x555767238f60 2024-05-21T15:24:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.506+0000 7fdca5e92640 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x555767238f60 0x555767635b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.507+0000 7fdc897fa640 10 client.17985 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:24:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.507+0000 7fdca5e92640 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x555767238f60 0x555767635b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a62c36d8a8ed90d server_cookie=ee85dd0ef2d45a23 in_seq=0 out_seq=0 2024-05-21T15:24:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.507+0000 7fdc897fa640 1 -- 172.21.15.78:0/166145871 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55576758c370 con 0x555767238f60 2024-05-21T15:24:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.507+0000 7fdc897fa640 10 client.17985 handle_command_reply: tid=0 2024-05-21T15:24:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.507+0000 7fdcaef95740 1 client.17985 shutdown 2024-05-21T15:24:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 -- 172.21.15.78:0/166145871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 msgr2=0x555767624e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767624e70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009bb0 tx=0x7fdc90003690 comp rx=0 tx=0).stop 2024-05-21T15:24:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 -- 172.21.15.78:0/166145871 shutdown_connections 2024-05-21T15:24:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x555767238f60 0x555767635b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672596b0 0x5557676253b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55576727a390 0x555767623310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdc897fa640 0 client.17985 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:24:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 --2- 172.21.15.78:0/166145871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767624e70 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 -- 172.21.15.78:0/166145871 >> 172.21.15.78:0/166145871 conn(0x55576726e4e0 msgr2=0x55576761aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 -- 172.21.15.78:0/166145871 shutdown_connections 2024-05-21T15:24:27.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 1 -- 172.21.15.78:0/166145871 wait complete. 2024-05-21T15:24:27.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.508+0000 7fdcaef95740 20 client.17985 trim_cache size 0 max 16384 2024-05-21T15:24:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdcaef95740 1 Processor -- start 2024-05-21T15:24:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdcaef95740 1 -- start start 2024-05-21T15:24:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555767238f60 0x555767625140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767614b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557672596b0 0x555767615090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdcaef95740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01aecc0 con 0x5557672514d0 2024-05-21T15:24:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdcaef95740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01b0800 con 0x555767238f60 2024-05-21T15:24:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdcaef95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01af390 con 0x5557672596b0 2024-05-21T15:24:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdca5e92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767614b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdca5e92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767614b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47932/0 (socket says 172.21.15.78:47932) 2024-05-21T15:24:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdca5691640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557672596b0 0x555767615090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdca5e92640 1 -- 172.21.15.78:0/1787299556 learned_addr learned my addr 172.21.15.78:0/1787299556 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.522+0000 7fdca5691640 1 --2- 172.21.15.78:0/1787299556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555767238f60 0x555767625140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdca5e92640 1 -- 172.21.15.78:0/1787299556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557672596b0 msgr2=0x555767615090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdca5e92640 1 --2- 172.21.15.78:0/1787299556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557672596b0 0x555767615090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdca5e92640 1 -- 172.21.15.78:0/1787299556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555767238f60 msgr2=0x555767625140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdca5e92640 1 --2- 172.21.15.78:0/1787299556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555767238f60 0x555767625140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdca5e92640 1 -- 172.21.15.78:0/1787299556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557676246b0 con 0x5557672514d0 2024-05-21T15:24:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdca5e92640 1 --2- 172.21.15.78:0/1787299556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767614b50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009900 tx=0x7fdc9c02a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7c389fc0e353b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdc8b7fe640 1 -- 172.21.15.78:0/1787299556 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c035040 con 0x5557672514d0 2024-05-21T15:24:27.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdc8b7fe640 1 -- 172.21.15.78:0/1787299556 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c04d070 con 0x5557672514d0 2024-05-21T15:24:27.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.523+0000 7fdc8b7fe640 1 -- 172.21.15.78:0/1787299556 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c048680 con 0x5557672514d0 2024-05-21T15:24:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 -- 172.21.15.78:0/1787299556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 msgr2=0x555767614b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 --2- 172.21.15.78:0/1787299556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767614b50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009900 tx=0x7fdc9c02a9b0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 -- 172.21.15.78:0/1787299556 shutdown_connections 2024-05-21T15:24:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 --2- 172.21.15.78:0/1787299556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557672596b0 0x555767615090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 --2- 172.21.15.78:0/1787299556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672514d0 0x555767614b50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 --2- 172.21.15.78:0/1787299556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555767238f60 0x555767625140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 -- 172.21.15.78:0/1787299556 >> 172.21.15.78:0/1787299556 conn(0x55576726e4e0 msgr2=0x555767624a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 -- 172.21.15.78:0/1787299556 shutdown_connections 2024-05-21T15:24:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 -- 172.21.15.78:0/1787299556 wait complete. 2024-05-21T15:24:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 -- start start 2024-05-21T15:24:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.524+0000 7fdcaef95740 1 Processor -- start 2024-05-21T15:24:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555767238f60 0x5557676282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 0x5557676287e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdcaef95740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x55576769bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdcaef95740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01b0130 con 0x5557672596b0 2024-05-21T15:24:27.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdcaef95740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01afa60 con 0x5557672514d0 2024-05-21T15:24:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdcaef95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01accc0 con 0x555767238f60 2024-05-21T15:24:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdcaef95740 10 client.0 _ll_register_callbacks cb 0x7fdc90011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x55576769bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x55576769bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47948/0 (socket says 172.21.15.78:47948) 2024-05-21T15:24:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 -- 172.21.15.78:0/3299265028 learned_addr learned my addr 172.21.15.78:0/3299265028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5e92640 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 0x5557676287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555767238f60 0x5557676282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 -- 172.21.15.78:0/3299265028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555767238f60 msgr2=0x5557676282a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555767238f60 0x5557676282a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 -- 172.21.15.78:0/3299265028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 msgr2=0x5557676287e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 0x5557676287e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.525+0000 7fdca5691640 1 -- 172.21.15.78:0/3299265028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555767629000 con 0x5557672596b0 2024-05-21T15:24:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdca5691640 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x55576769bcf0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdc90010010 tx=0x7fdc9002d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3701d0ced461048f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdc897fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdc897fa640 1 -- 172.21.15.78:0/3299265028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9002d2b0 con 0x5557672596b0 2024-05-21T15:24:27.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdc897fa640 1 -- 172.21.15.78:0/3299265028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fdc90005250 con 0x5557672596b0 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 1 -- 172.21.15.78:0/3299265028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9c04d070 con 0x5557672596b0 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdc897fa640 1 -- 172.21.15.78:0/3299265028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdc90044050 con 0x5557672596b0 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 10 client.17991 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 1 -- 172.21.15.78:0/3299265028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55576769d200 con 0x5557672596b0 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdc897fa640 1 -- 172.21.15.78:0/3299265028 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c04d070 con 0x5557672596b0 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdc897fa640 1 -- 172.21.15.78:0/3299265028 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fdc9003b040 con 0x5557672596b0 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 10 client.17991 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 10 client.17991 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:24:27.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 20 client.17991 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55576727a390 0x55576769f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 4 client.17991 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:24:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdcaef95740 1 -- 172.21.15.78:0/3299265028 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557675866c0 con 0x55576727a390 2024-05-21T15:24:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.526+0000 7fdca5e92640 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55576727a390 0x55576769f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdc897fa640 10 client.17991 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:24:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdca5e92640 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55576727a390 0x55576769f850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=683e2584ef2b9a9e server_cookie=4de457a8dcde2859 in_seq=0 out_seq=0 2024-05-21T15:24:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdc897fa640 1 -- 172.21.15.78:0/3299265028 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557675866c0 con 0x55576727a390 2024-05-21T15:24:27.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdc897fa640 10 client.17991 handle_command_reply: tid=0 2024-05-21T15:24:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 client.17991 shutdown 2024-05-21T15:24:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 -- 172.21.15.78:0/3299265028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 msgr2=0x55576769bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x55576769bcf0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdc90010010 tx=0x7fdc9002d040 comp rx=0 tx=0).stop 2024-05-21T15:24:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 -- 172.21.15.78:0/3299265028 shutdown_connections 2024-05-21T15:24:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55576727a390 0x55576769f850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5557672514d0 0x5557676287e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557672596b0 0x55576769bcf0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 --2- 172.21.15.78:0/3299265028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555767238f60 0x5557676282a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdc897fa640 0 client.17991 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:24:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 -- 172.21.15.78:0/3299265028 >> 172.21.15.78:0/3299265028 conn(0x55576726e4e0 msgr2=0x555767624890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 -- 172.21.15.78:0/3299265028 shutdown_connections 2024-05-21T15:24:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 1 -- 172.21.15.78:0/3299265028 wait complete. 2024-05-21T15:24:27.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.527+0000 7fdcaef95740 20 client.17991 trim_cache size 0 max 16384 2024-05-21T15:24:27.541 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:24:27.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 -- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc8c0a2d00 msgr2=0x7fdc8c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc8c0a2d00 0x7fdc8c0a5210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdc98098560 tx=0x7fdc980866c0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 -- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca010c370 msgr2=0x7fdca019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca010c370 0x7fdca019dfd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800eaa0 tx=0x7fdc9800ef60 comp rx=0 tx=0).stop 2024-05-21T15:24:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 -- 172.21.15.78:0/3890331996 shutdown_connections 2024-05-21T15:24:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdca0107a10 0x7fdca0199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdc8c0a2d00 0x7fdc8c0a5210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca010c370 0x7fdca019dfd0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 --2- 172.21.15.78:0/3890331996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdca0101ae0 0x7fdca01997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 -- 172.21.15.78:0/3890331996 >> 172.21.15.78:0/3890331996 conn(0x7fdca00770c0 msgr2=0x7fdca006a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 -- 172.21.15.78:0/3890331996 shutdown_connections 2024-05-21T15:24:27.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.528+0000 7fdca80f6640 1 -- 172.21.15.78:0/3890331996 wait complete. 2024-05-21T15:24:27.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:27 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1370362282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:24:27.545 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:24:27.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 -- 172.21.15.78:0/255247176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 msgr2=0x7f659410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 --2- 172.21.15.78:0/255247176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 0x7f659410ca20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f6584009900 tx=0x7f658402a410 comp rx=0 tx=0).stop 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 -- 172.21.15.78:0/255247176 shutdown_connections 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 --2- 172.21.15.78:0/255247176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f659410cff0 0x7f659410f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 --2- 172.21.15.78:0/255247176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 0x7f659410ca20 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 --2- 172.21.15.78:0/255247176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6594103ba0 0x7f6594103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 -- 172.21.15.78:0/255247176 >> 172.21.15.78:0/255247176 conn(0x7f65940770b0 msgr2=0x7f65940774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 -- 172.21.15.78:0/255247176 shutdown_connections 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.656+0000 7f659ae7a640 1 -- 172.21.15.78:0/255247176 wait complete. 2024-05-21T15:24:27.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f659ae7a640 1 Processor -- start 2024-05-21T15:24:27.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f659ae7a640 1 -- start start 2024-05-21T15:24:27.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f659ae7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594103ba0 0x7f6594101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f659ae7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 0x7f65941019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f659ae7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f659410cff0 0x7f6594101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f659ae7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6594112da0 con 0x7f659410a600 2024-05-21T15:24:27.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f659ae7a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6594112c20 con 0x7f659410cff0 2024-05-21T15:24:27.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f659ae7a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6594112f20 con 0x7f6594103ba0 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f658bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 0x7f65941019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f658bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 0x7f65941019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47958/0 (socket says 172.21.15.78:47958) 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f658bfff640 1 -- 172.21.15.78:0/4056174978 learned_addr learned my addr 172.21.15.78:0/4056174978 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f6599417640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f659410cff0 0x7f6594101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f6598c16640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594103ba0 0x7f6594101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f658bfff640 1 -- 172.21.15.78:0/4056174978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594103ba0 msgr2=0x7f6594101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f658bfff640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594103ba0 0x7f6594101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f658bfff640 1 -- 172.21.15.78:0/4056174978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f659410cff0 msgr2=0x7f6594101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f658bfff640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f659410cff0 0x7f6594101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f658bfff640 1 -- 172.21.15.78:0/4056174978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65941a7ae0 con 0x7f659410a600 2024-05-21T15:24:27.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.657+0000 7f6599417640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f659410cff0 0x7f6594101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.658+0000 7f658bfff640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 0x7f65941019a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f6584006010 tx=0x7f65840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.658+0000 7f6589ffb640 1 -- 172.21.15.78:0/4056174978 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658403f030 con 0x7f659410a600 2024-05-21T15:24:27.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.658+0000 7f6589ffb640 1 -- 172.21.15.78:0/4056174978 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6584003ce0 con 0x7f659410a600 2024-05-21T15:24:27.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.658+0000 7f659ae7a640 1 -- 172.21.15.78:0/4056174978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65941a7d70 con 0x7f659410a600 2024-05-21T15:24:27.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.658+0000 7f6589ffb640 1 -- 172.21.15.78:0/4056174978 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584035070 con 0x7f659410a600 2024-05-21T15:24:27.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.658+0000 7f659ae7a640 1 -- 172.21.15.78:0/4056174978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65941a81f0 con 0x7f659410a600 2024-05-21T15:24:27.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.660+0000 7f6589ffb640 1 -- 172.21.15.78:0/4056174978 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6584042030 con 0x7f659410a600 2024-05-21T15:24:27.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f65a1dff740 1 Processor -- start 2024-05-21T15:24:27.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f65a1dff740 1 -- start start 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 0x55e5819a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819a5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f65a1dff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e58190ac40 con 0x55e5815dd4d0 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f65a1dff740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e5818b2580 con 0x55e5815e56b0 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f65a1dff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e581906c00 con 0x55e581606390 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f6589ffb640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f657c0a2d80 0x7f657c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f6598c16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f6598c16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47962/0 (socket says 172.21.15.78:47962) 2024-05-21T15:24:27.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f6598c16640 1 -- 172.21.15.78:0/2807337539 learned_addr learned my addr 172.21.15.78:0/2807337539 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f658bfff640 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f6599417640 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 0x55e5819a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.661+0000 7f6589ffb640 1 -- 172.21.15.78:0/4056174978 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f65840ef9f0 con 0x7f659410a600 2024-05-21T15:24:27.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f6599417640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f657c0a2d80 0x7f657c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f658bfff640 1 -- 172.21.15.78:0/2807337539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 msgr2=0x55e5819a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f658bfff640 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 0x55e5819a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f658bfff640 1 -- 172.21.15.78:0/2807337539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 msgr2=0x55e5819a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f658bfff640 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f658bfff640 1 -- 172.21.15.78:0/2807337539 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e58161e3f0 con 0x55e5815e56b0 2024-05-21T15:24:27.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f6598c16640 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f6599417640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f657c0a2d80 0x7f657c0a5290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55e5819a1e90 tx=0x7f658c004ab0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f658bfff640 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819a57a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65840ed0b0 tx=0x7f6584003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276498b85d3af8e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f657affd640 1 -- 172.21.15.78:0/2807337539 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584049050 con 0x55e5815e56b0 2024-05-21T15:24:27.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f657affd640 1 -- 172.21.15.78:0/2807337539 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f658403b580 con 0x55e5815e56b0 2024-05-21T15:24:27.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.662+0000 7f657affd640 1 -- 172.21.15.78:0/2807337539 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840c4bb0 con 0x55e5815e56b0 2024-05-21T15:24:27.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 -- 172.21.15.78:0/2807337539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 msgr2=0x55e5819a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819a57a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f65840ed0b0 tx=0x7f6584003f90 comp rx=0 tx=0).stop 2024-05-21T15:24:27.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 -- 172.21.15.78:0/2807337539 shutdown_connections 2024-05-21T15:24:27.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819a57a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2807337539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 0x55e5819a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 -- 172.21.15.78:0/2807337539 >> 172.21.15.78:0/2807337539 conn(0x55e5815fa4e0 msgr2=0x55e5819a14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 -- 172.21.15.78:0/2807337539 shutdown_connections 2024-05-21T15:24:27.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.663+0000 7f65a1dff740 1 -- 172.21.15.78:0/2807337539 wait complete. 2024-05-21T15:24:27.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 1 -- start start 2024-05-21T15:24:27.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 1 Processor -- start 2024-05-21T15:24:27.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 0x55e5819af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5819b1e30 con 0x55e5815dd4d0 2024-05-21T15:24:27.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e5818c5c40 con 0x55e5815e56b0 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e5819b1fb0 con 0x55e581606390 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f65a1dff740 10 client.0 _ll_register_callbacks cb 0x55e58161dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f6598c16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f658bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f658bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:41488/0 (socket says 172.21.15.78:41488) 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f6598c16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47964/0 (socket says 172.21.15.78:47964) 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f658bfff640 1 -- 172.21.15.78:0/2336101750 learned_addr learned my addr 172.21.15.78:0/2336101750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.664+0000 7f6599417640 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 0x55e5819af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f658bfff640 1 -- 172.21.15.78:0/2336101750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 msgr2=0x55e5819af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f658bfff640 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 0x55e5819af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f658bfff640 1 -- 172.21.15.78:0/2336101750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 msgr2=0x55e5819b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f658bfff640 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f658bfff640 1 -- 172.21.15.78:0/2336101750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5816e87d0 con 0x55e5815e56b0 2024-05-21T15:24:27.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f6598c16640 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f658bfff640 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819b1310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f65840ed160 tx=0x7f65840c0470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899af7f813b1ee61 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f6573fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:24:27.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f6573fff640 1 -- 172.21.15.78:0/2336101750 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840bb8a0 con 0x55e5815e56b0 2024-05-21T15:24:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f6573fff640 1 -- 172.21.15.78:0/2336101750 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f65840ba240 con 0x55e5815e56b0 2024-05-21T15:24:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f65a1dff740 1 -- 172.21.15.78:0/2336101750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e581918370 con 0x55e5815e56b0 2024-05-21T15:24:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f6573fff640 1 -- 172.21.15.78:0/2336101750 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6584049030 con 0x55e5815e56b0 2024-05-21T15:24:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f65a1dff740 10 client.27191 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f65a1dff740 1 -- 172.21.15.78:0/2336101750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e5819b1e30 con 0x55e5815e56b0 2024-05-21T15:24:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.665+0000 7f6573fff640 1 -- 172.21.15.78:0/2336101750 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e581918370 con 0x55e5815e56b0 2024-05-21T15:24:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f6573fff640 1 -- 172.21.15.78:0/2336101750 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f65840bc230 con 0x55e5815e56b0 2024-05-21T15:24:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f65a1dff740 10 client.27191 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f65a1dff740 10 client.27191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:27.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f65a1dff740 20 client.27191 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e5815c4f60 0x55e5819c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f65a1dff740 4 client.27191 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:27.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f65a1dff740 1 -- 172.21.15.78:0/2336101750 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5819126c0 con 0x55e5815c4f60 2024-05-21T15:24:27.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f6598c16640 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e5815c4f60 0x55e5819c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f6573fff640 10 client.27191 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:27.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.666+0000 7f6598c16640 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e5815c4f60 0x55e5819c17a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ab8d87cf06fa4f server_cookie=cce03bd97774fc03 in_seq=0 out_seq=0 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f6573fff640 1 -- 172.21.15.78:0/2336101750 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5819126c0 con 0x55e5815c4f60 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f6573fff640 10 client.27191 handle_command_reply: tid=0 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 client.27191 shutdown 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 -- 172.21.15.78:0/2336101750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 msgr2=0x55e5819b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819b1310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f65840ed160 tx=0x7f65840c0470 comp rx=0 tx=0).stop 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 -- 172.21.15.78:0/2336101750 shutdown_connections 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e5815c4f60 0x55e5819c17a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815e56b0 0x55e5819b1310 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f6573fff640 0 client.27191 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:27.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e581606390 0x55e5819af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2336101750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 -- 172.21.15.78:0/2336101750 >> 172.21.15.78:0/2336101750 conn(0x55e5815fa4e0 msgr2=0x55e5819a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 -- 172.21.15.78:0/2336101750 shutdown_connections 2024-05-21T15:24:27.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 1 -- 172.21.15.78:0/2336101750 wait complete. 2024-05-21T15:24:27.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.667+0000 7f65a1dff740 20 client.27191 trim_cache size 0 max 16384 2024-05-21T15:24:27.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.681+0000 7f65a1dff740 1 Processor -- start 2024-05-21T15:24:27.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.681+0000 7f65a1dff740 1 -- start start 2024-05-21T15:24:27.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.681+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815fa4e0 0x55e5819af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.681+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815dd4d0 0x55e5819a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f65a1dff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65941b0ee0 con 0x55e5815fa4e0 2024-05-21T15:24:27.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f65a1dff740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65941b2a20 con 0x55e5815dd4d0 2024-05-21T15:24:27.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f65a1dff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f65941b15b0 con 0x55e5815c4f60 2024-05-21T15:24:27.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815fa4e0 0x55e5819af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815fa4e0 0x55e5819af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47966/0 (socket says 172.21.15.78:47966) 2024-05-21T15:24:27.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 -- 172.21.15.78:0/135146034 learned_addr learned my addr 172.21.15.78:0/135146034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f658bfff640 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6599417640 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815dd4d0 0x55e5819a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 -- 172.21.15.78:0/135146034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 msgr2=0x55e5819a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 -- 172.21.15.78:0/135146034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815dd4d0 msgr2=0x55e5819a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815dd4d0 0x55e5819a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 -- 172.21.15.78:0/135146034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5819a9410 con 0x55e5815fa4e0 2024-05-21T15:24:27.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f6598c16640 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815fa4e0 0x55e5819af1f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x55e5819a2460 tx=0x7f65900118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8052971d5bc428 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f657a7fc640 1 -- 172.21.15.78:0/135146034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590021030 con 0x55e5815fa4e0 2024-05-21T15:24:27.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f657a7fc640 1 -- 172.21.15.78:0/135146034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659001d9c0 con 0x55e5815fa4e0 2024-05-21T15:24:27.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.682+0000 7f657a7fc640 1 -- 172.21.15.78:0/135146034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590004a70 con 0x55e5815fa4e0 2024-05-21T15:24:27.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 -- 172.21.15.78:0/135146034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815fa4e0 msgr2=0x55e5819af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815fa4e0 0x55e5819af1f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x55e5819a2460 tx=0x7f65900118d0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 -- 172.21.15.78:0/135146034 shutdown_connections 2024-05-21T15:24:27.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815dd4d0 0x55e5819a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 --2- 172.21.15.78:0/135146034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815fa4e0 0x55e5819af1f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 -- 172.21.15.78:0/135146034 >> 172.21.15.78:0/135146034 conn(0x55e5819aabd0 msgr2=0x55e5819aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 -- 172.21.15.78:0/135146034 shutdown_connections 2024-05-21T15:24:27.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 -- 172.21.15.78:0/135146034 wait complete. 2024-05-21T15:24:27.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.683+0000 7f65a1dff740 1 -- start start 2024-05-21T15:24:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f65a1dff740 1 Processor -- start 2024-05-21T15:24:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819b7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f65a1dff740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815fa4e0 0x55e5819b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f65a1dff740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65941b2350 con 0x55e5815dd4d0 2024-05-21T15:24:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f65a1dff740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65941b1c80 con 0x55e5815fa4e0 2024-05-21T15:24:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f65a1dff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f65941aeee0 con 0x55e5815c4f60 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f65a1dff740 10 client.0 _ll_register_callbacks cb 0x7f659000d440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f658bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f658bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47980/0 (socket says 172.21.15.78:47980) 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f6599417640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815fa4e0 0x55e5819b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f658bfff640 1 -- 172.21.15.78:0/2396283844 learned_addr learned my addr 172.21.15.78:0/2396283844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f6598c16640 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f658bfff640 1 -- 172.21.15.78:0/2396283844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 msgr2=0x55e5819b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f658bfff640 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f658bfff640 1 -- 172.21.15.78:0/2396283844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815fa4e0 msgr2=0x55e5819b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f658bfff640 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815fa4e0 0x55e5819b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f658bfff640 1 -- 172.21.15.78:0/2396283844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5819b53f0 con 0x55e5815dd4d0 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f6599417640 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815fa4e0 0x55e5819b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.684+0000 7f6598c16640 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f658bfff640 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b48f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f65840bbc40 tx=0x7f65840ed5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f232eee0130c2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f6573fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f6573fff640 1 -- 172.21.15.78:0/2396283844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658403bcb0 con 0x55e5815dd4d0 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f6573fff640 1 -- 172.21.15.78:0/2396283844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f65840da040 con 0x55e5815dd4d0 2024-05-21T15:24:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 1 -- 172.21.15.78:0/2396283844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659001d9c0 con 0x55e5815dd4d0 2024-05-21T15:24:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f6573fff640 1 -- 172.21.15.78:0/2396283844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f65840bb8a0 con 0x55e5815dd4d0 2024-05-21T15:24:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 10 client.18009 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 1 -- 172.21.15.78:0/2396283844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e5819b53f0 con 0x55e5815dd4d0 2024-05-21T15:24:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f6573fff640 1 -- 172.21.15.78:0/2396283844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659001d9c0 con 0x55e5815dd4d0 2024-05-21T15:24:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f6573fff640 1 -- 172.21.15.78:0/2396283844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f65840bc1a0 con 0x55e5815dd4d0 2024-05-21T15:24:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 10 client.18009 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 20 client.18009 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e5815e56b0 0x55e581a2bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 4 client.18009 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:24:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.685+0000 7f65a1dff740 1 -- 172.21.15.78:0/2396283844 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e58188f820 con 0x55e5815e56b0 2024-05-21T15:24:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f6598c16640 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e5815e56b0 0x55e581a2bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f6573fff640 10 client.18009 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f6598c16640 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e5815e56b0 0x55e581a2bc00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82b81cfc004645a server_cookie=ea63d6e7d7d899a0 in_seq=0 out_seq=0 2024-05-21T15:24:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f6573fff640 1 -- 172.21.15.78:0/2396283844 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e58188f820 con 0x55e5815e56b0 2024-05-21T15:24:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f6573fff640 10 client.18009 handle_command_reply: tid=0 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 client.18009 shutdown 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 -- 172.21.15.78:0/2396283844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 msgr2=0x55e5819b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b48f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f65840bbc40 tx=0x7f65840ed5c0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 -- 172.21.15.78:0/2396283844 shutdown_connections 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e5815e56b0 0x55e581a2bc00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5815fa4e0 0x55e5819b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5815dd4d0 0x55e5819b48f0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f6573fff640 0 client.18009 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 --2- 172.21.15.78:0/2396283844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e5815c4f60 0x55e5819b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 -- 172.21.15.78:0/2396283844 >> 172.21.15.78:0/2396283844 conn(0x55e5819aabd0 msgr2=0x55e5819a56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 -- 172.21.15.78:0/2396283844 shutdown_connections 2024-05-21T15:24:27.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.686+0000 7f65a1dff740 1 -- 172.21.15.78:0/2396283844 wait complete. 2024-05-21T15:24:27.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f65a1dff740 20 client.18009 trim_cache size 0 max 16384 2024-05-21T15:24:27.704 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:24:27.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 -- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f657c0a2d80 msgr2=0x7f657c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f657c0a2d80 0x7f657c0a5290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55e5819a1e90 tx=0x7f658c004ab0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 -- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 msgr2=0x7f65941019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:27.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 0x7f65941019a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f6584006010 tx=0x7f65840042f0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 -- 172.21.15.78:0/4056174978 shutdown_connections 2024-05-21T15:24:27.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f659410cff0 0x7f6594101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f657c0a2d80 0x7f657c0a5290 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659410a600 0x7f65941019a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 --2- 172.21.15.78:0/4056174978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6594103ba0 0x7f6594101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:27.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 -- 172.21.15.78:0/4056174978 >> 172.21.15.78:0/4056174978 conn(0x7f65940770b0 msgr2=0x7f6594100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:27.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 -- 172.21.15.78:0/4056174978 shutdown_connections 2024-05-21T15:24:27.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:27.687+0000 7f659ae7a640 1 -- 172.21.15.78:0/4056174978 wait complete. 2024-05-21T15:24:27.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:27 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1370362282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:24:27.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:27 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1370362282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:24:28.708 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:24:28.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:24:28.708 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:24:28.729 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:28 smithi078 ceph-mon[31997]: pgmap v258: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-05-21T15:24:28.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:28 smithi084 ceph-mon[36334]: pgmap v258: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-05-21T15:24:28.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1301781838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4810c330 msgr2=0x7fdb4810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1301781838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4810c330 0x7fdb4810e750 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009ca0 tx=0x7fdb4002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1301781838 shutdown_connections 2024-05-21T15:24:28.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1301781838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4810c330 0x7fdb4810e750 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1301781838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb48101aa0 0x7fdb48101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1301781838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb481079d0 0x7fdb48101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1301781838 >> 172.21.15.78:0/1301781838 conn(0x7fdb480770b0 msgr2=0x7fdb480774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:28.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1301781838 shutdown_connections 2024-05-21T15:24:28.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.819+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1301781838 wait complete. 2024-05-21T15:24:28.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4cca2640 1 Processor -- start 2024-05-21T15:24:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4cca2640 1 -- start start 2024-05-21T15:24:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4cca2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb48101aa0 0x7fdb481986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4cca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb481079d0 0x7fdb48198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4cca2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb4810c330 0x7fdb4819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4cca2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48111d00 con 0x7fdb48101aa0 2024-05-21T15:24:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4cca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48111b80 con 0x7fdb481079d0 2024-05-21T15:24:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4cca2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48111e80 con 0x7fdb4810c330 2024-05-21T15:24:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb48101aa0 0x7fdb481986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb48101aa0 0x7fdb481986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53002/0 (socket says 172.21.15.78:53002) 2024-05-21T15:24:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb4659c640 1 -- 172.21.15.78:0/1328050941 learned_addr learned my addr 172.21.15.78:0/1328050941 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb4810c330 0x7fdb4819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.820+0000 7fdb45d9b640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb481079d0 0x7fdb48198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb4659c640 1 -- 172.21.15.78:0/1328050941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb4810c330 msgr2=0x7fdb4819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb4659c640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb4810c330 0x7fdb4819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb4659c640 1 -- 172.21.15.78:0/1328050941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb481079d0 msgr2=0x7fdb48198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb4659c640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb481079d0 0x7fdb48198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb4659c640 1 -- 172.21.15.78:0/1328050941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4819d540 con 0x7fdb48101aa0 2024-05-21T15:24:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb4659c640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb48101aa0 0x7fdb481986e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800e9c0 tx=0x7fdb3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb3f7fe640 1 -- 172.21.15.78:0/1328050941 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800cd80 con 0x7fdb48101aa0 2024-05-21T15:24:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb3f7fe640 1 -- 172.21.15.78:0/1328050941 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb38005500 con 0x7fdb48101aa0 2024-05-21T15:24:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1328050941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4819d830 con 0x7fdb48101aa0 2024-05-21T15:24:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb3f7fe640 1 -- 172.21.15.78:0/1328050941 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38010690 con 0x7fdb48101aa0 2024-05-21T15:24:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.821+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1328050941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481a51b0 con 0x7fdb48101aa0 2024-05-21T15:24:28.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.823+0000 7fdb3f7fe640 1 -- 172.21.15.78:0/1328050941 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdb380046c0 con 0x7fdb48101aa0 2024-05-21T15:24:28.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb53c24740 1 Processor -- start 2024-05-21T15:24:28.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb53c24740 1 -- start start 2024-05-21T15:24:28.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff6a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb53c24740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5597ff608990 con 0x5597ff2e3730 2024-05-21T15:24:28.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb53c24740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5597ff5b05b0 con 0x5597ff304410 2024-05-21T15:24:28.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb53c24740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5597ff604950 con 0x5597ff2db550 2024-05-21T15:24:28.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb46d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb4659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff6a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb46d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:54588/0 (socket says 172.21.15.78:54588) 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb46d9d640 1 -- 172.21.15.78:0/4261811172 learned_addr learned my addr 172.21.15.78:0/4261811172 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.824+0000 7fdb3f7fe640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdb340a2d00 0x7fdb340a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb46d9d640 1 -- 172.21.15.78:0/4261811172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 msgr2=0x5597ff6a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/4261811172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff6a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb46d9d640 1 -- 172.21.15.78:0/4261811172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 msgr2=0x5597ff6a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/4261811172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb46d9d640 1 -- 172.21.15.78:0/4261811172 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ff3e47e0 con 0x5597ff304410 2024-05-21T15:24:28.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb3f7fe640 1 -- 172.21.15.78:0/1328050941 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fdb380cf8c0 con 0x7fdb48101aa0 2024-05-21T15:24:28.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdb340a2d00 0x7fdb340a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/4261811172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a3620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009ca0 tx=0x7fdb4002f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d9d8fe35035e92 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:28.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb3cff9640 1 -- 172.21.15.78:0/4261811172 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40042040 con 0x5597ff304410 2024-05-21T15:24:28.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdb340a2d00 0x7fdb340a5210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb4004d000 tx=0x7fdb4004e000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:28.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb3cff9640 1 -- 172.21.15.78:0/4261811172 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb40036040 con 0x5597ff304410 2024-05-21T15:24:28.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.825+0000 7fdb3cff9640 1 -- 172.21.15.78:0/4261811172 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4002f810 con 0x5597ff304410 2024-05-21T15:24:28.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 -- 172.21.15.78:0/4261811172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 msgr2=0x5597ff6a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 --2- 172.21.15.78:0/4261811172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a3620 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009ca0 tx=0x7fdb4002f4c0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 -- 172.21.15.78:0/4261811172 shutdown_connections 2024-05-21T15:24:28.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 --2- 172.21.15.78:0/4261811172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 --2- 172.21.15.78:0/4261811172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff6a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 --2- 172.21.15.78:0/4261811172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a3620 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 -- 172.21.15.78:0/4261811172 >> 172.21.15.78:0/4261811172 conn(0x5597ff2f8560 msgr2=0x5597ff69fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:28.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 -- 172.21.15.78:0/4261811172 shutdown_connections 2024-05-21T15:24:28.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.826+0000 7fdb53c24740 1 -- 172.21.15.78:0/4261811172 wait complete. 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 1 -- start start 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 1 Processor -- start 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff6b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2e3730 0x5597ff6b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff304410 0x5597ff6b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5597ff6b0970 con 0x5597ff304410 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5597ff5c3b40 con 0x5597ff2e3730 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5597ff6b0af0 con 0x5597ff2db550 2024-05-21T15:24:28.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb53c24740 10 client.0 _ll_register_callbacks cb 0x5597ff5b80f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:28.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb46d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff304410 0x5597ff6b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb45d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2e3730 0x5597ff6b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb46d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff304410 0x5597ff6b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53026/0 (socket says 172.21.15.78:53026) 2024-05-21T15:24:28.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb46d9d640 1 -- 172.21.15.78:0/996893953 learned_addr learned my addr 172.21.15.78:0/996893953 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:28.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb4659c640 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff6b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb46d9d640 1 -- 172.21.15.78:0/996893953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 msgr2=0x5597ff6b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:28.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff6b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb46d9d640 1 -- 172.21.15.78:0/996893953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2e3730 msgr2=0x5597ff6b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:28.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2e3730 0x5597ff6b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb46d9d640 1 -- 172.21.15.78:0/996893953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ff444f70 con 0x5597ff304410 2024-05-21T15:24:28.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.827+0000 7fdb45d9b640 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2e3730 0x5597ff6b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:28.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff304410 0x5597ff6b0dc0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fdb4002a900 tx=0x7fdb40004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f76694ab01d3933 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:28.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:28.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb2dffb640 1 -- 172.21.15.78:0/996893953 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4005f060 con 0x5597ff304410 2024-05-21T15:24:28.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb2dffb640 1 -- 172.21.15.78:0/996893953 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fdb40004300 con 0x5597ff304410 2024-05-21T15:24:28.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 1 -- 172.21.15.78:0/996893953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb40036040 con 0x5597ff304410 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb2dffb640 1 -- 172.21.15.78:0/996893953 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdb40042040 con 0x5597ff304410 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 10 client.18021 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 1 -- 172.21.15.78:0/996893953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5597ff2dc170 con 0x5597ff304410 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb2dffb640 1 -- 172.21.15.78:0/996893953 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb40036040 con 0x5597ff304410 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb2dffb640 1 -- 172.21.15.78:0/996893953 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fdb400305c0 con 0x5597ff304410 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 10 client.18021 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 20 client.18021 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:28.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5597ff2c31f0 0x5597ff6c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 4 client.18021 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:28.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.828+0000 7fdb53c24740 1 -- 172.21.15.78:0/996893953 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597ff6160c0 con 0x5597ff2c31f0 2024-05-21T15:24:28.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.829+0000 7fdb4659c640 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5597ff2c31f0 0x5597ff6c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.829+0000 7fdb2dffb640 10 client.18021 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:28.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.829+0000 7fdb4659c640 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5597ff2c31f0 0x5597ff6c2290 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222f17087a66a852 server_cookie=e7ea79be86176c97 in_seq=0 out_seq=0 2024-05-21T15:24:28.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb2dffb640 1 -- 172.21.15.78:0/996893953 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5597ff6160c0 con 0x5597ff2c31f0 2024-05-21T15:24:28.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb2dffb640 10 client.18021 handle_command_reply: tid=0 2024-05-21T15:24:28.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 client.18021 shutdown 2024-05-21T15:24:28.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 -- 172.21.15.78:0/996893953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff304410 msgr2=0x5597ff6b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff304410 0x5597ff6b0dc0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fdb4002a900 tx=0x7fdb40004190 comp rx=0 tx=0).stop 2024-05-21T15:24:28.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 -- 172.21.15.78:0/996893953 shutdown_connections 2024-05-21T15:24:28.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5597ff2c31f0 0x5597ff6c2290 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2e3730 0x5597ff6b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff304410 0x5597ff6b0dc0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 --2- 172.21.15.78:0/996893953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff6b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb2dffb640 0 client.18021 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:28.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 -- 172.21.15.78:0/996893953 >> 172.21.15.78:0/996893953 conn(0x5597ff2f8560 msgr2=0x5597ff6a33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:28.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 -- 172.21.15.78:0/996893953 shutdown_connections 2024-05-21T15:24:28.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 1 -- 172.21.15.78:0/996893953 wait complete. 2024-05-21T15:24:28.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.830+0000 7fdb53c24740 20 client.18021 trim_cache size 0 max 16384 2024-05-21T15:24:28.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb53c24740 1 Processor -- start 2024-05-21T15:24:28.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb53c24740 1 -- start start 2024-05-21T15:24:28.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff69fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb53c24740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481ac970 con 0x5597ff2e3730 2024-05-21T15:24:28.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb53c24740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481ae4b0 con 0x5597ff304410 2024-05-21T15:24:28.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb53c24740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481ad040 con 0x5597ff2db550 2024-05-21T15:24:28.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb46d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53036/0 (socket says 172.21.15.78:53036) 2024-05-21T15:24:28.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.848+0000 7fdb45d9b640 1 -- 172.21.15.78:0/4235634053 learned_addr learned my addr 172.21.15.78:0/4235634053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:28.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb4659c640 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff69fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb46d9d640 1 -- 172.21.15.78:0/4235634053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 msgr2=0x5597ff69fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:28.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff69fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb46d9d640 1 -- 172.21.15.78:0/4235634053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 msgr2=0x5597ff6a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:28.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb46d9d640 1 -- 172.21.15.78:0/4235634053 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ff6af0c0 con 0x5597ff304410 2024-05-21T15:24:28.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb4659c640 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff69fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:24:28.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb46d9d640 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a9b60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009ca0 tx=0x7fdb40031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e2d9e1ab151a8c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:28.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb2ffff640 1 -- 172.21.15.78:0/4235634053 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4005f070 con 0x5597ff304410 2024-05-21T15:24:28.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb2ffff640 1 -- 172.21.15.78:0/4235634053 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb40036030 con 0x5597ff304410 2024-05-21T15:24:28.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.849+0000 7fdb2ffff640 1 -- 172.21.15.78:0/4235634053 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb400305c0 con 0x5597ff304410 2024-05-21T15:24:28.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 -- 172.21.15.78:0/4235634053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 msgr2=0x5597ff6a9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a9b60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009ca0 tx=0x7fdb40031000 comp rx=0 tx=0).stop 2024-05-21T15:24:28.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 -- 172.21.15.78:0/4235634053 shutdown_connections 2024-05-21T15:24:28.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff304410 0x5597ff6a9b60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 --2- 172.21.15.78:0/4235634053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff2db550 0x5597ff69fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 -- 172.21.15.78:0/4235634053 >> 172.21.15.78:0/4235634053 conn(0x5597ff2f8560 msgr2=0x5597ff69f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:28.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 -- 172.21.15.78:0/4235634053 shutdown_connections 2024-05-21T15:24:28.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 -- 172.21.15.78:0/4235634053 wait complete. 2024-05-21T15:24:28.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.850+0000 7fdb53c24740 1 -- start start 2024-05-21T15:24:28.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb53c24740 1 Processor -- start 2024-05-21T15:24:28.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2db550 0x5597ff6bcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb53c24740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff304410 0x5597ff6ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb53c24740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481adde0 con 0x5597ff2e3730 2024-05-21T15:24:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb53c24740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481ad710 con 0x5597ff2db550 2024-05-21T15:24:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb53c24740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481aa970 con 0x5597ff304410 2024-05-21T15:24:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb53c24740 10 client.0 _ll_register_callbacks cb 0x7fdb3000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb4659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2db550 0x5597ff6bcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6b9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53038/0 (socket says 172.21.15.78:53038) 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb4659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2db550 0x5597ff6bcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:54616/0 (socket says 172.21.15.78:54616) 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 -- 172.21.15.78:0/3734141075 learned_addr learned my addr 172.21.15.78:0/3734141075 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 -- 172.21.15.78:0/3734141075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff304410 msgr2=0x5597ff6ba4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff304410 0x5597ff6ba4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 -- 172.21.15.78:0/3734141075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2db550 msgr2=0x5597ff6bcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2db550 0x5597ff6bcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 -- 172.21.15.78:0/3734141075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ff6baac0 con 0x5597ff2e3730 2024-05-21T15:24:28.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb4659c640 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2db550 0x5597ff6bcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:28.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb45d9b640 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6b9f90 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fdb30011830 tx=0x7fdb30011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac697912cf747689 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:28.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.851+0000 7fdb2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:28.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb2dffb640 1 -- 172.21.15.78:0/3734141075 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30021060 con 0x5597ff2e3730 2024-05-21T15:24:28.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb2dffb640 1 -- 172.21.15.78:0/3734141075 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fdb30010c60 con 0x5597ff2e3730 2024-05-21T15:24:28.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 1 -- 172.21.15.78:0/3734141075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb40036030 con 0x5597ff2e3730 2024-05-21T15:24:28.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb2dffb640 1 -- 172.21.15.78:0/3734141075 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdb30028050 con 0x5597ff2e3730 2024-05-21T15:24:28.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 10 client.18027 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:28.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 1 -- 172.21.15.78:0/3734141075 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5597ff72a200 con 0x5597ff2e3730 2024-05-21T15:24:28.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb2dffb640 1 -- 172.21.15.78:0/3734141075 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb40036030 con 0x5597ff2e3730 2024-05-21T15:24:28.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb2dffb640 1 -- 172.21.15.78:0/3734141075 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fdb3001b070 con 0x5597ff2e3730 2024-05-21T15:24:28.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 10 client.18027 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:28.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:28.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 20 client.18027 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:28.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5597ff298630 0x5597ff72ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:28.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 4 client.18027 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:24:28.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb53c24740 1 -- 172.21.15.78:0/3734141075 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5597ff610410 con 0x5597ff298630 2024-05-21T15:24:28.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.852+0000 7fdb4659c640 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5597ff298630 0x5597ff72ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:28.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.853+0000 7fdb2dffb640 10 client.18027 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:28.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.853+0000 7fdb4659c640 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5597ff298630 0x5597ff72ca20 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63c7f4ca2a3b9bb server_cookie=99e98cda0098ae6f in_seq=0 out_seq=0 2024-05-21T15:24:28.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb2dffb640 1 -- 172.21.15.78:0/3734141075 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5597ff610410 con 0x5597ff298630 2024-05-21T15:24:28.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb2dffb640 10 client.18027 handle_command_reply: tid=0 2024-05-21T15:24:28.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 client.18027 shutdown 2024-05-21T15:24:28.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 -- 172.21.15.78:0/3734141075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 msgr2=0x5597ff6b9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6b9f90 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fdb30011830 tx=0x7fdb30011cf0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 -- 172.21.15.78:0/3734141075 shutdown_connections 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5597ff298630 0x5597ff72ca20 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5597ff304410 0x5597ff6ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb2dffb640 0 client.18027 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5597ff2e3730 0x5597ff6b9f90 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 --2- 172.21.15.78:0/3734141075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5597ff2db550 0x5597ff6bcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 -- 172.21.15.78:0/3734141075 >> 172.21.15.78:0/3734141075 conn(0x5597ff2f8560 msgr2=0x5597ff6a50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 -- 172.21.15.78:0/3734141075 shutdown_connections 2024-05-21T15:24:28.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 1 -- 172.21.15.78:0/3734141075 wait complete. 2024-05-21T15:24:28.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.854+0000 7fdb53c24740 20 client.18027 trim_cache size 0 max 16384 2024-05-21T15:24:28.868 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:24:28.868 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:24:28.868 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "085a32b3-b007-4c06-8d34-1d9522863f90", 2024-05-21T15:24:28.868 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:24:28.868 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:24:28.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdb340a2d00 msgr2=0x7fdb340a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdb340a2d00 0x7fdb340a5210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdb4004d000 tx=0x7fdb4004e000 comp rx=0 tx=0).stop 2024-05-21T15:24:28.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb48101aa0 msgr2=0x7fdb481986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb48101aa0 0x7fdb481986e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800e9c0 tx=0x7fdb3800ee80 comp rx=0 tx=0).stop 2024-05-21T15:24:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1328050941 shutdown_connections 2024-05-21T15:24:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb481079d0 0x7fdb48198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fdb340a2d00 0x7fdb340a5210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb4810c330 0x7fdb4819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 --2- 172.21.15.78:0/1328050941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb48101aa0 0x7fdb481986e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1328050941 >> 172.21.15.78:0/1328050941 conn(0x7fdb480770b0 msgr2=0x7fdb48103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:28.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1328050941 shutdown_connections 2024-05-21T15:24:28.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:28.855+0000 7fdb4cca2640 1 -- 172.21.15.78:0/1328050941 wait complete. 2024-05-21T15:24:28.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:24:28.872 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:24:28.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:28 smithi172 ceph-mon[36402]: pgmap v258: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 14 op/s 2024-05-21T15:24:29.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.021+0000 7f054523d640 1 -- 172.21.15.78:0/3230564691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010a600 msgr2=0x7f054010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.021+0000 7f054523d640 1 --2- 172.21.15.78:0/3230564691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010a600 0x7f054010ca20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0528009900 tx=0x7f052802a410 comp rx=0 tx=0).stop 2024-05-21T15:24:29.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.022+0000 7f054523d640 1 -- 172.21.15.78:0/3230564691 shutdown_connections 2024-05-21T15:24:29.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.022+0000 7f054523d640 1 --2- 172.21.15.78:0/3230564691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054010cff0 0x7f054010f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.022+0000 7f054523d640 1 --2- 172.21.15.78:0/3230564691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010a600 0x7f054010ca20 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.022+0000 7f054523d640 1 --2- 172.21.15.78:0/3230564691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0540103ba0 0x7f0540103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.022+0000 7f054523d640 1 -- 172.21.15.78:0/3230564691 >> 172.21.15.78:0/3230564691 conn(0x7f05400770b0 msgr2=0x7f05400774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:29.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.022+0000 7f054523d640 1 -- 172.21.15.78:0/3230564691 shutdown_connections 2024-05-21T15:24:29.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.022+0000 7f054523d640 1 -- 172.21.15.78:0/3230564691 wait complete. 2024-05-21T15:24:29.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.022+0000 7f054523d640 1 Processor -- start 2024-05-21T15:24:29.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f054523d640 1 -- start start 2024-05-21T15:24:29.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f054523d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0540103ba0 0x7f05401997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f054523d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f054010a600 0x7f0540199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f054523d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010cff0 0x7f054019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f054523d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0540112da0 con 0x7f054010cff0 2024-05-21T15:24:29.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f054523d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0540112c20 con 0x7f0540103ba0 2024-05-21T15:24:29.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f054523d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0540112f20 con 0x7f054010a600 2024-05-21T15:24:29.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010cff0 0x7f054019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f054010a600 0x7f0540199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010cff0 0x7f054019e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53058/0 (socket says 172.21.15.78:53058) 2024-05-21T15:24:29.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f054010a600 0x7f0540199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:58850/0 (socket says 172.21.15.78:58850) 2024-05-21T15:24:29.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053f59e640 1 -- 172.21.15.78:0/4243422184 learned_addr learned my addr 172.21.15.78:0/4243422184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:29.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053ed9d640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0540103ba0 0x7f05401997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053f59e640 1 -- 172.21.15.78:0/4243422184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f054010a600 msgr2=0x7f0540199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053f59e640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f054010a600 0x7f0540199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053f59e640 1 -- 172.21.15.78:0/4243422184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0540103ba0 msgr2=0x7f05401997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053f59e640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0540103ba0 0x7f05401997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053f59e640 1 -- 172.21.15.78:0/4243422184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054019e640 con 0x7f054010cff0 2024-05-21T15:24:29.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053e59c640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f054010a600 0x7f0540199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:29.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.023+0000 7f053ed9d640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0540103ba0 0x7f05401997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:29.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.024+0000 7f053f59e640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010cff0 0x7f054019e000 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f053000eb10 tx=0x7f053000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:29.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.024+0000 7f0537fff640 1 -- 172.21.15.78:0/4243422184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530004b20 con 0x7f054010cff0 2024-05-21T15:24:29.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.024+0000 7f054523d640 1 -- 172.21.15.78:0/4243422184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054019e930 con 0x7f054010cff0 2024-05-21T15:24:29.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.024+0000 7f0537fff640 1 -- 172.21.15.78:0/4243422184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0530004cc0 con 0x7f054010cff0 2024-05-21T15:24:29.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.024+0000 7f0537fff640 1 -- 172.21.15.78:0/4243422184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530010760 con 0x7f054010cff0 2024-05-21T15:24:29.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.024+0000 7f054523d640 1 -- 172.21.15.78:0/4243422184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054019ee00 con 0x7f054010cff0 2024-05-21T15:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.026+0000 7f054c1c4740 1 Processor -- start 2024-05-21T15:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.026+0000 7f0537fff640 1 -- 172.21.15.78:0/4243422184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0530010900 con 0x7f054010cff0 2024-05-21T15:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f054c1c4740 1 -- start start 2024-05-21T15:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4dfdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4dfe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4dfe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f054c1c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bcd4d63d00 con 0x55bcd4a5f390 2024-05-21T15:24:29.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f054c1c4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bcd4d0b4c0 con 0x55bcd4a364d0 2024-05-21T15:24:29.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f054c1c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bcd4d5fcc0 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4dfdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4dfdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53064/0 (socket says 172.21.15.78:53064) 2024-05-21T15:24:29.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053e59c640 1 -- 172.21.15.78:0/379410848 learned_addr learned my addr 172.21.15.78:0/379410848 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:29.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053ed9d640 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4dfe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053e59c640 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4dfe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053e59c640 1 -- 172.21.15.78:0/379410848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 msgr2=0x55bcd4dfe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:29.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053e59c640 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4dfe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053e59c640 1 -- 172.21.15.78:0/379410848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 msgr2=0x55bcd4dfe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:29.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.027+0000 7f053e59c640 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4dfe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f0537fff640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f052c0a2d00 0x7f052c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f053e59c640 1 -- 172.21.15.78:0/379410848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd4b417d0 con 0x55bcd4a5f390 2024-05-21T15:24:29.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f053ed9d640 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4dfe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:29.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f0537fff640 1 -- 172.21.15.78:0/4243422184 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f05300cf9c0 con 0x7f054010cff0 2024-05-21T15:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f053e59c640 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4dfdbe0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0528031490 tx=0x7f05280058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bee29517227546 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f053e59c640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f052c0a2d00 0x7f052c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f05357fa640 1 -- 172.21.15.78:0/379410848 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528035040 con 0x55bcd4a5f390 2024-05-21T15:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f05357fa640 1 -- 172.21.15.78:0/379410848 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05280042c0 con 0x55bcd4a5f390 2024-05-21T15:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.028+0000 7f05357fa640 1 -- 172.21.15.78:0/379410848 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052803b940 con 0x55bcd4a5f390 2024-05-21T15:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 -- 172.21.15.78:0/379410848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 msgr2=0x55bcd4dfdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4dfdbe0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0528031490 tx=0x7f05280058d0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f053e59c640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f052c0a2d00 0x7f052c0a5210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f052803bca0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 -- 172.21.15.78:0/379410848 shutdown_connections 2024-05-21T15:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4dfe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4dfe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 --2- 172.21.15.78:0/379410848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4dfdbe0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 -- 172.21.15.78:0/379410848 >> 172.21.15.78:0/379410848 conn(0x55bcd4a534e0 msgr2=0x55bcd4dfa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 -- 172.21.15.78:0/379410848 shutdown_connections 2024-05-21T15:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 -- 172.21.15.78:0/379410848 wait complete. 2024-05-21T15:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.029+0000 7f054c1c4740 1 -- start start 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f054c1c4740 1 Processor -- start 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 0x55bcd4e0e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e0e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a5f390 0x55bcd4e0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f054c1c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bcd4e0f490 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f054c1c4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bcd4d1eb90 con 0x55bcd4a5f390 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f054c1c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bcd4e0f610 con 0x55bcd4a364d0 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f054c1c4740 10 client.0 _ll_register_callbacks cb 0x55bcd4d5ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e0e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e0e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53072/0 (socket says 172.21.15.78:53072) 2024-05-21T15:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053e59c640 1 -- 172.21.15.78:0/2629511250 learned_addr learned my addr 172.21.15.78:0/2629511250 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053ed9d640 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 0x55bcd4e0e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053f59e640 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a5f390 0x55bcd4e0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053e59c640 1 -- 172.21.15.78:0/2629511250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 msgr2=0x55bcd4e0e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053e59c640 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 0x55bcd4e0e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053e59c640 1 -- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a5f390 msgr2=0x55bcd4e0c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053e59c640 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a5f390 0x55bcd4e0c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053e59c640 1 -- 172.21.15.78:0/2629511250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd4a773f0 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.030+0000 7f053f59e640 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a5f390 0x55bcd4e0c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f053e59c640 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e0e730 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f052802a450 tx=0x7f052803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32be097de9caefc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f05267fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f05267fc640 1 -- 172.21.15.78:0/2629511250 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528059070 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 1 -- 172.21.15.78:0/2629511250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05280042c0 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f05267fc640 1 -- 172.21.15.78:0/2629511250 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f052803b820 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f05267fc640 1 -- 172.21.15.78:0/2629511250 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0528035070 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 10 client.18039 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 1 -- 172.21.15.78:0/2629511250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bcd4a370f0 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f05267fc640 1 -- 172.21.15.78:0/2629511250 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05280042c0 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f05267fc640 1 -- 172.21.15.78:0/2629511250 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0528064030 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 10 client.18039 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 10 client.18039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 20 client.18039 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bcd4a1df60 0x55bcd4e1ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 4 client.18039 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:29.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.031+0000 7f054c1c4740 1 -- 172.21.15.78:0/2629511250 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcd4d71430 con 0x55bcd4a1df60 2024-05-21T15:24:29.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.032+0000 7f053ed9d640 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bcd4a1df60 0x55bcd4e1ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.032+0000 7f05267fc640 10 client.18039 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:29.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.032+0000 7f053ed9d640 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bcd4a1df60 0x55bcd4e1ee20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54fea244dece1a server_cookie=3ab139dfa7d91a41 in_seq=0 out_seq=0 2024-05-21T15:24:29.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f05267fc640 1 -- 172.21.15.78:0/2629511250 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bcd4d71430 con 0x55bcd4a1df60 2024-05-21T15:24:29.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f05267fc640 10 client.18039 handle_command_reply: tid=0 2024-05-21T15:24:29.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 client.18039 shutdown 2024-05-21T15:24:29.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 -- 172.21.15.78:0/2629511250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a3e6b0 msgr2=0x55bcd4e0e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e0e730 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f052802a450 tx=0x7f052803f000 comp rx=0 tx=0).stop 2024-05-21T15:24:29.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 -- 172.21.15.78:0/2629511250 shutdown_connections 2024-05-21T15:24:29.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bcd4a1df60 0x55bcd4e1ee20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a5f390 0x55bcd4e0c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e0e730 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f05267fc640 0 client.18039 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:29.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 --2- 172.21.15.78:0/2629511250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 0x55bcd4e0e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 -- 172.21.15.78:0/2629511250 >> 172.21.15.78:0/2629511250 conn(0x55bcd4a534e0 msgr2=0x55bcd4dff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:29.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 -- 172.21.15.78:0/2629511250 shutdown_connections 2024-05-21T15:24:29.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 1 -- 172.21.15.78:0/2629511250 wait complete. 2024-05-21T15:24:29.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.033+0000 7f054c1c4740 20 client.18039 trim_cache size 0 max 16384 2024-05-21T15:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.050+0000 7f054c1c4740 1 Processor -- start 2024-05-21T15:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f054c1c4740 1 -- start start 2024-05-21T15:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 0x55bcd4e0c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e05f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f054c1c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05401acaa0 con 0x55bcd4a5f390 2024-05-21T15:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f054c1c4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05401ae5e0 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f054c1c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05401ad170 con 0x55bcd4a364d0 2024-05-21T15:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53078/0 (socket says 172.21.15.78:53078) 2024-05-21T15:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053f59e640 1 -- 172.21.15.78:0/3684354685 learned_addr learned my addr 172.21.15.78:0/3684354685 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053ed9d640 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 0x55bcd4e0c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053e59c640 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e05f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053f59e640 1 -- 172.21.15.78:0/3684354685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 msgr2=0x55bcd4e0c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053f59e640 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 0x55bcd4e0c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053f59e640 1 -- 172.21.15.78:0/3684354685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a3e6b0 msgr2=0x55bcd4e05f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053f59e640 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e05f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.051+0000 7f053f59e640 1 -- 172.21.15.78:0/3684354685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd4dfb0d0 con 0x55bcd4a5f390 2024-05-21T15:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.052+0000 7f053f59e640 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e064b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f05300982b0 tx=0x7f0530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba99d21926fc9478 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.052+0000 7f0534ff9640 1 -- 172.21.15.78:0/3684354685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053009be50 con 0x55bcd4a5f390 2024-05-21T15:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.052+0000 7f0534ff9640 1 -- 172.21.15.78:0/3684354685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053009e040 con 0x55bcd4a5f390 2024-05-21T15:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.052+0000 7f0534ff9640 1 -- 172.21.15.78:0/3684354685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05300a4890 con 0x55bcd4a5f390 2024-05-21T15:24:29.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.052+0000 7f054c1c4740 1 -- 172.21.15.78:0/3684354685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 msgr2=0x55bcd4e064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.052+0000 7f054c1c4740 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e064b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f05300982b0 tx=0x7f0530004a40 comp rx=0 tx=0).stop 2024-05-21T15:24:29.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 -- 172.21.15.78:0/3684354685 shutdown_connections 2024-05-21T15:24:29.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e05f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e064b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 --2- 172.21.15.78:0/3684354685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a364d0 0x55bcd4e0c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 -- 172.21.15.78:0/3684354685 >> 172.21.15.78:0/3684354685 conn(0x55bcd4a534e0 msgr2=0x55bcd4e0bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 -- 172.21.15.78:0/3684354685 shutdown_connections 2024-05-21T15:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 -- 172.21.15.78:0/3684354685 wait complete. 2024-05-21T15:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 -- start start 2024-05-21T15:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 Processor -- start 2024-05-21T15:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4e7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05401adf10 con 0x55bcd4a5f390 2024-05-21T15:24:29.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f05401ad840 con 0x55bcd4a364d0 2024-05-21T15:24:29.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05401aaac0 con 0x55bcd4a3e6b0 2024-05-21T15:24:29.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f054c1c4740 10 client.0 _ll_register_callbacks cb 0x7f0538009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:29.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f053f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.053+0000 7f053e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e1f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53086/0 (socket says 172.21.15.78:53086) 2024-05-21T15:24:29.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053f59e640 1 -- 172.21.15.78:0/1183194560 learned_addr learned my addr 172.21.15.78:0/1183194560 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e7f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:58898/0 (socket says 172.21.15.78:58898) 2024-05-21T15:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053ed9d640 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4e7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053f59e640 1 -- 172.21.15.78:0/1183194560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 msgr2=0x55bcd4e7f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053f59e640 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e7f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053f59e640 1 -- 172.21.15.78:0/1183194560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 msgr2=0x55bcd4e7f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053f59e640 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4e7f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053f59e640 1 -- 172.21.15.78:0/1183194560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd4e1f850 con 0x55bcd4a5f390 2024-05-21T15:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053e59c640 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f053f59e640 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e1f260 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f05300cd130 tx=0x7f0530005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a5ea235ec94360 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f05267fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f05267fc640 1 -- 172.21.15.78:0/1183194560 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053007e030 con 0x55bcd4a5f390 2024-05-21T15:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f05267fc640 1 -- 172.21.15.78:0/1183194560 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f053009b2a0 con 0x55bcd4a5f390 2024-05-21T15:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f054c1c4740 1 -- 172.21.15.78:0/1183194560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f053009e040 con 0x55bcd4a5f390 2024-05-21T15:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f05267fc640 1 -- 172.21.15.78:0/1183194560 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f05300a4ac0 con 0x55bcd4a5f390 2024-05-21T15:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f054c1c4740 10 client.18045 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f054c1c4740 1 -- 172.21.15.78:0/1183194560 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bcd4e86950 con 0x55bcd4a5f390 2024-05-21T15:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.054+0000 7f05267fc640 1 -- 172.21.15.78:0/1183194560 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053009e040 con 0x55bcd4a5f390 2024-05-21T15:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f05267fc640 1 -- 172.21.15.78:0/1183194560 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f053009ad10 con 0x55bcd4a5f390 2024-05-21T15:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f054c1c4740 10 client.18045 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f054c1c4740 10 client.18045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f054c1c4740 20 client.18045 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f054c1c4740 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bcd49f33a0 0x55bcd4e88fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f054c1c4740 4 client.18045 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f054c1c4740 1 -- 172.21.15.78:0/1183194560 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bcd4d6b780 con 0x55bcd49f33a0 2024-05-21T15:24:29.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f053ed9d640 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bcd49f33a0 0x55bcd4e88fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:29.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f05267fc640 10 client.18045 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:29.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f053ed9d640 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bcd49f33a0 0x55bcd4e88fc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f09ca24db6e334 server_cookie=bf4d5fbee45f3ba1 in_seq=0 out_seq=0 2024-05-21T15:24:29.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f05267fc640 1 -- 172.21.15.78:0/1183194560 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bcd4d6b780 con 0x55bcd49f33a0 2024-05-21T15:24:29.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f05267fc640 10 client.18045 handle_command_reply: tid=0 2024-05-21T15:24:29.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.055+0000 7f054c1c4740 1 client.18045 shutdown 2024-05-21T15:24:29.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 -- 172.21.15.78:0/1183194560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 msgr2=0x55bcd4e1f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e1f260 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f05300cd130 tx=0x7f0530005b40 comp rx=0 tx=0).stop 2024-05-21T15:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 -- 172.21.15.78:0/1183194560 shutdown_connections 2024-05-21T15:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55bcd49f33a0 0x55bcd4e88fc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcd4a5f390 0x55bcd4e1f260 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bcd4a3e6b0 0x55bcd4e7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f05267fc640 0 client.18045 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 --2- 172.21.15.78:0/1183194560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd4a364d0 0x55bcd4e7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 -- 172.21.15.78:0/1183194560 >> 172.21.15.78:0/1183194560 conn(0x55bcd4a534e0 msgr2=0x55bcd4df9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 -- 172.21.15.78:0/1183194560 shutdown_connections 2024-05-21T15:24:29.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 1 -- 172.21.15.78:0/1183194560 wait complete. 2024-05-21T15:24:29.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054c1c4740 20 client.18045 trim_cache size 0 max 16384 2024-05-21T15:24:29.071 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:24:29.071 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:24:29.071 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:24:29.071 INFO:teuthology.orchestra.run.smithi078.stdout: "085a32b3-b007-4c06-8d34-1d9522863f90": { 2024-05-21T15:24:29.072 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:24:29.072 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "085a32b3-b007-4c06-8d34-1d9522863f90", 2024-05-21T15:24:29.072 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:24:29.072 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:24:29.072 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:24:29.072 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:24:29.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 -- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f052c0a2d00 msgr2=0x7f052c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f052c0a2d00 0x7f052c0a5210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0528000c00 tx=0x7f052803bca0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 -- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010cff0 msgr2=0x7f054019e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:29.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010cff0 0x7f054019e000 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f053000eb10 tx=0x7f053000c2d0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 -- 172.21.15.78:0/4243422184 shutdown_connections 2024-05-21T15:24:29.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f052c0a2d00 0x7f052c0a5210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f054010cff0 0x7f054019e000 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f054010a600 0x7f0540199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 --2- 172.21.15.78:0/4243422184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0540103ba0 0x7f05401997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 -- 172.21.15.78:0/4243422184 >> 172.21.15.78:0/4243422184 conn(0x7f05400770b0 msgr2=0x7f0540100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.056+0000 7f054523d640 1 -- 172.21.15.78:0/4243422184 shutdown_connections 2024-05-21T15:24:29.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:29.057+0000 7f054523d640 1 -- 172.21.15.78:0/4243422184 wait complete. 2024-05-21T15:24:29.075 INFO:tasks.cephfs.filesystem:scrub status for tag:085a32b3-b007-4c06-8d34-1d9522863f90 - {'path': '/', 'tag': '085a32b3-b007-4c06-8d34-1d9522863f90', 'options': 'recursive,force'} 2024-05-21T15:24:29.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:24:29.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:24:29.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:24:29.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:24:29.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:24:29.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:24:29.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:29 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:24:29.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:24:29.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:24:29.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:24:29.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:24:29.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:24:29.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:29 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:24:29.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:24:29.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:24:29.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:24:29.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:24:29.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:24:29.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:29.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:29 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:30.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:30 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:24:30.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:30 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:30.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:30 smithi078 ceph-mon[31997]: pgmap v259: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:30.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:30 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:24:30.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:30 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:30.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:30 smithi084 ceph-mon[36334]: pgmap v259: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:30.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:30 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:24:30.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:30 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:24:30.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:30 smithi172 ceph-mon[36402]: pgmap v259: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:32.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:32 smithi078 ceph-mon[31997]: pgmap v260: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T15:24:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:32 smithi084 ceph-mon[36334]: pgmap v260: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T15:24:32.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:32 smithi172 ceph-mon[36402]: pgmap v260: 97 pgs: 97 active+clean; 14 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T15:24:34.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:34 smithi078 ceph-mon[31997]: pgmap v261: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:24:34.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:34 smithi084 ceph-mon[36334]: pgmap v261: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:24:34.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:34 smithi172 ceph-mon[36402]: pgmap v261: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:24:36.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:36 smithi078 ceph-mon[31997]: pgmap v262: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:36.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:36 smithi084 ceph-mon[36334]: pgmap v262: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:36.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:36 smithi172 ceph-mon[36402]: pgmap v262: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:37.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:37 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:37.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:37 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:37.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:37 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:37 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:37 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:37.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:37 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:37.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:37 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:37.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:37 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:37.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:37 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:38.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:38 smithi078 ceph-mon[31997]: pgmap v263: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 39 KiB/s rd, 45 op/s 2024-05-21T15:24:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:38 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:38 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:24:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:38 smithi084 ceph-mon[36334]: pgmap v263: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 39 KiB/s rd, 45 op/s 2024-05-21T15:24:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:38 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:38 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:24:38.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:38 smithi172 ceph-mon[36402]: pgmap v263: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 39 KiB/s rd, 45 op/s 2024-05-21T15:24:38.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:38 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:38.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:38 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:24:40.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:40 smithi078 ceph-mon[31997]: pgmap v264: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:40 smithi084 ceph-mon[36334]: pgmap v264: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:40 smithi172 ceph-mon[36402]: pgmap v264: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:42.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:42 smithi078 ceph-mon[31997]: pgmap v265: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:42 smithi084 ceph-mon[36334]: pgmap v265: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:42.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:42 smithi172 ceph-mon[36402]: pgmap v265: 97 pgs: 97 active+clean; 14 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:24:44.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:44 smithi078 ceph-mon[31997]: pgmap v266: 97 pgs: 97 active+clean; 14 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:24:44.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:44 smithi084 ceph-mon[36334]: pgmap v266: 97 pgs: 97 active+clean; 14 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:24:44.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:44 smithi172 ceph-mon[36402]: pgmap v266: 97 pgs: 97 active+clean; 14 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:24:46.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:46 smithi078 ceph-mon[31997]: pgmap v267: 97 pgs: 97 active+clean; 14 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:46.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:46 smithi084 ceph-mon[36334]: pgmap v267: 97 pgs: 97 active+clean; 14 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:46.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:46 smithi172 ceph-mon[36402]: pgmap v267: 97 pgs: 97 active+clean; 14 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:48.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:48 smithi078 ceph-mon[31997]: pgmap v268: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:48.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:48 smithi084 ceph-mon[36334]: pgmap v268: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:48.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:48 smithi172 ceph-mon[36402]: pgmap v268: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:50.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:50 smithi078 ceph-mon[31997]: pgmap v269: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:50.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:50 smithi084 ceph-mon[36334]: pgmap v269: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:50.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:50 smithi172 ceph-mon[36402]: pgmap v269: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:52 smithi078 ceph-mon[31997]: pgmap v270: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:52 smithi084 ceph-mon[36334]: pgmap v270: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:52.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:52 smithi172 ceph-mon[36402]: pgmap v270: 97 pgs: 97 active+clean; 14 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:54.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:54 smithi078 ceph-mon[31997]: pgmap v271: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:54.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:54 smithi084 ceph-mon[36334]: pgmap v271: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:54.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:54 smithi172 ceph-mon[36402]: pgmap v271: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:56.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:56 smithi078 ceph-mon[31997]: pgmap v272: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:56.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:56 smithi084 ceph-mon[36334]: pgmap v272: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:56.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:56 smithi172 ceph-mon[36402]: pgmap v272: 97 pgs: 97 active+clean; 14 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:24:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:24:58 smithi078 ceph-mon[31997]: pgmap v273: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:58.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:24:58 smithi084 ceph-mon[36334]: pgmap v273: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:58.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:24:58 smithi172 ceph-mon[36402]: pgmap v273: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:24:59.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:24:59.077 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:24:59.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 -- 172.21.15.78:0/2221112735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6018103c10 msgr2=0x7f6018104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 --2- 172.21.15.78:0/2221112735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6018103c10 0x7f6018104020 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6008009900 tx=0x7f600802a410 comp rx=0 tx=0).stop 2024-05-21T15:24:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 -- 172.21.15.78:0/2221112735 shutdown_connections 2024-05-21T15:24:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 --2- 172.21.15.78:0/2221112735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f601810a670 0x7f601810ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 --2- 172.21.15.78:0/2221112735 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f601810d060 0x7f601810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 --2- 172.21.15.78:0/2221112735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6018103c10 0x7f6018104020 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 -- 172.21.15.78:0/2221112735 >> 172.21.15.78:0/2221112735 conn(0x7f60180fd210 msgr2=0x7f60180ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 -- 172.21.15.78:0/2221112735 shutdown_connections 2024-05-21T15:24:59.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.202+0000 7f601fabc640 1 -- 172.21.15.78:0/2221112735 wait complete. 2024-05-21T15:24:59.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601fabc640 1 Processor -- start 2024-05-21T15:24:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601fabc640 1 -- start start 2024-05-21T15:24:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601fabc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6018103c10 0x7f60181997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601fabc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601810a670 0x7f6018199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601fabc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f601810d060 0x7f601819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601fabc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6018112df0 con 0x7f601810a670 2024-05-21T15:24:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601fabc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6018112c70 con 0x7f601810d060 2024-05-21T15:24:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601fabc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6018112f70 con 0x7f6018103c10 2024-05-21T15:24:59.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d057640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601810a670 0x7f6018199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d057640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601810a670 0x7f6018199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37826/0 (socket says 172.21.15.78:37826) 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d057640 1 -- 172.21.15.78:0/440477323 learned_addr learned my addr 172.21.15.78:0/440477323 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d858640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6018103c10 0x7f60181997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601e059640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f601810d060 0x7f601819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d057640 1 -- 172.21.15.78:0/440477323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6018103c10 msgr2=0x7f60181997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d057640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6018103c10 0x7f60181997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d057640 1 -- 172.21.15.78:0/440477323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f601810d060 msgr2=0x7f601819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d057640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f601810d060 0x7f601819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.203+0000 7f601d057640 1 -- 172.21.15.78:0/440477323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601819e650 con 0x7f601810a670 2024-05-21T15:24:59.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.204+0000 7f601d057640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601810a670 0x7f6018199d50 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f600c00d950 tx=0x7f600c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.204+0000 7f6016ffd640 1 -- 172.21.15.78:0/440477323 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c017070 con 0x7f601810a670 2024-05-21T15:24:59.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.204+0000 7f601fabc640 1 -- 172.21.15.78:0/440477323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601819e940 con 0x7f601810a670 2024-05-21T15:24:59.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.204+0000 7f6016ffd640 1 -- 172.21.15.78:0/440477323 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600c009c30 con 0x7f601810a670 2024-05-21T15:24:59.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.204+0000 7f6016ffd640 1 -- 172.21.15.78:0/440477323 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0047a0 con 0x7f601810a670 2024-05-21T15:24:59.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.204+0000 7f601fabc640 1 -- 172.21.15.78:0/440477323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60181a62c0 con 0x7f601810a670 2024-05-21T15:24:59.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.206+0000 7f6016ffd640 1 -- 172.21.15.78:0/440477323 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f600c004020 con 0x7f601810a670 2024-05-21T15:24:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f60265fd740 1 Processor -- start 2024-05-21T15:24:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f60265fd740 1 -- start start 2024-05-21T15:24:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f60265fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a075178390 0x55a075516d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f6016ffd640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f60000a2d00 0x7f60000a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f60265fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075517270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f60265fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a0755177b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f60265fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a07547cd00 con 0x55a075178390 2024-05-21T15:24:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f60265fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0754244c0 con 0x55a0751576b0 2024-05-21T15:24:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f60265fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a075478cc0 con 0x55a07514f4d0 2024-05-21T15:24:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f601e059640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a075178390 0x55a075516d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f601e059640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a075178390 0x55a075516d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37828/0 (socket says 172.21.15.78:37828) 2024-05-21T15:24:59.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f601d858640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f60000a2d00 0x7f60000a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f601e059640 1 -- 172.21.15.78:0/3138226487 learned_addr learned my addr 172.21.15.78:0/3138226487 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f6016ffd640 1 -- 172.21.15.78:0/440477323 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f600c0cf150 con 0x7f601810a670 2024-05-21T15:24:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.207+0000 7f601d057640 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075517270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601e059640 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a0755177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601e059640 1 -- 172.21.15.78:0/3138226487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 msgr2=0x55a075517270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601e059640 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075517270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601e059640 1 -- 172.21.15.78:0/3138226487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 msgr2=0x55a0755177b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601e059640 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a0755177b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601e059640 1 -- 172.21.15.78:0/3138226487 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a07525a7d0 con 0x55a075178390 2024-05-21T15:24:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601d057640 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075517270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:24:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601d858640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f60000a2d00 0x7f60000a5210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6008004770 tx=0x7f60080058d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f601e059640 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a075178390 0x55a075516d30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6010012820 tx=0x7f6010009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d85e663ed9b1b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f5ffffff640 1 -- 172.21.15.78:0/3138226487 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010012e60 con 0x55a075178390 2024-05-21T15:24:59.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f5ffffff640 1 -- 172.21.15.78:0/3138226487 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6010003e50 con 0x55a075178390 2024-05-21T15:24:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.208+0000 7f5ffffff640 1 -- 172.21.15.78:0/3138226487 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60100043b0 con 0x55a075178390 2024-05-21T15:24:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 -- 172.21.15.78:0/3138226487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a075178390 msgr2=0x55a075516d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a075178390 0x55a075516d30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6010012820 tx=0x7f6010009b10 comp rx=0 tx=0).stop 2024-05-21T15:24:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 -- 172.21.15.78:0/3138226487 shutdown_connections 2024-05-21T15:24:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a0755177b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075517270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 --2- 172.21.15.78:0/3138226487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a075178390 0x55a075516d30 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 -- 172.21.15.78:0/3138226487 >> 172.21.15.78:0/3138226487 conn(0x55a07516c4e0 msgr2=0x55a0755134e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 -- 172.21.15.78:0/3138226487 shutdown_connections 2024-05-21T15:24:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.209+0000 7f60265fd740 1 -- 172.21.15.78:0/3138226487 wait complete. 2024-05-21T15:24:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 1 -- start start 2024-05-21T15:24:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 1 Processor -- start 2024-05-21T15:24:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075522e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0751576b0 0x55a075523390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a075178390 0x55a075521360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a075523fc0 con 0x55a0751576b0 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a075437b90 con 0x55a075178390 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a075524140 con 0x55a07514f4d0 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f60265fd740 10 client.0 _ll_register_callbacks cb 0x55a075473da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f601d057640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0751576b0 0x55a075523390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f601d057640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0751576b0 0x55a075523390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37844/0 (socket says 172.21.15.78:37844) 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f601d057640 1 -- 172.21.15.78:0/1687449005 learned_addr learned my addr 172.21.15.78:0/1687449005 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.210+0000 7f601d858640 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075522e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f601e059640 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a075178390 0x55a075521360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f601d057640 1 -- 172.21.15.78:0/1687449005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 msgr2=0x55a075522e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f601d057640 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075522e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f601d057640 1 -- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a075178390 msgr2=0x55a075521360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f601d057640 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a075178390 0x55a075521360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f601d057640 1 -- 172.21.15.78:0/1687449005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0751903f0 con 0x55a0751576b0 2024-05-21T15:24:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f601e059640 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a075178390 0x55a075521360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:24:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f601d057640 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0751576b0 0x55a075523390 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f600c005fd0 tx=0x7f600c09b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20a2ea1ccd176b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f5ffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:59.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1687449005 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0a0b20 con 0x55a0751576b0 2024-05-21T15:24:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1687449005 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f600c097c50 con 0x55a0751576b0 2024-05-21T15:24:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f60265fd740 1 -- 172.21.15.78:0/1687449005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6010003e50 con 0x55a0751576b0 2024-05-21T15:24:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1687449005 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f600c09bce0 con 0x55a0751576b0 2024-05-21T15:24:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f60265fd740 10 client.18057 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:59.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f60265fd740 1 -- 172.21.15.78:0/1687449005 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a0751500f0 con 0x55a0751576b0 2024-05-21T15:24:59.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.211+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1687449005 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6010003e50 con 0x55a0751576b0 2024-05-21T15:24:59.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1687449005 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f600c0cca90 con 0x55a0751576b0 2024-05-21T15:24:59.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f60265fd740 10 client.18057 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:59.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f60265fd740 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:59.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f60265fd740 20 client.18057 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f60265fd740 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a075136f60 0x55a075533b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f60265fd740 4 client.18057 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f60265fd740 1 -- 172.21.15.78:0/1687449005 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a07548a430 con 0x55a075136f60 2024-05-21T15:24:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f601d858640 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a075136f60 0x55a075533b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f5ffd7fa640 10 client.18057 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.212+0000 7f601d858640 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a075136f60 0x55a075533b20 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3789eff723abc050 server_cookie=4e35e5853008be44 in_seq=0 out_seq=0 2024-05-21T15:24:59.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1687449005 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a07548a430 con 0x55a075136f60 2024-05-21T15:24:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f5ffd7fa640 10 client.18057 handle_command_reply: tid=0 2024-05-21T15:24:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 client.18057 shutdown 2024-05-21T15:24:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 -- 172.21.15.78:0/1687449005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0751576b0 msgr2=0x55a075523390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0751576b0 0x55a075523390 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f600c005fd0 tx=0x7f600c09b1e0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 -- 172.21.15.78:0/1687449005 shutdown_connections 2024-05-21T15:24:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a075136f60 0x55a075533b20 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a075178390 0x55a075521360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0751576b0 0x55a075523390 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f5ffd7fa640 0 client.18057 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 --2- 172.21.15.78:0/1687449005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a07514f4d0 0x55a075522e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 -- 172.21.15.78:0/1687449005 >> 172.21.15.78:0/1687449005 conn(0x55a07516c4e0 msgr2=0x55a075518a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.213+0000 7f60265fd740 1 -- 172.21.15.78:0/1687449005 shutdown_connections 2024-05-21T15:24:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.214+0000 7f60265fd740 1 -- 172.21.15.78:0/1687449005 wait complete. 2024-05-21T15:24:59.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.214+0000 7f60265fd740 20 client.18057 trim_cache size 0 max 16384 2024-05-21T15:24:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.231+0000 7f60265fd740 1 Processor -- start 2024-05-21T15:24:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.231+0000 7f60265fd740 1 -- start start 2024-05-21T15:24:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f60265fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a0755135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f60265fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a075539330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f60265fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 0x55a075539870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f60265fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60181aedd0 con 0x55a07514f4d0 2024-05-21T15:24:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f60265fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f60181b0910 con 0x55a0751576b0 2024-05-21T15:24:59.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f60265fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60181af4a0 con 0x55a075178390 2024-05-21T15:24:59.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a0755135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a0755135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37850/0 (socket says 172.21.15.78:37850) 2024-05-21T15:24:59.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 -- 172.21.15.78:0/762588446 learned_addr learned my addr 172.21.15.78:0/762588446 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601e059640 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 0x55a075539870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d057640 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a075539330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 -- 172.21.15.78:0/762588446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 msgr2=0x55a075539870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 0x55a075539870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 -- 172.21.15.78:0/762588446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 msgr2=0x55a075539330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a075539330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 -- 172.21.15.78:0/762588446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a075520660 con 0x55a07514f4d0 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d057640 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a075539330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f601d858640 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a0755135b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f6008009230 tx=0x7f6008040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4193d01f9866e81 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.232+0000 7f5fff7fe640 1 -- 172.21.15.78:0/762588446 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008056070 con 0x55a07514f4d0 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f5fff7fe640 1 -- 172.21.15.78:0/762588446 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008004240 con 0x55a07514f4d0 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f5fff7fe640 1 -- 172.21.15.78:0/762588446 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008035040 con 0x55a07514f4d0 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 -- 172.21.15.78:0/762588446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 msgr2=0x55a0755135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a0755135b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f6008009230 tx=0x7f6008040000 comp rx=0 tx=0).stop 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 -- 172.21.15.78:0/762588446 shutdown_connections 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a075539330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 0x55a075539870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 --2- 172.21.15.78:0/762588446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a0755135b0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 -- 172.21.15.78:0/762588446 >> 172.21.15.78:0/762588446 conn(0x55a07516c4e0 msgr2=0x55a075512ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 -- 172.21.15.78:0/762588446 shutdown_connections 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.233+0000 7f60265fd740 1 -- 172.21.15.78:0/762588446 wait complete. 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 1 -- start start 2024-05-21T15:24:59.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 1 Processor -- start 2024-05-21T15:24:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a075529520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a075529a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 0x55a0755259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60181b0240 con 0x55a07514f4d0 2024-05-21T15:24:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f60181afb70 con 0x55a0751576b0 2024-05-21T15:24:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60181acdd0 con 0x55a075178390 2024-05-21T15:24:59.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f60265fd740 10 client.0 _ll_register_callbacks cb 0x7f600c00b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f601d858640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a075529520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f601d858640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a075529520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37852/0 (socket says 172.21.15.78:37852) 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.234+0000 7f601d858640 1 -- 172.21.15.78:0/1902641470 learned_addr learned my addr 172.21.15.78:0/1902641470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f601d858640 1 -- 172.21.15.78:0/1902641470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 msgr2=0x55a0755259e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f601d858640 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 0x55a0755259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f601d858640 1 -- 172.21.15.78:0/1902641470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 msgr2=0x55a075529a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f601d858640 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a075529a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f601d858640 1 -- 172.21.15.78:0/1902641470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a075526150 con 0x55a07514f4d0 2024-05-21T15:24:59.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f601d858640 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a075529520 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6008005e20 tx=0x7f6008031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d0fa283c6c7b52 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f5ffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1902641470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008056070 con 0x55a07514f4d0 2024-05-21T15:24:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1902641470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f6008051490 con 0x55a07514f4d0 2024-05-21T15:24:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f60265fd740 1 -- 172.21.15.78:0/1902641470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6008004240 con 0x55a07514f4d0 2024-05-21T15:24:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1902641470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f600805f050 con 0x55a07514f4d0 2024-05-21T15:24:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f60265fd740 10 client.18063 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f60265fd740 1 -- 172.21.15.78:0/1902641470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a07559ca00 con 0x55a07514f4d0 2024-05-21T15:24:59.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1902641470 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008004240 con 0x55a07514f4d0 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.235+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1902641470 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6008059020 con 0x55a07514f4d0 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 10 client.18063 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 20 client.18063 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a07510c3a0 0x55a07559f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 4 client.18063 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 1 -- 172.21.15.78:0/1902641470 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a075484780 con 0x55a07510c3a0 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f601d057640 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a07510c3a0 0x55a07559f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f5ffd7fa640 10 client.18063 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f601d057640 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a07510c3a0 0x55a07559f250 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc3fce6f913bf32 server_cookie=f07fea3879d866a9 in_seq=0 out_seq=0 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f5ffd7fa640 1 -- 172.21.15.78:0/1902641470 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a075484780 con 0x55a07510c3a0 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f5ffd7fa640 10 client.18063 handle_command_reply: tid=0 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 1 client.18063 shutdown 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 1 -- 172.21.15.78:0/1902641470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 msgr2=0x55a075529520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.236+0000 7f60265fd740 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a075529520 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6008005e20 tx=0x7f6008031630 comp rx=0 tx=0).stop 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 1 -- 172.21.15.78:0/1902641470 shutdown_connections 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55a07510c3a0 0x55a07559f250 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0751576b0 0x55a075529a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a075178390 0x55a0755259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 1 --2- 172.21.15.78:0/1902641470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a07514f4d0 0x55a075529520 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f5ffd7fa640 0 client.18063 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:24:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 1 -- 172.21.15.78:0/1902641470 >> 172.21.15.78:0/1902641470 conn(0x55a07516c4e0 msgr2=0x55a075516ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 1 -- 172.21.15.78:0/1902641470 shutdown_connections 2024-05-21T15:24:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 1 -- 172.21.15.78:0/1902641470 wait complete. 2024-05-21T15:24:59.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f60265fd740 20 client.18063 trim_cache size 0 max 16384 2024-05-21T15:24:59.250 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:24:59.250 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:24:59.250 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:24:59.250 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:24:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 -- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f60000a2d00 msgr2=0x7f60000a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f60000a2d00 0x7f60000a5210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6008004770 tx=0x7f60080058d0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 -- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601810a670 msgr2=0x7f6018199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601810a670 0x7f6018199d50 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f600c00d950 tx=0x7f600c00de10 comp rx=0 tx=0).stop 2024-05-21T15:24:59.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 -- 172.21.15.78:0/440477323 shutdown_connections 2024-05-21T15:24:59.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f601810d060 0x7f601819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f60000a2d00 0x7f60000a5210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f601810a670 0x7f6018199d50 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 --2- 172.21.15.78:0/440477323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6018103c10 0x7f60181997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 -- 172.21.15.78:0/440477323 >> 172.21.15.78:0/440477323 conn(0x7f60180fd210 msgr2=0x7f601810d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 -- 172.21.15.78:0/440477323 shutdown_connections 2024-05-21T15:24:59.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.237+0000 7f601fabc640 1 -- 172.21.15.78:0/440477323 wait complete. 2024-05-21T15:24:59.254 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:24:59.254 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:24:59.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.362+0000 7f749cc5a640 1 -- 172.21.15.78:0/910268722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498108b50 msgr2=0x7f7498100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.362+0000 7f749cc5a640 1 --2- 172.21.15.78:0/910268722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498108b50 0x7f7498100ac0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f7488009900 tx=0x7f748802a410 comp rx=0 tx=0).stop 2024-05-21T15:24:59.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.363+0000 7f749cc5a640 1 -- 172.21.15.78:0/910268722 shutdown_connections 2024-05-21T15:24:59.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.363+0000 7f749cc5a640 1 --2- 172.21.15.78:0/910268722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7498101000 0x7f7498101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.363+0000 7f749cc5a640 1 --2- 172.21.15.78:0/910268722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f749810d500 0x7f749810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.363+0000 7f749cc5a640 1 --2- 172.21.15.78:0/910268722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498108b50 0x7f7498100ac0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.363+0000 7f749cc5a640 1 -- 172.21.15.78:0/910268722 >> 172.21.15.78:0/910268722 conn(0x7f74980fc810 msgr2=0x7f74980fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.363+0000 7f749cc5a640 1 -- 172.21.15.78:0/910268722 shutdown_connections 2024-05-21T15:24:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.363+0000 7f749cc5a640 1 -- 172.21.15.78:0/910268722 wait complete. 2024-05-21T15:24:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.363+0000 7f749cc5a640 1 Processor -- start 2024-05-21T15:24:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749cc5a640 1 -- start start 2024-05-21T15:24:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749cc5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498101000 0x7f7498199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749cc5a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7498108b50 0x7f7498199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749cc5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f749810d500 0x7f749819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749cc5a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7498112e10 con 0x7f7498101000 2024-05-21T15:24:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749cc5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7498112c90 con 0x7f749810d500 2024-05-21T15:24:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749cc5a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7498112f90 con 0x7f7498108b50 2024-05-21T15:24:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498101000 0x7f7498199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498101000 0x7f7498199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37874/0 (socket says 172.21.15.78:37874) 2024-05-21T15:24:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 -- 172.21.15.78:0/3038298336 learned_addr learned my addr 172.21.15.78:0/3038298336 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f7496d9d640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f749810d500 0x7f749819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f7495d9b640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7498108b50 0x7f7498199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 -- 172.21.15.78:0/3038298336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7498108b50 msgr2=0x7f7498199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7498108b50 0x7f7498199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 -- 172.21.15.78:0/3038298336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f749810d500 msgr2=0x7f749819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f749810d500 0x7f749819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 -- 172.21.15.78:0/3038298336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749819e5f0 con 0x7f7498101000 2024-05-21T15:24:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f7495d9b640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7498108b50 0x7f7498199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:24:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f749659c640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498101000 0x7f7498199790 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f7488005940 tx=0x7f74880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.364+0000 7f74877fe640 1 -- 172.21.15.78:0/3038298336 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748803f030 con 0x7f7498101000 2024-05-21T15:24:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.365+0000 7f74877fe640 1 -- 172.21.15.78:0/3038298336 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7488003ce0 con 0x7f7498101000 2024-05-21T15:24:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.365+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749819e880 con 0x7f7498101000 2024-05-21T15:24:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.365+0000 7f74877fe640 1 -- 172.21.15.78:0/3038298336 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488035070 con 0x7f7498101000 2024-05-21T15:24:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.365+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749819edc0 con 0x7f7498101000 2024-05-21T15:24:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.366+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7498069140 con 0x7f7498101000 2024-05-21T15:24:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.368+0000 7f74877fe640 1 -- 172.21.15.78:0/3038298336 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7488042030 con 0x7f7498101000 2024-05-21T15:24:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.368+0000 7f74877fe640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74800a2d80 0x7f74800a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.369+0000 7f74877fe640 1 -- 172.21.15.78:0/3038298336 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f74880efd00 con 0x7f7498101000 2024-05-21T15:24:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.369+0000 7f7495d9b640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74800a2d80 0x7f74800a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.369+0000 7f7495d9b640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74800a2d80 0x7f74800a5290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f748c005d00 tx=0x7f748c005ed0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.372+0000 7f74877fe640 1 -- 172.21.15.78:0/3038298336 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f74880bc890 con 0x7f7498101000 2024-05-21T15:24:59.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.587+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f749819aae0 con 0x7f7498101000 2024-05-21T15:24:59.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.588+0000 7f74877fe640 1 -- 172.21.15.78:0/3038298336 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7f74880bc190 con 0x7f7498101000 2024-05-21T15:24:59.591 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:24:59.591 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:24:59.592 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:24:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74800a2d80 msgr2=0x7f74800a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74800a2d80 0x7f74800a5290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f748c005d00 tx=0x7f748c005ed0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498101000 msgr2=0x7f7498199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498101000 0x7f7498199790 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f7488005940 tx=0x7f74880042f0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 shutdown_connections 2024-05-21T15:24:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f749810d500 0x7f749819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f74800a2d80 0x7f74800a5290 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7498108b50 0x7f7498199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 --2- 172.21.15.78:0/3038298336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7498101000 0x7f7498199790 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 >> 172.21.15.78:0/3038298336 conn(0x7f74980fc810 msgr2=0x7f74980fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 shutdown_connections 2024-05-21T15:24:59.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.590+0000 7f749cc5a640 1 -- 172.21.15.78:0/3038298336 wait complete. 2024-05-21T15:24:59.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:24:59.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 -- 172.21.15.78:0/1605159386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35341079f0 msgr2=0x7f3534101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 --2- 172.21.15.78:0/1605159386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35341079f0 0x7f3534101c50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3524009ca0 tx=0x7f352402a950 comp rx=0 tx=0).stop 2024-05-21T15:24:59.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 -- 172.21.15.78:0/1605159386 shutdown_connections 2024-05-21T15:24:59.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 --2- 172.21.15.78:0/1605159386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3534102840 0x7f353410e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 --2- 172.21.15.78:0/1605159386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3534107e00 0x7f35341021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 --2- 172.21.15.78:0/1605159386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35341079f0 0x7f3534101c50 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 -- 172.21.15.78:0/1605159386 >> 172.21.15.78:0/1605159386 conn(0x7f3534076830 msgr2=0x7f3534076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 -- 172.21.15.78:0/1605159386 shutdown_connections 2024-05-21T15:24:59.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 -- 172.21.15.78:0/1605159386 wait complete. 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 Processor -- start 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.715+0000 7f353ba7a640 1 -- start start 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353ba7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35341079f0 0x7f3534198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353ba7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3534107e00 0x7f3534198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353ba7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f353419cb20 0x7f353419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353ba7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3534111d00 con 0x7f353419cb20 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353ba7a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3534111b80 con 0x7f3534107e00 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353ba7a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3534111e80 con 0x7f35341079f0 2024-05-21T15:24:59.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f353419cb20 0x7f353419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f353419cb20 0x7f353419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37878/0 (socket says 172.21.15.78:37878) 2024-05-21T15:24:59.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 -- 172.21.15.78:0/547361586 learned_addr learned my addr 172.21.15.78:0/547361586 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f3539816640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35341079f0 0x7f3534198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f3539015640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3534107e00 0x7f3534198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 -- 172.21.15.78:0/547361586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35341079f0 msgr2=0x7f3534198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35341079f0 0x7f3534198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 -- 172.21.15.78:0/547361586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3534107e00 msgr2=0x7f3534198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3534107e00 0x7f3534198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 -- 172.21.15.78:0/547361586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353419d570 con 0x7f353419cb20 2024-05-21T15:24:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353a017640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f353419cb20 0x7f353419cf30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f352c00d750 tx=0x7f352c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f3532ffd640 1 -- 172.21.15.78:0/547361586 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c017070 con 0x7f353419cb20 2024-05-21T15:24:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f353ba7a640 1 -- 172.21.15.78:0/547361586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353419d860 con 0x7f353419cb20 2024-05-21T15:24:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f3532ffd640 1 -- 172.21.15.78:0/547361586 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c005500 con 0x7f353419cb20 2024-05-21T15:24:59.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.716+0000 7f3532ffd640 1 -- 172.21.15.78:0/547361586 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c004d90 con 0x7f353419cb20 2024-05-21T15:24:59.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.717+0000 7f353ba7a640 1 -- 172.21.15.78:0/547361586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35341a5060 con 0x7f353419cb20 2024-05-21T15:24:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.719+0000 7f3532ffd640 1 -- 172.21.15.78:0/547361586 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f352c004f30 con 0x7f353419cb20 2024-05-21T15:24:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.719+0000 7f35429fe740 1 Processor -- start 2024-05-21T15:24:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f35429fe740 1 -- start start 2024-05-21T15:24:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f35429fe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 0x561bc5e6fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f35429fe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 0x561bc5e700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f35429fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ab06b0 0x561bc5e705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f35429fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561bc5dd5c40 con 0x561bc5ab06b0 2024-05-21T15:24:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f35429fe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561bc5d7d580 con 0x561bc5ad1390 2024-05-21T15:24:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f35429fe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561bc5dd1c00 con 0x561bc5aa84d0 2024-05-21T15:24:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ab06b0 0x561bc5e705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ab06b0 0x561bc5e705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37884/0 (socket says 172.21.15.78:37884) 2024-05-21T15:24:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 -- 172.21.15.78:0/4106639157 learned_addr learned my addr 172.21.15.78:0/4106639157 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539816640 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 0x561bc5e700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 0x561bc5e6fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3532ffd640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f351c0a2d00 0x7f351c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 -- 172.21.15.78:0/4106639157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 msgr2=0x561bc5e700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 0x561bc5e700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 -- 172.21.15.78:0/4106639157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 msgr2=0x561bc5e6fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 0x561bc5e6fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f353a017640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f351c0a2d00 0x7f351c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.720+0000 7f3539015640 1 -- 172.21.15.78:0/4106639157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bc5ae93f0 con 0x561bc5ab06b0 2024-05-21T15:24:59.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.721+0000 7f3532ffd640 1 -- 172.21.15.78:0/547361586 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f352c0cee50 con 0x7f353419cb20 2024-05-21T15:24:59.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.721+0000 7f3539015640 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ab06b0 0x561bc5e705f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f3528005880 tx=0x7f352800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a21e078a3fdec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.721+0000 7f351bfff640 1 -- 172.21.15.78:0/4106639157 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004020 con 0x561bc5ab06b0 2024-05-21T15:24:59.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.721+0000 7f351bfff640 1 -- 172.21.15.78:0/4106639157 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528005500 con 0x561bc5ab06b0 2024-05-21T15:24:59.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.721+0000 7f351bfff640 1 -- 172.21.15.78:0/4106639157 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004c40 con 0x561bc5ab06b0 2024-05-21T15:24:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 -- 172.21.15.78:0/4106639157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ab06b0 msgr2=0x561bc5e705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f353a017640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f351c0a2d00 0x7f351c0a5210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f352c0cc5c0 tx=0x7f352c004830 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ab06b0 0x561bc5e705f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f3528005880 tx=0x7f352800b790 comp rx=0 tx=0).stop 2024-05-21T15:24:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 -- 172.21.15.78:0/4106639157 shutdown_connections 2024-05-21T15:24:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ab06b0 0x561bc5e705f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 0x561bc5e700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 --2- 172.21.15.78:0/4106639157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 0x561bc5e6fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 -- 172.21.15.78:0/4106639157 >> 172.21.15.78:0/4106639157 conn(0x561bc5ac54e0 msgr2=0x561bc5e6c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 -- 172.21.15.78:0/4106639157 shutdown_connections 2024-05-21T15:24:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 -- 172.21.15.78:0/4106639157 wait complete. 2024-05-21T15:24:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 -- start start 2024-05-21T15:24:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.722+0000 7f35429fe740 1 Processor -- start 2024-05-21T15:24:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f35429fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e7c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f35429fe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ab06b0 0x561bc5e7c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f35429fe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 0x561bc5e88790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f35429fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561bc5e7f2b0 con 0x561bc5aa84d0 2024-05-21T15:24:59.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f35429fe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561bc5d90c40 con 0x561bc5ad1390 2024-05-21T15:24:59.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f35429fe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561bc5e7f430 con 0x561bc5ab06b0 2024-05-21T15:24:59.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f35429fe740 10 client.0 _ll_register_callbacks cb 0x561bc5ae8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:59.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e7c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e7c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37886/0 (socket says 172.21.15.78:37886) 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 -- 172.21.15.78:0/268924574 learned_addr learned my addr 172.21.15.78:0/268924574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539816640 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ab06b0 0x561bc5e7c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 0x561bc5e88790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 -- 172.21.15.78:0/268924574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ab06b0 msgr2=0x561bc5e7c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ab06b0 0x561bc5e7c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 -- 172.21.15.78:0/268924574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 msgr2=0x561bc5e88790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 0x561bc5e88790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 -- 172.21.15.78:0/268924574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bc5bb37d0 con 0x561bc5aa84d0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f3539015640 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e7c4b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3528004920 tx=0x7f3528001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ed9d58f0371d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.723+0000 7f35197fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35197fa640 1 -- 172.21.15.78:0/268924574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35280049c0 con 0x561bc5aa84d0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35197fa640 1 -- 172.21.15.78:0/268924574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f3528004b60 con 0x561bc5aa84d0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 1 -- 172.21.15.78:0/268924574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528005500 con 0x561bc5aa84d0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35197fa640 1 -- 172.21.15.78:0/268924574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3528004d60 con 0x561bc5aa84d0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 10 client.18081 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 1 -- 172.21.15.78:0/268924574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561bc5e8b930 con 0x561bc5aa84d0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35197fa640 1 -- 172.21.15.78:0/268924574 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528005500 con 0x561bc5aa84d0 2024-05-21T15:24:59.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35197fa640 1 -- 172.21.15.78:0/268924574 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f35280304f0 con 0x561bc5aa84d0 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 10 client.18081 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 10 client.18081 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 20 client.18081 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561bc5a8ff60 0x561bc5e8ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 4 client.18081 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.724+0000 7f35429fe740 1 -- 172.21.15.78:0/268924574 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bc5de3370 con 0x561bc5a8ff60 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.725+0000 7f3539816640 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561bc5a8ff60 0x561bc5e8ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.725+0000 7f3539816640 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561bc5a8ff60 0x561bc5e8ec90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1136ab99739d7a2e server_cookie=8cd792c9866fe6c2 in_seq=0 out_seq=0 2024-05-21T15:24:59.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.725+0000 7f35197fa640 10 client.18081 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35197fa640 1 -- 172.21.15.78:0/268924574 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561bc5de3370 con 0x561bc5a8ff60 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35197fa640 10 client.18081 handle_command_reply: tid=0 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 client.18081 shutdown 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 -- 172.21.15.78:0/268924574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 msgr2=0x561bc5e7c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e7c4b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3528004920 tx=0x7f3528001e50 comp rx=0 tx=0).stop 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 -- 172.21.15.78:0/268924574 shutdown_connections 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561bc5a8ff60 0x561bc5e8ec90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ad1390 0x561bc5e88790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ab06b0 0x561bc5e7c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 --2- 172.21.15.78:0/268924574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e7c4b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35197fa640 0 client.18081 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:24:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 -- 172.21.15.78:0/268924574 >> 172.21.15.78:0/268924574 conn(0x561bc5ac54e0 msgr2=0x561bc5e704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 -- 172.21.15.78:0/268924574 shutdown_connections 2024-05-21T15:24:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 1 -- 172.21.15.78:0/268924574 wait complete. 2024-05-21T15:24:59.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.726+0000 7f35429fe740 20 client.18081 trim_cache size 0 max 16384 2024-05-21T15:24:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f35429fe740 1 Processor -- start 2024-05-21T15:24:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f35429fe740 1 -- start start 2024-05-21T15:24:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f35429fe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 0x561bc5e7e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f35429fe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e75d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f35429fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ad1390 0x561bc5e88370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f35429fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35341acb20 con 0x561bc5ad1390 2024-05-21T15:24:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f35429fe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f35341ae660 con 0x561bc5ab06b0 2024-05-21T15:24:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f35429fe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35341ad1f0 con 0x561bc5aa84d0 2024-05-21T15:24:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ad1390 0x561bc5e88370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ad1390 0x561bc5e88370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37890/0 (socket says 172.21.15.78:37890) 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 -- 172.21.15.78:0/1203799601 learned_addr learned my addr 172.21.15.78:0/1203799601 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 0x561bc5e7e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539816640 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e75d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 -- 172.21.15.78:0/1203799601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 msgr2=0x561bc5e7e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 0x561bc5e7e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 -- 172.21.15.78:0/1203799601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 msgr2=0x561bc5e75d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e75d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539015640 1 -- 172.21.15.78:0/1203799601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bc5e7f440 con 0x561bc5ad1390 2024-05-21T15:24:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.740+0000 7f3539816640 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e75d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:24:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.741+0000 7f3539015640 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ad1390 0x561bc5e88370 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3528004fc0 tx=0x7f3528005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74425e5091743444 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.741+0000 7f351b7fe640 1 -- 172.21.15.78:0/1203799601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352800b390 con 0x561bc5ad1390 2024-05-21T15:24:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.741+0000 7f351b7fe640 1 -- 172.21.15.78:0/1203799601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528023030 con 0x561bc5ad1390 2024-05-21T15:24:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.741+0000 7f351b7fe640 1 -- 172.21.15.78:0/1203799601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004bb0 con 0x561bc5ad1390 2024-05-21T15:24:59.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.741+0000 7f35429fe740 1 -- 172.21.15.78:0/1203799601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ad1390 msgr2=0x561bc5e88370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.741+0000 7f35429fe740 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ad1390 0x561bc5e88370 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3528004fc0 tx=0x7f3528005fb0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.741+0000 7f35429fe740 1 -- 172.21.15.78:0/1203799601 shutdown_connections 2024-05-21T15:24:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.741+0000 7f35429fe740 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e75d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5ad1390 0x561bc5e88370 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 --2- 172.21.15.78:0/1203799601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5aa84d0 0x561bc5e7e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 -- 172.21.15.78:0/1203799601 >> 172.21.15.78:0/1203799601 conn(0x561bc5ac54e0 msgr2=0x561bc5e7b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 -- 172.21.15.78:0/1203799601 shutdown_connections 2024-05-21T15:24:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 -- 172.21.15.78:0/1203799601 wait complete. 2024-05-21T15:24:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 -- start start 2024-05-21T15:24:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 Processor -- start 2024-05-21T15:24:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e84980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e7b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ad1390 0x561bc5e7bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35341adf90 con 0x561bc5aa84d0 2024-05-21T15:24:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f35341ad8c0 con 0x561bc5ab06b0 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35341aab20 con 0x561bc5ad1390 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.742+0000 7f35429fe740 10 client.0 _ll_register_callbacks cb 0x7f352800bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e84980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e84980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37904/0 (socket says 172.21.15.78:37904) 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 -- 172.21.15.78:0/129276161 learned_addr learned my addr 172.21.15.78:0/129276161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539816640 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e7b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ad1390 0x561bc5e7bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 -- 172.21.15.78:0/129276161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ad1390 msgr2=0x561bc5e7bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ad1390 0x561bc5e7bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 -- 172.21.15.78:0/129276161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 msgr2=0x561bc5e7b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e7b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 -- 172.21.15.78:0/129276161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bc5e7c070 con 0x561bc5aa84d0 2024-05-21T15:24:59.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f3539015640 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e84980 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3528030910 tx=0x7f3528030c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b931d4a00851c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f35197fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:59.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f35197fa640 1 -- 172.21.15.78:0/129276161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004a80 con 0x561bc5aa84d0 2024-05-21T15:24:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f35197fa640 1 -- 172.21.15.78:0/129276161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f3528005230 con 0x561bc5aa84d0 2024-05-21T15:24:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f35429fe740 1 -- 172.21.15.78:0/129276161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528023030 con 0x561bc5aa84d0 2024-05-21T15:24:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f35197fa640 1 -- 172.21.15.78:0/129276161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3528005430 con 0x561bc5aa84d0 2024-05-21T15:24:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f35429fe740 10 client.18087 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.743+0000 7f35429fe740 1 -- 172.21.15.78:0/129276161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561bc5e84200 con 0x561bc5aa84d0 2024-05-21T15:24:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35197fa640 1 -- 172.21.15.78:0/129276161 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528023030 con 0x561bc5aa84d0 2024-05-21T15:24:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35197fa640 1 -- 172.21.15.78:0/129276161 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f352803e030 con 0x561bc5aa84d0 2024-05-21T15:24:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35429fe740 10 client.18087 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35429fe740 10 client.18087 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:24:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35429fe740 20 client.18087 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35429fe740 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561bc5a653a0 0x561bc5ef9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35429fe740 4 client.18087 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:24:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35429fe740 1 -- 172.21.15.78:0/129276161 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bc5ddd6c0 con 0x561bc5a653a0 2024-05-21T15:24:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f3539816640 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561bc5a653a0 0x561bc5ef9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f35197fa640 10 client.18087 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:24:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.744+0000 7f3539816640 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561bc5a653a0 0x561bc5ef9010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a23815d09762b35b server_cookie=3453f44296435bb2 in_seq=0 out_seq=0 2024-05-21T15:24:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35197fa640 1 -- 172.21.15.78:0/129276161 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bc5ddd6c0 con 0x561bc5a653a0 2024-05-21T15:24:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35197fa640 10 client.18087 handle_command_reply: tid=0 2024-05-21T15:24:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 client.18087 shutdown 2024-05-21T15:24:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 -- 172.21.15.78:0/129276161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 msgr2=0x561bc5e84980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e84980 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3528030910 tx=0x7f3528030c40 comp rx=0 tx=0).stop 2024-05-21T15:24:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 -- 172.21.15.78:0/129276161 shutdown_connections 2024-05-21T15:24:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x561bc5a653a0 0x561bc5ef9010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561bc5ab06b0 0x561bc5e7b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35197fa640 0 client.18087 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:24:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bc5ad1390 0x561bc5e7bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 --2- 172.21.15.78:0/129276161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561bc5aa84d0 0x561bc5e84980 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 -- 172.21.15.78:0/129276161 >> 172.21.15.78:0/129276161 conn(0x561bc5ac54e0 msgr2=0x561bc5e7e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 -- 172.21.15.78:0/129276161 shutdown_connections 2024-05-21T15:24:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 1 -- 172.21.15.78:0/129276161 wait complete. 2024-05-21T15:24:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f35429fe740 20 client.18087 trim_cache size 0 max 16384 2024-05-21T15:24:59.761 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:24:59.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f353ba7a640 1 -- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f351c0a2d00 msgr2=0x7f351c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f353ba7a640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f351c0a2d00 0x7f351c0a5210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f352c0cc5c0 tx=0x7f352c004830 comp rx=0 tx=0).stop 2024-05-21T15:24:59.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f353ba7a640 1 -- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f353419cb20 msgr2=0x7f353419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.745+0000 7f353ba7a640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f353419cb20 0x7f353419cf30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f352c00d750 tx=0x7f352c00dc10 comp rx=0 tx=0).stop 2024-05-21T15:24:59.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.746+0000 7f353ba7a640 1 -- 172.21.15.78:0/547361586 shutdown_connections 2024-05-21T15:24:59.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.746+0000 7f353ba7a640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3534107e00 0x7f3534198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.746+0000 7f353ba7a640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f351c0a2d00 0x7f351c0a5210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.746+0000 7f353ba7a640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f353419cb20 0x7f353419cf30 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.746+0000 7f353ba7a640 1 --2- 172.21.15.78:0/547361586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35341079f0 0x7f3534198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.746+0000 7f353ba7a640 1 -- 172.21.15.78:0/547361586 >> 172.21.15.78:0/547361586 conn(0x7f3534076830 msgr2=0x7f3534103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.746+0000 7f353ba7a640 1 -- 172.21.15.78:0/547361586 shutdown_connections 2024-05-21T15:24:59.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.746+0000 7f353ba7a640 1 -- 172.21.15.78:0/547361586 wait complete. 2024-05-21T15:24:59.765 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:24:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 -- 172.21.15.78:0/2356099487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3424107a00 msgr2=0x7f3424101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 --2- 172.21.15.78:0/2356099487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3424107a00 0x7f3424101590 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3414009ca0 tx=0x7f341402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 -- 172.21.15.78:0/2356099487 shutdown_connections 2024-05-21T15:24:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 --2- 172.21.15.78:0/2356099487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3424101ad0 0x7f3424101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 --2- 172.21.15.78:0/2356099487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f342410c360 0x7f342410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 --2- 172.21.15.78:0/2356099487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3424107a00 0x7f3424101590 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 -- 172.21.15.78:0/2356099487 >> 172.21.15.78:0/2356099487 conn(0x7f3424077100 msgr2=0x7f3424077540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 -- 172.21.15.78:0/2356099487 shutdown_connections 2024-05-21T15:24:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.875+0000 7f3428a06640 1 -- 172.21.15.78:0/2356099487 wait complete. 2024-05-21T15:24:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3428a06640 1 Processor -- start 2024-05-21T15:24:59.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3428a06640 1 -- start start 2024-05-21T15:24:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3428a06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3424101ad0 0x7f342419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3428a06640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3424107a00 0x7f342419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3428a06640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342410c360 0x7f342419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3428a06640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3424111d10 con 0x7f342410c360 2024-05-21T15:24:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3428a06640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3424111b90 con 0x7f3424107a00 2024-05-21T15:24:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3428a06640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3424111e90 con 0x7f3424101ad0 2024-05-21T15:24:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342410c360 0x7f342419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342410c360 0x7f342419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37920/0 (socket says 172.21.15.78:37920) 2024-05-21T15:24:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 -- 172.21.15.78:0/3621793366 learned_addr learned my addr 172.21.15.78:0/3621793366 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3421d9b640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3424107a00 0x7f342419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f342259c640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3424101ad0 0x7f342419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 -- 172.21.15.78:0/3621793366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3424101ad0 msgr2=0x7f342419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3424101ad0 0x7f342419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 -- 172.21.15.78:0/3621793366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3424107a00 msgr2=0x7f342419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3424107a00 0x7f342419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 -- 172.21.15.78:0/3621793366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34241a5910 con 0x7f342410c360 2024-05-21T15:24:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.876+0000 7f3422d9d640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342410c360 0x7f342419b110 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d750 tx=0x7f341c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.877+0000 7f341b7fe640 1 -- 172.21.15.78:0/3621793366 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c017070 con 0x7f342410c360 2024-05-21T15:24:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.877+0000 7f341b7fe640 1 -- 172.21.15.78:0/3621793366 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f341c005500 con 0x7f342410c360 2024-05-21T15:24:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.877+0000 7f3428a06640 1 -- 172.21.15.78:0/3621793366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34241a5c00 con 0x7f342410c360 2024-05-21T15:24:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.877+0000 7f341b7fe640 1 -- 172.21.15.78:0/3621793366 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c004d90 con 0x7f342410c360 2024-05-21T15:24:59.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.877+0000 7f3428a06640 1 -- 172.21.15.78:0/3621793366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241a61a0 con 0x7f342410c360 2024-05-21T15:24:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.879+0000 7f341b7fe640 1 -- 172.21.15.78:0/3621793366 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f341c004f30 con 0x7f342410c360 2024-05-21T15:24:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.879+0000 7f342f984740 1 Processor -- start 2024-05-21T15:24:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342f984740 1 -- start start 2024-05-21T15:24:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f341b7fe640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f34100a2d80 0x7f34100a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342f984740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b3ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342f984740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337774d0 0x55fa33b3f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342f984740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa3377f6b0 0x55fa33b3f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342f984740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa33aa4c40 con 0x55fa337a0390 2024-05-21T15:24:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342f984740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fa33a4c580 con 0x55fa3377f6b0 2024-05-21T15:24:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342f984740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fa33aa0c00 con 0x55fa337774d0 2024-05-21T15:24:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f341b7fe640 1 -- 172.21.15.78:0/3621793366 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f341c0cece0 con 0x7f342410c360 2024-05-21T15:24:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f3421d9b640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f34100a2d80 0x7f34100a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b3ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b3ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37926/0 (socket says 172.21.15.78:37926) 2024-05-21T15:24:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342259c640 1 -- 172.21.15.78:0/2796130442 learned_addr learned my addr 172.21.15.78:0/2796130442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f3421d9b640 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337774d0 0x55fa33b3f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f3422d9d640 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa3377f6b0 0x55fa33b3f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342259c640 1 -- 172.21.15.78:0/2796130442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337774d0 msgr2=0x55fa33b3f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342259c640 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337774d0 0x55fa33b3f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342259c640 1 -- 172.21.15.78:0/2796130442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa3377f6b0 msgr2=0x55fa33b3f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342259c640 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa3377f6b0 0x55fa33b3f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.880+0000 7f342259c640 1 -- 172.21.15.78:0/2796130442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa337b83f0 con 0x55fa337a0390 2024-05-21T15:24:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.881+0000 7f3421d9b640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f34100a2d80 0x7f34100a5290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3424100d70 tx=0x7f340c00ab50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.881+0000 7f342259c640 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b3ed20 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3414009ca0 tx=0x7f341402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd3cacf960223e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.881+0000 7f3418ff9640 1 -- 172.21.15.78:0/2796130442 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414049070 con 0x55fa337a0390 2024-05-21T15:24:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.881+0000 7f3418ff9640 1 -- 172.21.15.78:0/2796130442 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3414036040 con 0x55fa337a0390 2024-05-21T15:24:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.881+0000 7f3418ff9640 1 -- 172.21.15.78:0/2796130442 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341402f9b0 con 0x55fa337a0390 2024-05-21T15:24:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 -- 172.21.15.78:0/2796130442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 msgr2=0x55fa33b3ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b3ed20 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3414009ca0 tx=0x7f341402f470 comp rx=0 tx=0).stop 2024-05-21T15:24:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 -- 172.21.15.78:0/2796130442 shutdown_connections 2024-05-21T15:24:59.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa3377f6b0 0x55fa33b3f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337774d0 0x55fa33b3f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 --2- 172.21.15.78:0/2796130442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b3ed20 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 -- 172.21.15.78:0/2796130442 >> 172.21.15.78:0/2796130442 conn(0x55fa337944e0 msgr2=0x55fa33b3b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 -- 172.21.15.78:0/2796130442 shutdown_connections 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 -- 172.21.15.78:0/2796130442 wait complete. 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 -- start start 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.882+0000 7f342f984740 1 Processor -- start 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b49350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b49890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b4f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa33b4bfb0 con 0x55fa337a0390 2024-05-21T15:24:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55fa33a5fc40 con 0x55fa337774d0 2024-05-21T15:24:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fa33b4c130 con 0x55fa3377f6b0 2024-05-21T15:24:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 10 client.0 _ll_register_callbacks cb 0x55fa337b7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b4f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b4f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37936/0 (socket says 172.21.15.78:37936) 2024-05-21T15:24:59.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 -- 172.21.15.78:0/413757472 learned_addr learned my addr 172.21.15.78:0/413757472 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342259c640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b49350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3421d9b640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b49890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 -- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 msgr2=0x55fa33b49890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b49890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 -- 172.21.15.78:0/413757472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 msgr2=0x55fa33b49350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b49350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 -- 172.21.15.78:0/413757472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa338827d0 con 0x55fa337a0390 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342259c640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:24:59.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3421d9b640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:24:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3422d9d640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b4f450 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f341c09af20 tx=0x7f341c09ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e560a2ecc155f5b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3409ffb640 1 -- 172.21.15.78:0/413757472 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c020040 con 0x55fa337a0390 2024-05-21T15:24:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3409ffb640 1 -- 172.21.15.78:0/413757472 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f341c0cc620 con 0x55fa337a0390 2024-05-21T15:24:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 1 -- 172.21.15.78:0/413757472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3414036040 con 0x55fa337a0390 2024-05-21T15:24:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f3409ffb640 1 -- 172.21.15.78:0/413757472 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f341c09f520 con 0x55fa337a0390 2024-05-21T15:24:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 10 client.18099 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.883+0000 7f342f984740 1 -- 172.21.15.78:0/413757472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fa33b58790 con 0x55fa337a0390 2024-05-21T15:24:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f3409ffb640 1 -- 172.21.15.78:0/413757472 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3414036040 con 0x55fa337a0390 2024-05-21T15:24:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f3409ffb640 1 -- 172.21.15.78:0/413757472 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f341c09f170 con 0x55fa337a0390 2024-05-21T15:24:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f342f984740 10 client.18099 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f342f984740 10 client.18099 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:24:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f342f984740 20 client.18099 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f342f984740 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55fa3375ef60 0x55fa33b5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f342f984740 4 client.18099 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:24:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f342f984740 1 -- 172.21.15.78:0/413757472 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa33ab2370 con 0x55fa3375ef60 2024-05-21T15:24:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f342259c640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55fa3375ef60 0x55fa33b5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f342259c640 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55fa3375ef60 0x55fa33b5bd30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a909bda8d8e2926 server_cookie=4e29885858c809db in_seq=0 out_seq=0 2024-05-21T15:24:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.884+0000 7f3409ffb640 10 client.18099 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:24:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f3409ffb640 1 -- 172.21.15.78:0/413757472 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa33ab2370 con 0x55fa3375ef60 2024-05-21T15:24:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f3409ffb640 10 client.18099 handle_command_reply: tid=0 2024-05-21T15:24:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f342f984740 1 client.18099 shutdown 2024-05-21T15:24:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f342f984740 1 -- 172.21.15.78:0/413757472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 msgr2=0x55fa33b4f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f342f984740 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b4f450 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f341c09af20 tx=0x7f341c09ac70 comp rx=0 tx=0).stop 2024-05-21T15:24:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f342f984740 1 -- 172.21.15.78:0/413757472 shutdown_connections 2024-05-21T15:24:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f342f984740 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55fa3375ef60 0x55fa33b5bd30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f342f984740 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b4f450 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f3409ffb640 0 client.18099 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:24:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f342f984740 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.885+0000 7f342f984740 1 --2- 172.21.15.78:0/413757472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.886+0000 7f342f984740 1 -- 172.21.15.78:0/413757472 >> 172.21.15.78:0/413757472 conn(0x55fa337944e0 msgr2=0x55fa33b40a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.886+0000 7f342f984740 1 -- 172.21.15.78:0/413757472 shutdown_connections 2024-05-21T15:24:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.886+0000 7f342f984740 1 -- 172.21.15.78:0/413757472 wait complete. 2024-05-21T15:24:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.886+0000 7f342f984740 20 client.18099 trim_cache size 0 max 16384 2024-05-21T15:24:59.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f342f984740 1 Processor -- start 2024-05-21T15:24:59.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f342f984740 1 -- start start 2024-05-21T15:24:59.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f342f984740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b48440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f342f984740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b4f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f342f984740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b40ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f342f984740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34241aea90 con 0x55fa337a0390 2024-05-21T15:24:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f342f984740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f34241b05d0 con 0x55fa337774d0 2024-05-21T15:24:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f342f984740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34241af160 con 0x55fa3377f6b0 2024-05-21T15:24:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b40ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b40ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37952/0 (socket says 172.21.15.78:37952) 2024-05-21T15:24:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.899+0000 7f3422d9d640 1 -- 172.21.15.78:0/2497815180 learned_addr learned my addr 172.21.15.78:0/2497815180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f3421d9b640 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b4f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f342259c640 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b48440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f3422d9d640 1 -- 172.21.15.78:0/2497815180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 msgr2=0x55fa33b4f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f3422d9d640 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b4f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f3422d9d640 1 -- 172.21.15.78:0/2497815180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 msgr2=0x55fa33b48440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f3422d9d640 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b48440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f3422d9d640 1 -- 172.21.15.78:0/2497815180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa33b40310 con 0x55fa337a0390 2024-05-21T15:24:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f342259c640 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b48440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:24:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f3422d9d640 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b40ed0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f341c09b1b0 tx=0x7f341c0975f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da5da37909c2caa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f340bfff640 1 -- 172.21.15.78:0/2497815180 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c0b9030 con 0x55fa337a0390 2024-05-21T15:24:59.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f340bfff640 1 -- 172.21.15.78:0/2497815180 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f341c0a3de0 con 0x55fa337a0390 2024-05-21T15:24:59.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.900+0000 7f340bfff640 1 -- 172.21.15.78:0/2497815180 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c020070 con 0x55fa337a0390 2024-05-21T15:24:59.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- 172.21.15.78:0/2497815180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 msgr2=0x55fa33b40ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b40ed0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f341c09b1b0 tx=0x7f341c0975f0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- 172.21.15.78:0/2497815180 shutdown_connections 2024-05-21T15:24:59.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa337a0390 0x55fa33b40ed0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa3377f6b0 0x55fa33b4f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 --2- 172.21.15.78:0/2497815180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b48440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- 172.21.15.78:0/2497815180 >> 172.21.15.78:0/2497815180 conn(0x55fa337944e0 msgr2=0x55fa33b407c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- 172.21.15.78:0/2497815180 shutdown_connections 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- 172.21.15.78:0/2497815180 wait complete. 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- start start 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 Processor -- start 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b55750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa3377f6b0 0x55fa33b50aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337a0390 0x55fa33b50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34241aff00 con 0x55fa3377f6b0 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f34241af830 con 0x55fa337774d0 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34241aca90 con 0x55fa337a0390 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f342f984740 10 client.0 _ll_register_callbacks cb 0x7f341c09b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f3421d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa3377f6b0 0x55fa33b50aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f3421d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa3377f6b0 0x55fa33b50aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37960/0 (socket says 172.21.15.78:37960) 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.901+0000 7f3421d9b640 1 -- 172.21.15.78:0/2816406972 learned_addr learned my addr 172.21.15.78:0/2816406972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f342259c640 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b55750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3422d9d640 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337a0390 0x55fa33b50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3421d9b640 1 -- 172.21.15.78:0/2816406972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337a0390 msgr2=0x55fa33b50fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3421d9b640 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337a0390 0x55fa33b50fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3421d9b640 1 -- 172.21.15.78:0/2816406972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 msgr2=0x55fa33b55750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:24:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3421d9b640 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b55750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3421d9b640 1 -- 172.21.15.78:0/2816406972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa33b51520 con 0x55fa3377f6b0 2024-05-21T15:24:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3421d9b640 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa3377f6b0 0x55fa33b50aa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f342406b3b0 tx=0x7f340c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda109f678a97894 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:24:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:24:59.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3409ffb640 1 -- 172.21.15.78:0/2816406972 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c012db0 con 0x55fa3377f6b0 2024-05-21T15:24:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3409ffb640 1 -- 172.21.15.78:0/2816406972 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f340c021400 con 0x55fa3377f6b0 2024-05-21T15:24:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f342f984740 1 -- 172.21.15.78:0/2816406972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341c0a3de0 con 0x55fa3377f6b0 2024-05-21T15:24:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3409ffb640 1 -- 172.21.15.78:0/2816406972 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f340c021600 con 0x55fa3377f6b0 2024-05-21T15:24:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f342f984740 10 client.18105 fetch_fsmap learned FSMap version 37 2024-05-21T15:24:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f342f984740 1 -- 172.21.15.78:0/2816406972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fa33bc3c70 con 0x55fa3377f6b0 2024-05-21T15:24:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.902+0000 7f3409ffb640 1 -- 172.21.15.78:0/2816406972 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f341c0a3de0 con 0x55fa3377f6b0 2024-05-21T15:24:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f3409ffb640 1 -- 172.21.15.78:0/2816406972 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f340c029070 con 0x55fa3377f6b0 2024-05-21T15:24:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f342f984740 10 client.18105 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f342f984740 10 client.18105 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f342f984740 20 client.18105 populate_metadata read hostname 'smithi078' 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f342f984740 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55fa337343a0 0x55fa33bc64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f342f984740 4 client.18105 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f342f984740 1 -- 172.21.15.78:0/2816406972 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa33aac6c0 con 0x55fa337343a0 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f342259c640 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55fa337343a0 0x55fa33bc64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f3409ffb640 10 client.18105 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.903+0000 7f342259c640 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55fa337343a0 0x55fa33bc64e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32474058fe89ef7 server_cookie=cc9231e171cc3203 in_seq=0 out_seq=0 2024-05-21T15:24:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3409ffb640 1 -- 172.21.15.78:0/2816406972 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa33aac6c0 con 0x55fa337343a0 2024-05-21T15:24:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3409ffb640 10 client.18105 handle_command_reply: tid=0 2024-05-21T15:24:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 client.18105 shutdown 2024-05-21T15:24:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 -- 172.21.15.78:0/2816406972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa3377f6b0 msgr2=0x55fa33b50aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa3377f6b0 0x55fa33b50aa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f342406b3b0 tx=0x7f340c004730 comp rx=0 tx=0).stop 2024-05-21T15:24:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 -- 172.21.15.78:0/2816406972 shutdown_connections 2024-05-21T15:24:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55fa337343a0 0x55fa33bc64e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fa337a0390 0x55fa33b50fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3409ffb640 0 client.18105 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:24:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa3377f6b0 0x55fa33b50aa0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 --2- 172.21.15.78:0/2816406972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55fa337774d0 0x55fa33b55750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 -- 172.21.15.78:0/2816406972 >> 172.21.15.78:0/2816406972 conn(0x55fa337944e0 msgr2=0x55fa33b481a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 -- 172.21.15.78:0/2816406972 shutdown_connections 2024-05-21T15:24:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 1 -- 172.21.15.78:0/2816406972 wait complete. 2024-05-21T15:24:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f342f984740 20 client.18105 trim_cache size 0 max 16384 2024-05-21T15:24:59.919 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:24:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 -- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f34100a2d80 msgr2=0x7f34100a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f34100a2d80 0x7f34100a5290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3424100d70 tx=0x7f340c00ab50 comp rx=0 tx=0).stop 2024-05-21T15:24:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 -- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342410c360 msgr2=0x7f342419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:24:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342410c360 0x7f342419b110 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d750 tx=0x7f341c00dc10 comp rx=0 tx=0).stop 2024-05-21T15:24:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 -- 172.21.15.78:0/3621793366 shutdown_connections 2024-05-21T15:24:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3424107a00 0x7f342419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f34100a2d80 0x7f34100a5290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f342410c360 0x7f342419b110 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 --2- 172.21.15.78:0/3621793366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3424101ad0 0x7f342419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:24:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.904+0000 7f3428a06640 1 -- 172.21.15.78:0/3621793366 >> 172.21.15.78:0/3621793366 conn(0x7f3424077100 msgr2=0x7f3424103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:24:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.905+0000 7f3428a06640 1 -- 172.21.15.78:0/3621793366 shutdown_connections 2024-05-21T15:24:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:24:59.905+0000 7f3428a06640 1 -- 172.21.15.78:0/3621793366 wait complete. 2024-05-21T15:24:59.922 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:25:00.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 -- 172.21.15.78:0/2355313782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 msgr2=0x7f4210101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 --2- 172.21.15.78:0/2355313782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 0x7f4210101590 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4200009900 tx=0x7f420002a410 comp rx=0 tx=0).stop 2024-05-21T15:25:00.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 -- 172.21.15.78:0/2355313782 shutdown_connections 2024-05-21T15:25:00.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 --2- 172.21.15.78:0/2355313782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f421010c360 0x7f421010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 --2- 172.21.15.78:0/2355313782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4210101ad0 0x7f4210101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 --2- 172.21.15.78:0/2355313782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 0x7f4210101590 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 -- 172.21.15.78:0/2355313782 >> 172.21.15.78:0/2355313782 conn(0x7f4210077100 msgr2=0x7f4210077540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 -- 172.21.15.78:0/2355313782 shutdown_connections 2024-05-21T15:25:00.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.031+0000 7f42160de640 1 -- 172.21.15.78:0/2355313782 wait complete. 2024-05-21T15:25:00.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f42160de640 1 Processor -- start 2024-05-21T15:25:00.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f42160de640 1 -- start start 2024-05-21T15:25:00.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f42160de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4210101ad0 0x7f4210198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f42160de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 0x7f4210198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f42160de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f421010c360 0x7f421019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f42160de640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4210111d10 con 0x7f4210107a00 2024-05-21T15:25:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f42160de640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4210111b90 con 0x7f4210101ad0 2024-05-21T15:25:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f42160de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4210111e90 con 0x7f421010c360 2024-05-21T15:25:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f4206ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 0x7f4210198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f4206ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 0x7f4210198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37986/0 (socket says 172.21.15.78:37986) 2024-05-21T15:25:00.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f4206ffd640 1 -- 172.21.15.78:0/1780854504 learned_addr learned my addr 172.21.15.78:0/1780854504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f420ffff640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f421010c360 0x7f421019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f420f7fe640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4210101ad0 0x7f4210198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f4206ffd640 1 -- 172.21.15.78:0/1780854504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f421010c360 msgr2=0x7f421019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f4206ffd640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f421010c360 0x7f421019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f4206ffd640 1 -- 172.21.15.78:0/1780854504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4210101ad0 msgr2=0x7f4210198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f4206ffd640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4210101ad0 0x7f4210198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.032+0000 7f4206ffd640 1 -- 172.21.15.78:0/1780854504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421019d4f0 con 0x7f4210107a00 2024-05-21T15:25:00.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.033+0000 7f4206ffd640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 0x7f4210198c70 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c9a0 tx=0x7f41f800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.033+0000 7f420d7fa640 1 -- 172.21.15.78:0/1780854504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8004220 con 0x7f4210107a00 2024-05-21T15:25:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.033+0000 7f42160de640 1 -- 172.21.15.78:0/1780854504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421019d7e0 con 0x7f4210107a00 2024-05-21T15:25:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.033+0000 7f420d7fa640 1 -- 172.21.15.78:0/1780854504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f80043c0 con 0x7f4210107a00 2024-05-21T15:25:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.033+0000 7f420d7fa640 1 -- 172.21.15.78:0/1780854504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8004a00 con 0x7f4210107a00 2024-05-21T15:25:00.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.033+0000 7f42160de640 1 -- 172.21.15.78:0/1780854504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421019dce0 con 0x7f4210107a00 2024-05-21T15:25:00.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.035+0000 7f420d7fa640 1 -- 172.21.15.78:0/1780854504 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f41f80079b0 con 0x7f4210107a00 2024-05-21T15:25:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f421cf7d740 1 Processor -- start 2024-05-21T15:25:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f421cf7d740 1 -- start start 2024-05-21T15:25:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c78390 0x556ad5016bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c4f4d0 0x556ad5017110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c576b0 0x556ad5017650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f421cf7d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556ad4f7cc40 con 0x556ad4c4f4d0 2024-05-21T15:25:00.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f421cf7d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556ad4f24580 con 0x556ad4c576b0 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f421cf7d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556ad4f78c00 con 0x556ad4c78390 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f420d7fa640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41fc0a2d00 0x7f41fc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f420ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c78390 0x556ad5016bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f420ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c78390 0x556ad5016bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:48908/0 (socket says 172.21.15.78:48908) 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f420ffff640 1 -- 172.21.15.78:0/1434014015 learned_addr learned my addr 172.21.15.78:0/1434014015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f420f7fe640 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c4f4d0 0x556ad5017110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f420ffff640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41fc0a2d00 0x7f41fc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f420d7fa640 1 -- 172.21.15.78:0/1780854504 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f41f80cf2f0 con 0x7f4210107a00 2024-05-21T15:25:00.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.036+0000 7f4206ffd640 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c576b0 0x556ad5017650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f420f7fe640 1 -- 172.21.15.78:0/1434014015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c78390 msgr2=0x556ad5016bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f420f7fe640 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c78390 0x556ad5016bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f420f7fe640 1 -- 172.21.15.78:0/1434014015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c576b0 msgr2=0x556ad5017650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f420f7fe640 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c576b0 0x556ad5017650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f420f7fe640 1 -- 172.21.15.78:0/1434014015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad4c903f0 con 0x556ad4c4f4d0 2024-05-21T15:25:00.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f4206ffd640 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c576b0 0x556ad5017650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:00.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f420ffff640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41fc0a2d00 0x7f41fc0a5210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4208013bd0 tx=0x7f4208007990 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f420f7fe640 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c4f4d0 0x556ad5017110 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4200006010 tx=0x7f4200005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73be66b4e39a0e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f4205ffb640 1 -- 172.21.15.78:0/1434014015 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420003f070 con 0x556ad4c4f4d0 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f4205ffb640 1 -- 172.21.15.78:0/1434014015 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42000318b0 con 0x556ad4c4f4d0 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.037+0000 7f4205ffb640 1 -- 172.21.15.78:0/1434014015 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200035070 con 0x556ad4c4f4d0 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 -- 172.21.15.78:0/1434014015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c4f4d0 msgr2=0x556ad5017110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c4f4d0 0x556ad5017110 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f4200006010 tx=0x7f4200005e80 comp rx=0 tx=0).stop 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 -- 172.21.15.78:0/1434014015 shutdown_connections 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c576b0 0x556ad5017650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c4f4d0 0x556ad5017110 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 --2- 172.21.15.78:0/1434014015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c78390 0x556ad5016bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 -- 172.21.15.78:0/1434014015 >> 172.21.15.78:0/1434014015 conn(0x556ad4c6c4e0 msgr2=0x556ad50134d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 -- 172.21.15.78:0/1434014015 shutdown_connections 2024-05-21T15:25:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.038+0000 7f421cf7d740 1 -- 172.21.15.78:0/1434014015 wait complete. 2024-05-21T15:25:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 1 -- start start 2024-05-21T15:25:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 1 Processor -- start 2024-05-21T15:25:00.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c4f4d0 0x556ad5025980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c576b0 0x556ad5025ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c78390 0x556ad502b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556ad5028450 con 0x556ad4c78390 2024-05-21T15:25:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556ad4f37c40 con 0x556ad4c4f4d0 2024-05-21T15:25:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556ad50285d0 con 0x556ad4c576b0 2024-05-21T15:25:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f421cf7d740 10 client.0 _ll_register_callbacks cb 0x556ad4c8fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:00.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f420ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c78390 0x556ad502b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f420ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c78390 0x556ad502b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38008/0 (socket says 172.21.15.78:38008) 2024-05-21T15:25:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f420ffff640 1 -- 172.21.15.78:0/76923535 learned_addr learned my addr 172.21.15.78:0/76923535 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f420f7fe640 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c4f4d0 0x556ad5025980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.039+0000 7f4206ffd640 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c576b0 0x556ad5025ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f420ffff640 1 -- 172.21.15.78:0/76923535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c576b0 msgr2=0x556ad5025ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f420ffff640 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c576b0 0x556ad5025ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f420ffff640 1 -- 172.21.15.78:0/76923535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c4f4d0 msgr2=0x556ad5025980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f420ffff640 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c4f4d0 0x556ad5025980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f420ffff640 1 -- 172.21.15.78:0/76923535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad4d5a7d0 con 0x556ad4c78390 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f420ffff640 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c78390 0x556ad502b890 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4208004fd0 tx=0x7f4208007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda5ff3fc39c5839 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f41eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f41eeffd640 1 -- 172.21.15.78:0/76923535 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208004440 con 0x556ad4c78390 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f41eeffd640 1 -- 172.21.15.78:0/76923535 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f42080045e0 con 0x556ad4c78390 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f421cf7d740 1 -- 172.21.15.78:0/76923535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42000318b0 con 0x556ad4c78390 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f41eeffd640 1 -- 172.21.15.78:0/76923535 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f42080047e0 con 0x556ad4c78390 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f421cf7d740 10 client.18117 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:00.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f421cf7d740 1 -- 172.21.15.78:0/76923535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556ad5034b60 con 0x556ad4c78390 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.040+0000 7f41eeffd640 1 -- 172.21.15.78:0/76923535 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42000318b0 con 0x556ad4c78390 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.041+0000 7f41eeffd640 1 -- 172.21.15.78:0/76923535 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f420802d030 con 0x556ad4c78390 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.041+0000 7f421cf7d740 10 client.18117 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.041+0000 7f421cf7d740 10 client.18117 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.041+0000 7f421cf7d740 20 client.18117 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.041+0000 7f421cf7d740 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x556ad4c36f60 0x556ad50380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.041+0000 7f421cf7d740 4 client.18117 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.041+0000 7f421cf7d740 1 -- 172.21.15.78:0/76923535 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad4f8a370 con 0x556ad4c36f60 2024-05-21T15:25:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.041+0000 7f420f7fe640 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x556ad4c36f60 0x556ad50380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.042+0000 7f41eeffd640 10 client.18117 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:25:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.042+0000 7f420f7fe640 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x556ad4c36f60 0x556ad50380c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e3ee63cee02327f server_cookie=5b8612000ff8987a in_seq=0 out_seq=0 2024-05-21T15:25:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.042+0000 7f41eeffd640 1 -- 172.21.15.78:0/76923535 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ad4f8a370 con 0x556ad4c36f60 2024-05-21T15:25:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.042+0000 7f41eeffd640 10 client.18117 handle_command_reply: tid=0 2024-05-21T15:25:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.042+0000 7f421cf7d740 1 client.18117 shutdown 2024-05-21T15:25:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 -- 172.21.15.78:0/76923535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c78390 msgr2=0x556ad502b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c78390 0x556ad502b890 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4208004fd0 tx=0x7f4208007690 comp rx=0 tx=0).stop 2024-05-21T15:25:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 -- 172.21.15.78:0/76923535 shutdown_connections 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x556ad4c36f60 0x556ad50380c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c78390 0x556ad502b890 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c576b0 0x556ad5025ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f41eeffd640 0 client.18117 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 --2- 172.21.15.78:0/76923535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c4f4d0 0x556ad5025980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 -- 172.21.15.78:0/76923535 >> 172.21.15.78:0/76923535 conn(0x556ad4c6c4e0 msgr2=0x556ad5011b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 -- 172.21.15.78:0/76923535 shutdown_connections 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 1 -- 172.21.15.78:0/76923535 wait complete. 2024-05-21T15:25:00.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.043+0000 7f421cf7d740 20 client.18117 trim_cache size 0 max 16384 2024-05-21T15:25:00.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.059+0000 7f421cf7d740 1 Processor -- start 2024-05-21T15:25:00.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.059+0000 7f421cf7d740 1 -- start start 2024-05-21T15:25:00.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 0x556ad502b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad5012250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad503d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f421cf7d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42101ac850 con 0x556ad4c576b0 2024-05-21T15:25:00.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f421cf7d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f42101ae390 con 0x556ad4c36f60 2024-05-21T15:25:00.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f421cf7d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f42101acf20 con 0x556ad4c4f4d0 2024-05-21T15:25:00.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad503d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad503d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38020/0 (socket says 172.21.15.78:38020) 2024-05-21T15:25:00.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 -- 172.21.15.78:0/3202228901 learned_addr learned my addr 172.21.15.78:0/3202228901 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420f7fe640 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 0x556ad502b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f4206ffd640 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad5012250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 -- 172.21.15.78:0/3202228901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 msgr2=0x556ad5012250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad5012250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 -- 172.21.15.78:0/3202228901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 msgr2=0x556ad502b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 0x556ad502b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 -- 172.21.15.78:0/3202228901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad50140c0 con 0x556ad4c576b0 2024-05-21T15:25:00.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.060+0000 7f420ffff640 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad503d880 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f42080135c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2dcce581170758 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.061+0000 7f42057fa640 1 -- 172.21.15.78:0/3202228901 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420801f3f0 con 0x556ad4c576b0 2024-05-21T15:25:00.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.061+0000 7f42057fa640 1 -- 172.21.15.78:0/3202228901 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4208027400 con 0x556ad4c576b0 2024-05-21T15:25:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.061+0000 7f42057fa640 1 -- 172.21.15.78:0/3202228901 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208038650 con 0x556ad4c576b0 2024-05-21T15:25:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 -- 172.21.15.78:0/3202228901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 msgr2=0x556ad503d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad503d880 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4208006010 tx=0x7f42080135c0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 -- 172.21.15.78:0/3202228901 shutdown_connections 2024-05-21T15:25:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad503d880 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad5012250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3202228901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 0x556ad502b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 -- 172.21.15.78:0/3202228901 >> 172.21.15.78:0/3202228901 conn(0x556ad4c6c4e0 msgr2=0x556ad5011b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 -- 172.21.15.78:0/3202228901 shutdown_connections 2024-05-21T15:25:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 -- 172.21.15.78:0/3202228901 wait complete. 2024-05-21T15:25:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.062+0000 7f421cf7d740 1 -- start start 2024-05-21T15:25:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f421cf7d740 1 Processor -- start 2024-05-21T15:25:00.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 0x556ad5032010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad502d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f421cf7d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad502d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f421cf7d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42101adcc0 con 0x556ad4c576b0 2024-05-21T15:25:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f421cf7d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f42101ad5f0 con 0x556ad4c36f60 2024-05-21T15:25:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f421cf7d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f42101aa850 con 0x556ad4c4f4d0 2024-05-21T15:25:00.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f421cf7d740 10 client.0 _ll_register_callbacks cb 0x7f4208013630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f4206ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad502d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f4206ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad502d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:48934/0 (socket says 172.21.15.78:48934) 2024-05-21T15:25:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f420ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad502d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f4206ffd640 1 -- 172.21.15.78:0/3087028329 learned_addr learned my addr 172.21.15.78:0/3087028329 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.063+0000 7f420f7fe640 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 0x556ad5032010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f4206ffd640 1 -- 172.21.15.78:0/3087028329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 msgr2=0x556ad5032010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f4206ffd640 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 0x556ad5032010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f4206ffd640 1 -- 172.21.15.78:0/3087028329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 msgr2=0x556ad502d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f4206ffd640 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad502d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f4206ffd640 1 -- 172.21.15.78:0/3087028329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad502de10 con 0x556ad4c4f4d0 2024-05-21T15:25:00.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f420ffff640 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad502d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f4206ffd640 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad502d360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41f8097be0 tx=0x7f41f800a550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda00e8337b44c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f41eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:25:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f41eeffd640 1 -- 172.21.15.78:0/3087028329 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f80cf040 con 0x556ad4c4f4d0 2024-05-21T15:25:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f421cf7d740 1 -- 172.21.15.78:0/3087028329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4208027400 con 0x556ad4c4f4d0 2024-05-21T15:25:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f41eeffd640 1 -- 172.21.15.78:0/3087028329 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f41f8097df0 con 0x556ad4c4f4d0 2024-05-21T15:25:00.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f41eeffd640 1 -- 172.21.15.78:0/3087028329 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f41f801d040 con 0x556ad4c4f4d0 2024-05-21T15:25:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f421cf7d740 10 client.27226 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.064+0000 7f421cf7d740 1 -- 172.21.15.78:0/3087028329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556ad50a01a0 con 0x556ad4c4f4d0 2024-05-21T15:25:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f41eeffd640 1 -- 172.21.15.78:0/3087028329 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4208027400 con 0x556ad4c4f4d0 2024-05-21T15:25:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f41eeffd640 1 -- 172.21.15.78:0/3087028329 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f41f8007ca0 con 0x556ad4c4f4d0 2024-05-21T15:25:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f421cf7d740 10 client.27226 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f421cf7d740 10 client.27226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:25:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f421cf7d740 20 client.27226 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x556ad4c78390 0x556ad50a2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f421cf7d740 4 client.27226 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:25:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f421cf7d740 1 -- 172.21.15.78:0/3087028329 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ad50137a0 con 0x556ad4c78390 2024-05-21T15:25:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.065+0000 7f420f7fe640 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x556ad4c78390 0x556ad50a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f41eeffd640 10 client.27226 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:25:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f420f7fe640 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x556ad4c78390 0x556ad50a2890 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c7482ebd8acfb0f server_cookie=d547e78972487a8 in_seq=0 out_seq=0 2024-05-21T15:25:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f41eeffd640 1 -- 172.21.15.78:0/3087028329 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ad50137a0 con 0x556ad4c78390 2024-05-21T15:25:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f41eeffd640 10 client.27226 handle_command_reply: tid=0 2024-05-21T15:25:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 client.27226 shutdown 2024-05-21T15:25:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 -- 172.21.15.78:0/3087028329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 msgr2=0x556ad502d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad502d360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41f8097be0 tx=0x7f41f800a550 comp rx=0 tx=0).stop 2024-05-21T15:25:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 -- 172.21.15.78:0/3087028329 shutdown_connections 2024-05-21T15:25:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x556ad4c78390 0x556ad50a2890 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556ad4c576b0 0x556ad502d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f41eeffd640 0 client.27226 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:25:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ad4c4f4d0 0x556ad502d360 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 --2- 172.21.15.78:0/3087028329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556ad4c36f60 0x556ad5032010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 -- 172.21.15.78:0/3087028329 >> 172.21.15.78:0/3087028329 conn(0x556ad4c6c4e0 msgr2=0x556ad5024040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 -- 172.21.15.78:0/3087028329 shutdown_connections 2024-05-21T15:25:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 1 -- 172.21.15.78:0/3087028329 wait complete. 2024-05-21T15:25:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.066+0000 7f421cf7d740 20 client.27226 trim_cache size 0 max 16384 2024-05-21T15:25:00.081 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:25:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 -- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41fc0a2d00 msgr2=0x7f41fc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41fc0a2d00 0x7f41fc0a5210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4208013bd0 tx=0x7f4208007990 comp rx=0 tx=0).stop 2024-05-21T15:25:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 -- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 msgr2=0x7f4210198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 0x7f4210198c70 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c9a0 tx=0x7f41f800ce60 comp rx=0 tx=0).stop 2024-05-21T15:25:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 -- 172.21.15.78:0/1780854504 shutdown_connections 2024-05-21T15:25:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41fc0a2d00 0x7f41fc0a5210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f421010c360 0x7f421019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4210107a00 0x7f4210198c70 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 --2- 172.21.15.78:0/1780854504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4210101ad0 0x7f4210198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 -- 172.21.15.78:0/1780854504 >> 172.21.15.78:0/1780854504 conn(0x7f4210077100 msgr2=0x7f4210103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 -- 172.21.15.78:0/1780854504 shutdown_connections 2024-05-21T15:25:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.067+0000 7f42160de640 1 -- 172.21.15.78:0/1780854504 wait complete. 2024-05-21T15:25:00.085 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:25:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 -- 172.21.15.78:0/3713952532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8101770 msgr2=0x7f97d810f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 --2- 172.21.15.78:0/3713952532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8101770 0x7f97d810f5c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009ca0 tx=0x7f97d002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 -- 172.21.15.78:0/3713952532 shutdown_connections 2024-05-21T15:25:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 --2- 172.21.15.78:0/3713952532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97d8108f70 0x7f97d8101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 --2- 172.21.15.78:0/3713952532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8101770 0x7f97d810f5c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 --2- 172.21.15.78:0/3713952532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8108ad0 0x7f97d8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 -- 172.21.15.78:0/3713952532 >> 172.21.15.78:0/3713952532 conn(0x7f97d8076830 msgr2=0x7f97d8076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 -- 172.21.15.78:0/3713952532 shutdown_connections 2024-05-21T15:25:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.205+0000 7f97dd61b640 1 -- 172.21.15.78:0/3713952532 wait complete. 2024-05-21T15:25:00.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.206+0000 7f97dd61b640 1 Processor -- start 2024-05-21T15:25:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.206+0000 7f97dd61b640 1 -- start start 2024-05-21T15:25:00.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.206+0000 7f97dd61b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97d8101770 0x7f97d81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.206+0000 7f97dd61b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8108ad0 0x7f97d8199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.206+0000 7f97dd61b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8108f70 0x7f97d819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.206+0000 7f97dd61b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8112de0 con 0x7f97d8108ad0 2024-05-21T15:25:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.206+0000 7f97dd61b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8112c60 con 0x7f97d8101770 2024-05-21T15:25:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.206+0000 7f97dd61b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8112f60 con 0x7f97d8108f70 2024-05-21T15:25:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8108ad0 0x7f97d8199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8108ad0 0x7f97d8199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38038/0 (socket says 172.21.15.78:38038) 2024-05-21T15:25:00.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 -- 172.21.15.78:0/530988767 learned_addr learned my addr 172.21.15.78:0/530988767 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97d8101770 0x7f97d81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d77fe640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8108f70 0x7f97d819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 -- 172.21.15.78:0/530988767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8108f70 msgr2=0x7f97d819dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8108f70 0x7f97d819dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 -- 172.21.15.78:0/530988767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97d8101770 msgr2=0x7f97d81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97d8101770 0x7f97d81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 -- 172.21.15.78:0/530988767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d819e600 con 0x7f97d8108ad0 2024-05-21T15:25:00.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d77fe640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8108f70 0x7f97d819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:00.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97d8101770 0x7f97d81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:00.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97d67fc640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8108ad0 0x7f97d8199d00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00d770 tx=0x7f97cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97bffff640 1 -- 172.21.15.78:0/530988767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc017070 con 0x7f97d8108ad0 2024-05-21T15:25:00.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97dd61b640 1 -- 172.21.15.78:0/530988767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d819e8f0 con 0x7f97d8108ad0 2024-05-21T15:25:00.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97bffff640 1 -- 172.21.15.78:0/530988767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97cc009c30 con 0x7f97d8108ad0 2024-05-21T15:25:00.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97bffff640 1 -- 172.21.15.78:0/530988767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc00f620 con 0x7f97d8108ad0 2024-05-21T15:25:00.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.207+0000 7f97dd61b640 1 -- 172.21.15.78:0/530988767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d81a6270 con 0x7f97d8108ad0 2024-05-21T15:25:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.210+0000 7f97e45a7740 1 Processor -- start 2024-05-21T15:25:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.210+0000 7f97e45a7740 1 -- start start 2024-05-21T15:25:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.210+0000 7f97bffff640 1 -- 172.21.15.78:0/530988767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f97cc01a070 con 0x7f97d8108ad0 2024-05-21T15:25:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a5fdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a5fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 0x55ac4a5fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97e45a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac4a563c40 con 0x55ac4a25f390 2024-05-21T15:25:00.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97e45a7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ac4a50b580 con 0x55ac4a23e6b0 2024-05-21T15:25:00.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97e45a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac4a55fc00 con 0x55ac4a2364d0 2024-05-21T15:25:00.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a5fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a5fdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38042/0 (socket says 172.21.15.78:38042) 2024-05-21T15:25:00.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 -- 172.21.15.78:0/2735456915 learned_addr learned my addr 172.21.15.78:0/2735456915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d67fc640 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 0x55ac4a5fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a5fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 -- 172.21.15.78:0/2735456915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 msgr2=0x55ac4a5fe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a5fe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 -- 172.21.15.78:0/2735456915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 msgr2=0x55ac4a5fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 0x55ac4a5fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 -- 172.21.15.78:0/2735456915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4a2773f0 con 0x55ac4a25f390 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97bffff640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f97c00a2d00 0x7f97c00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d77fe640 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a5fdd20 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009ca0 tx=0x7f97d0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f6ee60dd603cd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f97c00a2d00 0x7f97c00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.211+0000 7f97bffff640 1 -- 172.21.15.78:0/530988767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f97cc0cf010 con 0x7f97d8108ad0 2024-05-21T15:25:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.212+0000 7f97bd7fa640 1 -- 172.21.15.78:0/2735456915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0003970 con 0x55ac4a25f390 2024-05-21T15:25:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.212+0000 7f97bd7fa640 1 -- 172.21.15.78:0/2735456915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0036050 con 0x55ac4a25f390 2024-05-21T15:25:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.212+0000 7f97bd7fa640 1 -- 172.21.15.78:0/2735456915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0004330 con 0x55ac4a25f390 2024-05-21T15:25:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.212+0000 7f97e45a7740 1 -- 172.21.15.78:0/2735456915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 msgr2=0x55ac4a5fdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.212+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f97c00a2d00 0x7f97c00a5210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55ac4a5ff2e0 tx=0x7f97c800b710 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.212+0000 7f97e45a7740 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a5fdd20 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009ca0 tx=0x7f97d0009800 comp rx=0 tx=0).stop 2024-05-21T15:25:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 -- 172.21.15.78:0/2735456915 shutdown_connections 2024-05-21T15:25:00.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 0x55ac4a5fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a5fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 --2- 172.21.15.78:0/2735456915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a5fdd20 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 -- 172.21.15.78:0/2735456915 >> 172.21.15.78:0/2735456915 conn(0x55ac4a2534e0 msgr2=0x55ac4a5fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 -- 172.21.15.78:0/2735456915 shutdown_connections 2024-05-21T15:25:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 -- 172.21.15.78:0/2735456915 wait complete. 2024-05-21T15:25:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 -- start start 2024-05-21T15:25:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 Processor -- start 2024-05-21T15:25:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a608370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a60e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ac4a60ae90 con 0x55ac4a25f390 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ac4a51ec40 con 0x55ac4a23e6b0 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac4a60b010 con 0x55ac4a2364d0 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97e45a7740 10 client.0 _ll_register_callbacks cb 0x55ac4a276be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a60e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a60e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38058/0 (socket says 172.21.15.78:38058) 2024-05-21T15:25:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.213+0000 7f97d77fe640 1 -- 172.21.15.78:0/162295410 learned_addr learned my addr 172.21.15.78:0/162295410 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a608370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97d67fc640 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97d77fe640 1 -- 172.21.15.78:0/162295410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 msgr2=0x55ac4a608370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97d77fe640 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a608370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97d77fe640 1 -- 172.21.15.78:0/162295410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 msgr2=0x55ac4a6088b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97d77fe640 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6088b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97d77fe640 1 -- 172.21.15.78:0/162295410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4a3417d0 con 0x55ac4a25f390 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97d77fe640 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a60e250 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009bf0 tx=0x7f97d0031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c0e394dc37631d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97baff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97baff5640 1 -- 172.21.15.78:0/162295410 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0051060 con 0x55ac4a25f390 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97baff5640 1 -- 172.21.15.78:0/162295410 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f97d00040d0 con 0x55ac4a25f390 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97e45a7740 1 -- 172.21.15.78:0/162295410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d0036050 con 0x55ac4a25f390 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97baff5640 1 -- 172.21.15.78:0/162295410 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97d0058050 con 0x55ac4a25f390 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97e45a7740 10 client.18135 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97e45a7740 1 -- 172.21.15.78:0/162295410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ac4a60ae90 con 0x55ac4a25f390 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.214+0000 7f97baff5640 1 -- 172.21.15.78:0/162295410 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0036050 con 0x55ac4a25f390 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97baff5640 1 -- 172.21.15.78:0/162295410 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f97d0042070 con 0x55ac4a25f390 2024-05-21T15:25:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97e45a7740 10 client.18135 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97e45a7740 10 client.18135 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97e45a7740 20 client.18135 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97e45a7740 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ac4a21df60 0x55ac4a61a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97e45a7740 4 client.18135 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97e45a7740 1 -- 172.21.15.78:0/162295410 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac4a571370 con 0x55ac4a21df60 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ac4a21df60 0x55ac4a61a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97baff5640 10 client.18135 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.215+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ac4a21df60 0x55ac4a61a690 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=387ea73da431cf2c server_cookie=778d3b54543126cd in_seq=0 out_seq=0 2024-05-21T15:25:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97baff5640 1 -- 172.21.15.78:0/162295410 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac4a571370 con 0x55ac4a21df60 2024-05-21T15:25:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97baff5640 10 client.18135 handle_command_reply: tid=0 2024-05-21T15:25:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 client.18135 shutdown 2024-05-21T15:25:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 -- 172.21.15.78:0/162295410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 msgr2=0x55ac4a60e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a60e250 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009bf0 tx=0x7f97d0031950 comp rx=0 tx=0).stop 2024-05-21T15:25:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 -- 172.21.15.78:0/162295410 shutdown_connections 2024-05-21T15:25:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ac4a21df60 0x55ac4a61a690 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97baff5640 0 client.18135 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:25:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a25f390 0x55ac4a60e250 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 --2- 172.21.15.78:0/162295410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a608370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 -- 172.21.15.78:0/162295410 >> 172.21.15.78:0/162295410 conn(0x55ac4a2534e0 msgr2=0x55ac4a5ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.216+0000 7f97e45a7740 1 -- 172.21.15.78:0/162295410 shutdown_connections 2024-05-21T15:25:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.217+0000 7f97e45a7740 1 -- 172.21.15.78:0/162295410 wait complete. 2024-05-21T15:25:00.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.217+0000 7f97e45a7740 20 client.18135 trim_cache size 0 max 16384 2024-05-21T15:25:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97e45a7740 1 Processor -- start 2024-05-21T15:25:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97e45a7740 1 -- start start 2024-05-21T15:25:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a21df60 0x55ac4a60a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a602920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a61fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97e45a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81aed50 con 0x55ac4a23e6b0 2024-05-21T15:25:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97e45a7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b0890 con 0x55ac4a21df60 2024-05-21T15:25:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97e45a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81af420 con 0x55ac4a2364d0 2024-05-21T15:25:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a61fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a61fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38066/0 (socket says 172.21.15.78:38066) 2024-05-21T15:25:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d77fe640 1 -- 172.21.15.78:0/2212322817 learned_addr learned my addr 172.21.15.78:0/2212322817 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a21df60 0x55ac4a60a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d77fe640 1 -- 172.21.15.78:0/2212322817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 msgr2=0x55ac4a602920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d67fc640 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a602920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d77fe640 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a602920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d77fe640 1 -- 172.21.15.78:0/2212322817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a21df60 msgr2=0x55ac4a60a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d77fe640 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a21df60 0x55ac4a60a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.230+0000 7f97d77fe640 1 -- 172.21.15.78:0/2212322817 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4a6087d0 con 0x55ac4a23e6b0 2024-05-21T15:25:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97d77fe640 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a61fe30 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f97d002fc30 tx=0x7f97d00319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ba0edf4769e37 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97bcff9640 1 -- 172.21.15.78:0/2212322817 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0051070 con 0x55ac4a23e6b0 2024-05-21T15:25:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97bcff9640 1 -- 172.21.15.78:0/2212322817 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0036040 con 0x55ac4a23e6b0 2024-05-21T15:25:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97bcff9640 1 -- 172.21.15.78:0/2212322817 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0030a20 con 0x55ac4a23e6b0 2024-05-21T15:25:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97e45a7740 1 -- 172.21.15.78:0/2212322817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 msgr2=0x55ac4a61fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97e45a7740 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a61fe30 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f97d002fc30 tx=0x7f97d00319c0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97e45a7740 1 -- 172.21.15.78:0/2212322817 shutdown_connections 2024-05-21T15:25:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97e45a7740 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a61fe30 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97e45a7740 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a2364d0 0x55ac4a602920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97e45a7740 1 --2- 172.21.15.78:0/2212322817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a21df60 0x55ac4a60a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.231+0000 7f97e45a7740 1 -- 172.21.15.78:0/2212322817 >> 172.21.15.78:0/2212322817 conn(0x55ac4a2534e0 msgr2=0x55ac4a5f9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 -- 172.21.15.78:0/2212322817 shutdown_connections 2024-05-21T15:25:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 -- 172.21.15.78:0/2212322817 wait complete. 2024-05-21T15:25:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 -- start start 2024-05-21T15:25:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 Processor -- start 2024-05-21T15:25:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a21df60 0x55ac4a614100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a2364d0 0x55ac4a614640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6110d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b01c0 con 0x55ac4a23e6b0 2024-05-21T15:25:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81afaf0 con 0x55ac4a2364d0 2024-05-21T15:25:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81acd50 con 0x55ac4a21df60 2024-05-21T15:25:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97e45a7740 10 client.0 _ll_register_callbacks cb 0x7f97d00093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6110d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38082/0 (socket says 172.21.15.78:38082) 2024-05-21T15:25:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97d77fe640 1 -- 172.21.15.78:0/1298430247 learned_addr learned my addr 172.21.15.78:0/1298430247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.232+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a21df60 0x55ac4a614100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97d77fe640 1 -- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a21df60 msgr2=0x55ac4a614100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97d77fe640 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a21df60 0x55ac4a614100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97d77fe640 1 -- 172.21.15.78:0/1298430247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a2364d0 msgr2=0x55ac4a614640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97d77fe640 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a2364d0 0x55ac4a614640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97d77fe640 1 -- 172.21.15.78:0/1298430247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4a614c30 con 0x55ac4a23e6b0 2024-05-21T15:25:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a21df60 0x55ac4a614100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97d77fe640 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6110d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f97d004e040 tx=0x7f97d002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e6e2dc04d4e335 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97baff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97baff5640 1 -- 172.21.15.78:0/1298430247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0051040 con 0x55ac4a23e6b0 2024-05-21T15:25:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97baff5640 1 -- 172.21.15.78:0/1298430247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f97d002f810 con 0x55ac4a23e6b0 2024-05-21T15:25:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97e45a7740 1 -- 172.21.15.78:0/1298430247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d0036040 con 0x55ac4a23e6b0 2024-05-21T15:25:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97baff5640 1 -- 172.21.15.78:0/1298430247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97d002fa10 con 0x55ac4a23e6b0 2024-05-21T15:25:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97e45a7740 10 client.18141 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97e45a7740 1 -- 172.21.15.78:0/1298430247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ac4a682e70 con 0x55ac4a23e6b0 2024-05-21T15:25:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97baff5640 1 -- 172.21.15.78:0/1298430247 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0036040 con 0x55ac4a23e6b0 2024-05-21T15:25:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97baff5640 1 -- 172.21.15.78:0/1298430247 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f97d006b030 con 0x55ac4a23e6b0 2024-05-21T15:25:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97e45a7740 10 client.18141 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97e45a7740 10 client.18141 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:25:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97e45a7740 20 client.18141 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.233+0000 7f97e45a7740 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ac4a25f390 0x55ac4a6854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 4 client.18141 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:25:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 -- 172.21.15.78:0/1298430247 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac4a56b6c0 con 0x55ac4a25f390 2024-05-21T15:25:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ac4a25f390 0x55ac4a6854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97baff5640 10 client.18141 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:25:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97d6ffd640 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ac4a25f390 0x55ac4a6854e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d473272cd658c46b server_cookie=a01c3a973a26e612 in_seq=0 out_seq=0 2024-05-21T15:25:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97baff5640 1 -- 172.21.15.78:0/1298430247 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac4a56b6c0 con 0x55ac4a25f390 2024-05-21T15:25:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97baff5640 10 client.18141 handle_command_reply: tid=0 2024-05-21T15:25:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 client.18141 shutdown 2024-05-21T15:25:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 -- 172.21.15.78:0/1298430247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 msgr2=0x55ac4a6110d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6110d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f97d004e040 tx=0x7f97d002f310 comp rx=0 tx=0).stop 2024-05-21T15:25:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 -- 172.21.15.78:0/1298430247 shutdown_connections 2024-05-21T15:25:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55ac4a25f390 0x55ac4a6854e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ac4a2364d0 0x55ac4a614640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ac4a23e6b0 0x55ac4a6110d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 --2- 172.21.15.78:0/1298430247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac4a21df60 0x55ac4a614100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97baff5640 0 client.18141 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:25:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 -- 172.21.15.78:0/1298430247 >> 172.21.15.78:0/1298430247 conn(0x55ac4a2534e0 msgr2=0x55ac4a5f9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 -- 172.21.15.78:0/1298430247 shutdown_connections 2024-05-21T15:25:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.234+0000 7f97e45a7740 1 -- 172.21.15.78:0/1298430247 wait complete. 2024-05-21T15:25:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97e45a7740 20 client.18141 trim_cache size 0 max 16384 2024-05-21T15:25:00.255 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:25:00.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 -- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f97c00a2d00 msgr2=0x7f97c00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f97c00a2d00 0x7f97c00a5210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55ac4a5ff2e0 tx=0x7f97c800b710 comp rx=0 tx=0).stop 2024-05-21T15:25:00.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 -- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8108ad0 msgr2=0x7f97d8199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8108ad0 0x7f97d8199d00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00d770 tx=0x7f97cc00dc30 comp rx=0 tx=0).stop 2024-05-21T15:25:00.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 -- 172.21.15.78:0/530988767 shutdown_connections 2024-05-21T15:25:00.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f97c00a2d00 0x7f97c00a5210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8108f70 0x7f97d819dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97d8108ad0 0x7f97d8199d00 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 --2- 172.21.15.78:0/530988767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97d8101770 0x7f97d81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 -- 172.21.15.78:0/530988767 >> 172.21.15.78:0/530988767 conn(0x7f97d8076830 msgr2=0x7f97d80ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 -- 172.21.15.78:0/530988767 shutdown_connections 2024-05-21T15:25:00.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.235+0000 7f97dd61b640 1 -- 172.21.15.78:0/530988767 wait complete. 2024-05-21T15:25:00.258 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:25:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 -- 172.21.15.78:0/3325057407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 msgr2=0x7fe50410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/3325057407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 0x7fe50410f520 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009ca0 tx=0x7fe4fc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 -- 172.21.15.78:0/3325057407 shutdown_connections 2024-05-21T15:25:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/3325057407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe504108f40 0x7fe504101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/3325057407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 0x7fe50410f520 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/3325057407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe504108aa0 0x7fe504100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 -- 172.21.15.78:0/3325057407 >> 172.21.15.78:0/3325057407 conn(0x7fe504076830 msgr2=0x7fe504076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 -- 172.21.15.78:0/3325057407 shutdown_connections 2024-05-21T15:25:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.370+0000 7fe50ab9c640 1 -- 172.21.15.78:0/3325057407 wait complete. 2024-05-21T15:25:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe50ab9c640 1 Processor -- start 2024-05-21T15:25:00.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe50ab9c640 1 -- start start 2024-05-21T15:25:00.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe50ab9c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 0x7fe504199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe50ab9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe504108aa0 0x7fe504199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe50ab9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe504108f40 0x7fe50419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe50ab9c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe504112de0 con 0x7fe5041016d0 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe50ab9c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe504112c60 con 0x7fe504108aa0 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe50ab9c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe504112f60 con 0x7fe504108f40 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe508938640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 0x7fe504199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe508938640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 0x7fe504199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38088/0 (socket says 172.21.15.78:38088) 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe508938640 1 -- 172.21.15.78:0/2290078217 learned_addr learned my addr 172.21.15.78:0/2290078217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe4fbfff640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe504108aa0 0x7fe504199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe509139640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe504108f40 0x7fe50419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe508938640 1 -- 172.21.15.78:0/2290078217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe504108f40 msgr2=0x7fe50419df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe508938640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe504108f40 0x7fe50419df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe508938640 1 -- 172.21.15.78:0/2290078217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe504108aa0 msgr2=0x7fe504199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe508938640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe504108aa0 0x7fe504199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe508938640 1 -- 172.21.15.78:0/2290078217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50419e5a0 con 0x7fe5041016d0 2024-05-21T15:25:00.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.371+0000 7fe4fbfff640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe504108aa0 0x7fe504199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:00.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.372+0000 7fe508938640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 0x7fe504199740 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe50000e9f0 tx=0x7fe50000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.372+0000 7fe4f9ffb640 1 -- 172.21.15.78:0/2290078217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50000cd80 con 0x7fe5041016d0 2024-05-21T15:25:00.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.372+0000 7fe50ab9c640 1 -- 172.21.15.78:0/2290078217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50419e890 con 0x7fe5041016d0 2024-05-21T15:25:00.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.372+0000 7fe4f9ffb640 1 -- 172.21.15.78:0/2290078217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe500005500 con 0x7fe5041016d0 2024-05-21T15:25:00.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.372+0000 7fe4f9ffb640 1 -- 172.21.15.78:0/2290078217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe500010690 con 0x7fe5041016d0 2024-05-21T15:25:00.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.372+0000 7fe50ab9c640 1 -- 172.21.15.78:0/2290078217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50419ed90 con 0x7fe5041016d0 2024-05-21T15:25:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.374+0000 7fe4f9ffb640 1 -- 172.21.15.78:0/2290078217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5000046c0 con 0x7fe5041016d0 2024-05-21T15:25:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.374+0000 7fe511a3d740 1 Processor -- start 2024-05-21T15:25:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe511a3d740 1 -- start start 2024-05-21T15:25:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd3cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 0x55cfefd3d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef97d6b0 0x55cfefd3d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe511a3d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cfefca2c40 con 0x55cfef99e390 2024-05-21T15:25:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe511a3d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cfefc4a580 con 0x55cfef9754d0 2024-05-21T15:25:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe511a3d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cfefc9ec00 con 0x55cfef97d6b0 2024-05-21T15:25:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe4f9ffb640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4ec0a2d00 0x7fe4ec0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd3cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd3cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38096/0 (socket says 172.21.15.78:38096) 2024-05-21T15:25:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 -- 172.21.15.78:0/86910662 learned_addr learned my addr 172.21.15.78:0/86910662 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe508938640 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 0x55cfefd3d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe4fbfff640 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef97d6b0 0x55cfefd3d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4ec0a2d00 0x7fe4ec0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe4f9ffb640 1 -- 172.21.15.78:0/2290078217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fe5000cfa40 con 0x7fe5041016d0 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 -- 172.21.15.78:0/86910662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef97d6b0 msgr2=0x55cfefd3d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef97d6b0 0x55cfefd3d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 -- 172.21.15.78:0/86910662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 msgr2=0x55cfefd3d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 0x55cfefd3d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.375+0000 7fe509139640 1 -- 172.21.15.78:0/86910662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfef9b63f0 con 0x55cfef99e390 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.376+0000 7fe509139640 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd3cd20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc004af0 tx=0x7fe4fc031f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2fff28b574289b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.376+0000 7fe4eaffd640 1 -- 172.21.15.78:0/86910662 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc004040 con 0x55cfef99e390 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.376+0000 7fe509139640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4ec0a2d00 0x7fe4ec0a5210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc04a000 tx=0x7fe4fc04b000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.376+0000 7fe4eaffd640 1 -- 172.21.15.78:0/86910662 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc04e070 con 0x55cfef99e390 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.376+0000 7fe4eaffd640 1 -- 172.21.15.78:0/86910662 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc02fb40 con 0x55cfef99e390 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 -- 172.21.15.78:0/86910662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 msgr2=0x55cfefd3cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd3cd20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc004af0 tx=0x7fe4fc031f20 comp rx=0 tx=0).stop 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 -- 172.21.15.78:0/86910662 shutdown_connections 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 0x55cfefd3d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef97d6b0 0x55cfefd3d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 --2- 172.21.15.78:0/86910662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd3cd20 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 -- 172.21.15.78:0/86910662 >> 172.21.15.78:0/86910662 conn(0x55cfef9924e0 msgr2=0x55cfefd394d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 -- 172.21.15.78:0/86910662 shutdown_connections 2024-05-21T15:25:00.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 -- 172.21.15.78:0/86910662 wait complete. 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 -- start start 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.377+0000 7fe511a3d740 1 Processor -- start 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 0x55cfefd473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 0x55cfefd47900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd4d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe511a3d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cfefd49ef0 con 0x55cfef99e390 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe511a3d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cfefc5dc40 con 0x55cfef97d6b0 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe511a3d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cfefd4a070 con 0x55cfef9754d0 2024-05-21T15:25:00.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe511a3d740 10 client.0 _ll_register_callbacks cb 0x55cfef9b5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd4d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd4d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38100/0 (socket says 172.21.15.78:38100) 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 -- 172.21.15.78:0/2018257624 learned_addr learned my addr 172.21.15.78:0/2018257624 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe508938640 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 0x55cfefd473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe4fbfff640 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 0x55cfefd47900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 -- 172.21.15.78:0/2018257624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 msgr2=0x55cfefd473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 0x55cfefd473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 -- 172.21.15.78:0/2018257624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 msgr2=0x55cfefd47900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 0x55cfefd47900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 -- 172.21.15.78:0/2018257624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfefa807d0 con 0x55cfef99e390 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe509139640 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd4d320 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009ca0 tx=0x7fe4fc031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0091cc6ec5e9639 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe4e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe4e3fff640 1 -- 172.21.15.78:0/2018257624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc02faf0 con 0x55cfef99e390 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe4e3fff640 1 -- 172.21.15.78:0/2018257624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fe4fc003fa0 con 0x55cfef99e390 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.378+0000 7fe511a3d740 1 -- 172.21.15.78:0/2018257624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4fc04e070 con 0x55cfef99e390 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe4e3fff640 1 -- 172.21.15.78:0/2018257624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe4fc0041a0 con 0x55cfef99e390 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe511a3d740 10 client.18153 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:00.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe511a3d740 1 -- 172.21.15.78:0/2018257624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cfefd49ef0 con 0x55cfef99e390 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe4e3fff640 1 -- 172.21.15.78:0/2018257624 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc04e070 con 0x55cfef99e390 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe4e3fff640 1 -- 172.21.15.78:0/2018257624 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe4fc030ae0 con 0x55cfef99e390 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe511a3d740 10 client.18153 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe511a3d740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe511a3d740 20 client.18153 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe511a3d740 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cfef95cf60 0x55cfefd59770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe511a3d740 4 client.18153 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe511a3d740 1 -- 172.21.15.78:0/2018257624 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfefcb0370 con 0x55cfef95cf60 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.379+0000 7fe508938640 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cfef95cf60 0x55cfefd59770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.380+0000 7fe4e3fff640 10 client.18153 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.380+0000 7fe508938640 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cfef95cf60 0x55cfefd59770 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea7c920f86cca1b server_cookie=f5da41dac0cde8a0 in_seq=0 out_seq=0 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.380+0000 7fe4e3fff640 1 -- 172.21.15.78:0/2018257624 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cfefcb0370 con 0x55cfef95cf60 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.380+0000 7fe4e3fff640 10 client.18153 handle_command_reply: tid=0 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 client.18153 shutdown 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 -- 172.21.15.78:0/2018257624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 msgr2=0x55cfefd4d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd4d320 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009ca0 tx=0x7fe4fc031b10 comp rx=0 tx=0).stop 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 -- 172.21.15.78:0/2018257624 shutdown_connections 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cfef95cf60 0x55cfefd59770 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 0x55cfefd47900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe4e3fff640 0 client.18153 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd4d320 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 --2- 172.21.15.78:0/2018257624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 0x55cfefd473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 -- 172.21.15.78:0/2018257624 >> 172.21.15.78:0/2018257624 conn(0x55cfef9924e0 msgr2=0x55cfefd3ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 -- 172.21.15.78:0/2018257624 shutdown_connections 2024-05-21T15:25:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 1 -- 172.21.15.78:0/2018257624 wait complete. 2024-05-21T15:25:00.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.381+0000 7fe511a3d740 20 client.18153 trim_cache size 0 max 16384 2024-05-21T15:25:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.394+0000 7fe511a3d740 1 Processor -- start 2024-05-21T15:25:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.394+0000 7fe511a3d740 1 -- start start 2024-05-21T15:25:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 0x55cfefd3f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 0x55cfefd46060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd5ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe511a3d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041aec70 con 0x55cfef99e390 2024-05-21T15:25:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe511a3d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b07b0 con 0x55cfef97d6b0 2024-05-21T15:25:00.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe511a3d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041af340 con 0x55cfef9754d0 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd5ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd5ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38108/0 (socket says 172.21.15.78:38108) 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 -- 172.21.15.78:0/4269812234 learned_addr learned my addr 172.21.15.78:0/4269812234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe4fbfff640 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 0x55cfefd46060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 -- 172.21.15.78:0/4269812234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 msgr2=0x55cfefd3f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe508938640 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 0x55cfefd3f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 0x55cfefd3f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 -- 172.21.15.78:0/4269812234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 msgr2=0x55cfefd46060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 0x55cfefd46060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 -- 172.21.15.78:0/4269812234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfefd3e2c0 con 0x55cfef99e390 2024-05-21T15:25:00.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe509139640 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd5ef30 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc031df0 tx=0x7fe4fc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6c7f6fda96fe83 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe4ea7fc640 1 -- 172.21.15.78:0/4269812234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc03e040 con 0x55cfef99e390 2024-05-21T15:25:00.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe4ea7fc640 1 -- 172.21.15.78:0/4269812234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc04e040 con 0x55cfef99e390 2024-05-21T15:25:00.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.395+0000 7fe4ea7fc640 1 -- 172.21.15.78:0/4269812234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc049830 con 0x55cfef99e390 2024-05-21T15:25:00.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 -- 172.21.15.78:0/4269812234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 msgr2=0x55cfefd5ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd5ef30 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc031df0 tx=0x7fe4fc009a20 comp rx=0 tx=0).stop 2024-05-21T15:25:00.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 -- 172.21.15.78:0/4269812234 shutdown_connections 2024-05-21T15:25:00.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef97d6b0 0x55cfefd46060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef99e390 0x55cfefd5ef30 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 --2- 172.21.15.78:0/4269812234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef9754d0 0x55cfefd3f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 -- 172.21.15.78:0/4269812234 >> 172.21.15.78:0/4269812234 conn(0x55cfef9924e0 msgr2=0x55cfefd45950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 -- 172.21.15.78:0/4269812234 shutdown_connections 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 -- 172.21.15.78:0/4269812234 wait complete. 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 -- start start 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.396+0000 7fe511a3d740 1 Processor -- start 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 0x55cfefd4ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef97d6b0 0x55cfefd4f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe511a3d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef99e390 0x55cfefdb9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe511a3d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b00e0 con 0x55cfef97d6b0 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe511a3d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041afa10 con 0x55cfef9754d0 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe511a3d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041acc70 con 0x55cfef99e390 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe511a3d740 10 client.0 _ll_register_callbacks cb 0x7fe50009bd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef97d6b0 0x55cfefd4f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef97d6b0 0x55cfefd4f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38124/0 (socket says 172.21.15.78:38124) 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 -- 172.21.15.78:0/3209687804 learned_addr learned my addr 172.21.15.78:0/3209687804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:00.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe509139640 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef99e390 0x55cfefdb9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe508938640 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 0x55cfefd4ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 -- 172.21.15.78:0/3209687804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef99e390 msgr2=0x55cfefdb9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef99e390 0x55cfefdb9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 -- 172.21.15.78:0/3209687804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 msgr2=0x55cfefd4ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 0x55cfefd4ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 -- 172.21.15.78:0/3209687804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfefdba0f0 con 0x55cfef97d6b0 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe509139640 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef99e390 0x55cfefdb9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4fbfff640 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef97d6b0 0x55cfefd4f140 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009c20 tx=0x7fe4f40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f225d3a35cee61 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:00.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.397+0000 7fe4e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe4e3fff640 1 -- 172.21.15.78:0/3209687804 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f401e470 con 0x55cfef97d6b0 2024-05-21T15:25:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe4e3fff640 1 -- 172.21.15.78:0/3209687804 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7fe4f401e610 con 0x55cfef97d6b0 2024-05-21T15:25:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 1 -- 172.21.15.78:0/3209687804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4fc04e040 con 0x55cfef97d6b0 2024-05-21T15:25:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe4e3fff640 1 -- 172.21.15.78:0/3209687804 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe4f401e810 con 0x55cfef97d6b0 2024-05-21T15:25:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 10 client.18159 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 1 -- 172.21.15.78:0/3209687804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cfefdc1450 con 0x55cfef97d6b0 2024-05-21T15:25:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe4e3fff640 1 -- 172.21.15.78:0/3209687804 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4fc04e040 con 0x55cfef97d6b0 2024-05-21T15:25:00.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe4e3fff640 1 -- 172.21.15.78:0/3209687804 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe4f4024030 con 0x55cfef97d6b0 2024-05-21T15:25:00.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 10 client.18159 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:00.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:00.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 20 client.18159 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:00.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cfef9323a0 0x55cfefdc3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:00.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 4 client.18159 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:25:00.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe511a3d740 1 -- 172.21.15.78:0/3209687804 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfefcaa6c0 con 0x55cfef9323a0 2024-05-21T15:25:00.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.398+0000 7fe508938640 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cfef9323a0 0x55cfefdc3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:00.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe4e3fff640 10 client.18159 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:00.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe508938640 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cfef9323a0 0x55cfefdc3c40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0215289d6e273a server_cookie=8b84a8ba36b24e6b in_seq=0 out_seq=0 2024-05-21T15:25:00.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe4e3fff640 1 -- 172.21.15.78:0/3209687804 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfefcaa6c0 con 0x55cfef9323a0 2024-05-21T15:25:00.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe4e3fff640 10 client.18159 handle_command_reply: tid=0 2024-05-21T15:25:00.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 client.18159 shutdown 2024-05-21T15:25:00.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 -- 172.21.15.78:0/3209687804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef97d6b0 msgr2=0x55cfefd4f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef97d6b0 0x55cfefd4f140 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009c20 tx=0x7fe4f40039c0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 -- 172.21.15.78:0/3209687804 shutdown_connections 2024-05-21T15:25:00.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55cfef9323a0 0x55cfefdc3c40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfef99e390 0x55cfefdb9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe4e3fff640 0 client.18159 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:00.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cfef97d6b0 0x55cfefd4f140 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 --2- 172.21.15.78:0/3209687804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cfef9754d0 0x55cfefd4ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 -- 172.21.15.78:0/3209687804 >> 172.21.15.78:0/3209687804 conn(0x55cfef9924e0 msgr2=0x55cfefd3efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 -- 172.21.15.78:0/3209687804 shutdown_connections 2024-05-21T15:25:00.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.399+0000 7fe511a3d740 1 -- 172.21.15.78:0/3209687804 wait complete. 2024-05-21T15:25:00.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe511a3d740 20 client.18159 trim_cache size 0 max 16384 2024-05-21T15:25:00.414 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:25:00.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 -- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4ec0a2d00 msgr2=0x7fe4ec0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4ec0a2d00 0x7fe4ec0a5210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc04a000 tx=0x7fe4fc04b000 comp rx=0 tx=0).stop 2024-05-21T15:25:00.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 -- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 msgr2=0x7fe504199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:00.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 0x7fe504199740 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe50000e9f0 tx=0x7fe50000edf0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 -- 172.21.15.78:0/2290078217 shutdown_connections 2024-05-21T15:25:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe504108aa0 0x7fe504199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe4ec0a2d00 0x7fe4ec0a5210 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe504108f40 0x7fe50419df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 --2- 172.21.15.78:0/2290078217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5041016d0 0x7fe504199740 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 -- 172.21.15.78:0/2290078217 >> 172.21.15.78:0/2290078217 conn(0x7fe504076830 msgr2=0x7fe5040ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:00.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 -- 172.21.15.78:0/2290078217 shutdown_connections 2024-05-21T15:25:00.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:00.400+0000 7fe50ab9c640 1 -- 172.21.15.78:0/2290078217 wait complete. 2024-05-21T15:25:00.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:00 smithi078 ceph-mon[31997]: pgmap v274: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:00.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:00 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3038298336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:25:00.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:00 smithi084 ceph-mon[36334]: pgmap v274: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:00.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:00 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3038298336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:25:00.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:00 smithi172 ceph-mon[36402]: pgmap v274: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:00.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:00 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3038298336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:25:01.419 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:25:01.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:25:01.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:25:01.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 -- 172.21.15.78:0/795748628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4103260 msgr2=0x7f7af410f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 --2- 172.21.15.78:0/795748628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4103260 0x7f7af410f5e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009ca0 tx=0x7f7aec02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 -- 172.21.15.78:0/795748628 shutdown_connections 2024-05-21T15:25:01.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 --2- 172.21.15.78:0/795748628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4103260 0x7f7af410f5e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 --2- 172.21.15.78:0/795748628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af4108f60 0x7f7af4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 --2- 172.21.15.78:0/795748628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7af4108ac0 0x7f7af41026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 -- 172.21.15.78:0/795748628 >> 172.21.15.78:0/795748628 conn(0x7f7af40fe310 msgr2=0x7f7af4100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 -- 172.21.15.78:0/795748628 shutdown_connections 2024-05-21T15:25:01.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.550+0000 7f7af97e0640 1 -- 172.21.15.78:0/795748628 wait complete. 2024-05-21T15:25:01.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af97e0640 1 Processor -- start 2024-05-21T15:25:01.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af97e0640 1 -- start start 2024-05-21T15:25:01.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af97e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af4103260 0x7f7af41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af97e0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7af4108ac0 0x7f7af4199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af97e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4108f60 0x7f7af419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af97e0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4112e20 con 0x7f7af4108f60 2024-05-21T15:25:01.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af97e0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4112ca0 con 0x7f7af4108ac0 2024-05-21T15:25:01.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af97e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4112fa0 con 0x7f7af4103260 2024-05-21T15:25:01.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4108f60 0x7f7af419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af4103260 0x7f7af41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4108f60 0x7f7af419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38146/0 (socket says 172.21.15.78:38146) 2024-05-21T15:25:01.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7af37fe640 1 -- 172.21.15.78:0/1863416947 learned_addr learned my addr 172.21.15.78:0/1863416947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.551+0000 7f7aebfff640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7af4108ac0 0x7f7af4199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af37fe640 1 -- 172.21.15.78:0/1863416947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af4103260 msgr2=0x7f7af41997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af37fe640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af4103260 0x7f7af41997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af37fe640 1 -- 172.21.15.78:0/1863416947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7af4108ac0 msgr2=0x7f7af4199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af37fe640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7af4108ac0 0x7f7af4199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af37fe640 1 -- 172.21.15.78:0/1863416947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af419e600 con 0x7f7af4108f60 2024-05-21T15:25:01.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af37fe640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4108f60 0x7f7af419dfc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7aec031a20 tx=0x7f7aec031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af0ff9640 1 -- 172.21.15.78:0/1863416947 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec003c30 con 0x7f7af4108f60 2024-05-21T15:25:01.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af0ff9640 1 -- 172.21.15.78:0/1863416947 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aec003dd0 con 0x7f7af4108f60 2024-05-21T15:25:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af97e0640 1 -- 172.21.15.78:0/1863416947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af419e890 con 0x7f7af4108f60 2024-05-21T15:25:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af0ff9640 1 -- 172.21.15.78:0/1863416947 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec02fc40 con 0x7f7af4108f60 2024-05-21T15:25:01.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.552+0000 7f7af97e0640 1 -- 172.21.15.78:0/1863416947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af41a6270 con 0x7f7af4108f60 2024-05-21T15:25:01.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7af0ff9640 1 -- 172.21.15.78:0/1863416947 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7aec004050 con 0x7f7af4108f60 2024-05-21T15:25:01.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7b00322740 1 Processor -- start 2024-05-21T15:25:01.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7b00322740 1 -- start start 2024-05-21T15:25:01.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7b00322740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d29410 0x5623680c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7b00322740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7b00322740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 0x5623680c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7b00322740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56236802d990 con 0x562367d00550 2024-05-21T15:25:01.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7b00322740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562367fd55b0 con 0x562367d29410 2024-05-21T15:25:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.555+0000 7f7b00322740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562368029950 con 0x562367d08730 2024-05-21T15:25:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7af0ff9640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ae00a2d80 0x7f7ae00a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38156/0 (socket says 172.21.15.78:38156) 2024-05-21T15:25:01.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7af2ffd640 1 -- 172.21.15.78:0/2344826253 learned_addr learned my addr 172.21.15.78:0/2344826253 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7aebfff640 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 0x5623680c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7aebfff640 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d29410 0x5623680c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7af37fe640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ae00a2d80 0x7f7ae00a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7af0ff9640 1 -- 172.21.15.78:0/1863416947 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f7aec0f0500 con 0x7f7af4108f60 2024-05-21T15:25:01.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7aebfff640 1 -- 172.21.15.78:0/2344826253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d29410 msgr2=0x5623680c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7aebfff640 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d29410 0x5623680c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7aebfff640 1 -- 172.21.15.78:0/2344826253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 msgr2=0x5623680c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7aebfff640 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7aebfff640 1 -- 172.21.15.78:0/2344826253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562367e097e0 con 0x562367d08730 2024-05-21T15:25:01.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:01.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7af37fe640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ae00a2d80 0x7f7ae00a5290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0ee1c0 tx=0x7f7aec0e7630 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.556+0000 7f7aebfff640 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 0x5623680c90a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009710 tx=0x7f7adc0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c64ddbe72a0c21ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.557+0000 7f7ae97fa640 1 -- 172.21.15.78:0/2344826253 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc004050 con 0x562367d08730 2024-05-21T15:25:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.557+0000 7f7ae97fa640 1 -- 172.21.15.78:0/2344826253 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7adc005500 con 0x562367d08730 2024-05-21T15:25:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.557+0000 7f7ae97fa640 1 -- 172.21.15.78:0/2344826253 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc004b90 con 0x562367d08730 2024-05-21T15:25:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.557+0000 7f7b00322740 1 -- 172.21.15.78:0/2344826253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 msgr2=0x5623680c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.557+0000 7f7b00322740 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 0x5623680c90a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009710 tx=0x7f7adc0096a0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 -- 172.21.15.78:0/2344826253 shutdown_connections 2024-05-21T15:25:01.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 0x5623680c90a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 --2- 172.21.15.78:0/2344826253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d29410 0x5623680c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 -- 172.21.15.78:0/2344826253 >> 172.21.15.78:0/2344826253 conn(0x562367d1d560 msgr2=0x5623680c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 -- 172.21.15.78:0/2344826253 shutdown_connections 2024-05-21T15:25:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 -- 172.21.15.78:0/2344826253 wait complete. 2024-05-21T15:25:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 -- start start 2024-05-21T15:25:01.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 Processor -- start 2024-05-21T15:25:01.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 0x5623680d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d08730 0x5623680d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d29410 0x5623680d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5623680d5970 con 0x562367d29410 2024-05-21T15:25:01.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562367fe8b40 con 0x562367d08730 2024-05-21T15:25:01.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5623680d5af0 con 0x562367d00550 2024-05-21T15:25:01.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.558+0000 7f7b00322740 10 client.0 _ll_register_callbacks cb 0x562367fdd0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:01.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d29410 0x5623680d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d29410 0x5623680d2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38170/0 (socket says 172.21.15.78:38170) 2024-05-21T15:25:01.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 -- 172.21.15.78:0/215178390 learned_addr learned my addr 172.21.15.78:0/215178390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 0x5623680d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d08730 0x5623680d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 -- 172.21.15.78:0/215178390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 msgr2=0x5623680d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 0x5623680d4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 -- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d08730 msgr2=0x5623680d4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d08730 0x5623680d4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 -- 172.21.15.78:0/215178390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562367e69f70 con 0x562367d29410 2024-05-21T15:25:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d08730 0x5623680d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7aebfff640 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d29410 0x5623680d2ba0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7adc004870 tx=0x7f7adc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179f38f54e2d780 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7ada7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7ada7fc640 1 -- 172.21.15.78:0/215178390 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00f850 con 0x562367d29410 2024-05-21T15:25:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7ada7fc640 1 -- 172.21.15.78:0/215178390 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f7adc00f9f0 con 0x562367d29410 2024-05-21T15:25:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7b00322740 1 -- 172.21.15.78:0/215178390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7adc005500 con 0x562367d29410 2024-05-21T15:25:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7ada7fc640 1 -- 172.21.15.78:0/215178390 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7adc035050 con 0x562367d29410 2024-05-21T15:25:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7b00322740 10 client.18171 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.559+0000 7f7b00322740 1 -- 172.21.15.78:0/215178390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562367d01170 con 0x562367d29410 2024-05-21T15:25:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7ada7fc640 1 -- 172.21.15.78:0/215178390 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7adc005500 con 0x562367d29410 2024-05-21T15:25:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7ada7fc640 1 -- 172.21.15.78:0/215178390 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7adc01d030 con 0x562367d29410 2024-05-21T15:25:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7b00322740 10 client.18171 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7b00322740 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7b00322740 20 client.18171 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7b00322740 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562367ce81f0 0x5623680e5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7b00322740 4 client.18171 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7b00322740 1 -- 172.21.15.78:0/215178390 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56236803b0c0 con 0x562367ce81f0 2024-05-21T15:25:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.560+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562367ce81f0 0x5623680e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7ada7fc640 10 client.18171 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562367ce81f0 0x5623680e5040 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed92c1c4c5b2019 server_cookie=bdb48d3aba78d3d6 in_seq=0 out_seq=0 2024-05-21T15:25:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7ada7fc640 1 -- 172.21.15.78:0/215178390 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56236803b0c0 con 0x562367ce81f0 2024-05-21T15:25:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7ada7fc640 10 client.18171 handle_command_reply: tid=0 2024-05-21T15:25:01.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7b00322740 1 client.18171 shutdown 2024-05-21T15:25:01.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7b00322740 1 -- 172.21.15.78:0/215178390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d29410 msgr2=0x5623680d2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7b00322740 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d29410 0x5623680d2ba0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7adc004870 tx=0x7f7adc004610 comp rx=0 tx=0).stop 2024-05-21T15:25:01.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7b00322740 1 -- 172.21.15.78:0/215178390 shutdown_connections 2024-05-21T15:25:01.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.561+0000 7f7b00322740 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562367ce81f0 0x5623680e5040 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.562+0000 7f7b00322740 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367d08730 0x5623680d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.562+0000 7f7b00322740 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d29410 0x5623680d2ba0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.562+0000 7f7ada7fc640 0 client.18171 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:01.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.562+0000 7f7b00322740 1 --2- 172.21.15.78:0/215178390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 0x5623680d4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.562+0000 7f7b00322740 1 -- 172.21.15.78:0/215178390 >> 172.21.15.78:0/215178390 conn(0x562367d1d560 msgr2=0x5623680ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.562+0000 7f7b00322740 1 -- 172.21.15.78:0/215178390 shutdown_connections 2024-05-21T15:25:01.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.562+0000 7f7b00322740 1 -- 172.21.15.78:0/215178390 wait complete. 2024-05-21T15:25:01.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.562+0000 7f7b00322740 20 client.18171 trim_cache size 0 max 16384 2024-05-21T15:25:01.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7b00322740 1 Processor -- start 2024-05-21T15:25:01.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7b00322740 1 -- start start 2024-05-21T15:25:01.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7b00322740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7b00322740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7b00322740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 0x5623680cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7b00322740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41aec80 con 0x562367d00550 2024-05-21T15:25:01.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7b00322740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41b07c0 con 0x562367ce81f0 2024-05-21T15:25:01.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7b00322740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41af350 con 0x562367d08730 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7af2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680d30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38182/0 (socket says 172.21.15.78:38182) 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.580+0000 7f7af2ffd640 1 -- 172.21.15.78:0/1943280760 learned_addr learned my addr 172.21.15.78:0/1943280760 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7aebfff640 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7af2ffd640 1 -- 172.21.15.78:0/1943280760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 msgr2=0x5623680cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 0x5623680cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7af2ffd640 1 -- 172.21.15.78:0/1943280760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 msgr2=0x5623680ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7af2ffd640 1 -- 172.21.15.78:0/1943280760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623680d5ae0 con 0x562367d00550 2024-05-21T15:25:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7aebfff640 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:25:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680d30f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x5623680c9be0 tx=0x7f7ae400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bd2ad86d9d8d1e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7ae8ff9640 1 -- 172.21.15.78:0/1943280760 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4010b60 con 0x562367d00550 2024-05-21T15:25:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7ae8ff9640 1 -- 172.21.15.78:0/1943280760 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae4023070 con 0x562367d00550 2024-05-21T15:25:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.581+0000 7f7ae8ff9640 1 -- 172.21.15.78:0/1943280760 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4004c60 con 0x562367d00550 2024-05-21T15:25:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 -- 172.21.15.78:0/1943280760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 msgr2=0x5623680d30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680d30f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x5623680c9be0 tx=0x7f7ae400b410 comp rx=0 tx=0).stop 2024-05-21T15:25:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 -- 172.21.15.78:0/1943280760 shutdown_connections 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d08730 0x5623680cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d00550 0x5623680d30f0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 --2- 172.21.15.78:0/1943280760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 -- 172.21.15.78:0/1943280760 >> 172.21.15.78:0/1943280760 conn(0x562367d1d560 msgr2=0x5623680c8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 -- 172.21.15.78:0/1943280760 shutdown_connections 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 -- 172.21.15.78:0/1943280760 wait complete. 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 -- start start 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.582+0000 7f7b00322740 1 Processor -- start 2024-05-21T15:25:01.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 0x5623680d8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d08730 0x5623680d8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41b00f0 con 0x562367d08730 2024-05-21T15:25:01.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41afa20 con 0x562367ce81f0 2024-05-21T15:25:01.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41acc80 con 0x562367d00550 2024-05-21T15:25:01.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 10 client.0 _ll_register_callbacks cb 0x7f7ae400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:01.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d08730 0x5623680d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d08730 0x5623680d8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38196/0 (socket says 172.21.15.78:38196) 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 -- 172.21.15.78:0/3637296703 learned_addr learned my addr 172.21.15.78:0/3637296703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 0x5623680d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 -- 172.21.15.78:0/3637296703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 msgr2=0x5623680d8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 0x5623680d8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 -- 172.21.15.78:0/3637296703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 msgr2=0x5623680daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 -- 172.21.15.78:0/3637296703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623680d8dc0 con 0x562367d08730 2024-05-21T15:25:01.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7aebfff640 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d08730 0x5623680d8880 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7adc01ee20 tx=0x7f7adc00e3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd954bf773236c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7ada7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:01.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7ada7fc640 1 -- 172.21.15.78:0/3637296703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc004680 con 0x562367d08730 2024-05-21T15:25:01.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7ada7fc640 1 -- 172.21.15.78:0/3637296703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f7adc00fc90 con 0x562367d08730 2024-05-21T15:25:01.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 1 -- 172.21.15.78:0/3637296703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae4023070 con 0x562367d08730 2024-05-21T15:25:01.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7ada7fc640 1 -- 172.21.15.78:0/3637296703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7adc004c20 con 0x562367d08730 2024-05-21T15:25:01.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 10 client.18177 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:01.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.583+0000 7f7b00322740 1 -- 172.21.15.78:0/3637296703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56236814cd70 con 0x562367d08730 2024-05-21T15:25:01.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7ada7fc640 1 -- 172.21.15.78:0/3637296703 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae4023070 con 0x562367d08730 2024-05-21T15:25:01.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7ada7fc640 1 -- 172.21.15.78:0/3637296703 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7adc01d040 con 0x562367d08730 2024-05-21T15:25:01.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7b00322740 10 client.18177 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:01.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7b00322740 10 client.18177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:01.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7b00322740 20 client.18177 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:01.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7b00322740 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562367d29410 0x56236814f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7b00322740 4 client.18177 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:25:01.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7b00322740 1 -- 172.21.15.78:0/3637296703 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562368035410 con 0x562367d29410 2024-05-21T15:25:01.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562367d29410 0x56236814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7ada7fc640 10 client.18177 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.584+0000 7f7af2ffd640 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562367d29410 0x56236814f560 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed7421d110e37f1e server_cookie=8452b0a1104bc4d0 in_seq=0 out_seq=0 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.585+0000 7f7ada7fc640 1 -- 172.21.15.78:0/3637296703 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562368035410 con 0x562367d29410 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.585+0000 7f7ada7fc640 10 client.18177 handle_command_reply: tid=0 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.585+0000 7f7b00322740 1 client.18177 shutdown 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 -- 172.21.15.78:0/3637296703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d08730 msgr2=0x5623680d8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d08730 0x5623680d8880 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7adc01ee20 tx=0x7f7adc00e3e0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 -- 172.21.15.78:0/3637296703 shutdown_connections 2024-05-21T15:25:01.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x562367d29410 0x56236814f560 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562367d08730 0x5623680d8880 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562367d00550 0x5623680d8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 --2- 172.21.15.78:0/3637296703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562367ce81f0 0x5623680daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 -- 172.21.15.78:0/3637296703 >> 172.21.15.78:0/3637296703 conn(0x562367d1d560 msgr2=0x5623680c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7ada7fc640 0 client.18177 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 -- 172.21.15.78:0/3637296703 shutdown_connections 2024-05-21T15:25:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 1 -- 172.21.15.78:0/3637296703 wait complete. 2024-05-21T15:25:01.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7b00322740 20 client.18177 trim_cache size 0 max 16384 2024-05-21T15:25:01.602 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:25:01.602 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:25:01.602 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "4e4dab00-e93b-4004-96b7-8a53267c45b4", 2024-05-21T15:25:01.602 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:25:01.602 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:25:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7af97e0640 1 -- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ae00a2d80 msgr2=0x7f7ae00a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7af97e0640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ae00a2d80 0x7f7ae00a5290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0ee1c0 tx=0x7f7aec0e7630 comp rx=0 tx=0).stop 2024-05-21T15:25:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7af97e0640 1 -- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4108f60 msgr2=0x7f7af419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7af97e0640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4108f60 0x7f7af419dfc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7aec031a20 tx=0x7f7aec031a50 comp rx=0 tx=0).stop 2024-05-21T15:25:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7af97e0640 1 -- 172.21.15.78:0/1863416947 shutdown_connections 2024-05-21T15:25:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7af97e0640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7af4108ac0 0x7f7af4199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.586+0000 7f7af97e0640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7ae00a2d80 0x7f7ae00a5290 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.587+0000 7f7af97e0640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7af4108f60 0x7f7af419dfc0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.587+0000 7f7af97e0640 1 --2- 172.21.15.78:0/1863416947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af4103260 0x7f7af41997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.587+0000 7f7af97e0640 1 -- 172.21.15.78:0/1863416947 >> 172.21.15.78:0/1863416947 conn(0x7f7af40fe310 msgr2=0x7f7af4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.587+0000 7f7af97e0640 1 -- 172.21.15.78:0/1863416947 shutdown_connections 2024-05-21T15:25:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.587+0000 7f7af97e0640 1 -- 172.21.15.78:0/1863416947 wait complete. 2024-05-21T15:25:01.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:25:01.605 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:25:01.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.717+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2523647565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 msgr2=0x7f19f010e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.717+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2523647565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 0x7f19f010e770 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009ca0 tx=0x7f19e802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.717+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2523647565 shutdown_connections 2024-05-21T15:25:01.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.717+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2523647565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0102230 0x7f19f0102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.717+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2523647565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 0x7f19f010e770 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.717+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2523647565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f01079f0 0x7f19f0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.717+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2523647565 >> 172.21.15.78:0/2523647565 conn(0x7f19f0076830 msgr2=0x7f19f0076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2523647565 shutdown_connections 2024-05-21T15:25:01.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2523647565 wait complete. 2024-05-21T15:25:01.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 Processor -- start 2024-05-21T15:25:01.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 -- start start 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0102230 0x7f19f019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f01079f0 0x7f19f019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 0x7f19f01a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0111d00 con 0x7f19f010c350 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0111b80 con 0x7f19f0102230 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.718+0000 7f19f7ce9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0111e80 con 0x7f19f01079f0 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 0x7f19f01a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 0x7f19f01a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38202/0 (socket says 172.21.15.78:38202) 2024-05-21T15:25:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 -- 172.21.15.78:0/2987734284 learned_addr learned my addr 172.21.15.78:0/2987734284 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f5a85640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0102230 0x7f19f019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f5284640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f01079f0 0x7f19f019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 -- 172.21.15.78:0/2987734284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f01079f0 msgr2=0x7f19f019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f01079f0 0x7f19f019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 -- 172.21.15.78:0/2987734284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0102230 msgr2=0x7f19f019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0102230 0x7f19f019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 -- 172.21.15.78:0/2987734284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f019b320 con 0x7f19f010c350 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f6286640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 0x7f19f01a55e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f19e8031a20 tx=0x7f19e8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19e6ffd640 1 -- 172.21.15.78:0/2987734284 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8003c30 con 0x7f19f010c350 2024-05-21T15:25:01.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19e6ffd640 1 -- 172.21.15.78:0/2987734284 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8003dd0 con 0x7f19f010c350 2024-05-21T15:25:01.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2987734284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f01a5b80 con 0x7f19f010c350 2024-05-21T15:25:01.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19e6ffd640 1 -- 172.21.15.78:0/2987734284 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e802fcf0 con 0x7f19f010c350 2024-05-21T15:25:01.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.719+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2987734284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01a60d0 con 0x7f19f010c350 2024-05-21T15:25:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19e6ffd640 1 -- 172.21.15.78:0/2987734284 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f19e8004050 con 0x7f19f010c350 2024-05-21T15:25:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19feb8d740 1 Processor -- start 2024-05-21T15:25:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19feb8d740 1 -- start start 2024-05-21T15:25:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de00d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de01270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da416b0 0x55581de017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19feb8d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55581dd66d00 con 0x55581da394d0 2024-05-21T15:25:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19feb8d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55581dd0e4c0 con 0x55581da62390 2024-05-21T15:25:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.722+0000 7f19feb8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55581dd62cc0 con 0x55581da416b0 2024-05-21T15:25:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de01270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de01270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38214/0 (socket says 172.21.15.78:38214) 2024-05-21T15:25:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 -- 172.21.15.78:0/2314066511 learned_addr learned my addr 172.21.15.78:0/2314066511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5284640 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de00d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5284640 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da416b0 0x55581de017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19e6ffd640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f19dc0a2d00 0x7f19dc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 -- 172.21.15.78:0/2314066511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da416b0 msgr2=0x55581de017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da416b0 0x55581de017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 -- 172.21.15.78:0/2314066511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 msgr2=0x55581de00d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f6286640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f19dc0a2d00 0x7f19dc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de00d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 -- 172.21.15.78:0/2314066511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581db447d0 con 0x55581da394d0 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5284640 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de00d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f5a85640 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de01270 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00bc10 tx=0x7f19ec0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afca39e94c2064c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19e6ffd640 1 -- 172.21.15.78:0/2987734284 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f19e80f0780 con 0x7f19f010c350 2024-05-21T15:25:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19dbfff640 1 -- 172.21.15.78:0/2314066511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec005020 con 0x55581da394d0 2024-05-21T15:25:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19f6286640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f19dc0a2d00 0x7f19dc0a5210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19e80bcd40 tx=0x7f19e80e78b0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19dbfff640 1 -- 172.21.15.78:0/2314066511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19ec0047a0 con 0x55581da394d0 2024-05-21T15:25:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.723+0000 7f19dbfff640 1 -- 172.21.15.78:0/2314066511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec011750 con 0x55581da394d0 2024-05-21T15:25:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 -- 172.21.15.78:0/2314066511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 msgr2=0x55581de01270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de01270 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00bc10 tx=0x7f19ec0107a0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 -- 172.21.15.78:0/2314066511 shutdown_connections 2024-05-21T15:25:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da416b0 0x55581de017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de01270 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2314066511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de00d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 -- 172.21.15.78:0/2314066511 >> 172.21.15.78:0/2314066511 conn(0x55581da564e0 msgr2=0x55581ddfd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 -- 172.21.15.78:0/2314066511 shutdown_connections 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.724+0000 7f19feb8d740 1 -- 172.21.15.78:0/2314066511 wait complete. 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 1 -- start start 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 1 Processor -- start 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de0ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de0d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da62390 0x55581de0b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55581de0dfc0 con 0x55581da394d0 2024-05-21T15:25:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55581dd21b90 con 0x55581da416b0 2024-05-21T15:25:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55581de0e140 con 0x55581da62390 2024-05-21T15:25:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19feb8d740 10 client.0 _ll_register_callbacks cb 0x55581dd5dda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19f5284640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de0ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.725+0000 7f19f5284640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de0ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38224/0 (socket says 172.21.15.78:38224) 2024-05-21T15:25:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5284640 1 -- 172.21.15.78:0/2343461874 learned_addr learned my addr 172.21.15.78:0/2343461874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5a85640 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de0d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5284640 1 -- 172.21.15.78:0/2343461874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da62390 msgr2=0x55581de0b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5284640 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da62390 0x55581de0b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5284640 1 -- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 msgr2=0x55581de0d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5284640 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de0d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5284640 1 -- 172.21.15.78:0/2343461874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581da7a3f0 con 0x55581da394d0 2024-05-21T15:25:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5a85640 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de0d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:25:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19f5284640 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de0ce60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f19f00746f0 tx=0x7f19e00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df19ce3932007e75 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19d97fa640 1 -- 172.21.15.78:0/2343461874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0005020 con 0x55581da394d0 2024-05-21T15:25:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19d97fa640 1 -- 172.21.15.78:0/2343461874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f19e00051c0 con 0x55581da394d0 2024-05-21T15:25:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19feb8d740 1 -- 172.21.15.78:0/2343461874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ec0047a0 con 0x55581da394d0 2024-05-21T15:25:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19d97fa640 1 -- 172.21.15.78:0/2343461874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f19e00053c0 con 0x55581da394d0 2024-05-21T15:25:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19feb8d740 10 client.18189 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19feb8d740 1 -- 172.21.15.78:0/2343461874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55581da3a0f0 con 0x55581da394d0 2024-05-21T15:25:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.726+0000 7f19d97fa640 1 -- 172.21.15.78:0/2343461874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19ec0047a0 con 0x55581da394d0 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19d97fa640 1 -- 172.21.15.78:0/2343461874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f19e000f040 con 0x55581da394d0 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19feb8d740 10 client.18189 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19feb8d740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19feb8d740 20 client.18189 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55581da20f60 0x55581de1dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19feb8d740 4 client.18189 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19feb8d740 1 -- 172.21.15.78:0/2343461874 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55581dd74430 con 0x55581da20f60 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19f5a85640 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55581da20f60 0x55581de1dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19f5a85640 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55581da20f60 0x55581de1dd30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f97e9086e28eae14 server_cookie=5259a6d22e4909c5 in_seq=0 out_seq=0 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.727+0000 7f19d97fa640 10 client.18189 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19d97fa640 1 -- 172.21.15.78:0/2343461874 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55581dd74430 con 0x55581da20f60 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19d97fa640 10 client.18189 handle_command_reply: tid=0 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 client.18189 shutdown 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 -- 172.21.15.78:0/2343461874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 msgr2=0x55581de0ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de0ce60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f19f00746f0 tx=0x7f19e00107a0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 -- 172.21.15.78:0/2343461874 shutdown_connections 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55581da20f60 0x55581de1dd30 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de0d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da62390 0x55581de0b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19d97fa640 0 client.18189 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 --2- 172.21.15.78:0/2343461874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da394d0 0x55581de0ce60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 -- 172.21.15.78:0/2343461874 >> 172.21.15.78:0/2343461874 conn(0x55581da564e0 msgr2=0x55581de02a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 -- 172.21.15.78:0/2343461874 shutdown_connections 2024-05-21T15:25:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.728+0000 7f19feb8d740 1 -- 172.21.15.78:0/2343461874 wait complete. 2024-05-21T15:25:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.729+0000 7f19feb8d740 20 client.18189 trim_cache size 0 max 16384 2024-05-21T15:25:01.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19feb8d740 1 Processor -- start 2024-05-21T15:25:01.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19feb8d740 1 -- start start 2024-05-21T15:25:01.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 0x55581de234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de23a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da62390 0x55581de0ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19feb8d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01aed90 con 0x55581da62390 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19feb8d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01b08d0 con 0x55581da416b0 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19feb8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01af460 con 0x55581da394d0 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19f5284640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da62390 0x55581de0ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19f5284640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da62390 0x55581de0ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38234/0 (socket says 172.21.15.78:38234) 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.746+0000 7f19f5284640 1 -- 172.21.15.78:0/3717907057 learned_addr learned my addr 172.21.15.78:0/3717907057 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19f5284640 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 0x55581de234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19f5a85640 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de23a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19f5284640 1 -- 172.21.15.78:0/3717907057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 msgr2=0x55581de234f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19f5284640 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 0x55581de234f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19f5284640 1 -- 172.21.15.78:0/3717907057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 msgr2=0x55581de23a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19f5284640 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de23a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19f5284640 1 -- 172.21.15.78:0/3717907057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581ddf78c0 con 0x55581da62390 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19f5284640 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da62390 0x55581de0ab70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f19e0000c00 tx=0x7f19e0005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221af17b255c64d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19db7fe640 1 -- 172.21.15.78:0/3717907057 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0005620 con 0x55581da62390 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19db7fe640 1 -- 172.21.15.78:0/3717907057 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e001d8a0 con 0x55581da62390 2024-05-21T15:25:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.747+0000 7f19db7fe640 1 -- 172.21.15.78:0/3717907057 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00243f0 con 0x55581da62390 2024-05-21T15:25:01.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 -- 172.21.15.78:0/3717907057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da62390 msgr2=0x55581de0ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da62390 0x55581de0ab70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f19e0000c00 tx=0x7f19e0005f70 comp rx=0 tx=0).stop 2024-05-21T15:25:01.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 -- 172.21.15.78:0/3717907057 shutdown_connections 2024-05-21T15:25:01.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da416b0 0x55581de23a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da62390 0x55581de0ab70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 --2- 172.21.15.78:0/3717907057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 0x55581de234f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 -- 172.21.15.78:0/3717907057 >> 172.21.15.78:0/3717907057 conn(0x55581da564e0 msgr2=0x55581de014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 -- 172.21.15.78:0/3717907057 shutdown_connections 2024-05-21T15:25:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 -- 172.21.15.78:0/3717907057 wait complete. 2024-05-21T15:25:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 -- start start 2024-05-21T15:25:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.748+0000 7f19feb8d740 1 Processor -- start 2024-05-21T15:25:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 0x55581de17920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da416b0 0x55581de17e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19feb8d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de14e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19feb8d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01b0200 con 0x55581da416b0 2024-05-21T15:25:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19feb8d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01afb30 con 0x55581da62390 2024-05-21T15:25:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19feb8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01acd90 con 0x55581da394d0 2024-05-21T15:25:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19feb8d740 10 client.0 _ll_register_callbacks cb 0x7f19e000ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da416b0 0x55581de17e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da416b0 0x55581de17e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38246/0 (socket says 172.21.15.78:38246) 2024-05-21T15:25:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 -- 172.21.15.78:0/251681342 learned_addr learned my addr 172.21.15.78:0/251681342 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5284640 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 0x55581de17920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5284640 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de14e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 -- 172.21.15.78:0/251681342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 msgr2=0x55581de17920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 0x55581de17920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 -- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 msgr2=0x55581de14e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de14e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 -- 172.21.15.78:0/251681342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581de154f0 con 0x55581da416b0 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5284640 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de14e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19f5a85640 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da416b0 0x55581de17e80 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00bb30 tx=0x7f19ec00b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e49ebc1953d865 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:01.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.749+0000 7f19d97fa640 1 -- 172.21.15.78:0/251681342 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec02c020 con 0x55581da416b0 2024-05-21T15:25:01.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19d97fa640 1 -- 172.21.15.78:0/251681342 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8658+0+0 (secure 0 0 0) 0x7f19ec005250 con 0x55581da416b0 2024-05-21T15:25:01.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 1 -- 172.21.15.78:0/251681342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e001d8a0 con 0x55581da416b0 2024-05-21T15:25:01.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19d97fa640 1 -- 172.21.15.78:0/251681342 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f19ec037050 con 0x55581da416b0 2024-05-21T15:25:01.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 10 client.18195 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:01.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 1 -- 172.21.15.78:0/251681342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55581de85a20 con 0x55581da416b0 2024-05-21T15:25:01.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19d97fa640 1 -- 172.21.15.78:0/251681342 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e001d8a0 con 0x55581da416b0 2024-05-21T15:25:01.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19d97fa640 1 -- 172.21.15.78:0/251681342 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f19ec004800 con 0x55581da416b0 2024-05-21T15:25:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 10 client.18195 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 20 client.18195 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55581d9f63a0 0x55581de87f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 4 client.18195 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:25:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19feb8d740 1 -- 172.21.15.78:0/251681342 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55581dd6e780 con 0x55581d9f63a0 2024-05-21T15:25:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.750+0000 7f19f5284640 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55581d9f63a0 0x55581de87f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19d97fa640 10 client.18195 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19f5284640 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55581d9f63a0 0x55581de87f60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7f27c88e1f230d server_cookie=49da423188f7c929 in_seq=0 out_seq=0 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19d97fa640 1 -- 172.21.15.78:0/251681342 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55581dd6e780 con 0x55581d9f63a0 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19d97fa640 10 client.18195 handle_command_reply: tid=0 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 client.18195 shutdown 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 -- 172.21.15.78:0/251681342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da416b0 msgr2=0x55581de17e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da416b0 0x55581de17e80 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00bb30 tx=0x7f19ec00b710 comp rx=0 tx=0).stop 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 -- 172.21.15.78:0/251681342 shutdown_connections 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55581d9f63a0 0x55581de87f60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55581da62390 0x55581de14e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55581da416b0 0x55581de17e80 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19d97fa640 0 client.18195 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 --2- 172.21.15.78:0/251681342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55581da394d0 0x55581de17920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 -- 172.21.15.78:0/251681342 >> 172.21.15.78:0/251681342 conn(0x55581da564e0 msgr2=0x55581de224e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 -- 172.21.15.78:0/251681342 shutdown_connections 2024-05-21T15:25:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 1 -- 172.21.15.78:0/251681342 wait complete. 2024-05-21T15:25:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.751+0000 7f19feb8d740 20 client.18195 trim_cache size 0 max 16384 2024-05-21T15:25:01.765 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:25:01.765 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:25:01.765 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:25:01.765 INFO:teuthology.orchestra.run.smithi078.stdout: "4e4dab00-e93b-4004-96b7-8a53267c45b4": { 2024-05-21T15:25:01.765 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:25:01.765 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "4e4dab00-e93b-4004-96b7-8a53267c45b4", 2024-05-21T15:25:01.765 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:25:01.765 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:25:01.766 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:25:01.766 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:25:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f19dc0a2d00 msgr2=0x7f19dc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f19dc0a2d00 0x7f19dc0a5210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19e80bcd40 tx=0x7f19e80e78b0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 msgr2=0x7f19f01a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 0x7f19f01a55e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f19e8031a20 tx=0x7f19e8031a50 comp rx=0 tx=0).stop 2024-05-21T15:25:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2987734284 shutdown_connections 2024-05-21T15:25:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f19dc0a2d00 0x7f19dc0a5210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19f010c350 0x7f19f01a55e0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f01079f0 0x7f19f019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 --2- 172.21.15.78:0/2987734284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f19f0102230 0x7f19f019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2987734284 >> 172.21.15.78:0/2987734284 conn(0x7f19f0076830 msgr2=0x7f19f0103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2987734284 shutdown_connections 2024-05-21T15:25:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:01.752+0000 7f19f7ce9640 1 -- 172.21.15.78:0/2987734284 wait complete. 2024-05-21T15:25:01.768 INFO:tasks.cephfs.filesystem:scrub status for tag:4e4dab00-e93b-4004-96b7-8a53267c45b4 - {'path': '/', 'tag': '4e4dab00-e93b-4004-96b7-8a53267c45b4', 'options': 'recursive,force'} 2024-05-21T15:25:02.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: pgmap v275: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:02.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:25:02.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:02 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: pgmap v275: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:25:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:02 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: pgmap v275: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:25:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:02.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:02.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:02.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:02.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:02.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:02 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:04.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:04 smithi078 ceph-mon[31997]: pgmap v276: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 56 op/s 2024-05-21T15:25:04.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:04 smithi084 ceph-mon[36334]: pgmap v276: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 56 op/s 2024-05-21T15:25:04.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:04 smithi172 ceph-mon[36402]: pgmap v276: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 56 op/s 2024-05-21T15:25:06.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:06 smithi078 ceph-mon[31997]: pgmap v277: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 54 op/s 2024-05-21T15:25:06.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:06 smithi084 ceph-mon[36334]: pgmap v277: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 54 op/s 2024-05-21T15:25:06.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:06 smithi172 ceph-mon[36402]: pgmap v277: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 54 op/s 2024-05-21T15:25:07.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:07 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:07.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:07 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:07.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:07 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:07.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:07 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:07.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:07 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:07.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:07 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:07.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:07 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:07.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:07 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:07.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:07 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:08 smithi078 ceph-mon[31997]: pgmap v278: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 62 op/s 2024-05-21T15:25:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:08 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:08 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:08 smithi078 ceph-mon[31997]: osdmap e95: 12 total, 12 up, 12 in 2024-05-21T15:25:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:08 smithi084 ceph-mon[36334]: pgmap v278: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 62 op/s 2024-05-21T15:25:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:08 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:08 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:08 smithi084 ceph-mon[36334]: osdmap e95: 12 total, 12 up, 12 in 2024-05-21T15:25:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:08 smithi172 ceph-mon[36402]: pgmap v278: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 KiB/s wr, 62 op/s 2024-05-21T15:25:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:08 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:08 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:08 smithi172 ceph-mon[36402]: osdmap e95: 12 total, 12 up, 12 in 2024-05-21T15:25:10.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:10 smithi078 ceph-mon[31997]: pgmap v280: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 70 op/s 2024-05-21T15:25:10.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:10 smithi084 ceph-mon[36334]: pgmap v280: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 70 op/s 2024-05-21T15:25:10.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:10 smithi172 ceph-mon[36402]: pgmap v280: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 70 op/s 2024-05-21T15:25:11.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:11 smithi084 ceph-mon[36334]: pgmap v281: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 70 op/s 2024-05-21T15:25:11.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:11 smithi172 ceph-mon[36402]: pgmap v281: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 70 op/s 2024-05-21T15:25:12.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:11 smithi078 ceph-mon[31997]: pgmap v281: 97 pgs: 97 active+clean; 14 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 70 op/s 2024-05-21T15:25:14.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:13 smithi078 ceph-mon[31997]: pgmap v282: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 204 B/s wr, 17 op/s 2024-05-21T15:25:14.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:13 smithi084 ceph-mon[36334]: pgmap v282: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 204 B/s wr, 17 op/s 2024-05-21T15:25:14.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:13 smithi172 ceph-mon[36402]: pgmap v282: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 204 B/s wr, 17 op/s 2024-05-21T15:25:16.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:15 smithi078 ceph-mon[31997]: pgmap v283: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 204 B/s wr, 15 op/s 2024-05-21T15:25:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:15 smithi084 ceph-mon[36334]: pgmap v283: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 204 B/s wr, 15 op/s 2024-05-21T15:25:16.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:15 smithi172 ceph-mon[36402]: pgmap v283: 97 pgs: 97 active+clean; 13 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 204 B/s wr, 15 op/s 2024-05-21T15:25:17.684 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:17 smithi172 ceph-mon[36402]: osdmap e96: 12 total, 12 up, 12 in 2024-05-21T15:25:17.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:17 smithi078 ceph-mon[31997]: osdmap e96: 12 total, 12 up, 12 in 2024-05-21T15:25:17.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:17 smithi084 ceph-mon[36334]: osdmap e96: 12 total, 12 up, 12 in 2024-05-21T15:25:18.683 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:18 smithi172 ceph-mon[36402]: pgmap v285: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 219 B/s wr, 11 op/s 2024-05-21T15:25:18.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:18 smithi078 ceph-mon[31997]: pgmap v285: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 219 B/s wr, 11 op/s 2024-05-21T15:25:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:18 smithi084 ceph-mon[36334]: pgmap v285: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 219 B/s wr, 11 op/s 2024-05-21T15:25:20.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:20 smithi078 ceph-mon[31997]: pgmap v286: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:25:20.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:20 smithi084 ceph-mon[36334]: pgmap v286: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:25:20.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:20 smithi172 ceph-mon[36402]: pgmap v286: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:25:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:22 smithi078 ceph-mon[31997]: pgmap v287: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:25:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:22 smithi084 ceph-mon[36334]: pgmap v287: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:25:22.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:22 smithi172 ceph-mon[36402]: pgmap v287: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:25:24.735 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:24 smithi172 ceph-mon[36402]: pgmap v288: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:25:24.736 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:24 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:24 smithi078 ceph-mon[31997]: pgmap v288: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:25:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:24 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:24 smithi084 ceph-mon[36334]: pgmap v288: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:25:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:24 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:25:25.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:25 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:25:25.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:25 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:25:25.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:25 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:25:25.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:25 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:25:25.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:25 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:25:25.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:25 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:25:25.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:25 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:25:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:25 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:25:25.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:25 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:25:26.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:26 smithi078 ceph-mon[31997]: pgmap v289: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:25:26.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:26 smithi084 ceph-mon[36334]: pgmap v289: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:25:26.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:26 smithi172 ceph-mon[36402]: pgmap v289: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T15:25:28.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:28 smithi078 ceph-mon[31997]: pgmap v290: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 13 op/s 2024-05-21T15:25:28.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:28 smithi084 ceph-mon[36334]: pgmap v290: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 13 op/s 2024-05-21T15:25:28.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:28 smithi172 ceph-mon[36402]: pgmap v290: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 97 B/s wr, 13 op/s 2024-05-21T15:25:30.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:30 smithi078 ceph-mon[31997]: pgmap v291: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:30.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:30 smithi084 ceph-mon[36334]: pgmap v291: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:30.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:30 smithi172 ceph-mon[36402]: pgmap v291: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:31.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:25:31.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:25:31.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 -- 172.21.15.78:0/2449847475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 msgr2=0x7ff0f810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/2449847475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 0x7ff0f810e750 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009ca0 tx=0x7ff0f002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 -- 172.21.15.78:0/2449847475 shutdown_connections 2024-05-21T15:25:31.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/2449847475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f8101aa0 0x7ff0f8101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/2449847475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 0x7ff0f810e750 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/2449847475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f81079d0 0x7ff0f8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 -- 172.21.15.78:0/2449847475 >> 172.21.15.78:0/2449847475 conn(0x7ff0f80770b0 msgr2=0x7ff0f80774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 -- 172.21.15.78:0/2449847475 shutdown_connections 2024-05-21T15:25:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.882+0000 7ff0fdd77640 1 -- 172.21.15.78:0/2449847475 wait complete. 2024-05-21T15:25:31.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.883+0000 7ff0fdd77640 1 Processor -- start 2024-05-21T15:25:31.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.883+0000 7ff0fdd77640 1 -- start start 2024-05-21T15:25:31.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.883+0000 7ff0fdd77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8101aa0 0x7ff0f81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.883+0000 7ff0fdd77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f81079d0 0x7ff0f8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0fdd77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 0x7ff0f819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0fdd77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f8111d00 con 0x7ff0f810c330 2024-05-21T15:25:31.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0fdd77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f8111b80 con 0x7ff0f81079d0 2024-05-21T15:25:31.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0fdd77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f8111e80 con 0x7ff0f8101aa0 2024-05-21T15:25:31.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 0x7ff0f819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 0x7ff0f819cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41470/0 (socket says 172.21.15.78:41470) 2024-05-21T15:25:31.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 -- 172.21.15.78:0/782553923 learned_addr learned my addr 172.21.15.78:0/782553923 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:31.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f6ffd640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f81079d0 0x7ff0f8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f77fe640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8101aa0 0x7ff0f81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 -- 172.21.15.78:0/782553923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8101aa0 msgr2=0x7ff0f81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8101aa0 0x7ff0f81986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 -- 172.21.15.78:0/782553923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f81079d0 msgr2=0x7ff0f8198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f81079d0 0x7ff0f8198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 -- 172.21.15.78:0/782553923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f819d500 con 0x7ff0f810c330 2024-05-21T15:25:31.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 0x7ff0f819cec0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0031a20 tx=0x7ff0f0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f4ff9640 1 -- 172.21.15.78:0/782553923 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0003c30 con 0x7ff0f810c330 2024-05-21T15:25:31.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f4ff9640 1 -- 172.21.15.78:0/782553923 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f0003dd0 con 0x7ff0f810c330 2024-05-21T15:25:31.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0fdd77640 1 -- 172.21.15.78:0/782553923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f819d790 con 0x7ff0f810c330 2024-05-21T15:25:31.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0f4ff9640 1 -- 172.21.15.78:0/782553923 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f002fc40 con 0x7ff0f810c330 2024-05-21T15:25:31.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.884+0000 7ff0fdd77640 1 -- 172.21.15.78:0/782553923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f819dce0 con 0x7ff0f810c330 2024-05-21T15:25:31.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.887+0000 7ff0f4ff9640 1 -- 172.21.15.78:0/782553923 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff0f0004050 con 0x7ff0f810c330 2024-05-21T15:25:31.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.887+0000 7ff104c28740 1 Processor -- start 2024-05-21T15:25:31.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.887+0000 7ff104c28740 1 -- start start 2024-05-21T15:25:31.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.887+0000 7ff0f4ff9640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0e00a2d00 0x7ff0e00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff104c28740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff104c28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f6ffd640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0e00a2d00 0x7ff0e00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff104c28740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff104c28740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c621318d00 con 0x55c620ff36b0 2024-05-21T15:25:31.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff104c28740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c6212c04c0 con 0x55c621014390 2024-05-21T15:25:31.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff104c28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c621314cc0 con 0x55c620feb4d0 2024-05-21T15:25:31.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f4ff9640 1 -- 172.21.15.78:0/782553923 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7ff0f00f0550 con 0x7ff0f810c330 2024-05-21T15:25:31.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41482/0 (socket says 172.21.15.78:41482) 2024-05-21T15:25:31.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 -- 172.21.15.78:0/100845484 learned_addr learned my addr 172.21.15.78:0/100845484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:31.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f77fe640 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f6ffd640 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 -- 172.21.15.78:0/100845484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 msgr2=0x55c6213b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:31.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 -- 172.21.15.78:0/100845484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 msgr2=0x55c6213b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 -- 172.21.15.78:0/100845484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6210f67d0 con 0x55c620ff36b0 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f6ffd640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0e00a2d00 0x7ff0e00a5210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff0f810d610 tx=0x7ff0ec00bb20 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b3640 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0031f80 tx=0x7ff0f00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b985877bf4f06955 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0ddffb640 1 -- 172.21.15.78:0/100845484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00bcca0 con 0x55c620ff36b0 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0ddffb640 1 -- 172.21.15.78:0/100845484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f0036070 con 0x55c620ff36b0 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.888+0000 7ff0ddffb640 1 -- 172.21.15.78:0/100845484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0004730 con 0x55c620ff36b0 2024-05-21T15:25:31.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 -- 172.21.15.78:0/100845484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 msgr2=0x55c6213b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b3640 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0031f80 tx=0x7ff0f00094a0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 -- 172.21.15.78:0/100845484 shutdown_connections 2024-05-21T15:25:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b3640 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 --2- 172.21.15.78:0/100845484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 -- 172.21.15.78:0/100845484 >> 172.21.15.78:0/100845484 conn(0x55c6210084e0 msgr2=0x55c6213af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 -- 172.21.15.78:0/100845484 shutdown_connections 2024-05-21T15:25:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.889+0000 7ff104c28740 1 -- 172.21.15.78:0/100845484 wait complete. 2024-05-21T15:25:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.890+0000 7ff104c28740 1 -- start start 2024-05-21T15:25:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.890+0000 7ff104c28740 1 Processor -- start 2024-05-21T15:25:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.890+0000 7ff104c28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.890+0000 7ff104c28740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c620ff36b0 0x55c6213c1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.890+0000 7ff104c28740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c621014390 0x55c6213bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.890+0000 7ff104c28740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6213c2050 con 0x55c621014390 2024-05-21T15:25:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.890+0000 7ff104c28740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55c6212d3b90 con 0x55c620ff36b0 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.890+0000 7ff104c28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c6213c21d0 con 0x55c620feb4d0 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff104c28740 10 client.0 _ll_register_callbacks cb 0x55c62130fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c621014390 0x55c6213bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c620ff36b0 0x55c6213c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c621014390 0x55c6213bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41498/0 (socket says 172.21.15.78:41498) 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 -- 172.21.15.78:0/2970841496 learned_addr learned my addr 172.21.15.78:0/2970841496 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 -- 172.21.15.78:0/2970841496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 msgr2=0x55c6213c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:31.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213c10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 -- 172.21.15.78:0/2970841496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c620ff36b0 msgr2=0x55c6213c1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c620ff36b0 0x55c6213c1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 -- 172.21.15.78:0/2970841496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62102c3f0 con 0x55c621014390 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f77fe640 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f6ffd640 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c620ff36b0 0x55c6213c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c621014390 0x55c6213bf4f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00eddd0 tx=0x7ff0f00bad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b19f697debcd6c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/2970841496 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00d9070 con 0x55c621014390 2024-05-21T15:25:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/2970841496 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7ff0f00b9310 con 0x55c621014390 2024-05-21T15:25:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff104c28740 1 -- 172.21.15.78:0/2970841496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f0036070 con 0x55c621014390 2024-05-21T15:25:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/2970841496 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff0f00bafa0 con 0x55c621014390 2024-05-21T15:25:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff104c28740 10 client.18207 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.891+0000 7ff104c28740 1 -- 172.21.15.78:0/2970841496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c620fec0f0 con 0x55c621014390 2024-05-21T15:25:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/2970841496 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f0036070 con 0x55c621014390 2024-05-21T15:25:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/2970841496 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff0f00bcf00 con 0x55c621014390 2024-05-21T15:25:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff104c28740 10 client.18207 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff104c28740 10 client.18207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff104c28740 20 client.18207 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff104c28740 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c620fd2f60 0x55c6213d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff104c28740 4 client.18207 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff104c28740 1 -- 172.21.15.78:0/2970841496 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c621326430 con 0x55c620fd2f60 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.892+0000 7ff0f77fe640 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c620fd2f60 0x55c6213d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.893+0000 7ff0d6ffd640 10 client.18207 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.893+0000 7ff0f77fe640 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c620fd2f60 0x55c6213d1a10 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a6a96768df568b server_cookie=591ae31103f8b7e9 in_seq=0 out_seq=0 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.893+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/2970841496 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c621326430 con 0x55c620fd2f60 2024-05-21T15:25:31.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.893+0000 7ff0d6ffd640 10 client.18207 handle_command_reply: tid=0 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.893+0000 7ff104c28740 1 client.18207 shutdown 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 -- 172.21.15.78:0/2970841496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c621014390 msgr2=0x55c6213bf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c621014390 0x55c6213bf4f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00eddd0 tx=0x7ff0f00bad60 comp rx=0 tx=0).stop 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 -- 172.21.15.78:0/2970841496 shutdown_connections 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c620fd2f60 0x55c6213d1a10 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c620ff36b0 0x55c6213c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c621014390 0x55c6213bf4f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 --2- 172.21.15.78:0/2970841496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff0d6ffd640 0 client.18207 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 -- 172.21.15.78:0/2970841496 >> 172.21.15.78:0/2970841496 conn(0x55c6210084e0 msgr2=0x55c6213b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 -- 172.21.15.78:0/2970841496 shutdown_connections 2024-05-21T15:25:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 1 -- 172.21.15.78:0/2970841496 wait complete. 2024-05-21T15:25:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.894+0000 7ff104c28740 20 client.18207 trim_cache size 0 max 16384 2024-05-21T15:25:31.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.912+0000 7ff104c28740 1 Processor -- start 2024-05-21T15:25:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.912+0000 7ff104c28740 1 -- start start 2024-05-21T15:25:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.912+0000 7ff104c28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.912+0000 7ff104c28740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.912+0000 7ff104c28740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.912+0000 7ff104c28740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f81acab0 con 0x55c620ff36b0 2024-05-21T15:25:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.912+0000 7ff104c28740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f81ae5f0 con 0x55c621014390 2024-05-21T15:25:31.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.912+0000 7ff104c28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f81ad180 con 0x55c620feb4d0 2024-05-21T15:25:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41504/0 (socket says 172.21.15.78:41504) 2024-05-21T15:25:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49010/0 (socket says 172.21.15.78:49010) 2024-05-21T15:25:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f6ffd640 1 -- 172.21.15.78:0/4121666635 learned_addr learned my addr 172.21.15.78:0/4121666635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f7fff640 1 -- 172.21.15.78:0/4121666635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 msgr2=0x55c6213bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/4121666635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f7fff640 1 -- 172.21.15.78:0/4121666635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 msgr2=0x55c6213b8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/4121666635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f7fff640 1 -- 172.21.15.78:0/4121666635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6213b42c0 con 0x55c621014390 2024-05-21T15:25:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/4121666635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b92b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00b8980 tx=0x7ff0f002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3361290f9255c86 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0dd7fa640 1 -- 172.21.15.78:0/4121666635 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00d9020 con 0x55c621014390 2024-05-21T15:25:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0dd7fa640 1 -- 172.21.15.78:0/4121666635 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f0036070 con 0x55c621014390 2024-05-21T15:25:31.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.913+0000 7ff0dd7fa640 1 -- 172.21.15.78:0/4121666635 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00bcca0 con 0x55c621014390 2024-05-21T15:25:31.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 -- 172.21.15.78:0/4121666635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 msgr2=0x55c6213b92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 --2- 172.21.15.78:0/4121666635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b92b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00b8980 tx=0x7ff0f002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 -- 172.21.15.78:0/4121666635 shutdown_connections 2024-05-21T15:25:31.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 --2- 172.21.15.78:0/4121666635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c6213b92b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 --2- 172.21.15.78:0/4121666635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620ff36b0 0x55c6213b8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 --2- 172.21.15.78:0/4121666635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620feb4d0 0x55c6213bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 -- 172.21.15.78:0/4121666635 >> 172.21.15.78:0/4121666635 conn(0x55c6210084e0 msgr2=0x55c6213bed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 -- 172.21.15.78:0/4121666635 shutdown_connections 2024-05-21T15:25:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 -- 172.21.15.78:0/4121666635 wait complete. 2024-05-21T15:25:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.914+0000 7ff104c28740 1 -- start start 2024-05-21T15:25:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff104c28740 1 Processor -- start 2024-05-21T15:25:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff104c28740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620feb4d0 0x55c6213c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff104c28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620ff36b0 0x55c6213c2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff104c28740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c621437d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff104c28740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f81adf20 con 0x55c620feb4d0 2024-05-21T15:25:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff104c28740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f81ad850 con 0x55c621014390 2024-05-21T15:25:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff104c28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f81aaab0 con 0x55c620ff36b0 2024-05-21T15:25:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff104c28740 10 client.0 _ll_register_callbacks cb 0x7ff0e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff0f77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620feb4d0 0x55c6213c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff0f77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620feb4d0 0x55c6213c2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41518/0 (socket says 172.21.15.78:41518) 2024-05-21T15:25:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff0f77fe640 1 -- 172.21.15.78:0/3023731863 learned_addr learned my addr 172.21.15.78:0/3023731863 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:31.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff0f6ffd640 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620ff36b0 0x55c6213c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.915+0000 7ff0f7fff640 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c621437d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0f77fe640 1 -- 172.21.15.78:0/3023731863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620ff36b0 msgr2=0x55c6213c2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:31.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0f77fe640 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620ff36b0 0x55c6213c2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0f77fe640 1 -- 172.21.15.78:0/3023731863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 msgr2=0x55c621437d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:31.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0f77fe640 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c621437d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0f77fe640 1 -- 172.21.15.78:0/3023731863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6213c3210 con 0x55c620feb4d0 2024-05-21T15:25:31.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0f77fe640 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620feb4d0 0x55c6213c2400 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8004700 tx=0x7ff0e8012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c681b5e3c1b0cac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:31.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/3023731863 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8005220 con 0x55c620feb4d0 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/3023731863 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7ff0e80053c0 con 0x55c620feb4d0 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff104c28740 1 -- 172.21.15.78:0/3023731863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f0036070 con 0x55c620feb4d0 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/3023731863 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff0e8030050 con 0x55c620feb4d0 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff104c28740 10 client.18213 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff104c28740 1 -- 172.21.15.78:0/3023731863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c6214393a0 con 0x55c620feb4d0 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/3023731863 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f0036070 con 0x55c620feb4d0 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.916+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/3023731863 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e800fa90 con 0x55c620feb4d0 2024-05-21T15:25:31.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 10 client.18213 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:31.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 10 client.18213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:31.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 20 client.18213 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:31.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c620fa83a0 0x55c62143bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:31.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 4 client.18213 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:25:31.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 -- 172.21.15.78:0/3023731863 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c621320780 con 0x55c620fa83a0 2024-05-21T15:25:31.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff0f6ffd640 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c620fa83a0 0x55c62143bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff0d6ffd640 10 client.18213 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff0f6ffd640 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c620fa83a0 0x55c62143bb90 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b87b8f3c8f3dbb server_cookie=e44462d0fa5a04b in_seq=0 out_seq=0 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff0d6ffd640 1 -- 172.21.15.78:0/3023731863 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c621320780 con 0x55c620fa83a0 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff0d6ffd640 10 client.18213 handle_command_reply: tid=0 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 client.18213 shutdown 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 -- 172.21.15.78:0/3023731863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620feb4d0 msgr2=0x55c6213c2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620feb4d0 0x55c6213c2400 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8004700 tx=0x7ff0e8012d00 comp rx=0 tx=0).stop 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 -- 172.21.15.78:0/3023731863 shutdown_connections 2024-05-21T15:25:31.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55c620fa83a0 0x55c62143bb90 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c621014390 0x55c621437d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.917+0000 7ff104c28740 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c620ff36b0 0x55c6213c2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff104c28740 1 --2- 172.21.15.78:0/3023731863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c620feb4d0 0x55c6213c2400 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0d6ffd640 0 client.18213 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff104c28740 1 -- 172.21.15.78:0/3023731863 >> 172.21.15.78:0/3023731863 conn(0x55c6210084e0 msgr2=0x55c6213bddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff104c28740 1 -- 172.21.15.78:0/3023731863 shutdown_connections 2024-05-21T15:25:31.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff104c28740 1 -- 172.21.15.78:0/3023731863 wait complete. 2024-05-21T15:25:31.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff104c28740 20 client.18213 trim_cache size 0 max 16384 2024-05-21T15:25:31.933 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:25:31.933 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:25:31.933 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:25:31.933 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:25:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 -- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0e00a2d00 msgr2=0x7ff0e00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0e00a2d00 0x7ff0e00a5210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff0f810d610 tx=0x7ff0ec00bb20 comp rx=0 tx=0).stop 2024-05-21T15:25:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 -- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 msgr2=0x7ff0f819cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 0x7ff0f819cec0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0031a20 tx=0x7ff0f0031a50 comp rx=0 tx=0).stop 2024-05-21T15:25:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 -- 172.21.15.78:0/782553923 shutdown_connections 2024-05-21T15:25:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0f81079d0 0x7ff0f8198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff0e00a2d00 0x7ff0e00a5210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0f810c330 0x7ff0f819cec0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 --2- 172.21.15.78:0/782553923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8101aa0 0x7ff0f81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:31.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 -- 172.21.15.78:0/782553923 >> 172.21.15.78:0/782553923 conn(0x7ff0f80770b0 msgr2=0x7ff0f8103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:31.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 -- 172.21.15.78:0/782553923 shutdown_connections 2024-05-21T15:25:31.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:31.918+0000 7ff0fdd77640 1 -- 172.21.15.78:0/782553923 wait complete. 2024-05-21T15:25:31.936 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:25:31.937 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:25:32.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 -- 172.21.15.78:0/2631727406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67881079d0 msgr2=0x7f6788101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 --2- 172.21.15.78:0/2631727406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67881079d0 0x7f6788101560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6778009860 tx=0x7f677802a340 comp rx=0 tx=0).stop 2024-05-21T15:25:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 -- 172.21.15.78:0/2631727406 shutdown_connections 2024-05-21T15:25:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 --2- 172.21.15.78:0/2631727406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f678810c330 0x7f678810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 --2- 172.21.15.78:0/2631727406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6788101aa0 0x7f6788101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 --2- 172.21.15.78:0/2631727406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67881079d0 0x7f6788101560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 -- 172.21.15.78:0/2631727406 >> 172.21.15.78:0/2631727406 conn(0x7f67880770c0 msgr2=0x7f6788077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 -- 172.21.15.78:0/2631727406 shutdown_connections 2024-05-21T15:25:32.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.049+0000 7f678d51d640 1 -- 172.21.15.78:0/2631727406 wait complete. 2024-05-21T15:25:32.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f678d51d640 1 Processor -- start 2024-05-21T15:25:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f678d51d640 1 -- start start 2024-05-21T15:25:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f678d51d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6788101aa0 0x7f6788198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f678d51d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67881079d0 0x7f67881988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f678d51d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f678810c330 0x7f6788198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f678d51d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111ce0 con 0x7f678810c330 2024-05-21T15:25:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f678d51d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111b60 con 0x7f67881079d0 2024-05-21T15:25:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f678d51d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6788111e60 con 0x7f6788101aa0 2024-05-21T15:25:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f67877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f678810c330 0x7f6788198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f67877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f678810c330 0x7f6788198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41538/0 (socket says 172.21.15.78:41538) 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f67877fe640 1 -- 172.21.15.78:0/2161436051 learned_addr learned my addr 172.21.15.78:0/2161436051 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f6786ffd640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6788101aa0 0x7f6788198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.050+0000 7f677e7fc640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67881079d0 0x7f67881988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6786ffd640 1 -- 172.21.15.78:0/2161436051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67881079d0 msgr2=0x7f67881988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6786ffd640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67881079d0 0x7f67881988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6786ffd640 1 -- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f678810c330 msgr2=0x7f6788198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6786ffd640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f678810c330 0x7f6788198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6786ffd640 1 -- 172.21.15.78:0/2161436051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67881a7a20 con 0x7f6788101aa0 2024-05-21T15:25:32.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f67877fe640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f678810c330 0x7f6788198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:32.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6786ffd640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6788101aa0 0x7f6788198370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6778035040 tx=0x7f6778033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6784ff9640 1 -- 172.21.15.78:0/2161436051 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677803f020 con 0x7f6788101aa0 2024-05-21T15:25:32.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6784ff9640 1 -- 172.21.15.78:0/2161436051 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6778030550 con 0x7f6788101aa0 2024-05-21T15:25:32.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881a7cb0 con 0x7f6788101aa0 2024-05-21T15:25:32.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f6784ff9640 1 -- 172.21.15.78:0/2161436051 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778037040 con 0x7f6788101aa0 2024-05-21T15:25:32.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.051+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a81a0 con 0x7f6788101aa0 2024-05-21T15:25:32.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.052+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6788102c80 con 0x7f6788101aa0 2024-05-21T15:25:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.053+0000 7f6784ff9640 1 -- 172.21.15.78:0/2161436051 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6778042020 con 0x7f6788101aa0 2024-05-21T15:25:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.054+0000 7f6784ff9640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67740a2d00 0x7f67740a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.054+0000 7f677e7fc640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67740a2d00 0x7f67740a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.054+0000 7f6784ff9640 1 -- 172.21.15.78:0/2161436051 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f67780ef800 con 0x7f6788101aa0 2024-05-21T15:25:32.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.055+0000 7f677e7fc640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67740a2d00 0x7f67740a5210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f678810d860 tx=0x7f6770009260 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.058+0000 7f6784ff9640 1 -- 172.21.15.78:0/2161436051 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6778049020 con 0x7f6788101aa0 2024-05-21T15:25:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.278+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6788069f50 con 0x7f6788101aa0 2024-05-21T15:25:32.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.279+0000 7f6784ff9640 1 -- 172.21.15.78:0/2161436051 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7f6788069f50 con 0x7f6788101aa0 2024-05-21T15:25:32.284 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:25:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:25:32.284 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:25:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67740a2d00 msgr2=0x7f67740a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67740a2d00 0x7f67740a5210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f678810d860 tx=0x7f6770009260 comp rx=0 tx=0).stop 2024-05-21T15:25:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6788101aa0 msgr2=0x7f6788198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6788101aa0 0x7f6788198370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6778035040 tx=0x7f6778033b60 comp rx=0 tx=0).stop 2024-05-21T15:25:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 shutdown_connections 2024-05-21T15:25:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67881079d0 0x7f67881988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f67740a2d00 0x7f67740a5210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f678810c330 0x7f6788198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 --2- 172.21.15.78:0/2161436051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6788101aa0 0x7f6788198370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.281+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 >> 172.21.15.78:0/2161436051 conn(0x7f67880770c0 msgr2=0x7f6788103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.282+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 shutdown_connections 2024-05-21T15:25:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.282+0000 7f678d51d640 1 -- 172.21.15.78:0/2161436051 wait complete. 2024-05-21T15:25:32.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:25:32.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/3433285252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 msgr2=0x7fd5c410e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/3433285252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 0x7fd5c410e750 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009ca0 tx=0x7fd5bc02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/3433285252 shutdown_connections 2024-05-21T15:25:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/3433285252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 0x7fd5c410e750 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/3433285252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c4101aa0 0x7fd5c4101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/3433285252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c41079d0 0x7fd5c4101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/3433285252 >> 172.21.15.78:0/3433285252 conn(0x7fd5c40770b0 msgr2=0x7fd5c40774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/3433285252 shutdown_connections 2024-05-21T15:25:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.410+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/3433285252 wait complete. 2024-05-21T15:25:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5cb8fe640 1 Processor -- start 2024-05-21T15:25:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5cb8fe640 1 -- start start 2024-05-21T15:25:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5cb8fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c4101aa0 0x7fd5c4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5cb8fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c41079d0 0x7fd5c4199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5cb8fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 0x7fd5c419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5cb8fe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c4111d00 con 0x7fd5c410c330 2024-05-21T15:25:32.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5cb8fe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c4111b80 con 0x7fd5c4101aa0 2024-05-21T15:25:32.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5cb8fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c4111e80 con 0x7fd5c41079d0 2024-05-21T15:25:32.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c9e9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 0x7fd5c419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c9e9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 0x7fd5c419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41564/0 (socket says 172.21.15.78:41564) 2024-05-21T15:25:32.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c9e9b640 1 -- 172.21.15.78:0/958725470 learned_addr learned my addr 172.21.15.78:0/958725470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c41079d0 0x7fd5c4199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c969a640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c4101aa0 0x7fd5c4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c9e9b640 1 -- 172.21.15.78:0/958725470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c41079d0 msgr2=0x7fd5c4199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c9e9b640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c41079d0 0x7fd5c4199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c9e9b640 1 -- 172.21.15.78:0/958725470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c4101aa0 msgr2=0x7fd5c4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c9e9b640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c4101aa0 0x7fd5c4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.411+0000 7fd5c9e9b640 1 -- 172.21.15.78:0/958725470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c419e570 con 0x7fd5c410c330 2024-05-21T15:25:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.412+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c41079d0 0x7fd5c4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.412+0000 7fd5c9e9b640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 0x7fd5c419df30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc031a20 tx=0x7fd5bc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.412+0000 7fd5ba7fc640 1 -- 172.21.15.78:0/958725470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc003b70 con 0x7fd5c410c330 2024-05-21T15:25:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.412+0000 7fd5ba7fc640 1 -- 172.21.15.78:0/958725470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc003d10 con 0x7fd5c410c330 2024-05-21T15:25:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.412+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/958725470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c419e800 con 0x7fd5c410c330 2024-05-21T15:25:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.412+0000 7fd5ba7fc640 1 -- 172.21.15.78:0/958725470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc02fc40 con 0x7fd5c410c330 2024-05-21T15:25:32.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.412+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/958725470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c41a61e0 con 0x7fd5c410c330 2024-05-21T15:25:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5ba7fc640 1 -- 172.21.15.78:0/958725470 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd5bc004050 con 0x7fd5c410c330 2024-05-21T15:25:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5d27a5740 1 Processor -- start 2024-05-21T15:25:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5d27a5740 1 -- start start 2024-05-21T15:25:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400dc6390 0x558401164d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x558401165260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x5584011657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584010cac40 con 0x558400d9d4d0 2024-05-21T15:25:32.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558401072580 con 0x558400dc6390 2024-05-21T15:25:32.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5584010c6c00 con 0x558400da56b0 2024-05-21T15:25:32.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5c969a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x558401165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5c969a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x558401165260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41566/0 (socket says 172.21.15.78:41566) 2024-05-21T15:25:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5c969a640 1 -- 172.21.15.78:0/2627697291 learned_addr learned my addr 172.21.15.78:0/2627697291 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400dc6390 0x558401164d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.415+0000 7fd5ba7fc640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd5b00a2d80 0x7fd5b00a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c9e9b640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd5b00a2d80 0x7fd5b00a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c969a640 1 -- 172.21.15.78:0/2627697291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 msgr2=0x5584011657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c969a640 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x5584011657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c969a640 1 -- 172.21.15.78:0/2627697291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400dc6390 msgr2=0x558401164d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c969a640 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400dc6390 0x558401164d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c969a640 1 -- 172.21.15.78:0/2627697291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558400dde3f0 con 0x558400d9d4d0 2024-05-21T15:25:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400dc6390 0x558401164d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5ba7fc640 1 -- 172.21.15.78:0/958725470 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fd5bc0f0460 con 0x7fd5c410c330 2024-05-21T15:25:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c969a640 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x558401165260 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009c90 tx=0x7fd5c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fb1328d563ea3d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5c9e9b640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd5b00a2d80 0x7fd5b00a5290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0bc8b0 tx=0x7fd5bc033040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5af7fe640 1 -- 172.21.15.78:0/2627697291 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c001e070 con 0x558400d9d4d0 2024-05-21T15:25:32.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5af7fe640 1 -- 172.21.15.78:0/2627697291 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c0019b10 con 0x558400d9d4d0 2024-05-21T15:25:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.416+0000 7fd5af7fe640 1 -- 172.21.15.78:0/2627697291 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000ea30 con 0x558400d9d4d0 2024-05-21T15:25:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 -- 172.21.15.78:0/2627697291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 msgr2=0x558401165260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x558401165260 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009c90 tx=0x7fd5c0011810 comp rx=0 tx=0).stop 2024-05-21T15:25:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 -- 172.21.15.78:0/2627697291 shutdown_connections 2024-05-21T15:25:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x5584011657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x558401165260 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/2627697291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400dc6390 0x558401164d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 -- 172.21.15.78:0/2627697291 >> 172.21.15.78:0/2627697291 conn(0x558400dba4e0 msgr2=0x5584011614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 -- 172.21.15.78:0/2627697291 shutdown_connections 2024-05-21T15:25:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.417+0000 7fd5d27a5740 1 -- 172.21.15.78:0/2627697291 wait complete. 2024-05-21T15:25:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 1 -- start start 2024-05-21T15:25:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 1 Processor -- start 2024-05-21T15:25:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401170de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401171320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400dc6390 0x55840116f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558401171fb0 con 0x558400dc6390 2024-05-21T15:25:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558401085c40 con 0x558400d9d4d0 2024-05-21T15:25:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558401172130 con 0x558400da56b0 2024-05-21T15:25:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5d27a5740 10 client.0 _ll_register_callbacks cb 0x558400dddbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5c8e99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400dc6390 0x55840116f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5c8e99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400dc6390 0x55840116f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41570/0 (socket says 172.21.15.78:41570) 2024-05-21T15:25:32.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5c8e99640 1 -- 172.21.15.78:0/1901408739 learned_addr learned my addr 172.21.15.78:0/1901408739 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5c969a640 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401171320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.418+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401170de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5c969a640 1 -- 172.21.15.78:0/1901408739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 msgr2=0x558401170de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5c969a640 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401170de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5c969a640 1 -- 172.21.15.78:0/1901408739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400dc6390 msgr2=0x55840116f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5c969a640 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400dc6390 0x55840116f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5c969a640 1 -- 172.21.15.78:0/1901408739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558400ea87d0 con 0x558400da56b0 2024-05-21T15:25:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400dc6390 0x55840116f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5c969a640 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401171320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5584011662e0 tx=0x7fd5c0009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2269266e41100c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5acff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-05-21T15:25:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5acff9640 1 -- 172.21.15.78:0/1901408739 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0029020 con 0x558400da56b0 2024-05-21T15:25:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5acff9640 1 -- 172.21.15.78:0/1901408739 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fd5c0005040 con 0x558400da56b0 2024-05-21T15:25:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5d27a5740 1 -- 172.21.15.78:0/1901408739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c0019b10 con 0x558400da56b0 2024-05-21T15:25:32.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5acff9640 1 -- 172.21.15.78:0/1901408739 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd5c0034050 con 0x558400da56b0 2024-05-21T15:25:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5d27a5740 10 client.27313 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5d27a5740 1 -- 172.21.15.78:0/1901408739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55840117e830 con 0x558400da56b0 2024-05-21T15:25:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.419+0000 7fd5acff9640 1 -- 172.21.15.78:0/1901408739 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c0019b10 con 0x558400da56b0 2024-05-21T15:25:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5acff9640 1 -- 172.21.15.78:0/1901408739 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd5c0004be0 con 0x558400da56b0 2024-05-21T15:25:32.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5d27a5740 10 client.27313 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5d27a5740 10 client.27313 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:25:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5d27a5740 20 client.27313 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x558400d84f60 0x558401181be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5d27a5740 4 client.27313 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5d27a5740 1 -- 172.21.15.78:0/1901408739 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584010d8370 con 0x558400d84f60 2024-05-21T15:25:32.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x558400d84f60 0x558401181be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5acff9640 10 client.27313 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:25:32.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.420+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x558400d84f60 0x558401181be0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7baff174c9a938a server_cookie=a10ab7ae891a1652 in_seq=0 out_seq=0 2024-05-21T15:25:32.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5acff9640 1 -- 172.21.15.78:0/1901408739 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584010d8370 con 0x558400d84f60 2024-05-21T15:25:32.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5acff9640 10 client.27313 handle_command_reply: tid=0 2024-05-21T15:25:32.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 client.27313 shutdown 2024-05-21T15:25:32.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 -- 172.21.15.78:0/1901408739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 msgr2=0x558401171320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401171320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5584011662e0 tx=0x7fd5c0009420 comp rx=0 tx=0).stop 2024-05-21T15:25:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 -- 172.21.15.78:0/1901408739 shutdown_connections 2024-05-21T15:25:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x558400d84f60 0x558401181be0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400dc6390 0x55840116f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5acff9640 0 client.27313 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:25:32.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401171320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/1901408739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401170de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 -- 172.21.15.78:0/1901408739 >> 172.21.15.78:0/1901408739 conn(0x558400dba4e0 msgr2=0x558401166a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 -- 172.21.15.78:0/1901408739 shutdown_connections 2024-05-21T15:25:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 1 -- 172.21.15.78:0/1901408739 wait complete. 2024-05-21T15:25:32.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.421+0000 7fd5d27a5740 20 client.27313 trim_cache size 0 max 16384 2024-05-21T15:25:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5d27a5740 1 Processor -- start 2024-05-21T15:25:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5d27a5740 1 -- start start 2024-05-21T15:25:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400d84f60 0x558401171120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401187210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400da56b0 0x558401187750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41aec50 con 0x558400da56b0 2024-05-21T15:25:32.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41b0790 con 0x558400d9d4d0 2024-05-21T15:25:32.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41af320 con 0x558400d84f60 2024-05-21T15:25:32.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5c8e99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400da56b0 0x558401187750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5c969a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401187210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5c8e99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400d84f60 0x558401171120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5c969a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401187210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49076/0 (socket says 172.21.15.78:49076) 2024-05-21T15:25:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5c8e99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400d84f60 0x558401171120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44600/0 (socket says 172.21.15.78:44600) 2024-05-21T15:25:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.435+0000 7fd5c969a640 1 -- 172.21.15.78:0/642174469 learned_addr learned my addr 172.21.15.78:0/642174469 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5c969a640 1 -- 172.21.15.78:0/642174469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400d84f60 msgr2=0x558401171120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5c969a640 1 --2- 172.21.15.78:0/642174469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400d84f60 0x558401171120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5c969a640 1 -- 172.21.15.78:0/642174469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400da56b0 msgr2=0x558401187750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5c969a640 1 --2- 172.21.15.78:0/642174469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400da56b0 0x558401187750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5c969a640 1 -- 172.21.15.78:0/642174469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584011620c0 con 0x558400d9d4d0 2024-05-21T15:25:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/642174469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400d84f60 0x558401171120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5c969a640 1 --2- 172.21.15.78:0/642174469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401187210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00059f0 tx=0x7fd5c0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f0364db7399447a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5aeffd640 1 -- 172.21.15.78:0/642174469 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0029040 con 0x558400d9d4d0 2024-05-21T15:25:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5aeffd640 1 -- 172.21.15.78:0/642174469 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c000e860 con 0x558400d9d4d0 2024-05-21T15:25:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.436+0000 7fd5aeffd640 1 -- 172.21.15.78:0/642174469 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c001e040 con 0x558400d9d4d0 2024-05-21T15:25:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 -- 172.21.15.78:0/642174469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 msgr2=0x558401187210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/642174469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401187210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00059f0 tx=0x7fd5c0005fd0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 -- 172.21.15.78:0/642174469 shutdown_connections 2024-05-21T15:25:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/642174469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d9d4d0 0x558401187210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/642174469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400da56b0 0x558401187750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/642174469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400d84f60 0x558401171120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 -- 172.21.15.78:0/642174469 >> 172.21.15.78:0/642174469 conn(0x558400dba4e0 msgr2=0x558401170a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 -- 172.21.15.78:0/642174469 shutdown_connections 2024-05-21T15:25:32.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 -- 172.21.15.78:0/642174469 wait complete. 2024-05-21T15:25:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 -- start start 2024-05-21T15:25:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.437+0000 7fd5d27a5740 1 Processor -- start 2024-05-21T15:25:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d84f60 0x558401172380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x5584011728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5d27a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401172e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41b00c0 con 0x558400d9d4d0 2024-05-21T15:25:32.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41af9f0 con 0x558400d84f60 2024-05-21T15:25:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5d27a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c41acc50 con 0x558400da56b0 2024-05-21T15:25:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5d27a5740 10 client.0 _ll_register_callbacks cb 0x7fd5c001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d84f60 0x558401172380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c969a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x5584011728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d84f60 0x558401172380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49090/0 (socket says 172.21.15.78:49090) 2024-05-21T15:25:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 -- 172.21.15.78:0/3766088205 learned_addr learned my addr 172.21.15.78:0/3766088205 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401172e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 -- 172.21.15.78:0/3766088205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 msgr2=0x558401172e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401172e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 -- 172.21.15.78:0/3766088205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 msgr2=0x5584011728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x5584011728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 -- 172.21.15.78:0/3766088205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584011e7e80 con 0x558400d84f60 2024-05-21T15:25:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5c8e99640 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d84f60 0x558401172380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x558401182180 tx=0x7fd5b4028040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e25ff70a8b30a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5acff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:25:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5acff9640 1 -- 172.21.15.78:0/3766088205 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4019040 con 0x558400d84f60 2024-05-21T15:25:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5d27a5740 1 -- 172.21.15.78:0/3766088205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c000e860 con 0x558400d84f60 2024-05-21T15:25:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.438+0000 7fd5acff9640 1 -- 172.21.15.78:0/3766088205 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fd5b400fbe0 con 0x558400d84f60 2024-05-21T15:25:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5acff9640 1 -- 172.21.15.78:0/3766088205 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd5b4037050 con 0x558400d84f60 2024-05-21T15:25:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5d27a5740 10 client.27386 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5d27a5740 1 -- 172.21.15.78:0/3766088205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5584011e9100 con 0x558400d84f60 2024-05-21T15:25:32.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5acff9640 1 -- 172.21.15.78:0/3766088205 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c000e860 con 0x558400d84f60 2024-05-21T15:25:32.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5acff9640 1 -- 172.21.15.78:0/3766088205 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd5b4020430 con 0x558400d84f60 2024-05-21T15:25:32.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5d27a5740 10 client.27386 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:32.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5d27a5740 10 client.27386 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:25:32.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5d27a5740 20 client.27386 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:32.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x558400dc6390 0x5584011eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5d27a5740 4 client.27386 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:25:32.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5d27a5740 1 -- 172.21.15.78:0/3766088205 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584010d26c0 con 0x558400dc6390 2024-05-21T15:25:32.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.439+0000 7fd5c969a640 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x558400dc6390 0x5584011eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5acff9640 10 client.27386 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:25:32.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5c969a640 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x558400dc6390 0x5584011eb8c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=888b72a6d3ffc31 server_cookie=6fca44dbd1b63682 in_seq=0 out_seq=0 2024-05-21T15:25:32.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5acff9640 1 -- 172.21.15.78:0/3766088205 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584010d26c0 con 0x558400dc6390 2024-05-21T15:25:32.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5acff9640 10 client.27386 handle_command_reply: tid=0 2024-05-21T15:25:32.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 client.27386 shutdown 2024-05-21T15:25:32.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 -- 172.21.15.78:0/3766088205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d84f60 msgr2=0x558401172380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d84f60 0x558401172380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x558401182180 tx=0x7fd5b4028040 comp rx=0 tx=0).stop 2024-05-21T15:25:32.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 -- 172.21.15.78:0/3766088205 shutdown_connections 2024-05-21T15:25:32.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x558400dc6390 0x5584011eb8c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558400da56b0 0x558401172e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5acff9640 0 client.27386 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:25:32.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558400d9d4d0 0x5584011728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 --2- 172.21.15.78:0/3766088205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558400d84f60 0x558401172380 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 -- 172.21.15.78:0/3766088205 >> 172.21.15.78:0/3766088205 conn(0x558400dba4e0 msgr2=0x558401164ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 -- 172.21.15.78:0/3766088205 shutdown_connections 2024-05-21T15:25:32.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 1 -- 172.21.15.78:0/3766088205 wait complete. 2024-05-21T15:25:32.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.440+0000 7fd5d27a5740 20 client.27386 trim_cache size 0 max 16384 2024-05-21T15:25:32.462 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:25:32.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd5b00a2d80 msgr2=0x7fd5b00a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd5b00a2d80 0x7fd5b00a5290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0bc8b0 tx=0x7fd5bc033040 comp rx=0 tx=0).stop 2024-05-21T15:25:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 msgr2=0x7fd5c419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 0x7fd5c419df30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc031a20 tx=0x7fd5bc031a50 comp rx=0 tx=0).stop 2024-05-21T15:25:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/958725470 shutdown_connections 2024-05-21T15:25:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fd5b00a2d80 0x7fd5b00a5290 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5c410c330 0x7fd5c419df30 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c41079d0 0x7fd5c4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 --2- 172.21.15.78:0/958725470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5c4101aa0 0x7fd5c4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/958725470 >> 172.21.15.78:0/958725470 conn(0x7fd5c40770b0 msgr2=0x7fd5c4068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/958725470 shutdown_connections 2024-05-21T15:25:32.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.441+0000 7fd5cb8fe640 1 -- 172.21.15.78:0/958725470 wait complete. 2024-05-21T15:25:32.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:25:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 -- 172.21.15.78:0/2684547070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc1010c330 msgr2=0x7efc1010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 --2- 172.21.15.78:0/2684547070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc1010c330 0x7efc1010e750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc08009be0 tx=0x7efc0802a6c0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 -- 172.21.15.78:0/2684547070 shutdown_connections 2024-05-21T15:25:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 --2- 172.21.15.78:0/2684547070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc1010c330 0x7efc1010e750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 --2- 172.21.15.78:0/2684547070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc10101aa0 0x7efc10101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 --2- 172.21.15.78:0/2684547070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc101079d0 0x7efc10101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 -- 172.21.15.78:0/2684547070 >> 172.21.15.78:0/2684547070 conn(0x7efc100770b0 msgr2=0x7efc100774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 -- 172.21.15.78:0/2684547070 shutdown_connections 2024-05-21T15:25:32.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.577+0000 7efc14b65640 1 -- 172.21.15.78:0/2684547070 wait complete. 2024-05-21T15:25:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.578+0000 7efc14b65640 1 Processor -- start 2024-05-21T15:25:32.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.578+0000 7efc14b65640 1 -- start start 2024-05-21T15:25:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.578+0000 7efc14b65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc10101aa0 0x7efc1019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.578+0000 7efc14b65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc101079d0 0x7efc1019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.578+0000 7efc14b65640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc1010c330 0x7efc1019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.578+0000 7efc14b65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc10111ce0 con 0x7efc10101aa0 2024-05-21T15:25:32.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.578+0000 7efc14b65640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc10111b60 con 0x7efc1010c330 2024-05-21T15:25:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.578+0000 7efc14b65640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc10111e60 con 0x7efc101079d0 2024-05-21T15:25:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc10101aa0 0x7efc1019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc10101aa0 0x7efc1019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41614/0 (socket says 172.21.15.78:41614) 2024-05-21T15:25:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 -- 172.21.15.78:0/566660333 learned_addr learned my addr 172.21.15.78:0/566660333 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0ed9d640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc1010c330 0x7efc1019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc07fff640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc101079d0 0x7efc1019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 -- 172.21.15.78:0/566660333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc101079d0 msgr2=0x7efc1019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc101079d0 0x7efc1019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 -- 172.21.15.78:0/566660333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc1010c330 msgr2=0x7efc1019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc1010c330 0x7efc1019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 -- 172.21.15.78:0/566660333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc101a5860 con 0x7efc10101aa0 2024-05-21T15:25:32.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc0e59c640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc10101aa0 0x7efc1019a520 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7efc0000e9c0 tx=0x7efc0000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc077fe640 1 -- 172.21.15.78:0/566660333 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0000ccc0 con 0x7efc10101aa0 2024-05-21T15:25:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc14b65640 1 -- 172.21.15.78:0/566660333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc101a5b50 con 0x7efc10101aa0 2024-05-21T15:25:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc077fe640 1 -- 172.21.15.78:0/566660333 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc00005500 con 0x7efc10101aa0 2024-05-21T15:25:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc077fe640 1 -- 172.21.15.78:0/566660333 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00010430 con 0x7efc10101aa0 2024-05-21T15:25:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.579+0000 7efc14b65640 1 -- 172.21.15.78:0/566660333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc101a60c0 con 0x7efc10101aa0 2024-05-21T15:25:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.582+0000 7efc077fe640 1 -- 172.21.15.78:0/566660333 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efc000105f0 con 0x7efc10101aa0 2024-05-21T15:25:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.582+0000 7efc1ba14740 1 Processor -- start 2024-05-21T15:25:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.582+0000 7efc1ba14740 1 -- start start 2024-05-21T15:25:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 0x55d8856efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d8856f0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d8856f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc1ba14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d885655c40 con 0x55d8853284d0 2024-05-21T15:25:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc1ba14740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d8855fd580 con 0x55d8853306b0 2024-05-21T15:25:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc1ba14740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d885651c00 con 0x55d885351390 2024-05-21T15:25:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc0e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d8856f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc0e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d8856f0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41626/0 (socket says 172.21.15.78:41626) 2024-05-21T15:25:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc0e59c640 1 -- 172.21.15.78:0/1579876333 learned_addr learned my addr 172.21.15.78:0/1579876333 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc0ed9d640 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 0x55d8856efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc07fff640 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d8856f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc077fe640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efbfc0a2d00 0x7efbfc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc07fff640 1 -- 172.21.15.78:0/1579876333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 msgr2=0x55d8856efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc07fff640 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 0x55d8856efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc07fff640 1 -- 172.21.15.78:0/1579876333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 msgr2=0x55d8856f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc07fff640 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d8856f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc07fff640 1 -- 172.21.15.78:0/1579876333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8853693f0 con 0x55d8853306b0 2024-05-21T15:25:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc0e59c640 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d8856f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:32.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc0ed9d640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efbfc0a2d00 0x7efbfc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc077fe640 1 -- 172.21.15.78:0/566660333 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7efc000cf890 con 0x7efc10101aa0 2024-05-21T15:25:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.583+0000 7efc07fff640 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d8856f07a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc10074f80 tx=0x7efbf8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0389253435ecce server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.584+0000 7efc04ff9640 1 -- 172.21.15.78:0/1579876333 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8009db0 con 0x55d8853306b0 2024-05-21T15:25:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.584+0000 7efc04ff9640 1 -- 172.21.15.78:0/1579876333 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf80035f0 con 0x55d8853306b0 2024-05-21T15:25:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.584+0000 7efc04ff9640 1 -- 172.21.15.78:0/1579876333 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80196b0 con 0x55d8853306b0 2024-05-21T15:25:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.584+0000 7efc1ba14740 1 -- 172.21.15.78:0/1579876333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 msgr2=0x55d8856f07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.584+0000 7efc1ba14740 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d8856f07a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc10074f80 tx=0x7efbf8011810 comp rx=0 tx=0).stop 2024-05-21T15:25:32.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.584+0000 7efc0ed9d640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efbfc0a2d00 0x7efbfc0a5210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc08004af0 tx=0x7efc08009500 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 -- 172.21.15.78:0/1579876333 shutdown_connections 2024-05-21T15:25:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d8856f07a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d8856f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 --2- 172.21.15.78:0/1579876333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 0x55d8856efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 -- 172.21.15.78:0/1579876333 >> 172.21.15.78:0/1579876333 conn(0x55d8853454e0 msgr2=0x55d8856ec4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 -- 172.21.15.78:0/1579876333 shutdown_connections 2024-05-21T15:25:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 -- 172.21.15.78:0/1579876333 wait complete. 2024-05-21T15:25:32.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 -- start start 2024-05-21T15:25:32.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 Processor -- start 2024-05-21T15:25:32.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856fa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8856fd070 con 0x55d8853306b0 2024-05-21T15:25:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d885610c40 con 0x55d885351390 2024-05-21T15:25:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d8856fd1f0 con 0x55d8853284d0 2024-05-21T15:25:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.585+0000 7efc1ba14740 10 client.0 _ll_register_callbacks cb 0x55d885368be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856fa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41636/0 (socket says 172.21.15.78:41636) 2024-05-21T15:25:32.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 -- 172.21.15.78:0/2391356109 learned_addr learned my addr 172.21.15.78:0/2391356109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc0ed9d640 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc0e59c640 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 -- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 msgr2=0x55d8856fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 -- 172.21.15.78:0/2391356109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 msgr2=0x55d885700520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 -- 172.21.15.78:0/2391356109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8854337d0 con 0x55d8853306b0 2024-05-21T15:25:32.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc0e59c640 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:32.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc07fff640 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856fa900 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7efbf801b040 tx=0x7efbf80094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a705d786484ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efbf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:32.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efbf5ffb640 1 -- 172.21.15.78:0/2391356109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8027030 con 0x55d8853306b0 2024-05-21T15:25:32.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc1ba14740 1 -- 172.21.15.78:0/2391356109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf80035f0 con 0x55d8853306b0 2024-05-21T15:25:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efbf5ffb640 1 -- 172.21.15.78:0/2391356109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7efbf8016050 con 0x55d8853306b0 2024-05-21T15:25:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efbf5ffb640 1 -- 172.21.15.78:0/2391356109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efbf8031050 con 0x55d8853306b0 2024-05-21T15:25:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc1ba14740 10 client.18249 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.586+0000 7efc1ba14740 1 -- 172.21.15.78:0/2391356109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d885709810 con 0x55d8853306b0 2024-05-21T15:25:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efbf5ffb640 1 -- 172.21.15.78:0/2391356109 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf80035f0 con 0x55d8853306b0 2024-05-21T15:25:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efbf5ffb640 1 -- 172.21.15.78:0/2391356109 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7efbf8019a00 con 0x55d8853306b0 2024-05-21T15:25:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efc1ba14740 10 client.18249 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:32.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efc1ba14740 10 client.18249 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:25:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efc1ba14740 20 client.18249 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d88530ff60 0x55d88570cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efc1ba14740 4 client.18249 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efc1ba14740 1 -- 172.21.15.78:0/2391356109 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d885663370 con 0x55d88530ff60 2024-05-21T15:25:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efc0e59c640 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d88530ff60 0x55d88570cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efbf5ffb640 10 client.18249 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:25:32.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.587+0000 7efc0e59c640 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d88530ff60 0x55d88570cbd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a028578c7078a3f0 server_cookie=d7cbdd0c330eb8f5 in_seq=0 out_seq=0 2024-05-21T15:25:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efbf5ffb640 1 -- 172.21.15.78:0/2391356109 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d885663370 con 0x55d88530ff60 2024-05-21T15:25:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efbf5ffb640 10 client.18249 handle_command_reply: tid=0 2024-05-21T15:25:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 client.18249 shutdown 2024-05-21T15:25:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 -- 172.21.15.78:0/2391356109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 msgr2=0x55d8856fa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856fa900 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7efbf801b040 tx=0x7efbf80094e0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 -- 172.21.15.78:0/2391356109 shutdown_connections 2024-05-21T15:25:32.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d88530ff60 0x55d88570cbd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efbf5ffb640 0 client.18249 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:25:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856fa900 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2391356109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 -- 172.21.15.78:0/2391356109 >> 172.21.15.78:0/2391356109 conn(0x55d8853454e0 msgr2=0x55d8856f1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 -- 172.21.15.78:0/2391356109 shutdown_connections 2024-05-21T15:25:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.588+0000 7efc1ba14740 1 -- 172.21.15.78:0/2391356109 wait complete. 2024-05-21T15:25:32.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.589+0000 7efc1ba14740 20 client.18249 trim_cache size 0 max 16384 2024-05-21T15:25:32.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:32 smithi078 ceph-mon[31997]: pgmap v292: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:32.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:32 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2161436051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:25:32.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.602+0000 7efc1ba14740 1 Processor -- start 2024-05-21T15:25:32.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.602+0000 7efc1ba14740 1 -- start start 2024-05-21T15:25:32.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856ec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc1ba14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc101aed50 con 0x55d8853306b0 2024-05-21T15:25:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc1ba14740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc101b0890 con 0x55d885351390 2024-05-21T15:25:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc1ba14740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc101af420 con 0x55d8853284d0 2024-05-21T15:25:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41644/0 (socket says 172.21.15.78:41644) 2024-05-21T15:25:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 -- 172.21.15.78:0/2007400589 learned_addr learned my addr 172.21.15.78:0/2007400589 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc0e59c640 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc0ed9d640 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 -- 172.21.15.78:0/2007400589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 msgr2=0x55d8856ec540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856ec540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 -- 172.21.15.78:0/2007400589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 msgr2=0x55d885700420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 -- 172.21.15.78:0/2007400589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8856fb0a0 con 0x55d8853306b0 2024-05-21T15:25:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc0ed9d640 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:25:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efc07fff640 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856f20e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004f50 tx=0x7efbf8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417c5cc8916eb53a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efbf7fff640 1 -- 172.21.15.78:0/2007400589 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8027040 con 0x55d8853306b0 2024-05-21T15:25:32.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efbf7fff640 1 -- 172.21.15.78:0/2007400589 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf803a070 con 0x55d8853306b0 2024-05-21T15:25:32.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.603+0000 7efbf7fff640 1 -- 172.21.15.78:0/2007400589 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8022b40 con 0x55d8853306b0 2024-05-21T15:25:32.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 -- 172.21.15.78:0/2007400589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 msgr2=0x55d8856f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856f20e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7efbf8004f50 tx=0x7efbf8009260 comp rx=0 tx=0).stop 2024-05-21T15:25:32.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 -- 172.21.15.78:0/2007400589 shutdown_connections 2024-05-21T15:25:32.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d885351390 0x55d885700420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853306b0 0x55d8856f20e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 --2- 172.21.15.78:0/2007400589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8853284d0 0x55d8856ec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 -- 172.21.15.78:0/2007400589 >> 172.21.15.78:0/2007400589 conn(0x55d8853454e0 msgr2=0x55d8856f19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 -- 172.21.15.78:0/2007400589 shutdown_connections 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.604+0000 7efc1ba14740 1 -- 172.21.15.78:0/2007400589 wait complete. 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 1 -- start start 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 1 Processor -- start 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d885701c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d885702140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 0x55d885772dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc101b01c0 con 0x55d8853284d0 2024-05-21T15:25:32.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efc101afaf0 con 0x55d8853306b0 2024-05-21T15:25:32.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc101acd50 con 0x55d885351390 2024-05-21T15:25:32.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc1ba14740 10 client.0 _ll_register_callbacks cb 0x7efc000d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:32.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d885701c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d885701c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41660/0 (socket says 172.21.15.78:41660) 2024-05-21T15:25:32.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 -- 172.21.15.78:0/675671011 learned_addr learned my addr 172.21.15.78:0/675671011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc07fff640 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d885702140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0ed9d640 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 0x55d885772dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 -- 172.21.15.78:0/675671011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 msgr2=0x55d885772dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 0x55d885772dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 -- 172.21.15.78:0/675671011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 msgr2=0x55d885702140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d885702140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 -- 172.21.15.78:0/675671011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8857027e0 con 0x55d8853284d0 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc07fff640 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d885702140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efc0e59c640 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d885701c00 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7efc0009ba20 tx=0x7efc000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbd34dbfec15195 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.605+0000 7efbf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efbf5ffb640 1 -- 172.21.15.78:0/675671011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0009b680 con 0x55d8853284d0 2024-05-21T15:25:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efbf5ffb640 1 -- 172.21.15.78:0/675671011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7efc000a4500 con 0x55d8853284d0 2024-05-21T15:25:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 1 -- 172.21.15.78:0/675671011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf803a070 con 0x55d8853284d0 2024-05-21T15:25:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efbf5ffb640 1 -- 172.21.15.78:0/675671011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efc0009c420 con 0x55d8853284d0 2024-05-21T15:25:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 10 client.18255 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 1 -- 172.21.15.78:0/675671011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d885774360 con 0x55d8853284d0 2024-05-21T15:25:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efbf5ffb640 1 -- 172.21.15.78:0/675671011 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf803a070 con 0x55d8853284d0 2024-05-21T15:25:32.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efbf5ffb640 1 -- 172.21.15.78:0/675671011 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7efc000ba030 con 0x55d8853284d0 2024-05-21T15:25:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 10 client.18255 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 10 client.18255 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:25:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 20 client.18255 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d8852e53a0 0x55d885776a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 4 client.18255 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:25:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc1ba14740 1 -- 172.21.15.78:0/675671011 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d88565d6c0 con 0x55d8852e53a0 2024-05-21T15:25:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.606+0000 7efc07fff640 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d8852e53a0 0x55d885776a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efbf5ffb640 10 client.18255 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:25:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc07fff640 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d8852e53a0 0x55d885776a70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46ff73773dee94d2 server_cookie=5ee7347ebeb391a8 in_seq=0 out_seq=0 2024-05-21T15:25:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efbf5ffb640 1 -- 172.21.15.78:0/675671011 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d88565d6c0 con 0x55d8852e53a0 2024-05-21T15:25:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efbf5ffb640 10 client.18255 handle_command_reply: tid=0 2024-05-21T15:25:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 client.18255 shutdown 2024-05-21T15:25:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 -- 172.21.15.78:0/675671011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 msgr2=0x55d885701c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d885701c00 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7efc0009ba20 tx=0x7efc000048f0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 -- 172.21.15.78:0/675671011 shutdown_connections 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55d8852e53a0 0x55d885776a70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d8853306b0 0x55d885702140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efbf5ffb640 0 client.18255 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d885351390 0x55d885772dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 --2- 172.21.15.78:0/675671011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8853284d0 0x55d885701c00 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 -- 172.21.15.78:0/675671011 >> 172.21.15.78:0/675671011 conn(0x55d8853454e0 msgr2=0x55d8856f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 -- 172.21.15.78:0/675671011 shutdown_connections 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 1 -- 172.21.15.78:0/675671011 wait complete. 2024-05-21T15:25:32.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.607+0000 7efc1ba14740 20 client.18255 trim_cache size 0 max 16384 2024-05-21T15:25:32.628 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:25:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 -- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efbfc0a2d00 msgr2=0x7efbfc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efbfc0a2d00 0x7efbfc0a5210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc08004af0 tx=0x7efc08009500 comp rx=0 tx=0).stop 2024-05-21T15:25:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 -- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc10101aa0 msgr2=0x7efc1019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc10101aa0 0x7efc1019a520 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7efc0000e9c0 tx=0x7efc0000edc0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 -- 172.21.15.78:0/566660333 shutdown_connections 2024-05-21T15:25:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc1010c330 0x7efc1019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7efbfc0a2d00 0x7efbfc0a5210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc101079d0 0x7efc1019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 --2- 172.21.15.78:0/566660333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc10101aa0 0x7efc1019a520 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 -- 172.21.15.78:0/566660333 >> 172.21.15.78:0/566660333 conn(0x7efc100770b0 msgr2=0x7efc10103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 -- 172.21.15.78:0/566660333 shutdown_connections 2024-05-21T15:25:32.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.608+0000 7efc14b65640 1 -- 172.21.15.78:0/566660333 wait complete. 2024-05-21T15:25:32.631 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:25:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 -- 172.21.15.78:0/3130973451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38108f70 msgr2=0x7f0d381010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/3130973451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38108f70 0x7f0d381010e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009900 tx=0x7f0d2802a410 comp rx=0 tx=0).stop 2024-05-21T15:25:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 -- 172.21.15.78:0/3130973451 shutdown_connections 2024-05-21T15:25:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/3130973451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d38101770 0x7f0d3810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/3130973451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38108f70 0x7f0d381010e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/3130973451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38108ad0 0x7f0d38100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 -- 172.21.15.78:0/3130973451 >> 172.21.15.78:0/3130973451 conn(0x7f0d38076830 msgr2=0x7f0d38076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 -- 172.21.15.78:0/3130973451 shutdown_connections 2024-05-21T15:25:32.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.744+0000 7f0d3fd96640 1 -- 172.21.15.78:0/3130973451 wait complete. 2024-05-21T15:25:32.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3fd96640 1 Processor -- start 2024-05-21T15:25:32.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3fd96640 1 -- start start 2024-05-21T15:25:32.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3fd96640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38108ad0 0x7f0d38199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3fd96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38199590 0x7f0d3819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3fd96640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3819ddb0 0x7f0d3819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3fd96640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38112e10 con 0x7f0d38108ad0 2024-05-21T15:25:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3fd96640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38112c90 con 0x7f0d3819ddb0 2024-05-21T15:25:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3fd96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38112f90 con 0x7f0d38199590 2024-05-21T15:25:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3db32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38108ad0 0x7f0d38199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3db32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38108ad0 0x7f0d38199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41684/0 (socket says 172.21.15.78:41684) 2024-05-21T15:25:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3db32640 1 -- 172.21.15.78:0/439222232 learned_addr learned my addr 172.21.15.78:0/439222232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3d331640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38199590 0x7f0d3819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.745+0000 7f0d3e333640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3819ddb0 0x7f0d3819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3db32640 1 -- 172.21.15.78:0/439222232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38199590 msgr2=0x7f0d3819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3db32640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38199590 0x7f0d3819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3db32640 1 -- 172.21.15.78:0/439222232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3819ddb0 msgr2=0x7f0d3819e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3db32640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3819ddb0 0x7f0d3819e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3db32640 1 -- 172.21.15.78:0/439222232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d381a5b20 con 0x7f0d38108ad0 2024-05-21T15:25:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3d331640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38199590 0x7f0d3819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3e333640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3819ddb0 0x7f0d3819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3db32640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38108ad0 0x7f0d38199050 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400e9c0 tx=0x7f0d3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d2effd640 1 -- 172.21.15.78:0/439222232 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400cd80 con 0x7f0d38108ad0 2024-05-21T15:25:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3fd96640 1 -- 172.21.15.78:0/439222232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d381a5cf0 con 0x7f0d38108ad0 2024-05-21T15:25:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d2effd640 1 -- 172.21.15.78:0/439222232 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d34005500 con 0x7f0d38108ad0 2024-05-21T15:25:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d2effd640 1 -- 172.21.15.78:0/439222232 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34010690 con 0x7f0d38108ad0 2024-05-21T15:25:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.746+0000 7f0d3fd96640 1 -- 172.21.15.78:0/439222232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d381a61b0 con 0x7f0d38108ad0 2024-05-21T15:25:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d2effd640 1 -- 172.21.15.78:0/439222232 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0d340046c0 con 0x7f0d38108ad0 2024-05-21T15:25:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d46c42740 1 Processor -- start 2024-05-21T15:25:32.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d46c42740 1 -- start start 2024-05-21T15:25:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 0x55692cb3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 0x55692cb3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d46c42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55692caa0c40 con 0x55692c79c390 2024-05-21T15:25:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d46c42740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55692ca48580 con 0x55692c77b6b0 2024-05-21T15:25:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d46c42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55692ca9cc00 con 0x55692c7734d0 2024-05-21T15:25:32.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d3e333640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d3e333640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41686/0 (socket says 172.21.15.78:41686) 2024-05-21T15:25:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d3e333640 1 -- 172.21.15.78:0/3171195462 learned_addr learned my addr 172.21.15.78:0/3171195462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d3db32640 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 0x55692cb3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.749+0000 7f0d3d331640 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 0x55692cb3b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d2effd640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0d240a2d00 0x7f0d240a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d3e333640 1 -- 172.21.15.78:0/3171195462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 msgr2=0x55692cb3b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d3e333640 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 0x55692cb3b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d3e333640 1 -- 172.21.15.78:0/3171195462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 msgr2=0x55692cb3b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d3e333640 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 0x55692cb3b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d3e333640 1 -- 172.21.15.78:0/3171195462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692c7b43f0 con 0x55692c79c390 2024-05-21T15:25:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d2effd640 1 -- 172.21.15.78:0/439222232 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f0d340cf960 con 0x7f0d38108ad0 2024-05-21T15:25:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d3e333640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0d240a2d00 0x7f0d240a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d3e333640 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3ad20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0d30010620 tx=0x7f0d30010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ff2c09834cf81b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d23fff640 1 -- 172.21.15.78:0/3171195462 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30019070 con 0x55692c79c390 2024-05-21T15:25:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d23fff640 1 -- 172.21.15.78:0/3171195462 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30022070 con 0x55692c79c390 2024-05-21T15:25:32.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d23fff640 1 -- 172.21.15.78:0/3171195462 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011b20 con 0x55692c79c390 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.750+0000 7f0d3e333640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0d240a2d00 0x7f0d240a5210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d30021000 tx=0x7f0d3001f000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 -- 172.21.15.78:0/3171195462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 msgr2=0x55692cb3ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3ad20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0d30010620 tx=0x7f0d30010ae0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 -- 172.21.15.78:0/3171195462 shutdown_connections 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 0x55692cb3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 0x55692cb3b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3171195462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3ad20 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 -- 172.21.15.78:0/3171195462 >> 172.21.15.78:0/3171195462 conn(0x55692c7904e0 msgr2=0x55692cb374d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 -- 172.21.15.78:0/3171195462 shutdown_connections 2024-05-21T15:25:32.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.751+0000 7f0d46c42740 1 -- 172.21.15.78:0/3171195462 wait complete. 2024-05-21T15:25:32.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 1 -- start start 2024-05-21T15:25:32.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 1 Processor -- start 2024-05-21T15:25:32.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c7734d0 0x55692cb45420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 0x55692cb45980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c79c390 0x55692cb4b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55692cb47fb0 con 0x55692c7734d0 2024-05-21T15:25:32.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55692ca5bc40 con 0x55692c79c390 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55692cb48130 con 0x55692c77b6b0 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d46c42740 10 client.0 _ll_register_callbacks cb 0x55692c7b3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d3db32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c7734d0 0x55692cb45420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d3db32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c7734d0 0x55692cb45420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41694/0 (socket says 172.21.15.78:41694) 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d3db32640 1 -- 172.21.15.78:0/3260337981 learned_addr learned my addr 172.21.15.78:0/3260337981 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.752+0000 7f0d3e333640 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c79c390 0x55692cb4b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d3d331640 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 0x55692cb45980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d3db32640 1 -- 172.21.15.78:0/3260337981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 msgr2=0x55692cb45980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d3db32640 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 0x55692cb45980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d3db32640 1 -- 172.21.15.78:0/3260337981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c79c390 msgr2=0x55692cb4b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d3db32640 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c79c390 0x55692cb4b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d3db32640 1 -- 172.21.15.78:0/3260337981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692c87e7d0 con 0x55692c7734d0 2024-05-21T15:25:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d3db32640 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c7734d0 0x55692cb45420 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0d3409bd80 tx=0x7f0d34004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2492a2039b97f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d217fa640 1 -- 172.21.15.78:0/3260337981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3409b750 con 0x55692c7734d0 2024-05-21T15:25:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d217fa640 1 -- 172.21.15.78:0/3260337981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f0d340a47d0 con 0x55692c7734d0 2024-05-21T15:25:32.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d46c42740 1 -- 172.21.15.78:0/3260337981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d30022070 con 0x55692c7734d0 2024-05-21T15:25:32.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d217fa640 1 -- 172.21.15.78:0/3260337981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0d3409c4f0 con 0x55692c7734d0 2024-05-21T15:25:32.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d46c42740 10 client.18267 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:32.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d46c42740 1 -- 172.21.15.78:0/3260337981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55692cb54830 con 0x55692c7734d0 2024-05-21T15:25:32.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d217fa640 1 -- 172.21.15.78:0/3260337981 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30022070 con 0x55692c7734d0 2024-05-21T15:25:32.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d217fa640 1 -- 172.21.15.78:0/3260337981 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0d34062050 con 0x55692c7734d0 2024-05-21T15:25:32.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d46c42740 10 client.18267 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:32.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.753+0000 7f0d46c42740 10 client.18267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:25:32.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.754+0000 7f0d46c42740 20 client.18267 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:32.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.754+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55692c75af60 0x55692cb57d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.754+0000 7f0d46c42740 4 client.18267 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:32.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.754+0000 7f0d46c42740 1 -- 172.21.15.78:0/3260337981 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55692caae370 con 0x55692c75af60 2024-05-21T15:25:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.754+0000 7f0d3d331640 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55692c75af60 0x55692cb57d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.755+0000 7f0d217fa640 10 client.18267 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:25:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.755+0000 7f0d3d331640 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55692c75af60 0x55692cb57d10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad0b320cbfdb9bcd server_cookie=b93c916789ef188 in_seq=0 out_seq=0 2024-05-21T15:25:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.755+0000 7f0d217fa640 1 -- 172.21.15.78:0/3260337981 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55692caae370 con 0x55692c75af60 2024-05-21T15:25:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.755+0000 7f0d217fa640 10 client.18267 handle_command_reply: tid=0 2024-05-21T15:25:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.755+0000 7f0d46c42740 1 client.18267 shutdown 2024-05-21T15:25:32.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 -- 172.21.15.78:0/3260337981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c7734d0 msgr2=0x55692cb45420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c7734d0 0x55692cb45420 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0d3409bd80 tx=0x7f0d34004a30 comp rx=0 tx=0).stop 2024-05-21T15:25:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 -- 172.21.15.78:0/3260337981 shutdown_connections 2024-05-21T15:25:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55692c75af60 0x55692cb57d10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c79c390 0x55692cb4b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 0x55692cb45980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d217fa640 0 client.18267 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:25:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 --2- 172.21.15.78:0/3260337981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c7734d0 0x55692cb45420 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 -- 172.21.15.78:0/3260337981 >> 172.21.15.78:0/3260337981 conn(0x55692c7904e0 msgr2=0x55692cb3ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 -- 172.21.15.78:0/3260337981 shutdown_connections 2024-05-21T15:25:32.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 1 -- 172.21.15.78:0/3260337981 wait complete. 2024-05-21T15:25:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.756+0000 7f0d46c42740 20 client.18267 trim_cache size 0 max 16384 2024-05-21T15:25:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.769+0000 7f0d46c42740 1 Processor -- start 2024-05-21T15:25:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.769+0000 7f0d46c42740 1 -- start start 2024-05-21T15:25:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c7734d0 0x55692cb4b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 0x55692cb441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d46c42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381aeea0 con 0x55692c79c390 2024-05-21T15:25:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d46c42740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b09e0 con 0x55692c7734d0 2024-05-21T15:25:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d46c42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381af570 con 0x55692c77b6b0 2024-05-21T15:25:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41700/0 (socket says 172.21.15.78:41700) 2024-05-21T15:25:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 -- 172.21.15.78:0/2334311457 learned_addr learned my addr 172.21.15.78:0/2334311457 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3d331640 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 0x55692cb441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3db32640 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c7734d0 0x55692cb4b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 -- 172.21.15.78:0/2334311457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 msgr2=0x55692cb441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 0x55692cb441f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 -- 172.21.15.78:0/2334311457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c7734d0 msgr2=0x55692cb4b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c7734d0 0x55692cb4b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 -- 172.21.15.78:0/2334311457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692cb380a0 con 0x55692c79c390 2024-05-21T15:25:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d3e333640 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3e9d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0d30013ed0 tx=0x7f0d30013f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe532708111d0a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d237fe640 1 -- 172.21.15.78:0/2334311457 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30019040 con 0x55692c79c390 2024-05-21T15:25:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.770+0000 7f0d237fe640 1 -- 172.21.15.78:0/2334311457 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30022040 con 0x55692c79c390 2024-05-21T15:25:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d237fe640 1 -- 172.21.15.78:0/2334311457 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011440 con 0x55692c79c390 2024-05-21T15:25:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 -- 172.21.15.78:0/2334311457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 msgr2=0x55692cb3e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3e9d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0d30013ed0 tx=0x7f0d30013f00 comp rx=0 tx=0).stop 2024-05-21T15:25:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 -- 172.21.15.78:0/2334311457 shutdown_connections 2024-05-21T15:25:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb3e9d0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c77b6b0 0x55692cb441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2334311457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c7734d0 0x55692cb4b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 -- 172.21.15.78:0/2334311457 >> 172.21.15.78:0/2334311457 conn(0x55692c7904e0 msgr2=0x55692cb43ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 -- 172.21.15.78:0/2334311457 shutdown_connections 2024-05-21T15:25:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.771+0000 7f0d46c42740 1 -- 172.21.15.78:0/2334311457 wait complete. 2024-05-21T15:25:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 1 -- start start 2024-05-21T15:25:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 1 Processor -- start 2024-05-21T15:25:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 0x55692cb51630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 0x55692cb51b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb4ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381b0310 con 0x55692c79c390 2024-05-21T15:25:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381afc40 con 0x55692c77b6b0 2024-05-21T15:25:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d381acea0 con 0x55692c7734d0 2024-05-21T15:25:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d46c42740 10 client.0 _ll_register_callbacks cb 0x7f0d3400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:32.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d3e333640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb4ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d3e333640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb4ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41716/0 (socket says 172.21.15.78:41716) 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d3e333640 1 -- 172.21.15.78:0/2225674650 learned_addr learned my addr 172.21.15.78:0/2225674650 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d3d331640 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 0x55692cb51b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.772+0000 7f0d3db32640 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 0x55692cb51630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d3e333640 1 -- 172.21.15.78:0/2225674650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 msgr2=0x55692cb51630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d3e333640 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 0x55692cb51630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d3e333640 1 -- 172.21.15.78:0/2225674650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 msgr2=0x55692cb51b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d3e333640 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 0x55692cb51b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d3e333640 1 -- 172.21.15.78:0/2225674650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692cb4d0f0 con 0x55692c79c390 2024-05-21T15:25:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d3e333640 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb4ca80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0d30010b20 tx=0x7f0d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb69d80bd6610aa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d217fa640 1 -- 172.21.15.78:0/2225674650 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30019040 con 0x55692c79c390 2024-05-21T15:25:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d217fa640 1 -- 172.21.15.78:0/2225674650 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f0d3002b050 con 0x55692c79c390 2024-05-21T15:25:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d46c42740 1 -- 172.21.15.78:0/2225674650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d30022040 con 0x55692c79c390 2024-05-21T15:25:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d217fa640 1 -- 172.21.15.78:0/2225674650 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0d30013ac0 con 0x55692c79c390 2024-05-21T15:25:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d46c42740 10 client.18273 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d46c42740 1 -- 172.21.15.78:0/2225674650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55692cbbf990 con 0x55692c79c390 2024-05-21T15:25:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d217fa640 1 -- 172.21.15.78:0/2225674650 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30022040 con 0x55692c79c390 2024-05-21T15:25:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.773+0000 7f0d217fa640 1 -- 172.21.15.78:0/2225674650 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0d30030560 con 0x55692c79c390 2024-05-21T15:25:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d46c42740 10 client.18273 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d46c42740 10 client.18273 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:25:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d46c42740 20 client.18273 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55692c7303a0 0x55692cbc1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d46c42740 4 client.18273 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d46c42740 1 -- 172.21.15.78:0/2225674650 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55692caa86c0 con 0x55692c7303a0 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d3db32640 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55692c7303a0 0x55692cbc1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d217fa640 10 client.18273 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d3db32640 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55692c7303a0 0x55692cbc1f10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7ab40cfb9adf5b7 server_cookie=c3915c35ea37610e in_seq=0 out_seq=0 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d217fa640 1 -- 172.21.15.78:0/2225674650 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55692caa86c0 con 0x55692c7303a0 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d217fa640 10 client.18273 handle_command_reply: tid=0 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.774+0000 7f0d46c42740 1 client.18273 shutdown 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 -- 172.21.15.78:0/2225674650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 msgr2=0x55692cb4ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb4ca80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0d30010b20 tx=0x7f0d30004a40 comp rx=0 tx=0).stop 2024-05-21T15:25:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 -- 172.21.15.78:0/2225674650 shutdown_connections 2024-05-21T15:25:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55692c7303a0 0x55692cbc1f10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55692c77b6b0 0x55692cb51b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55692c79c390 0x55692cb4ca80 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 --2- 172.21.15.78:0/2225674650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692c7734d0 0x55692cb51630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d217fa640 0 client.18273 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:25:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 -- 172.21.15.78:0/2225674650 >> 172.21.15.78:0/2225674650 conn(0x55692c7904e0 msgr2=0x55692cb3cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 -- 172.21.15.78:0/2225674650 shutdown_connections 2024-05-21T15:25:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 1 -- 172.21.15.78:0/2225674650 wait complete. 2024-05-21T15:25:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d46c42740 20 client.18273 trim_cache size 0 max 16384 2024-05-21T15:25:32.794 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:25:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 -- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0d240a2d00 msgr2=0x7f0d240a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0d240a2d00 0x7f0d240a5210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d30021000 tx=0x7f0d3001f000 comp rx=0 tx=0).stop 2024-05-21T15:25:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 -- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38108ad0 msgr2=0x7f0d38199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38108ad0 0x7f0d38199050 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400e9c0 tx=0x7f0d3400ee80 comp rx=0 tx=0).stop 2024-05-21T15:25:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 -- 172.21.15.78:0/439222232 shutdown_connections 2024-05-21T15:25:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d3819ddb0 0x7f0d3819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0d240a2d00 0x7f0d240a5210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d38199590 0x7f0d3819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 --2- 172.21.15.78:0/439222232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d38108ad0 0x7f0d38199050 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.775+0000 7f0d3fd96640 1 -- 172.21.15.78:0/439222232 >> 172.21.15.78:0/439222232 conn(0x7f0d38076830 msgr2=0x7f0d3810ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.776+0000 7f0d3fd96640 1 -- 172.21.15.78:0/439222232 shutdown_connections 2024-05-21T15:25:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.776+0000 7f0d3fd96640 1 -- 172.21.15.78:0/439222232 wait complete. 2024-05-21T15:25:32.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:25:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:32 smithi084 ceph-mon[36334]: pgmap v292: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:32.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:32 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2161436051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:25:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 -- 172.21.15.78:0/1051802160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c1079d0 msgr2=0x7f5a6c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1051802160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c1079d0 0x7f5a6c101560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009860 tx=0x7f5a5c02a340 comp rx=0 tx=0).stop 2024-05-21T15:25:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 -- 172.21.15.78:0/1051802160 shutdown_connections 2024-05-21T15:25:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1051802160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a6c101aa0 0x7f5a6c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1051802160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6c10c330 0x7f5a6c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1051802160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c1079d0 0x7f5a6c101560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 -- 172.21.15.78:0/1051802160 >> 172.21.15.78:0/1051802160 conn(0x7f5a6c0770b0 msgr2=0x7f5a6c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 -- 172.21.15.78:0/1051802160 shutdown_connections 2024-05-21T15:25:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.908+0000 7f5a729aa640 1 -- 172.21.15.78:0/1051802160 wait complete. 2024-05-21T15:25:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a729aa640 1 Processor -- start 2024-05-21T15:25:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a729aa640 1 -- start start 2024-05-21T15:25:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a729aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6c101aa0 0x7f5a6c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a729aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c1079d0 0x7f5a6c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a729aa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a6c10c330 0x7f5a6c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a729aa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c111ce0 con 0x7f5a6c101aa0 2024-05-21T15:25:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a729aa640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c111b60 con 0x7f5a6c10c330 2024-05-21T15:25:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a729aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c111e60 con 0x7f5a6c1079d0 2024-05-21T15:25:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.909+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6c101aa0 0x7f5a6c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6c101aa0 0x7f5a6c198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41742/0 (socket says 172.21.15.78:41742) 2024-05-21T15:25:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c1079d0 0x7f5a6c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6bfff640 1 -- 172.21.15.78:0/1912868862 learned_addr learned my addr 172.21.15.78:0/1912868862 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a70f47640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a6c10c330 0x7f5a6c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6bfff640 1 -- 172.21.15.78:0/1912868862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c1079d0 msgr2=0x7f5a6c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c1079d0 0x7f5a6c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6bfff640 1 -- 172.21.15.78:0/1912868862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a6c10c330 msgr2=0x7f5a6c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a6c10c330 0x7f5a6c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6bfff640 1 -- 172.21.15.78:0/1912868862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c19d4c0 con 0x7f5a6c101aa0 2024-05-21T15:25:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6c101aa0 0x7f5a6c198660 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c000c00 tx=0x7f5a5c033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a697fa640 1 -- 172.21.15.78:0/1912868862 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c03f020 con 0x7f5a6c101aa0 2024-05-21T15:25:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a729aa640 1 -- 172.21.15.78:0/1912868862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c19d750 con 0x7f5a6c101aa0 2024-05-21T15:25:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a697fa640 1 -- 172.21.15.78:0/1912868862 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a5c032770 con 0x7f5a6c101aa0 2024-05-21T15:25:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a697fa640 1 -- 172.21.15.78:0/1912868862 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c037040 con 0x7f5a6c101aa0 2024-05-21T15:25:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.910+0000 7f5a729aa640 1 -- 172.21.15.78:0/1912868862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c19dc40 con 0x7f5a6c101aa0 2024-05-21T15:25:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a697fa640 1 -- 172.21.15.78:0/1912868862 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5a5c042020 con 0x7f5a6c101aa0 2024-05-21T15:25:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a79852740 1 Processor -- start 2024-05-21T15:25:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a79852740 1 -- start start 2024-05-21T15:25:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a79852740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b6383d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a79852740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b6384260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a79852740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a79852740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5559b62e9c40 con 0x5559b5fe5390 2024-05-21T15:25:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a79852740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5559b6291580 con 0x5559b5fc46b0 2024-05-21T15:25:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a79852740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559b62e5c00 con 0x5559b5fbc4d0 2024-05-21T15:25:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a70f47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b6383d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b6384260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a697fa640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a540a2d00 0x7f5a540a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63847a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:49220/0 (socket says 172.21.15.78:49220) 2024-05-21T15:25:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.913+0000 7f5a6b7fe640 1 -- 172.21.15.78:0/2509498158 learned_addr learned my addr 172.21.15.78:0/2509498158 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a70f47640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a540a2d00 0x7f5a540a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a6b7fe640 1 -- 172.21.15.78:0/2509498158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 msgr2=0x5559b6384260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/2509498158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b6384260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a6b7fe640 1 -- 172.21.15.78:0/2509498158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 msgr2=0x5559b6383d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/2509498158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b6383d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a697fa640 1 -- 172.21.15.78:0/1912868862 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f5a5c0ef450 con 0x7f5a6c101aa0 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a6b7fe640 1 -- 172.21.15.78:0/2509498158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b5ffd3f0 con 0x5559b5fc46b0 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/2509498158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63847a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009c60 tx=0x7f5a60011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b145a273e06b089 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a70f47640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a540a2d00 0x7f5a540a5210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5559b6380e90 tx=0x7f5a64004ab0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a527fc640 1 -- 172.21.15.78:0/2509498158 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6001e070 con 0x5559b5fc46b0 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a527fc640 1 -- 172.21.15.78:0/2509498158 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60009e10 con 0x5559b5fc46b0 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.914+0000 7f5a527fc640 1 -- 172.21.15.78:0/2509498158 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600196b0 con 0x5559b5fc46b0 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 -- 172.21.15.78:0/2509498158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 msgr2=0x5559b63847a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 --2- 172.21.15.78:0/2509498158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63847a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009c60 tx=0x7f5a60011810 comp rx=0 tx=0).stop 2024-05-21T15:25:32.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 -- 172.21.15.78:0/2509498158 shutdown_connections 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 --2- 172.21.15.78:0/2509498158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63847a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 --2- 172.21.15.78:0/2509498158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b6384260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 --2- 172.21.15.78:0/2509498158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b6383d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 -- 172.21.15.78:0/2509498158 >> 172.21.15.78:0/2509498158 conn(0x5559b5fd94e0 msgr2=0x5559b63804d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 -- 172.21.15.78:0/2509498158 shutdown_connections 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 -- 172.21.15.78:0/2509498158 wait complete. 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.915+0000 7f5a79852740 1 -- start start 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a79852740 1 Processor -- start 2024-05-21T15:25:32.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a79852740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b638e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a79852740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b638e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a79852740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fe5390 0x5559b63944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a79852740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5559b6390fb0 con 0x5559b5fbc4d0 2024-05-21T15:25:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a79852740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5559b62a4c40 con 0x5559b5fc46b0 2024-05-21T15:25:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a79852740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559b6391130 con 0x5559b5fe5390 2024-05-21T15:25:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a79852740 10 client.0 _ll_register_callbacks cb 0x5559b5ffcbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b638e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b638e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41766/0 (socket says 172.21.15.78:41766) 2024-05-21T15:25:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b638e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6bfff640 1 -- 172.21.15.78:0/4153652460 learned_addr learned my addr 172.21.15.78:0/4153652460 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a70f47640 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fe5390 0x5559b63944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6b7fe640 1 -- 172.21.15.78:0/4153652460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fe5390 msgr2=0x5559b63944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fe5390 0x5559b63944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6b7fe640 1 -- 172.21.15.78:0/4153652460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 msgr2=0x5559b638e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b638e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.916+0000 7f5a6b7fe640 1 -- 172.21.15.78:0/4153652460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b60c77d0 con 0x5559b5fc46b0 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b638e900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009230 tx=0x7f5a600056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91b24bd25a90677 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/4153652460 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6001e040 con 0x5559b5fc46b0 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/4153652460 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f5a60016040 con 0x5559b5fc46b0 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 1 -- 172.21.15.78:0/4153652460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a60009e10 con 0x5559b5fc46b0 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/4153652460 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5a60031050 con 0x5559b5fc46b0 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 10 client.27431 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 1 -- 172.21.15.78:0/4153652460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5559b639d790 con 0x5559b5fc46b0 2024-05-21T15:25:32.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/4153652460 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60009e10 con 0x5559b5fc46b0 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/4153652460 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5a60003cf0 con 0x5559b5fc46b0 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 10 client.27431 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 10 client.27431 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 20 client.27431 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5559b5fa3f60 0x5559b63a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 4 client.27431 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.917+0000 7f5a79852740 1 -- 172.21.15.78:0/4153652460 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559b62f7370 con 0x5559b5fa3f60 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.918+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5559b5fa3f60 0x5559b63a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.918+0000 7f5a4b7fe640 10 client.27431 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:25:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.918+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5559b5fa3f60 0x5559b63a0af0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a2cc4345959edf4 server_cookie=ad15bde428774ea3 in_seq=0 out_seq=0 2024-05-21T15:25:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/4153652460 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5559b62f7370 con 0x5559b5fa3f60 2024-05-21T15:25:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a4b7fe640 10 client.27431 handle_command_reply: tid=0 2024-05-21T15:25:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 client.27431 shutdown 2024-05-21T15:25:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 -- 172.21.15.78:0/4153652460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 msgr2=0x5559b638e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b638e900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009230 tx=0x7f5a600056a0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 -- 172.21.15.78:0/4153652460 shutdown_connections 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5559b5fa3f60 0x5559b63a0af0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b638e900 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fe5390 0x5559b63944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 --2- 172.21.15.78:0/4153652460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b638e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a4b7fe640 0 client.27431 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 -- 172.21.15.78:0/4153652460 >> 172.21.15.78:0/4153652460 conn(0x5559b5fd94e0 msgr2=0x5559b6385a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 -- 172.21.15.78:0/4153652460 shutdown_connections 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 1 -- 172.21.15.78:0/4153652460 wait complete. 2024-05-21T15:25:32.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.919+0000 7f5a79852740 20 client.27431 trim_cache size 0 max 16384 2024-05-21T15:25:32.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:32 smithi172 ceph-mon[36402]: pgmap v292: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:32.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:32 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2161436051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:25:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a79852740 1 Processor -- start 2024-05-21T15:25:32.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a79852740 1 -- start start 2024-05-21T15:25:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a79852740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b638da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a79852740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63a6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a79852740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b63a6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a79852740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1aca80 con 0x5559b5fe5390 2024-05-21T15:25:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a79852740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ae5c0 con 0x5559b5fc46b0 2024-05-21T15:25:32.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a79852740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ad150 con 0x5559b5fbc4d0 2024-05-21T15:25:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a70f47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b63a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a70f47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b63a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41774/0 (socket says 172.21.15.78:41774) 2024-05-21T15:25:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a70f47640 1 -- 172.21.15.78:0/2651728298 learned_addr learned my addr 172.21.15.78:0/2651728298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b638da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.933+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a70f47640 1 -- 172.21.15.78:0/2651728298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 msgr2=0x5559b638da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a70f47640 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b638da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a70f47640 1 -- 172.21.15.78:0/2651728298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 msgr2=0x5559b63a6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a70f47640 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63a6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a70f47640 1 -- 172.21.15.78:0/2651728298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b6383ab0 con 0x5559b5fe5390 2024-05-21T15:25:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b638da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a70f47640 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b63a6690 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5a6401cde0 tx=0x7f5a6400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c583470ac550439f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a51ffb640 1 -- 172.21.15.78:0/2651728298 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a640045b0 con 0x5559b5fe5390 2024-05-21T15:25:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a51ffb640 1 -- 172.21.15.78:0/2651728298 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a64004750 con 0x5559b5fe5390 2024-05-21T15:25:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.934+0000 7f5a51ffb640 1 -- 172.21.15.78:0/2651728298 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64020740 con 0x5559b5fe5390 2024-05-21T15:25:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 -- 172.21.15.78:0/2651728298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 msgr2=0x5559b63a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b63a6690 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5a6401cde0 tx=0x7f5a6400f000 comp rx=0 tx=0).stop 2024-05-21T15:25:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 -- 172.21.15.78:0/2651728298 shutdown_connections 2024-05-21T15:25:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fc46b0 0x5559b63a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fe5390 0x5559b63a6690 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 --2- 172.21.15.78:0/2651728298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fbc4d0 0x5559b638da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 -- 172.21.15.78:0/2651728298 >> 172.21.15.78:0/2651728298 conn(0x5559b5fd94e0 msgr2=0x5559b6389360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 -- 172.21.15.78:0/2651728298 shutdown_connections 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 -- 172.21.15.78:0/2651728298 wait complete. 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 -- start start 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.935+0000 7f5a79852740 1 Processor -- start 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a79852740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b6395350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a79852740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fc46b0 0x5559b6395890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a79852740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fe5390 0x5559b6395dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a79852740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1adef0 con 0x5559b5fbc4d0 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a79852740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ad820 con 0x5559b5fe5390 2024-05-21T15:25:32.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a79852740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1aaa80 con 0x5559b5fc46b0 2024-05-21T15:25:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a79852740 10 client.0 _ll_register_callbacks cb 0x7f5a6001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b6395350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a70f47640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fe5390 0x5559b6395dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b6395350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41786/0 (socket says 172.21.15.78:41786) 2024-05-21T15:25:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 -- 172.21.15.78:0/918959508 learned_addr learned my addr 172.21.15.78:0/918959508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fc46b0 0x5559b6395890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 -- 172.21.15.78:0/918959508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fc46b0 msgr2=0x5559b6395890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fc46b0 0x5559b6395890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 -- 172.21.15.78:0/918959508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fe5390 msgr2=0x5559b6395dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fe5390 0x5559b6395dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 -- 172.21.15.78:0/918959508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b6406cf0 con 0x5559b5fbc4d0 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a6bfff640 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b6395350 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0eccd0 tx=0x7f5a5c0b8190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c23a7b3c143895 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.936+0000 7f5a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/918959508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c005d20 con 0x5559b5fbc4d0 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/918959508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f5a5c004240 con 0x5559b5fbc4d0 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 1 -- 172.21.15.78:0/918959508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a64004750 con 0x5559b5fbc4d0 2024-05-21T15:25:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/918959508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5a5c0bc0f0 con 0x5559b5fbc4d0 2024-05-21T15:25:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 10 client.18291 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 1 -- 172.21.15.78:0/918959508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5559b6407f40 con 0x5559b5fbc4d0 2024-05-21T15:25:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/918959508 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a64004750 con 0x5559b5fbc4d0 2024-05-21T15:25:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/918959508 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5a5c03a030 con 0x5559b5fbc4d0 2024-05-21T15:25:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 10 client.18291 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 10 client.18291 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:25:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 20 client.18291 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5559b5f793a0 0x5559b640a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 4 client.18291 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:25:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a79852740 1 -- 172.21.15.78:0/918959508 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559b62f16c0 con 0x5559b5f793a0 2024-05-21T15:25:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.937+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5559b5f793a0 0x5559b640a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a4b7fe640 10 client.18291 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:25:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a6b7fe640 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5559b5f793a0 0x5559b640a650 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c9ae9576a308ef8 server_cookie=456a0f29566baaa6 in_seq=0 out_seq=0 2024-05-21T15:25:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a4b7fe640 1 -- 172.21.15.78:0/918959508 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559b62f16c0 con 0x5559b5f793a0 2024-05-21T15:25:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a4b7fe640 10 client.18291 handle_command_reply: tid=0 2024-05-21T15:25:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 client.18291 shutdown 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 -- 172.21.15.78:0/918959508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 msgr2=0x5559b6395350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b6395350 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0eccd0 tx=0x7f5a5c0b8190 comp rx=0 tx=0).stop 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 -- 172.21.15.78:0/918959508 shutdown_connections 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x5559b5f793a0 0x5559b640a650 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5559b5fe5390 0x5559b6395dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559b5fc46b0 0x5559b6395890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a4b7fe640 0 client.18291 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 --2- 172.21.15.78:0/918959508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559b5fbc4d0 0x5559b6395350 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 -- 172.21.15.78:0/918959508 >> 172.21.15.78:0/918959508 conn(0x5559b5fd94e0 msgr2=0x5559b6383cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 -- 172.21.15.78:0/918959508 shutdown_connections 2024-05-21T15:25:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 1 -- 172.21.15.78:0/918959508 wait complete. 2024-05-21T15:25:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.938+0000 7f5a79852740 20 client.18291 trim_cache size 0 max 16384 2024-05-21T15:25:32.953 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:25:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 -- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a540a2d00 msgr2=0x7f5a540a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a540a2d00 0x7f5a540a5210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5559b6380e90 tx=0x7f5a64004ab0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 -- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6c101aa0 msgr2=0x7f5a6c198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6c101aa0 0x7f5a6c198660 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c000c00 tx=0x7f5a5c033d10 comp rx=0 tx=0).stop 2024-05-21T15:25:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 -- 172.21.15.78:0/1912868862 shutdown_connections 2024-05-21T15:25:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a6c10c330 0x7f5a6c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a540a2d00 0x7f5a540a5210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a6c1079d0 0x7f5a6c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 --2- 172.21.15.78:0/1912868862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6c101aa0 0x7f5a6c198660 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 -- 172.21.15.78:0/1912868862 >> 172.21.15.78:0/1912868862 conn(0x7f5a6c0770b0 msgr2=0x7f5a6c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 -- 172.21.15.78:0/1912868862 shutdown_connections 2024-05-21T15:25:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:32.939+0000 7f5a729aa640 1 -- 172.21.15.78:0/1912868862 wait complete. 2024-05-21T15:25:32.956 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:25:33.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 -- 172.21.15.78:0/3749119344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f115410c330 msgr2=0x7f115410e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 --2- 172.21.15.78:0/3749119344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f115410c330 0x7f115410e750 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f114c009ca0 tx=0x7f114c02a8c0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 -- 172.21.15.78:0/3749119344 shutdown_connections 2024-05-21T15:25:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 --2- 172.21.15.78:0/3749119344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f115410c330 0x7f115410e750 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 --2- 172.21.15.78:0/3749119344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1154101aa0 0x7f1154101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 --2- 172.21.15.78:0/3749119344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11541079d0 0x7f1154101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 -- 172.21.15.78:0/3749119344 >> 172.21.15.78:0/3749119344 conn(0x7f11540770b0 msgr2=0x7f11540774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 -- 172.21.15.78:0/3749119344 shutdown_connections 2024-05-21T15:25:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.067+0000 7f1159d4c640 1 -- 172.21.15.78:0/3749119344 wait complete. 2024-05-21T15:25:33.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1159d4c640 1 Processor -- start 2024-05-21T15:25:33.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1159d4c640 1 -- start start 2024-05-21T15:25:33.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1159d4c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1154101aa0 0x7f11541985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1159d4c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11541079d0 0x7f1154198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1159d4c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f115410c330 0x7f115419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1159d4c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1154111d30 con 0x7f11541079d0 2024-05-21T15:25:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1159d4c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1154111bb0 con 0x7f115410c330 2024-05-21T15:25:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1159d4c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1154111eb0 con 0x7f1154101aa0 2024-05-21T15:25:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1152ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11541079d0 0x7f1154198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1152ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11541079d0 0x7f1154198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41808/0 (socket says 172.21.15.78:41808) 2024-05-21T15:25:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1152ffd640 1 -- 172.21.15.78:0/1821775508 learned_addr learned my addr 172.21.15.78:0/1821775508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f11537fe640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1154101aa0 0x7f11541985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.068+0000 7f1153fff640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f115410c330 0x7f115419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1152ffd640 1 -- 172.21.15.78:0/1821775508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1154101aa0 msgr2=0x7f11541985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1152ffd640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1154101aa0 0x7f11541985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1152ffd640 1 -- 172.21.15.78:0/1821775508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f115410c330 msgr2=0x7f115419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1152ffd640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f115410c330 0x7f115419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1152ffd640 1 -- 172.21.15.78:0/1821775508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115419d450 con 0x7f11541079d0 2024-05-21T15:25:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1153fff640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f115410c330 0x7f115419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1152ffd640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11541079d0 0x7f1154198b50 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f113c00d740 tx=0x7f113c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1150ff9640 1 -- 172.21.15.78:0/1821775508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c017070 con 0x7f11541079d0 2024-05-21T15:25:33.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1150ff9640 1 -- 172.21.15.78:0/1821775508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c00de50 con 0x7f11541079d0 2024-05-21T15:25:33.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1159d4c640 1 -- 172.21.15.78:0/1821775508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115419d740 con 0x7f11541079d0 2024-05-21T15:25:33.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1150ff9640 1 -- 172.21.15.78:0/1821775508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c003e00 con 0x7f11541079d0 2024-05-21T15:25:33.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.069+0000 7f1159d4c640 1 -- 172.21.15.78:0/1821775508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a50c0 con 0x7f11541079d0 2024-05-21T15:25:33.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1160bfc740 1 Processor -- start 2024-05-21T15:25:33.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1160bfc740 1 -- start start 2024-05-21T15:25:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1150ff9640 1 -- 172.21.15.78:0/1821775508 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f113c004020 con 0x7f11541079d0 2024-05-21T15:25:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c087b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c0880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c0885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1160bfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f71bfedc40 con 0x55f71bce9390 2024-05-21T15:25:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1160bfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f71bf95580 con 0x55f71bcc04d0 2024-05-21T15:25:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1160bfc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f71bfe9c00 con 0x55f71bcc86b0 2024-05-21T15:25:33.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1153fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c087b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1153fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c087b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41814/0 (socket says 172.21.15.78:41814) 2024-05-21T15:25:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1153fff640 1 -- 172.21.15.78:0/2208349224 learned_addr learned my addr 172.21.15.78:0/2208349224 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f1152ffd640 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c0885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.072+0000 7f11537fe640 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c0880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1153fff640 1 -- 172.21.15.78:0/2208349224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 msgr2=0x55f71c0885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1153fff640 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c0885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1153fff640 1 -- 172.21.15.78:0/2208349224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 msgr2=0x55f71c0880b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:33.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1153fff640 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c0880b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1153fff640 1 -- 172.21.15.78:0/2208349224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71bd013f0 con 0x55f71bce9390 2024-05-21T15:25:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1153fff640 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c087b70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f114c009ca0 tx=0x7f114c0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16230282509ba6a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1150ff9640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11400a2d00 0x7f11400a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1149ffb640 1 -- 172.21.15.78:0/2208349224 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c003970 con 0x55f71bce9390 2024-05-21T15:25:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1149ffb640 1 -- 172.21.15.78:0/2208349224 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c036040 con 0x55f71bce9390 2024-05-21T15:25:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1149ffb640 1 -- 172.21.15.78:0/2208349224 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c004450 con 0x55f71bce9390 2024-05-21T15:25:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f11537fe640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11400a2d00 0x7f11400a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.073+0000 7f1150ff9640 1 -- 172.21.15.78:0/1821775508 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f113c0cee90 con 0x7f11541079d0 2024-05-21T15:25:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 -- 172.21.15.78:0/2208349224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 msgr2=0x55f71c087b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f11537fe640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11400a2d00 0x7f11400a5210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f11541029d0 tx=0x7f1144005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c087b70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f114c009ca0 tx=0x7f114c0094a0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 -- 172.21.15.78:0/2208349224 shutdown_connections 2024-05-21T15:25:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c0880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c0885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2208349224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c087b70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 -- 172.21.15.78:0/2208349224 >> 172.21.15.78:0/2208349224 conn(0x55f71bcdd4e0 msgr2=0x55f71c0844d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 -- 172.21.15.78:0/2208349224 shutdown_connections 2024-05-21T15:25:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 -- 172.21.15.78:0/2208349224 wait complete. 2024-05-21T15:25:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.074+0000 7f1160bfc740 1 -- start start 2024-05-21T15:25:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1160bfc740 1 Processor -- start 2024-05-21T15:25:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c094500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc86b0 0x55f71c094a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bce9390 0x55f71c0a0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1160bfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f71c097250 con 0x55f71bcc04d0 2024-05-21T15:25:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1160bfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f71bfa8c40 con 0x55f71bcc86b0 2024-05-21T15:25:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1160bfc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f71c0973d0 con 0x55f71bce9390 2024-05-21T15:25:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1160bfc740 10 client.0 _ll_register_callbacks cb 0x55f71bd00be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f11537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c094500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f11537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c094500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41822/0 (socket says 172.21.15.78:41822) 2024-05-21T15:25:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f11537fe640 1 -- 172.21.15.78:0/3501331112 learned_addr learned my addr 172.21.15.78:0/3501331112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1152ffd640 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc86b0 0x55f71c094a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1153fff640 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bce9390 0x55f71c0a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f11537fe640 1 -- 172.21.15.78:0/3501331112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bce9390 msgr2=0x55f71c0a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f11537fe640 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bce9390 0x55f71c0a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f11537fe640 1 -- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc86b0 msgr2=0x55f71c094a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f11537fe640 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc86b0 0x55f71c094a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f11537fe640 1 -- 172.21.15.78:0/3501331112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f71bdcb7d0 con 0x55f71bcc04d0 2024-05-21T15:25:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.075+0000 7f1152ffd640 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc86b0 0x55f71c094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f11537fe640 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c094500 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f11440045f0 tx=0x7f1144004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c087d0ff274bba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1132ffd640 1 -- 172.21.15.78:0/3501331112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114401f640 con 0x55f71bcc04d0 2024-05-21T15:25:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1132ffd640 1 -- 172.21.15.78:0/3501331112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f114401f7e0 con 0x55f71bcc04d0 2024-05-21T15:25:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 1 -- 172.21.15.78:0/3501331112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c036040 con 0x55f71bcc04d0 2024-05-21T15:25:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1132ffd640 1 -- 172.21.15.78:0/3501331112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f114401f9e0 con 0x55f71bcc04d0 2024-05-21T15:25:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 10 client.18303 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:33.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 1 -- 172.21.15.78:0/3501331112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f71c0a3970 con 0x55f71bcc04d0 2024-05-21T15:25:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1132ffd640 1 -- 172.21.15.78:0/3501331112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c036040 con 0x55f71bcc04d0 2024-05-21T15:25:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1132ffd640 1 -- 172.21.15.78:0/3501331112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1144027070 con 0x55f71bcc04d0 2024-05-21T15:25:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 10 client.18303 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 20 client.18303 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f71bca7f60 0x55f71c0a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 4 client.18303 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.076+0000 7f1160bfc740 1 -- 172.21.15.78:0/3501331112 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f71bffb370 con 0x55f71bca7f60 2024-05-21T15:25:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.077+0000 7f1152ffd640 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f71bca7f60 0x55f71c0a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.077+0000 7f1152ffd640 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f71bca7f60 0x55f71c0a6f10 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9eda31139269800 server_cookie=fc543c0c8b179613 in_seq=0 out_seq=0 2024-05-21T15:25:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.077+0000 7f1132ffd640 10 client.18303 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1132ffd640 1 -- 172.21.15.78:0/3501331112 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f71bffb370 con 0x55f71bca7f60 2024-05-21T15:25:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1132ffd640 10 client.18303 handle_command_reply: tid=0 2024-05-21T15:25:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 client.18303 shutdown 2024-05-21T15:25:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 -- 172.21.15.78:0/3501331112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 msgr2=0x55f71c094500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c094500 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f11440045f0 tx=0x7f1144004730 comp rx=0 tx=0).stop 2024-05-21T15:25:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 -- 172.21.15.78:0/3501331112 shutdown_connections 2024-05-21T15:25:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f71bca7f60 0x55f71c0a6f10 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc86b0 0x55f71c094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bce9390 0x55f71c0a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1132ffd640 0 client.18303 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3501331112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c094500 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 -- 172.21.15.78:0/3501331112 >> 172.21.15.78:0/3501331112 conn(0x55f71bcdd4e0 msgr2=0x55f71c0884e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 -- 172.21.15.78:0/3501331112 shutdown_connections 2024-05-21T15:25:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 1 -- 172.21.15.78:0/3501331112 wait complete. 2024-05-21T15:25:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.078+0000 7f1160bfc740 20 client.18303 trim_cache size 0 max 16384 2024-05-21T15:25:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.091+0000 7f1160bfc740 1 Processor -- start 2024-05-21T15:25:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1160bfc740 1 -- start start 2024-05-21T15:25:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c0935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c096120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bce9390 0x55f71c0a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1160bfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ac8b0 con 0x55f71bcc04d0 2024-05-21T15:25:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1160bfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ae3f0 con 0x55f71bce9390 2024-05-21T15:25:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1160bfc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11541acf80 con 0x55f71bcc86b0 2024-05-21T15:25:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c0935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c0935f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41828/0 (socket says 172.21.15.78:41828) 2024-05-21T15:25:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 -- 172.21.15.78:0/3613346168 learned_addr learned my addr 172.21.15.78:0/3613346168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1153fff640 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bce9390 0x55f71c0a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f1152ffd640 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c096120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 -- 172.21.15.78:0/3613346168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 msgr2=0x55f71c096120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c096120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 -- 172.21.15.78:0/3613346168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bce9390 msgr2=0x55f71c0a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:33.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bce9390 0x55f71c0a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 -- 172.21.15.78:0/3613346168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71c0973c0 con 0x55f71bcc04d0 2024-05-21T15:25:33.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11537fe640 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c0935f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1144012cb0 tx=0x7f1144003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40fb8a1f80a00e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:33.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.092+0000 7f11497fa640 1 -- 172.21.15.78:0/3613346168 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114401f840 con 0x55f71bcc04d0 2024-05-21T15:25:33.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f11497fa640 1 -- 172.21.15.78:0/3613346168 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114401fb80 con 0x55f71bcc04d0 2024-05-21T15:25:33.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f11497fa640 1 -- 172.21.15.78:0/3613346168 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114401d800 con 0x55f71bcc04d0 2024-05-21T15:25:33.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 -- 172.21.15.78:0/3613346168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 msgr2=0x55f71c0935f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c0935f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1144012cb0 tx=0x7f1144003e50 comp rx=0 tx=0).stop 2024-05-21T15:25:33.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 -- 172.21.15.78:0/3613346168 shutdown_connections 2024-05-21T15:25:33.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bce9390 0x55f71c0a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c096120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 --2- 172.21.15.78:0/3613346168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bcc04d0 0x55f71c0935f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 -- 172.21.15.78:0/3613346168 >> 172.21.15.78:0/3613346168 conn(0x55f71bcdd4e0 msgr2=0x55f71c095a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:33.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 -- 172.21.15.78:0/3613346168 shutdown_connections 2024-05-21T15:25:33.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.093+0000 7f1160bfc740 1 -- 172.21.15.78:0/3613346168 wait complete. 2024-05-21T15:25:33.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 1 -- start start 2024-05-21T15:25:33.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 1 Processor -- start 2024-05-21T15:25:33.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c08a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c08ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c08b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11541add20 con 0x55f71bce9390 2024-05-21T15:25:33.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ad650 con 0x55f71bcc04d0 2024-05-21T15:25:33.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11541aa8b0 con 0x55f71bcc86b0 2024-05-21T15:25:33.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1160bfc740 10 client.0 _ll_register_callbacks cb 0x7f1144009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:33.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1153fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c08b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f11537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c08a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1153fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c08b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41842/0 (socket says 172.21.15.78:41842) 2024-05-21T15:25:33.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1152ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c08ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.094+0000 7f1153fff640 1 -- 172.21.15.78:0/2019057102 learned_addr learned my addr 172.21.15.78:0/2019057102 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:33.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f11537fe640 1 -- 172.21.15.78:0/2019057102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 msgr2=0x55f71c08ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:33.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f11537fe640 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c08ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f11537fe640 1 -- 172.21.15.78:0/2019057102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 msgr2=0x55f71c08b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:33.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f11537fe640 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c08b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f11537fe640 1 -- 172.21.15.78:0/2019057102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f71c09b7b0 con 0x55f71bcc04d0 2024-05-21T15:25:33.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1153fff640 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c08b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:33.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f11537fe640 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c08a910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1144009b80 tx=0x7f1144004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f17078c858619c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:33.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:25:33.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1132ffd640 1 -- 172.21.15.78:0/2019057102 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144040070 con 0x55f71bcc04d0 2024-05-21T15:25:33.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1132ffd640 1 -- 172.21.15.78:0/2019057102 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f1144005450 con 0x55f71bcc04d0 2024-05-21T15:25:33.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1160bfc740 1 -- 172.21.15.78:0/2019057102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114401fb80 con 0x55f71bcc04d0 2024-05-21T15:25:33.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1132ffd640 1 -- 172.21.15.78:0/2019057102 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f114404c050 con 0x55f71bcc04d0 2024-05-21T15:25:33.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1160bfc740 10 client.27455 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:33.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1160bfc740 1 -- 172.21.15.78:0/2019057102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f71c09ca10 con 0x55f71bcc04d0 2024-05-21T15:25:33.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.095+0000 7f1132ffd640 1 -- 172.21.15.78:0/2019057102 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114401fb80 con 0x55f71bcc04d0 2024-05-21T15:25:33.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1132ffd640 1 -- 172.21.15.78:0/2019057102 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1144038070 con 0x55f71bcc04d0 2024-05-21T15:25:33.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1160bfc740 10 client.27455 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:33.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1160bfc740 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:33.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1160bfc740 20 client.27455 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:33.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f71bc7d3a0 0x55f71c113690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:33.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1160bfc740 4 client.27455 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:25:33.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1160bfc740 1 -- 172.21.15.78:0/2019057102 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f71bff56c0 con 0x55f71bc7d3a0 2024-05-21T15:25:33.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1152ffd640 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f71bc7d3a0 0x55f71c113690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:33.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1132ffd640 10 client.27455 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1152ffd640 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f71bc7d3a0 0x55f71c113690 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1666697a6194858a server_cookie=6b4a4578850674a1 in_seq=0 out_seq=0 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1132ffd640 1 -- 172.21.15.78:0/2019057102 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f71bff56c0 con 0x55f71bc7d3a0 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1132ffd640 10 client.27455 handle_command_reply: tid=0 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.096+0000 7f1160bfc740 1 client.27455 shutdown 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 -- 172.21.15.78:0/2019057102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 msgr2=0x55f71c08a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c08a910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1144009b80 tx=0x7f1144004450 comp rx=0 tx=0).stop 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 -- 172.21.15.78:0/2019057102 shutdown_connections 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55f71bc7d3a0 0x55f71c113690 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f71bce9390 0x55f71c08b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1132ffd640 0 client.27455 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:33.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f71bcc86b0 0x55f71c08ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 --2- 172.21.15.78:0/2019057102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f71bcc04d0 0x55f71c08a910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 -- 172.21.15.78:0/2019057102 >> 172.21.15.78:0/2019057102 conn(0x55f71bcdd4e0 msgr2=0x55f71c093350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:33.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 -- 172.21.15.78:0/2019057102 shutdown_connections 2024-05-21T15:25:33.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 1 -- 172.21.15.78:0/2019057102 wait complete. 2024-05-21T15:25:33.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1160bfc740 20 client.27455 trim_cache size 0 max 16384 2024-05-21T15:25:33.113 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:25:33.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 -- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11400a2d00 msgr2=0x7f11400a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11400a2d00 0x7f11400a5210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f11541029d0 tx=0x7f1144005f90 comp rx=0 tx=0).stop 2024-05-21T15:25:33.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 -- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11541079d0 msgr2=0x7f1154198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11541079d0 0x7f1154198b50 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f113c00d740 tx=0x7f113c00dc00 comp rx=0 tx=0).stop 2024-05-21T15:25:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 -- 172.21.15.78:0/1821775508 shutdown_connections 2024-05-21T15:25:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f115410c330 0x7f115419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f11400a2d00 0x7f11400a5210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11541079d0 0x7f1154198b50 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 --2- 172.21.15.78:0/1821775508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1154101aa0 0x7f11541985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 -- 172.21.15.78:0/1821775508 >> 172.21.15.78:0/1821775508 conn(0x7f11540770b0 msgr2=0x7f1154103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 -- 172.21.15.78:0/1821775508 shutdown_connections 2024-05-21T15:25:33.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:33.097+0000 7f1159d4c640 1 -- 172.21.15.78:0/1821775508 wait complete. 2024-05-21T15:25:34.116 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:25:34.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:25:34.117 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:25:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.229+0000 7f00df321640 1 -- 172.21.15.78:0/1632658085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d81079d0 msgr2=0x7f00d8101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.229+0000 7f00df321640 1 --2- 172.21.15.78:0/1632658085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d81079d0 0x7f00d8101560 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009900 tx=0x7f00d402a410 comp rx=0 tx=0).stop 2024-05-21T15:25:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.229+0000 7f00df321640 1 -- 172.21.15.78:0/1632658085 shutdown_connections 2024-05-21T15:25:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.229+0000 7f00df321640 1 --2- 172.21.15.78:0/1632658085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d810c330 0x7f00d810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.229+0000 7f00df321640 1 --2- 172.21.15.78:0/1632658085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00d8101aa0 0x7f00d8101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.229+0000 7f00df321640 1 --2- 172.21.15.78:0/1632658085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d81079d0 0x7f00d8101560 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.229+0000 7f00df321640 1 -- 172.21.15.78:0/1632658085 >> 172.21.15.78:0/1632658085 conn(0x7f00d80770b0 msgr2=0x7f00d80774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.230+0000 7f00df321640 1 -- 172.21.15.78:0/1632658085 shutdown_connections 2024-05-21T15:25:34.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.230+0000 7f00df321640 1 -- 172.21.15.78:0/1632658085 wait complete. 2024-05-21T15:25:34.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.230+0000 7f00df321640 1 Processor -- start 2024-05-21T15:25:34.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.230+0000 7f00df321640 1 -- start start 2024-05-21T15:25:34.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.230+0000 7f00df321640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d8101aa0 0x7f00d819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00df321640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d81079d0 0x7f00d819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00df321640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00d810c330 0x7f00d819b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00df321640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8111ce0 con 0x7f00d8101aa0 2024-05-21T15:25:34.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00df321640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8111b60 con 0x7f00d81079d0 2024-05-21T15:25:34.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00df321640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8111e60 con 0x7f00d810c330 2024-05-21T15:25:34.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d8101aa0 0x7f00d819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d8101aa0 0x7f00d819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41870/0 (socket says 172.21.15.78:41870) 2024-05-21T15:25:34.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 -- 172.21.15.78:0/3805934665 learned_addr learned my addr 172.21.15.78:0/3805934665 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd8be640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00d810c330 0x7f00d819b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dc8bc640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d81079d0 0x7f00d819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 -- 172.21.15.78:0/3805934665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00d810c330 msgr2=0x7f00d819b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00d810c330 0x7f00d819b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 -- 172.21.15.78:0/3805934665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d81079d0 msgr2=0x7f00d819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d81079d0 0x7f00d819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 -- 172.21.15.78:0/3805934665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d81a5910 con 0x7f00d8101aa0 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d8101aa0 0x7f00d819a600 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f00d4005940 tx=0x7f00d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00ce7fc640 1 -- 172.21.15.78:0/3805934665 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d403f030 con 0x7f00d8101aa0 2024-05-21T15:25:34.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00ce7fc640 1 -- 172.21.15.78:0/3805934665 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d4003ad0 con 0x7f00d8101aa0 2024-05-21T15:25:34.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00ce7fc640 1 -- 172.21.15.78:0/3805934665 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4035070 con 0x7f00d8101aa0 2024-05-21T15:25:34.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00df321640 1 -- 172.21.15.78:0/3805934665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d81a5ba0 con 0x7f00d8101aa0 2024-05-21T15:25:34.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.231+0000 7f00df321640 1 -- 172.21.15.78:0/3805934665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d81a6090 con 0x7f00d8101aa0 2024-05-21T15:25:34.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.234+0000 7f00ce7fc640 1 -- 172.21.15.78:0/3805934665 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f00d4042030 con 0x7f00d8101aa0 2024-05-21T15:25:34.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.234+0000 7f00e61c7740 1 Processor -- start 2024-05-21T15:25:34.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.234+0000 7f00e61c7740 1 -- start start 2024-05-21T15:25:34.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.234+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e6078fb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.234+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 0x55e6078fbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.234+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e6078fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.234+0000 7f00e61c7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e607860990 con 0x55e60755c410 2024-05-21T15:25:34.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.234+0000 7f00e61c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e6078085b0 con 0x55e607533550 2024-05-21T15:25:34.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00e61c7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e60785c950 con 0x55e60753b730 2024-05-21T15:25:34.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e6078fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00ce7fc640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00c40a2d00 0x7f00c40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e6078fb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41886/0 (socket says 172.21.15.78:41886) 2024-05-21T15:25:34.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 -- 172.21.15.78:0/1963840400 learned_addr learned my addr 172.21.15.78:0/1963840400 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 0x55e6078fbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dc8bc640 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e6078fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00c40a2d00 0x7f00c40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 -- 172.21.15.78:0/1963840400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 msgr2=0x55e6078fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e6078fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 -- 172.21.15.78:0/1963840400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 msgr2=0x55e6078fbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 0x55e6078fbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 -- 172.21.15.78:0/1963840400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60763c7e0 con 0x55e60755c410 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dc8bc640 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e6078fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00ce7fc640 1 -- 172.21.15.78:0/3805934665 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f00d40efc60 con 0x7f00d8101aa0 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e6078fb620 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f00d0014a20 tx=0x7f00d0014ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a3a9312ee39ff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00c37fe640 1 -- 172.21.15.78:0/1963840400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0012c30 con 0x55e60755c410 2024-05-21T15:25:34.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00c37fe640 1 -- 172.21.15.78:0/1963840400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d0023070 con 0x55e60755c410 2024-05-21T15:25:34.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00c37fe640 1 -- 172.21.15.78:0/1963840400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d000e360 con 0x55e60755c410 2024-05-21T15:25:34.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.235+0000 7f00dd8be640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00c40a2d00 0x7f00c40a5210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f00d0022000 tx=0x7f00d0020000 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 -- 172.21.15.78:0/1963840400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 msgr2=0x55e6078fb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e6078fb620 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f00d0014a20 tx=0x7f00d0014ee0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 -- 172.21.15.78:0/1963840400 shutdown_connections 2024-05-21T15:25:34.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 0x55e6078fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e6078fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 --2- 172.21.15.78:0/1963840400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e6078fb620 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 -- 172.21.15.78:0/1963840400 >> 172.21.15.78:0/1963840400 conn(0x55e607550560 msgr2=0x55e6078f7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 -- 172.21.15.78:0/1963840400 shutdown_connections 2024-05-21T15:25:34.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.236+0000 7f00e61c7740 1 -- 172.21.15.78:0/1963840400 wait complete. 2024-05-21T15:25:34.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 1 -- start start 2024-05-21T15:25:34.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 1 Processor -- start 2024-05-21T15:25:34.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 0x55e607905d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e607906280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e60790be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e607908970 con 0x55e60755c410 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e60781bb40 con 0x55e607533550 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e607908af0 con 0x55e60753b730 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00e61c7740 10 client.0 _ll_register_callbacks cb 0x55e6078100f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00dd8be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e60790be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00dd8be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e60790be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41890/0 (socket says 172.21.15.78:41890) 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00dd8be640 1 -- 172.21.15.78:0/4063867296 learned_addr learned my addr 172.21.15.78:0/4063867296 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.237+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 0x55e607905d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00dc8bc640 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e607906280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00dd8be640 1 -- 172.21.15.78:0/4063867296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 msgr2=0x55e607906280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00dd8be640 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e607906280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00dd8be640 1 -- 172.21.15.78:0/4063867296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 msgr2=0x55e607905d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00dd8be640 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 0x55e607905d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00dd8be640 1 -- 172.21.15.78:0/4063867296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60769cf70 con 0x55e60755c410 2024-05-21T15:25:34.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00dd8be640 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e60790be60 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005b70 tx=0x7f00d0005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54424f85c9ddebc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00c0ff9640 1 -- 172.21.15.78:0/4063867296 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d000e360 con 0x55e60755c410 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00c0ff9640 1 -- 172.21.15.78:0/4063867296 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f00d001ed10 con 0x55e60755c410 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00e61c7740 1 -- 172.21.15.78:0/4063867296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d0023070 con 0x55e60755c410 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00c0ff9640 1 -- 172.21.15.78:0/4063867296 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f00d0012c10 con 0x55e60755c410 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00e61c7740 10 client.18321 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00e61c7740 1 -- 172.21.15.78:0/4063867296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e607534170 con 0x55e60755c410 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.238+0000 7f00c0ff9640 1 -- 172.21.15.78:0/4063867296 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d0023070 con 0x55e60755c410 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00c0ff9640 1 -- 172.21.15.78:0/4063867296 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f00d000f040 con 0x55e60755c410 2024-05-21T15:25:34.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00e61c7740 10 client.18321 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:34.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00e61c7740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:34.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00e61c7740 20 client.18321 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:34.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00e61c7740 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e60751b1f0 0x55e6079180d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00e61c7740 4 client.18321 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:34.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00e61c7740 1 -- 172.21.15.78:0/4063867296 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e60786e0c0 con 0x55e60751b1f0 2024-05-21T15:25:34.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e60751b1f0 0x55e6079180d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00c0ff9640 10 client.18321 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:34.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.239+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e60751b1f0 0x55e6079180d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6a16cdde576a04 server_cookie=f4a2326936b25c11 in_seq=0 out_seq=0 2024-05-21T15:25:34.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00c0ff9640 1 -- 172.21.15.78:0/4063867296 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e60786e0c0 con 0x55e60751b1f0 2024-05-21T15:25:34.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00c0ff9640 10 client.18321 handle_command_reply: tid=0 2024-05-21T15:25:34.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 client.18321 shutdown 2024-05-21T15:25:34.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 -- 172.21.15.78:0/4063867296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 msgr2=0x55e60790be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e60790be60 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005b70 tx=0x7f00d0005ba0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 -- 172.21.15.78:0/4063867296 shutdown_connections 2024-05-21T15:25:34.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e60751b1f0 0x55e6079180d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e60790be60 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00c0ff9640 0 client.18321 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:34.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e607906280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 --2- 172.21.15.78:0/4063867296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e607533550 0x55e607905d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 -- 172.21.15.78:0/4063867296 >> 172.21.15.78:0/4063867296 conn(0x55e607550560 msgr2=0x55e6078fb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 -- 172.21.15.78:0/4063867296 shutdown_connections 2024-05-21T15:25:34.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 1 -- 172.21.15.78:0/4063867296 wait complete. 2024-05-21T15:25:34.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.240+0000 7f00e61c7740 20 client.18321 trim_cache size 0 max 16384 2024-05-21T15:25:34.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00e61c7740 1 Processor -- start 2024-05-21T15:25:34.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00e61c7740 1 -- start start 2024-05-21T15:25:34.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e607533550 0x55e60791d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60753b730 0x55e60791ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e607902d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00e61c7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81aed00 con 0x55e60755c410 2024-05-21T15:25:34.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00e61c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b0840 con 0x55e60753b730 2024-05-21T15:25:34.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00e61c7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81af3d0 con 0x55e607533550 2024-05-21T15:25:34.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00dd8be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e607902d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00dd0bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e607533550 0x55e60791d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.259+0000 7f00dd8be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e607902d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41896/0 (socket says 172.21.15.78:41896) 2024-05-21T15:25:34.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dd8be640 1 -- 172.21.15.78:0/1137129479 learned_addr learned my addr 172.21.15.78:0/1137129479 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dc8bc640 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60753b730 0x55e60791ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dd8be640 1 -- 172.21.15.78:0/1137129479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e607533550 msgr2=0x55e60791d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dd8be640 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e607533550 0x55e60791d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dd8be640 1 -- 172.21.15.78:0/1137129479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60753b730 msgr2=0x55e60791ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dd8be640 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60753b730 0x55e60791ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dd8be640 1 -- 172.21.15.78:0/1137129479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e607906090 con 0x55e60755c410 2024-05-21T15:25:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e607533550 0x55e60791d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00dd8be640 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e607902d50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f00d001db60 tx=0x7f00d001da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc72039050cac7c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00c2ffd640 1 -- 172.21.15.78:0/1137129479 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d000e360 con 0x55e60755c410 2024-05-21T15:25:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00c2ffd640 1 -- 172.21.15.78:0/1137129479 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d0023040 con 0x55e60755c410 2024-05-21T15:25:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.260+0000 7f00c2ffd640 1 -- 172.21.15.78:0/1137129479 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0014160 con 0x55e60755c410 2024-05-21T15:25:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 -- 172.21.15.78:0/1137129479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 msgr2=0x55e607902d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e607902d50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f00d001db60 tx=0x7f00d001da80 comp rx=0 tx=0).stop 2024-05-21T15:25:34.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 -- 172.21.15.78:0/1137129479 shutdown_connections 2024-05-21T15:25:34.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60753b730 0x55e60791ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e60755c410 0x55e607902d50 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 --2- 172.21.15.78:0/1137129479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e607533550 0x55e60791d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 -- 172.21.15.78:0/1137129479 >> 172.21.15.78:0/1137129479 conn(0x55e607550560 msgr2=0x55e6078fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 -- 172.21.15.78:0/1137129479 shutdown_connections 2024-05-21T15:25:34.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 -- 172.21.15.78:0/1137129479 wait complete. 2024-05-21T15:25:34.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 -- start start 2024-05-21T15:25:34.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.261+0000 7f00e61c7740 1 Processor -- start 2024-05-21T15:25:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e607533550 0x55e607911ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e60790f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00e61c7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60755c410 0x55e60790f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00e61c7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b0170 con 0x55e607533550 2024-05-21T15:25:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00e61c7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81afaa0 con 0x55e60755c410 2024-05-21T15:25:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00e61c7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81acd00 con 0x55e60753b730 2024-05-21T15:25:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00e61c7740 10 client.0 _ll_register_callbacks cb 0x7f00d40bbc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e607533550 0x55e607911ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e607533550 0x55e607911ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41898/0 (socket says 172.21.15.78:41898) 2024-05-21T15:25:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 -- 172.21.15.78:0/3439538554 learned_addr learned my addr 172.21.15.78:0/3439538554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dc8bc640 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e60790f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd8be640 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60755c410 0x55e60790f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 -- 172.21.15.78:0/3439538554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 msgr2=0x55e60790f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e60790f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 -- 172.21.15.78:0/3439538554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60755c410 msgr2=0x55e60790f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60755c410 0x55e60790f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 -- 172.21.15.78:0/3439538554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60790fc90 con 0x55e607533550 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00dd0bd640 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e607533550 0x55e607911ea0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f00d40bbdf0 tx=0x7f00d40b84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00803697c1444b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.262+0000 7f00c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00c0ff9640 1 -- 172.21.15.78:0/3439538554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4049040 con 0x55e607533550 2024-05-21T15:25:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00c0ff9640 1 -- 172.21.15.78:0/3439538554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f00d40ba3f0 con 0x55e607533550 2024-05-21T15:25:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 1 -- 172.21.15.78:0/3439538554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d0023040 con 0x55e607533550 2024-05-21T15:25:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00c0ff9640 1 -- 172.21.15.78:0/3439538554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f00d40dc0b0 con 0x55e607533550 2024-05-21T15:25:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 10 client.18327 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 1 -- 172.21.15.78:0/3439538554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e607980120 con 0x55e607533550 2024-05-21T15:25:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00c0ff9640 1 -- 172.21.15.78:0/3439538554 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d0023040 con 0x55e607533550 2024-05-21T15:25:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00c0ff9640 1 -- 172.21.15.78:0/3439538554 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f00d40d6020 con 0x55e607533550 2024-05-21T15:25:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 10 client.18327 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 10 client.18327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 20 client.18327 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e6074f0630 0x55e607982800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 4 client.18327 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00e61c7740 1 -- 172.21.15.78:0/3439538554 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e607868410 con 0x55e6074f0630 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.263+0000 7f00dc8bc640 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e6074f0630 0x55e607982800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.264+0000 7f00c0ff9640 10 client.18327 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.264+0000 7f00dc8bc640 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e6074f0630 0x55e607982800 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3e1e6d678d0a6 server_cookie=3d910427010beab9 in_seq=0 out_seq=0 2024-05-21T15:25:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.264+0000 7f00c0ff9640 1 -- 172.21.15.78:0/3439538554 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e607868410 con 0x55e6074f0630 2024-05-21T15:25:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00c0ff9640 10 client.18327 handle_command_reply: tid=0 2024-05-21T15:25:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 client.18327 shutdown 2024-05-21T15:25:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 -- 172.21.15.78:0/3439538554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e607533550 msgr2=0x55e607911ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e607533550 0x55e607911ea0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f00d40bbdf0 tx=0x7f00d40b84c0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 -- 172.21.15.78:0/3439538554 shutdown_connections 2024-05-21T15:25:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55e6074f0630 0x55e607982800 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60755c410 0x55e60790f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00c0ff9640 0 client.18327 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60753b730 0x55e60790f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 --2- 172.21.15.78:0/3439538554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e607533550 0x55e607911ea0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 -- 172.21.15.78:0/3439538554 >> 172.21.15.78:0/3439538554 conn(0x55e607550560 msgr2=0x55e6078fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 -- 172.21.15.78:0/3439538554 shutdown_connections 2024-05-21T15:25:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 1 -- 172.21.15.78:0/3439538554 wait complete. 2024-05-21T15:25:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00e61c7740 20 client.18327 trim_cache size 0 max 16384 2024-05-21T15:25:34.280 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:25:34.280 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:25:34.280 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "19b5645c-f3dd-4d8e-985e-eca71538690a", 2024-05-21T15:25:34.280 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:25:34.280 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:25:34.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.265+0000 7f00df321640 1 -- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00c40a2d00 msgr2=0x7f00c40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00c40a2d00 0x7f00c40a5210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f00d0022000 tx=0x7f00d0020000 comp rx=0 tx=0).stop 2024-05-21T15:25:34.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 -- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d8101aa0 msgr2=0x7f00d819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d8101aa0 0x7f00d819a600 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f00d4005940 tx=0x7f00d4004270 comp rx=0 tx=0).stop 2024-05-21T15:25:34.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 -- 172.21.15.78:0/3805934665 shutdown_connections 2024-05-21T15:25:34.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d81079d0 0x7f00d819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f00c40a2d00 0x7f00c40a5210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00d810c330 0x7f00d819b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 --2- 172.21.15.78:0/3805934665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00d8101aa0 0x7f00d819a600 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 -- 172.21.15.78:0/3805934665 >> 172.21.15.78:0/3805934665 conn(0x7f00d80770b0 msgr2=0x7f00d8103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 -- 172.21.15.78:0/3805934665 shutdown_connections 2024-05-21T15:25:34.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.266+0000 7f00df321640 1 -- 172.21.15.78:0/3805934665 wait complete. 2024-05-21T15:25:34.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:25:34.283 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:25:34.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 -- 172.21.15.78:0/2877285941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c1079d0 msgr2=0x7f103c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 --2- 172.21.15.78:0/2877285941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c1079d0 0x7f103c101560 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1030009900 tx=0x7f103002a410 comp rx=0 tx=0).stop 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 -- 172.21.15.78:0/2877285941 shutdown_connections 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 --2- 172.21.15.78:0/2877285941 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f103c10c330 0x7f103c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 --2- 172.21.15.78:0/2877285941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f103c101aa0 0x7f103c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 --2- 172.21.15.78:0/2877285941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c1079d0 0x7f103c101560 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 -- 172.21.15.78:0/2877285941 >> 172.21.15.78:0/2877285941 conn(0x7f103c0770b0 msgr2=0x7f103c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 -- 172.21.15.78:0/2877285941 shutdown_connections 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 -- 172.21.15.78:0/2877285941 wait complete. 2024-05-21T15:25:34.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 Processor -- start 2024-05-21T15:25:34.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.399+0000 7f104355e640 1 -- start start 2024-05-21T15:25:34.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f104355e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f103c101aa0 0x7f103c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f104355e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f103c1079d0 0x7f103c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f104355e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c10c330 0x7f103c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f104355e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f103c111ce0 con 0x7f103c10c330 2024-05-21T15:25:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f104355e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f103c111b60 con 0x7f103c1079d0 2024-05-21T15:25:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f104355e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f103c111e60 con 0x7f103c101aa0 2024-05-21T15:25:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c10c330 0x7f103c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c10c330 0x7f103c19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41924/0 (socket says 172.21.15.78:41924) 2024-05-21T15:25:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 -- 172.21.15.78:0/2640821633 learned_addr learned my addr 172.21.15.78:0/2640821633 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f10412fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f103c101aa0 0x7f103c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1040af9640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f103c1079d0 0x7f103c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 -- 172.21.15.78:0/2640821633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f103c101aa0 msgr2=0x7f103c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f103c101aa0 0x7f103c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 -- 172.21.15.78:0/2640821633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f103c1079d0 msgr2=0x7f103c198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f103c1079d0 0x7f103c198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 -- 172.21.15.78:0/2640821633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c19d500 con 0x7f103c10c330 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f1041afb640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c10c330 0x7f103c19cec0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f103400eb10 tx=0x7f103400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f102e7fc640 1 -- 172.21.15.78:0/2640821633 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034004b20 con 0x7f103c10c330 2024-05-21T15:25:34.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.400+0000 7f104355e640 1 -- 172.21.15.78:0/2640821633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c19d7f0 con 0x7f103c10c330 2024-05-21T15:25:34.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.401+0000 7f102e7fc640 1 -- 172.21.15.78:0/2640821633 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1034004cc0 con 0x7f103c10c330 2024-05-21T15:25:34.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.401+0000 7f102e7fc640 1 -- 172.21.15.78:0/2640821633 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034010760 con 0x7f103c10c330 2024-05-21T15:25:34.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.401+0000 7f104355e640 1 -- 172.21.15.78:0/2640821633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1a5170 con 0x7f103c10c330 2024-05-21T15:25:34.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.403+0000 7f102e7fc640 1 -- 172.21.15.78:0/2640821633 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1034005020 con 0x7f103c10c330 2024-05-21T15:25:34.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.403+0000 7f104a408740 1 Processor -- start 2024-05-21T15:25:34.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f104a408740 1 -- start start 2024-05-21T15:25:34.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f104a408740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaed9390 0x5586ab277b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f102e7fc640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f10280a2d00 0x7f10280a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f104a408740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb04d0 0x5586ab2780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f104a408740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab278600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f104a408740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586ab1ddd00 con 0x5586aaeb86b0 2024-05-21T15:25:34.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f104a408740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5586ab1854c0 con 0x5586aaeb04d0 2024-05-21T15:25:34.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f104a408740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586ab1d9cc0 con 0x5586aaed9390 2024-05-21T15:25:34.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f10412fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaed9390 0x5586ab277b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f1040af9640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f10280a2d00 0x7f10280a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f1041afb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab278600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f10412fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaed9390 0x5586ab277b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.78:44864/0 (socket says 172.21.15.78:44864) 2024-05-21T15:25:34.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f10412fa640 1 -- 172.21.15.78:0/1502756240 learned_addr learned my addr 172.21.15.78:0/1502756240 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f102e7fc640 1 -- 172.21.15.78:0/2640821633 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f10340cfa70 con 0x7f103c10c330 2024-05-21T15:25:34.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f1040af9640 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb04d0 0x5586ab2780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f10412fa640 1 -- 172.21.15.78:0/1502756240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb04d0 msgr2=0x5586ab2780c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f10412fa640 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb04d0 0x5586ab2780c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f10412fa640 1 -- 172.21.15.78:0/1502756240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 msgr2=0x5586ab278600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f10412fa640 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab278600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f10412fa640 1 -- 172.21.15.78:0/1502756240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586aafbb7d0 con 0x5586aaed9390 2024-05-21T15:25:34.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.404+0000 7f1041afb640 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab278600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:34.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.405+0000 7f1040af9640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f10280a2d00 0x7f10280a5210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f103c10d6f0 tx=0x7f1038013880 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.405+0000 7f10412fa640 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaed9390 0x5586ab277b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f103c1029d0 tx=0x7f10300095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=666ce6b7d58c48d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.405+0000 7f10277fe640 1 -- 172.21.15.78:0/1502756240 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030040070 con 0x5586aaed9390 2024-05-21T15:25:34.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.405+0000 7f10277fe640 1 -- 172.21.15.78:0/1502756240 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10300317b0 con 0x5586aaed9390 2024-05-21T15:25:34.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.405+0000 7f10277fe640 1 -- 172.21.15.78:0/1502756240 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030035040 con 0x5586aaed9390 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 -- 172.21.15.78:0/1502756240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaed9390 msgr2=0x5586ab277b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaed9390 0x5586ab277b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f103c1029d0 tx=0x7f10300095e0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 -- 172.21.15.78:0/1502756240 shutdown_connections 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb04d0 0x5586ab2780c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab278600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 --2- 172.21.15.78:0/1502756240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaed9390 0x5586ab277b80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 -- 172.21.15.78:0/1502756240 >> 172.21.15.78:0/1502756240 conn(0x5586aaecd4e0 msgr2=0x5586ab2744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 -- 172.21.15.78:0/1502756240 shutdown_connections 2024-05-21T15:25:34.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 -- 172.21.15.78:0/1502756240 wait complete. 2024-05-21T15:25:34.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 -- start start 2024-05-21T15:25:34.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.406+0000 7f104a408740 1 Processor -- start 2024-05-21T15:25:34.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f104a408740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab284590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f104a408740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb86b0 0x5586ab284ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f104a408740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaed9390 0x5586ab28a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f104a408740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5586ab287260 con 0x5586aaed9390 2024-05-21T15:25:34.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f104a408740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5586ab198b90 con 0x5586aaeb86b0 2024-05-21T15:25:34.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f104a408740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586ab2873e0 con 0x5586aaeb04d0 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f104a408740 10 client.0 _ll_register_callbacks cb 0x5586ab1d4da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1041afb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaed9390 0x5586ab28a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1041afb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaed9390 0x5586ab28a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41944/0 (socket says 172.21.15.78:41944) 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1040af9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb86b0 0x5586ab284ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1041afb640 1 -- 172.21.15.78:0/1372200636 learned_addr learned my addr 172.21.15.78:0/1372200636 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f10412fa640 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab284590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1040af9640 1 -- 172.21.15.78:0/1372200636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 msgr2=0x5586ab284590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1040af9640 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab284590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1040af9640 1 -- 172.21.15.78:0/1372200636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaed9390 msgr2=0x5586ab28a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1040af9640 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaed9390 0x5586ab28a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1040af9640 1 -- 172.21.15.78:0/1372200636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586aaef13f0 con 0x5586aaeb86b0 2024-05-21T15:25:34.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1041afb640 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaed9390 0x5586ab28a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:34.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f10412fa640 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab284590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:25:34.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1040af9640 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb86b0 0x5586ab284ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f103c06b480 tx=0x7f1038004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c0606f46f65ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1024ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-05-21T15:25:34.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1024ff9640 1 -- 172.21.15.78:0/1372200636 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038010a40 con 0x5586aaeb86b0 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f104a408740 1 -- 172.21.15.78:0/1372200636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10300317b0 con 0x5586aaeb86b0 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.407+0000 7f1024ff9640 1 -- 172.21.15.78:0/1372200636 <== mon.2 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f1038010be0 con 0x5586aaeb86b0 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f1024ff9640 1 -- 172.21.15.78:0/1372200636 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1038010e20 con 0x5586aaeb86b0 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f104a408740 10 client.27485 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f104a408740 1 -- 172.21.15.78:0/1372200636 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5586aaeb10f0 con 0x5586aaeb86b0 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f1024ff9640 1 -- 172.21.15.78:0/1372200636 <== mon.2 v2:172.21.15.84:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10300317b0 con 0x5586aaeb86b0 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f1024ff9640 1 -- 172.21.15.78:0/1372200636 <== mon.2 v2:172.21.15.84:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f103801f070 con 0x5586aaeb86b0 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f104a408740 10 client.27485 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:34.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f104a408740 10 client.27485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:34.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f104a408740 20 client.27485 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:34.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f104a408740 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5586aae97f60 0x5586ab296d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f104a408740 4 client.27485 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:25:34.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f104a408740 1 -- 172.21.15.78:0/1372200636 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586ab1eb430 con 0x5586aae97f60 2024-05-21T15:25:34.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.408+0000 7f10412fa640 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5586aae97f60 0x5586ab296d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f1024ff9640 10 client.27485 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:34.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f10412fa640 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5586aae97f60 0x5586ab296d30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9af32b25398cac1 server_cookie=45d4817aedb0acc8 in_seq=0 out_seq=0 2024-05-21T15:25:34.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f1024ff9640 1 -- 172.21.15.78:0/1372200636 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586ab1eb430 con 0x5586aae97f60 2024-05-21T15:25:34.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f1024ff9640 10 client.27485 handle_command_reply: tid=0 2024-05-21T15:25:34.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 client.27485 shutdown 2024-05-21T15:25:34.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 -- 172.21.15.78:0/1372200636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb86b0 msgr2=0x5586ab284ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb86b0 0x5586ab284ad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f103c06b480 tx=0x7f1038004730 comp rx=0 tx=0).stop 2024-05-21T15:25:34.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 -- 172.21.15.78:0/1372200636 shutdown_connections 2024-05-21T15:25:34.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5586aae97f60 0x5586ab296d30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaeb86b0 0x5586ab284ad0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaed9390 0x5586ab28a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 --2- 172.21.15.78:0/1372200636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab284590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f1024ff9640 0 client.27485 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:34.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.409+0000 7f104a408740 1 -- 172.21.15.78:0/1372200636 >> 172.21.15.78:0/1372200636 conn(0x5586aaecd4e0 msgr2=0x5586ab272b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.410+0000 7f104a408740 1 -- 172.21.15.78:0/1372200636 shutdown_connections 2024-05-21T15:25:34.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.410+0000 7f104a408740 1 -- 172.21.15.78:0/1372200636 wait complete. 2024-05-21T15:25:34.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.410+0000 7f104a408740 20 client.27485 trim_cache size 0 max 16384 2024-05-21T15:25:34.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f104a408740 1 Processor -- start 2024-05-21T15:25:34.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f104a408740 1 -- start start 2024-05-21T15:25:34.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f104a408740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab28a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f104a408740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab29c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f104a408740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab29ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f104a408740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1ac970 con 0x5586aaeb86b0 2024-05-21T15:25:34.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f104a408740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1ae4b0 con 0x5586aaed9390 2024-05-21T15:25:34.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f104a408740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1ad040 con 0x5586aaeb04d0 2024-05-21T15:25:34.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f1040af9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab29c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f1040af9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab29c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41946/0 (socket says 172.21.15.78:41946) 2024-05-21T15:25:34.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f1040af9640 1 -- 172.21.15.78:0/4086624660 learned_addr learned my addr 172.21.15.78:0/4086624660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f1041afb640 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab29ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.428+0000 7f10412fa640 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab28a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1040af9640 1 -- 172.21.15.78:0/4086624660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 msgr2=0x5586ab28a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1040af9640 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab28a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1040af9640 1 -- 172.21.15.78:0/4086624660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 msgr2=0x5586ab29ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1040af9640 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab29ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1040af9640 1 -- 172.21.15.78:0/4086624660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ab2828d0 con 0x5586aaeb86b0 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1041afb640 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab29ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1040af9640 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab29c4f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f10380204f0 tx=0x7f1038009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e682ecd8a10efeaa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1026ffd640 1 -- 172.21.15.78:0/4086624660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038010840 con 0x5586aaeb86b0 2024-05-21T15:25:34.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1026ffd640 1 -- 172.21.15.78:0/4086624660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1038004c80 con 0x5586aaeb86b0 2024-05-21T15:25:34.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.429+0000 7f1026ffd640 1 -- 172.21.15.78:0/4086624660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038020ae0 con 0x5586aaeb86b0 2024-05-21T15:25:34.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 -- 172.21.15.78:0/4086624660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 msgr2=0x5586ab29c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab29c4f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f10380204f0 tx=0x7f1038009840 comp rx=0 tx=0).stop 2024-05-21T15:25:34.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 -- 172.21.15.78:0/4086624660 shutdown_connections 2024-05-21T15:25:34.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab29ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab29c4f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 --2- 172.21.15.78:0/4086624660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab28a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 -- 172.21.15.78:0/4086624660 >> 172.21.15.78:0/4086624660 conn(0x5586aaecd4e0 msgr2=0x5586ab279830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 -- 172.21.15.78:0/4086624660 shutdown_connections 2024-05-21T15:25:34.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 -- 172.21.15.78:0/4086624660 wait complete. 2024-05-21T15:25:34.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 -- start start 2024-05-21T15:25:34.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.430+0000 7f104a408740 1 Processor -- start 2024-05-21T15:25:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f104a408740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab27cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f104a408740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab27d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f104a408740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab2ff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f104a408740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1adde0 con 0x5586aaeb86b0 2024-05-21T15:25:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f104a408740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1ad710 con 0x5586aaed9390 2024-05-21T15:25:34.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f104a408740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1aa970 con 0x5586aaeb04d0 2024-05-21T15:25:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f104a408740 10 client.0 _ll_register_callbacks cb 0x7f1038009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:25:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab27d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f10412fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab27cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab27d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41954/0 (socket says 172.21.15.78:41954) 2024-05-21T15:25:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 -- 172.21.15.78:0/3527568631 learned_addr learned my addr 172.21.15.78:0/3527568631 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:25:34.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1041afb640 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab2ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 -- 172.21.15.78:0/3527568631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 msgr2=0x5586ab27cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab27cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 -- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 msgr2=0x5586ab2ff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:25:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab2ff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 -- 172.21.15.78:0/3527568631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ab2914d0 con 0x5586aaeb86b0 2024-05-21T15:25:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1041afb640 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab2ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f10412fa640 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab27cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:25:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1040af9640 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab27d4c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1038013df0 tx=0x7f1038004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbab155c8b65f40d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:25:34.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.431+0000 7f1024ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:25:34.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f1024ff9640 1 -- 172.21.15.78:0/3527568631 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038005a90 con 0x5586aaeb86b0 2024-05-21T15:25:34.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 1 -- 172.21.15.78:0/3527568631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1038004c80 con 0x5586aaeb86b0 2024-05-21T15:25:34.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f1024ff9640 1 -- 172.21.15.78:0/3527568631 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f1038020a90 con 0x5586aaeb86b0 2024-05-21T15:25:34.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f1024ff9640 1 -- 172.21.15.78:0/3527568631 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1038020520 con 0x5586aaeb86b0 2024-05-21T15:25:34.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 10 client.18345 fetch_fsmap learned FSMap version 37 2024-05-21T15:25:34.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 1 -- 172.21.15.78:0/3527568631 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5586ab3008a0 con 0x5586aaeb86b0 2024-05-21T15:25:34.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f1024ff9640 1 -- 172.21.15.78:0/3527568631 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1038004c80 con 0x5586aaeb86b0 2024-05-21T15:25:34.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f1024ff9640 1 -- 172.21.15.78:0/3527568631 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1038036070 con 0x5586aaeb86b0 2024-05-21T15:25:34.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 10 client.18345 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:25:34.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:25:34.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 20 client.18345 populate_metadata read hostname 'smithi078' 2024-05-21T15:25:34.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5586aae6d3a0 0x5586ab302f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:25:34.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 4 client.18345 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:25:34.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f104a408740 1 -- 172.21.15.78:0/3527568631 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586ab1e5780 con 0x5586aae6d3a0 2024-05-21T15:25:34.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.432+0000 7f10412fa640 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5586aae6d3a0 0x5586ab302f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:25:34.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f1024ff9640 10 client.18345 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:34.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f10412fa640 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5586aae6d3a0 0x5586ab302f10 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7387c47d5eccc387 server_cookie=c09b20b48e9727bb in_seq=0 out_seq=0 2024-05-21T15:25:34.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f1024ff9640 1 -- 172.21.15.78:0/3527568631 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5586ab1e5780 con 0x5586aae6d3a0 2024-05-21T15:25:34.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f1024ff9640 10 client.18345 handle_command_reply: tid=0 2024-05-21T15:25:34.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 client.18345 shutdown 2024-05-21T15:25:34.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 -- 172.21.15.78:0/3527568631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 msgr2=0x5586ab27d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab27d4c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1038013df0 tx=0x7f1038004930 comp rx=0 tx=0).stop 2024-05-21T15:25:34.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 -- 172.21.15.78:0/3527568631 shutdown_connections 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5586aae6d3a0 0x5586ab302f10 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5586aaed9390 0x5586ab2ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f1024ff9640 0 client.18345 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586aaeb86b0 0x5586ab27d4c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 --2- 172.21.15.78:0/3527568631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586aaeb04d0 0x5586ab27cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 -- 172.21.15.78:0/3527568631 >> 172.21.15.78:0/3527568631 conn(0x5586aaecd4e0 msgr2=0x5586ab282b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 -- 172.21.15.78:0/3527568631 shutdown_connections 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 1 -- 172.21.15.78:0/3527568631 wait complete. 2024-05-21T15:25:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.433+0000 7f104a408740 20 client.18345 trim_cache size 0 max 16384 2024-05-21T15:25:34.449 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:25:34.449 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:25:34.449 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:25:34.449 INFO:teuthology.orchestra.run.smithi078.stdout: "19b5645c-f3dd-4d8e-985e-eca71538690a": { 2024-05-21T15:25:34.449 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:25:34.449 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "19b5645c-f3dd-4d8e-985e-eca71538690a", 2024-05-21T15:25:34.450 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:25:34.450 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:25:34.450 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:25:34.450 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:25:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 -- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f10280a2d00 msgr2=0x7f10280a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f10280a2d00 0x7f10280a5210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f103c10d6f0 tx=0x7f1038013880 comp rx=0 tx=0).stop 2024-05-21T15:25:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 -- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c10c330 msgr2=0x7f103c19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:25:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c10c330 0x7f103c19cec0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f103400eb10 tx=0x7f103400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 -- 172.21.15.78:0/2640821633 shutdown_connections 2024-05-21T15:25:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f103c1079d0 0x7f103c198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f10280a2d00 0x7f10280a5210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f103c10c330 0x7f103c19cec0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 --2- 172.21.15.78:0/2640821633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f103c101aa0 0x7f103c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:25:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 -- 172.21.15.78:0/2640821633 >> 172.21.15.78:0/2640821633 conn(0x7f103c0770b0 msgr2=0x7f103c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:25:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 -- 172.21.15.78:0/2640821633 shutdown_connections 2024-05-21T15:25:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:25:34.434+0000 7f104355e640 1 -- 172.21.15.78:0/2640821633 wait complete. 2024-05-21T15:25:34.453 INFO:tasks.cephfs.filesystem:scrub status for tag:19b5645c-f3dd-4d8e-985e-eca71538690a - {'path': '/', 'tag': '19b5645c-f3dd-4d8e-985e-eca71538690a', 'options': 'recursive,force'} 2024-05-21T15:25:34.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:34 smithi078 ceph-mon[31997]: pgmap v293: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:25:34.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:34 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:34.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:34 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:34.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:34 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:34.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:34 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:34.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:34 smithi084 ceph-mon[36334]: pgmap v293: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:25:34.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:34 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:34.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:34 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:34.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:34 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:34.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:34 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:34.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:34 smithi172 ceph-mon[36402]: pgmap v293: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T15:25:34.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:34 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:34.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:34 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:34.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:34 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:34.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:34 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:25:35.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:35.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:35.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:35.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:25:35.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:35 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:25:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:35.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:25:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:35 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:25:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:25:35.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:35.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:35 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:25:36.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:36 smithi078 ceph-mon[31997]: pgmap v294: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T15:25:36.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:36 smithi084 ceph-mon[36334]: pgmap v294: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T15:25:36.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:36 smithi172 ceph-mon[36402]: pgmap v294: 97 pgs: 97 active+clean; 13 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T15:25:38.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:38 smithi078 ceph-mon[31997]: pgmap v295: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 op/s 2024-05-21T15:25:38.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:38 smithi084 ceph-mon[36334]: pgmap v295: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 op/s 2024-05-21T15:25:38.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:38 smithi172 ceph-mon[36402]: pgmap v295: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 op/s 2024-05-21T15:25:40.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:40 smithi078 ceph-mon[31997]: pgmap v296: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:40.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:40 smithi084 ceph-mon[36334]: pgmap v296: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:40.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:40 smithi172 ceph-mon[36402]: pgmap v296: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:42.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:42 smithi078 ceph-mon[31997]: pgmap v297: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:42.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:42 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:42.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:42 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:42.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:42 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:42 smithi084 ceph-mon[36334]: pgmap v297: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:42 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:42 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:42.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:42 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:42.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:42 smithi172 ceph-mon[36402]: pgmap v297: 97 pgs: 97 active+clean; 13 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:42.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:42 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:42.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:42 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:42.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:42 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:43.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:43 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:43.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:43 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:25:43.901 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:43 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:43.901 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:43 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:25:43.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:43 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:43.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:43 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:25:44.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:44 smithi078 ceph-mon[31997]: pgmap v298: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:25:44.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:44 smithi084 ceph-mon[36334]: pgmap v298: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:25:44.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:44 smithi172 ceph-mon[36402]: pgmap v298: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 op/s 2024-05-21T15:25:46.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:46 smithi078 ceph-mon[31997]: pgmap v299: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:46.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:46 smithi084 ceph-mon[36334]: pgmap v299: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:46.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:46 smithi172 ceph-mon[36402]: pgmap v299: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 op/s 2024-05-21T15:25:48.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:48 smithi078 ceph-mon[31997]: pgmap v300: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:25:48.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:48 smithi084 ceph-mon[36334]: pgmap v300: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:25:48.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:48 smithi172 ceph-mon[36402]: pgmap v300: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 op/s 2024-05-21T15:25:50.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:50 smithi078 ceph-mon[31997]: pgmap v301: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 op/s 2024-05-21T15:25:50.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:50 smithi084 ceph-mon[36334]: pgmap v301: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 op/s 2024-05-21T15:25:50.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:50 smithi172 ceph-mon[36402]: pgmap v301: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 op/s 2024-05-21T15:25:52.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:52 smithi078 ceph-mon[31997]: pgmap v302: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:52.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:52 smithi084 ceph-mon[36334]: pgmap v302: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:52.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:52 smithi172 ceph-mon[36402]: pgmap v302: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:54.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:54 smithi078 ceph-mon[31997]: pgmap v303: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:25:54.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:54 smithi084 ceph-mon[36334]: pgmap v303: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:25:54.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:54 smithi172 ceph-mon[36402]: pgmap v303: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:25:56.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:56 smithi078 ceph-mon[31997]: pgmap v304: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:56.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:56 smithi084 ceph-mon[36334]: pgmap v304: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:56.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:56 smithi172 ceph-mon[36402]: pgmap v304: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:25:58.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:25:58 smithi078 ceph-mon[31997]: pgmap v305: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:25:58.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:25:58 smithi084 ceph-mon[36334]: pgmap v305: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:25:58.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:25:58 smithi172 ceph-mon[36402]: pgmap v305: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:00.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:00 smithi078 ceph-mon[31997]: pgmap v306: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:00.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:00 smithi084 ceph-mon[36334]: pgmap v306: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:00.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:00 smithi172 ceph-mon[36402]: pgmap v306: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:02.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:02 smithi078 ceph-mon[31997]: pgmap v307: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:02.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:02 smithi084 ceph-mon[36334]: pgmap v307: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:02.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:02 smithi172 ceph-mon[36402]: pgmap v307: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:04.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:26:04.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:26:04.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 -- 172.21.15.78:0/457406135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00103210 msgr2=0x7f3a0010f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 --2- 172.21.15.78:0/457406135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00103210 0x7f3a0010f5e0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009ca0 tx=0x7f39f802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 -- 172.21.15.78:0/457406135 shutdown_connections 2024-05-21T15:26:04.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 --2- 172.21.15.78:0/457406135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00103210 0x7f3a0010f5e0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 --2- 172.21.15.78:0/457406135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a00108f60 0x7f3a00102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 --2- 172.21.15.78:0/457406135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a00108ac0 0x7f3a00102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 -- 172.21.15.78:0/457406135 >> 172.21.15.78:0/457406135 conn(0x7f3a000fe180 msgr2=0x7f3a001005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:04.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 -- 172.21.15.78:0/457406135 shutdown_connections 2024-05-21T15:26:04.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.581+0000 7f3a04de3640 1 -- 172.21.15.78:0/457406135 wait complete. 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.582+0000 7f3a04de3640 1 Processor -- start 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.582+0000 7f3a04de3640 1 -- start start 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.582+0000 7f3a04de3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a00103210 0x7f3a00199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.582+0000 7f3a04de3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a00108ac0 0x7f3a00199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.582+0000 7f3a04de3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00108f60 0x7f3a0019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.582+0000 7f3a04de3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00112e00 con 0x7f3a00108f60 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f3a04de3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00112c80 con 0x7f3a00103210 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f3a04de3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00112f80 con 0x7f3a00108ac0 2024-05-21T15:26:04.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00108f60 0x7f3a0019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00108f60 0x7f3a0019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49040/0 (socket says 172.21.15.78:49040) 2024-05-21T15:26:04.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 -- 172.21.15.78:0/928110982 learned_addr learned my addr 172.21.15.78:0/928110982 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:04.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39f7fff640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a00108ac0 0x7f3a00199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fe59c640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a00103210 0x7f3a00199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 -- 172.21.15.78:0/928110982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a00108ac0 msgr2=0x7f3a00199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a00108ac0 0x7f3a00199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 -- 172.21.15.78:0/928110982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a00103210 msgr2=0x7f3a00199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a00103210 0x7f3a00199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 -- 172.21.15.78:0/928110982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0019e5c0 con 0x7f3a00108f60 2024-05-21T15:26:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fe59c640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a00103210 0x7f3a00199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39fed9d640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00108f60 0x7f3a0019df80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f39f8031a20 tx=0x7f39f8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39f77fe640 1 -- 172.21.15.78:0/928110982 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8003c30 con 0x7f3a00108f60 2024-05-21T15:26:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f3a04de3640 1 -- 172.21.15.78:0/928110982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0019e850 con 0x7f3a00108f60 2024-05-21T15:26:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39f77fe640 1 -- 172.21.15.78:0/928110982 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f8003dd0 con 0x7f3a00108f60 2024-05-21T15:26:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f39f77fe640 1 -- 172.21.15.78:0/928110982 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f802fcf0 con 0x7f3a00108f60 2024-05-21T15:26:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.583+0000 7f3a04de3640 1 -- 172.21.15.78:0/928110982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0019eda0 con 0x7f3a00108f60 2024-05-21T15:26:04.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.586+0000 7f3a0bd65740 1 Processor -- start 2024-05-21T15:26:04.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f39f77fe640 1 -- 172.21.15.78:0/928110982 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f39f8004050 con 0x7f3a00108f60 2024-05-21T15:26:04.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f3a0bd65740 1 -- start start 2024-05-21T15:26:04.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7b1390 0x564d4db4fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 0x564d4db50270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7906b0 0x564d4db507b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d4dab5d00 con 0x564d4d7906b0 2024-05-21T15:26:04.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d4da5d4c0 con 0x564d4d7884d0 2024-05-21T15:26:04.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d4dab1cc0 con 0x564d4d7b1390 2024-05-21T15:26:04.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7906b0 0x564d4db507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7906b0 0x564d4db507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49052/0 (socket says 172.21.15.78:49052) 2024-05-21T15:26:04.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f39f7fff640 1 -- 172.21.15.78:0/2865411419 learned_addr learned my addr 172.21.15.78:0/2865411419 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:04.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f39f77fe640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f39ec0a2d00 0x7f39ec0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.587+0000 7f39f7fff640 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7b1390 0x564d4db4fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39fed9d640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f39ec0a2d00 0x7f39ec0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39fe59c640 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 0x564d4db50270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f7fff640 1 -- 172.21.15.78:0/2865411419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7b1390 msgr2=0x564d4db4fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f7fff640 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7b1390 0x564d4db4fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f7fff640 1 -- 172.21.15.78:0/2865411419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 msgr2=0x564d4db50270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f77fe640 1 -- 172.21.15.78:0/928110982 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f39f80f0830 con 0x7f3a00108f60 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f7fff640 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 0x564d4db50270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f7fff640 1 -- 172.21.15.78:0/2865411419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4d8937d0 con 0x564d4d7906b0 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39fe59c640 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 0x564d4db50270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f7fff640 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7906b0 0x564d4db507b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f39e8005e00 tx=0x7f39e800f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c206afbd830c68 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:04.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f4ff9640 1 -- 172.21.15.78:0/2865411419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8019070 con 0x564d4d7906b0 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39fed9d640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f39ec0a2d00 0x7f39ec0a5210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39f80bcb60 tx=0x7f39f80b22f0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f4ff9640 1 -- 172.21.15.78:0/2865411419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e80047a0 con 0x564d4d7906b0 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.588+0000 7f39f4ff9640 1 -- 172.21.15.78:0/2865411419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8011890 con 0x564d4d7906b0 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2865411419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7906b0 msgr2=0x564d4db507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7906b0 0x564d4db507b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f39e8005e00 tx=0x7f39e800f600 comp rx=0 tx=0).stop 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2865411419 shutdown_connections 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 0x564d4db50270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7906b0 0x564d4db507b0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2865411419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7b1390 0x564d4db4fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2865411419 >> 172.21.15.78:0/2865411419 conn(0x564d4d7a54e0 msgr2=0x564d4db4c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2865411419 shutdown_connections 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2865411419 wait complete. 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.589+0000 7f3a0bd65740 1 -- start start 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f3a0bd65740 1 Processor -- start 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 0x564d4db5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 0x564d4db5c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db5a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564d4db5d020 con 0x564d4d7b1390 2024-05-21T15:26:04.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564d4da70b90 con 0x564d4d7906b0 2024-05-21T15:26:04.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d4db5d1a0 con 0x564d4d7884d0 2024-05-21T15:26:04.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f3a0bd65740 10 client.0 _ll_register_callbacks cb 0x564d4daacda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:04.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db5a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db5a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49068/0 (socket says 172.21.15.78:49068) 2024-05-21T15:26:04.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 -- 172.21.15.78:0/2497875265 learned_addr learned my addr 172.21.15.78:0/2497875265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:04.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 0x564d4db5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39fe59c640 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 0x564d4db5c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 -- 172.21.15.78:0/2497875265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 msgr2=0x564d4db5be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 0x564d4db5be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 -- 172.21.15.78:0/2497875265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 msgr2=0x564d4db5c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 0x564d4db5c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 -- 172.21.15.78:0/2497875265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4d7c93f0 con 0x564d4d7b1390 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39f7fff640 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db5a310 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f39e8004ca0 tx=0x7f39e8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1121a2bf2c90d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.590+0000 7f39e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f39e5ffb640 1 -- 172.21.15.78:0/2497875265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8019070 con 0x564d4d7b1390 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f39e5ffb640 1 -- 172.21.15.78:0/2497875265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f39e80116b0 con 0x564d4d7b1390 2024-05-21T15:26:04.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2497875265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e80047a0 con 0x564d4d7b1390 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f39e5ffb640 1 -- 172.21.15.78:0/2497875265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f39e8034050 con 0x564d4d7b1390 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 10 client.18357 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2497875265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d4d7890f0 con 0x564d4d7b1390 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f39e5ffb640 1 -- 172.21.15.78:0/2497875265 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e80047a0 con 0x564d4d7b1390 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f39e5ffb640 1 -- 172.21.15.78:0/2497875265 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f39e801d5c0 con 0x564d4d7b1390 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 10 client.18357 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 20 client.18357 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:04.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d4d76ff60 0x564d4db6cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 4 client.18357 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.591+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2497875265 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d4dac3430 con 0x564d4d76ff60 2024-05-21T15:26:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.592+0000 7f39fe59c640 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d4d76ff60 0x564d4db6cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.592+0000 7f39e5ffb640 10 client.18357 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.592+0000 7f39fe59c640 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d4d76ff60 0x564d4db6cb70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99fcad2d4faac126 server_cookie=39e99a9347a2b37 in_seq=0 out_seq=0 2024-05-21T15:26:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f39e5ffb640 1 -- 172.21.15.78:0/2497875265 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d4dac3430 con 0x564d4d76ff60 2024-05-21T15:26:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f39e5ffb640 10 client.18357 handle_command_reply: tid=0 2024-05-21T15:26:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 client.18357 shutdown 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2497875265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 msgr2=0x564d4db5a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db5a310 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f39e8004ca0 tx=0x7f39e8015040 comp rx=0 tx=0).stop 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2497875265 shutdown_connections 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d4d76ff60 0x564d4db6cb70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 0x564d4db5c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db5a310 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f39e5ffb640 0 client.18357 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/2497875265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 0x564d4db5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2497875265 >> 172.21.15.78:0/2497875265 conn(0x564d4d7a54e0 msgr2=0x564d4db51a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2497875265 shutdown_connections 2024-05-21T15:26:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 1 -- 172.21.15.78:0/2497875265 wait complete. 2024-05-21T15:26:04.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.593+0000 7f3a0bd65740 20 client.18357 trim_cache size 0 max 16384 2024-05-21T15:26:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f3a0bd65740 1 Processor -- start 2024-05-21T15:26:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f3a0bd65740 1 -- start start 2024-05-21T15:26:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 0x564d4db72220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 0x564d4db72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db66600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a001aec60 con 0x564d4d7b1390 2024-05-21T15:26:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a001b07a0 con 0x564d4d7906b0 2024-05-21T15:26:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a001af330 con 0x564d4d7884d0 2024-05-21T15:26:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db66600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49078/0 (socket says 172.21.15.78:49078) 2024-05-21T15:26:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 -- 172.21.15.78:0/839467932 learned_addr learned my addr 172.21.15.78:0/839467932 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:04.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 --2- 172.21.15.78:0/839467932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 0x564d4db72220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 -- 172.21.15.78:0/839467932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 msgr2=0x564d4db72220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 --2- 172.21.15.78:0/839467932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 0x564d4db72220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 -- 172.21.15.78:0/839467932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 msgr2=0x564d4db72760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 --2- 172.21.15.78:0/839467932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 0x564d4db72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.611+0000 7f39f7fff640 1 -- 172.21.15.78:0/839467932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4db6cfd0 con 0x564d4d7b1390 2024-05-21T15:26:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.612+0000 7f39f7fff640 1 --2- 172.21.15.78:0/839467932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db66600 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f39e801dc10 tx=0x7f39e8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df67a4c49d6ed30 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.612+0000 7f39e7fff640 1 -- 172.21.15.78:0/839467932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8019040 con 0x564d4d7b1390 2024-05-21T15:26:04.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.612+0000 7f39e7fff640 1 -- 172.21.15.78:0/839467932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e80047a0 con 0x564d4d7b1390 2024-05-21T15:26:04.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.612+0000 7f39e7fff640 1 -- 172.21.15.78:0/839467932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8013020 con 0x564d4d7b1390 2024-05-21T15:26:04.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- 172.21.15.78:0/839467932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 msgr2=0x564d4db66600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/839467932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db66600 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f39e801dc10 tx=0x7f39e8005970 comp rx=0 tx=0).stop 2024-05-21T15:26:04.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- 172.21.15.78:0/839467932 shutdown_connections 2024-05-21T15:26:04.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/839467932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7906b0 0x564d4db72760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/839467932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4db66600 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/839467932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7884d0 0x564d4db72220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- 172.21.15.78:0/839467932 >> 172.21.15.78:0/839467932 conn(0x564d4d7a54e0 msgr2=0x564d4db5a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- 172.21.15.78:0/839467932 shutdown_connections 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- 172.21.15.78:0/839467932 wait complete. 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- start start 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 Processor -- start 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 0x564d4db63580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7906b0 0x564d4db63ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4dbd2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a001b00d0 con 0x564d4d7b1390 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a001afa00 con 0x564d4d7884d0 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.613+0000 7f3a0bd65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a001acc60 con 0x564d4d7906b0 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f3a0bd65740 10 client.0 _ll_register_callbacks cb 0x7f39e800ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4dbd2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4dbd2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49088/0 (socket says 172.21.15.78:49088) 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 -- 172.21.15.78:0/3645624749 learned_addr learned my addr 172.21.15.78:0/3645624749 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39fe59c640 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7906b0 0x564d4db63ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 -- 172.21.15.78:0/3645624749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7906b0 msgr2=0x564d4db63ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7906b0 0x564d4db63ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 -- 172.21.15.78:0/3645624749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 msgr2=0x564d4db63580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 0x564d4db63580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 -- 172.21.15.78:0/3645624749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4db64340 con 0x564d4d7b1390 2024-05-21T15:26:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39f7fff640 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4dbd2dd0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x564d4db62de0 tx=0x7f39e800fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2e0ec960e3bf1b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39e5ffb640 1 -- 172.21.15.78:0/3645624749 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8019040 con 0x564d4d7b1390 2024-05-21T15:26:04.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39e5ffb640 1 -- 172.21.15.78:0/3645624749 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f39e80053e0 con 0x564d4d7b1390 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f3a0bd65740 1 -- 172.21.15.78:0/3645624749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e80047a0 con 0x564d4d7b1390 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f39e5ffb640 1 -- 172.21.15.78:0/3645624749 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f39e8013030 con 0x564d4d7b1390 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f3a0bd65740 10 client.18363 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.614+0000 7f3a0bd65740 1 -- 172.21.15.78:0/3645624749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564d4dbd4240 con 0x564d4d7b1390 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f39e5ffb640 1 -- 172.21.15.78:0/3645624749 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e80047a0 con 0x564d4d7b1390 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f39e5ffb640 1 -- 172.21.15.78:0/3645624749 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f39e8011a60 con 0x564d4d7b1390 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f3a0bd65740 10 client.18363 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f3a0bd65740 10 client.18363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:04.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f3a0bd65740 20 client.18363 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d4d7453a0 0x564d4dbd6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f3a0bd65740 4 client.18363 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f3a0bd65740 1 -- 172.21.15.78:0/3645624749 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d4dabd780 con 0x564d4d7453a0 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f39fe59c640 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d4d7453a0 0x564d4dbd6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f39e5ffb640 10 client.18363 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.615+0000 7f39fe59c640 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d4d7453a0 0x564d4dbd6890 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c08d33b1eb7988 server_cookie=8b94c78afb2029e6 in_seq=0 out_seq=0 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f39e5ffb640 1 -- 172.21.15.78:0/3645624749 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564d4dabd780 con 0x564d4d7453a0 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f39e5ffb640 10 client.18363 handle_command_reply: tid=0 2024-05-21T15:26:04.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 client.18363 shutdown 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 -- 172.21.15.78:0/3645624749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 msgr2=0x564d4dbd2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4dbd2dd0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x564d4db62de0 tx=0x7f39e800fee0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 -- 172.21.15.78:0/3645624749 shutdown_connections 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564d4d7453a0 0x564d4dbd6890 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564d4d7b1390 0x564d4dbd2dd0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f39e5ffb640 0 client.18363 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d4d7906b0 0x564d4db63ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 --2- 172.21.15.78:0/3645624749 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564d4d7884d0 0x564d4db63580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 -- 172.21.15.78:0/3645624749 >> 172.21.15.78:0/3645624749 conn(0x564d4d7a54e0 msgr2=0x564d4db51720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 -- 172.21.15.78:0/3645624749 shutdown_connections 2024-05-21T15:26:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 1 -- 172.21.15.78:0/3645624749 wait complete. 2024-05-21T15:26:04.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a0bd65740 20 client.18363 trim_cache size 0 max 16384 2024-05-21T15:26:04.629 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:04.629 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:26:04.629 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:26:04.629 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a04de3640 1 -- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f39ec0a2d00 msgr2=0x7f39ec0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a04de3640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f39ec0a2d00 0x7f39ec0a5210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39f80bcb60 tx=0x7f39f80b22f0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a04de3640 1 -- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00108f60 msgr2=0x7f3a0019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.616+0000 7f3a04de3640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00108f60 0x7f3a0019df80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f39f8031a20 tx=0x7f39f8031a50 comp rx=0 tx=0).stop 2024-05-21T15:26:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.617+0000 7f3a04de3640 1 -- 172.21.15.78:0/928110982 shutdown_connections 2024-05-21T15:26:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.617+0000 7f3a04de3640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f39ec0a2d00 0x7f39ec0a5210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.617+0000 7f3a04de3640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a00108f60 0x7f3a0019df80 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.617+0000 7f3a04de3640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a00108ac0 0x7f3a00199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.617+0000 7f3a04de3640 1 --2- 172.21.15.78:0/928110982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a00103210 0x7f3a00199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.617+0000 7f3a04de3640 1 -- 172.21.15.78:0/928110982 >> 172.21.15.78:0/928110982 conn(0x7f3a000fe180 msgr2=0x7f3a00105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.617+0000 7f3a04de3640 1 -- 172.21.15.78:0/928110982 shutdown_connections 2024-05-21T15:26:04.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.617+0000 7f3a04de3640 1 -- 172.21.15.78:0/928110982 wait complete. 2024-05-21T15:26:04.632 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:26:04.633 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:26:04.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 -- 172.21.15.78:0/2729973860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878101b30 msgr2=0x7fb878101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/2729973860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878101b30 0x7fb878101f40 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb864009900 tx=0x7fb86402a410 comp rx=0 tx=0).stop 2024-05-21T15:26:04.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 -- 172.21.15.78:0/2729973860 shutdown_connections 2024-05-21T15:26:04.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/2729973860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb87810c370 0x7fb87810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/2729973860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878101b30 0x7fb878101f40 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/2729973860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb878107a10 0x7fb8781015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 -- 172.21.15.78:0/2729973860 >> 172.21.15.78:0/2729973860 conn(0x7fb878077130 msgr2=0x7fb878077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:04.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 -- 172.21.15.78:0/2729973860 shutdown_connections 2024-05-21T15:26:04.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 -- 172.21.15.78:0/2729973860 wait complete. 2024-05-21T15:26:04.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 Processor -- start 2024-05-21T15:26:04.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.740+0000 7fb87f0f9640 1 -- start start 2024-05-21T15:26:04.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87f0f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878101b30 0x7fb878199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87f0f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878107a10 0x7fb878199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87f0f9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb87810c370 0x7fb87819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87f0f9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb878111d20 con 0x7fb878107a10 2024-05-21T15:26:04.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87f0f9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb878111ba0 con 0x7fb878101b30 2024-05-21T15:26:04.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87f0f9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb878111ea0 con 0x7fb87810c370 2024-05-21T15:26:04.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878107a10 0x7fb878199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878107a10 0x7fb878199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49112/0 (socket says 172.21.15.78:49112) 2024-05-21T15:26:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 -- 172.21.15.78:0/1198527573 learned_addr learned my addr 172.21.15.78:0/1198527573 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87d696640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb87810c370 0x7fb87819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87ce95640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878101b30 0x7fb878199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 -- 172.21.15.78:0/1198527573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb87810c370 msgr2=0x7fb87819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb87810c370 0x7fb87819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 -- 172.21.15.78:0/1198527573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878101b30 msgr2=0x7fb878199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878101b30 0x7fb878199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 -- 172.21.15.78:0/1198527573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87819e5f0 con 0x7fb878107a10 2024-05-21T15:26:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb87ce95640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878101b30 0x7fb878199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb877fff640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878107a10 0x7fb878199cf0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb864005940 tx=0x7fb8640040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb875ffb640 1 -- 172.21.15.78:0/1198527573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86403f030 con 0x7fb878107a10 2024-05-21T15:26:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb875ffb640 1 -- 172.21.15.78:0/1198527573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8640044c0 con 0x7fb878107a10 2024-05-21T15:26:04.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.741+0000 7fb875ffb640 1 -- 172.21.15.78:0/1198527573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864035070 con 0x7fb878107a10 2024-05-21T15:26:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.742+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87819e880 con 0x7fb878107a10 2024-05-21T15:26:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.742+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87819ed20 con 0x7fb878107a10 2024-05-21T15:26:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.743+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb878069140 con 0x7fb878107a10 2024-05-21T15:26:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.744+0000 7fb875ffb640 1 -- 172.21.15.78:0/1198527573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb864042030 con 0x7fb878107a10 2024-05-21T15:26:04.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.745+0000 7fb875ffb640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb8680a2d80 0x7fb8680a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:04.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.745+0000 7fb87ce95640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb8680a2d80 0x7fb8680a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:04.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.745+0000 7fb875ffb640 1 -- 172.21.15.78:0/1198527573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fb8640ef880 con 0x7fb878107a10 2024-05-21T15:26:04.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.745+0000 7fb87ce95640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb8680a2d80 0x7fb8680a5290 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb878102a10 tx=0x7fb86c005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:04.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.749+0000 7fb875ffb640 1 -- 172.21.15.78:0/1198527573 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8640bc410 con 0x7fb878107a10 2024-05-21T15:26:04.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:04 smithi078 ceph-mon[31997]: pgmap v308: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:26:04.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:04 smithi084 ceph-mon[36334]: pgmap v308: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:26:04.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:04 smithi172 ceph-mon[36402]: pgmap v308: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:26:04.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.965+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8781a62f0 con 0x7fb878107a10 2024-05-21T15:26:04.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.966+0000 7fb875ffb640 1 -- 172.21.15.78:0/1198527573 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7fb8640bbd10 con 0x7fb878107a10 2024-05-21T15:26:04.970 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:26:04.970 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:26:04.970 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:26:04.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb8680a2d80 msgr2=0x7fb8680a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb8680a2d80 0x7fb8680a5290 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb878102a10 tx=0x7fb86c005f90 comp rx=0 tx=0).stop 2024-05-21T15:26:04.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878107a10 msgr2=0x7fb878199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:04.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878107a10 0x7fb878199cf0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb864005940 tx=0x7fb8640040c0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 shutdown_connections 2024-05-21T15:26:04.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fb8680a2d80 0x7fb8680a5290 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb87810c370 0x7fb87819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb878107a10 0x7fb878199cf0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 --2- 172.21.15.78:0/1198527573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb878101b30 0x7fb878199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.968+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 >> 172.21.15.78:0/1198527573 conn(0x7fb878077130 msgr2=0x7fb878073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.969+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 shutdown_connections 2024-05-21T15:26:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:04.969+0000 7fb87f0f9640 1 -- 172.21.15.78:0/1198527573 wait complete. 2024-05-21T15:26:04.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:26:05.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1226170049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 msgr2=0x7f7b98101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1226170049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 0x7f7b98101eb0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009900 tx=0x7f7b8002a410 comp rx=0 tx=0).stop 2024-05-21T15:26:05.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1226170049 shutdown_connections 2024-05-21T15:26:05.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1226170049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b9810c330 0x7f7b9810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1226170049 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 0x7f7b98101eb0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1226170049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b981079d0 0x7f7b98101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1226170049 >> 172.21.15.78:0/1226170049 conn(0x7f7b980770b0 msgr2=0x7f7b980774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1226170049 shutdown_connections 2024-05-21T15:26:05.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.094+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1226170049 wait complete. 2024-05-21T15:26:05.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.095+0000 7f7b9c8b8640 1 Processor -- start 2024-05-21T15:26:05.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.095+0000 7f7b9c8b8640 1 -- start start 2024-05-21T15:26:05.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.095+0000 7f7b9c8b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 0x7f7b981997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.095+0000 7f7b9c8b8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b981079d0 0x7f7b98199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.095+0000 7f7b9c8b8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9810c330 0x7f7b9819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.095+0000 7f7b9c8b8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98111ce0 con 0x7f7b98101aa0 2024-05-21T15:26:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.095+0000 7f7b9c8b8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98111b60 con 0x7f7b981079d0 2024-05-21T15:26:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.095+0000 7f7b9c8b8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98111e60 con 0x7f7b9810c330 2024-05-21T15:26:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 0x7f7b981997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 0x7f7b981997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49134/0 (socket says 172.21.15.78:49134) 2024-05-21T15:26:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 -- 172.21.15.78:0/1398672053 learned_addr learned my addr 172.21.15.78:0/1398672053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b96d9d640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9810c330 0x7f7b9819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b981079d0 0x7f7b98199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 -- 172.21.15.78:0/1398672053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9810c330 msgr2=0x7f7b9819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9810c330 0x7f7b9819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 -- 172.21.15.78:0/1398672053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b981079d0 msgr2=0x7f7b98199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b981079d0 0x7f7b98199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 -- 172.21.15.78:0/1398672053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9819e610 con 0x7f7b98101aa0 2024-05-21T15:26:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b96d9d640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9810c330 0x7f7b9819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b981079d0 0x7f7b98199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9659c640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 0x7f7b981997b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800ea50 tx=0x7f7b8800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b8f7fe640 1 -- 172.21.15.78:0/1398672053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800cdc0 con 0x7f7b98101aa0 2024-05-21T15:26:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b8f7fe640 1 -- 172.21.15.78:0/1398672053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b88005020 con 0x7f7b98101aa0 2024-05-21T15:26:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1398672053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9819e900 con 0x7f7b98101aa0 2024-05-21T15:26:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b8f7fe640 1 -- 172.21.15.78:0/1398672053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88011690 con 0x7f7b98101aa0 2024-05-21T15:26:05.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.096+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1398672053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9819edd0 con 0x7f7b98101aa0 2024-05-21T15:26:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.099+0000 7f7ba3835740 1 Processor -- start 2024-05-21T15:26:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.099+0000 7f7ba3835740 1 -- start start 2024-05-21T15:26:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x557262196d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x557262197260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 0x5572621977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7ba3835740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572620fcc40 con 0x557261dcf4d0 2024-05-21T15:26:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7ba3835740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5572620a4580 con 0x557261dd76b0 2024-05-21T15:26:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7ba3835740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5572620f8c00 con 0x557261df8390 2024-05-21T15:26:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b8f7fe640 1 -- 172.21.15.78:0/1398672053 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7b88011890 con 0x7f7b98101aa0 2024-05-21T15:26:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b9659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x557262197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b9659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x557262197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49140/0 (socket says 172.21.15.78:49140) 2024-05-21T15:26:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b9659c640 1 -- 172.21.15.78:0/958031369 learned_addr learned my addr 172.21.15.78:0/958031369 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b96d9d640 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x557262196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 0x5572621977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b9659c640 1 -- 172.21.15.78:0/958031369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 msgr2=0x557262196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b9659c640 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x557262196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b9659c640 1 -- 172.21.15.78:0/958031369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 msgr2=0x5572621977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b9659c640 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 0x5572621977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.100+0000 7f7b9659c640 1 -- 172.21.15.78:0/958031369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557261e103f0 con 0x557261dcf4d0 2024-05-21T15:26:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.101+0000 7f7b9659c640 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x557262197260 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7b880cce70 tx=0x7f7b880d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a05a89fcaf2d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.101+0000 7f7b8f7fe640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7b840a2d00 0x7f7b840a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.101+0000 7f7b8cff9640 1 -- 172.21.15.78:0/958031369 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b880de070 con 0x557261dcf4d0 2024-05-21T15:26:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.101+0000 7f7b96d9d640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7b840a2d00 0x7f7b840a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.101+0000 7f7b8cff9640 1 -- 172.21.15.78:0/958031369 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8809c3d0 con 0x557261dcf4d0 2024-05-21T15:26:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.101+0000 7f7b8cff9640 1 -- 172.21.15.78:0/958031369 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8809b810 con 0x557261dcf4d0 2024-05-21T15:26:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.101+0000 7f7b8f7fe640 1 -- 172.21.15.78:0/1398672053 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f7b880cf840 con 0x7f7b98101aa0 2024-05-21T15:26:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 -- 172.21.15.78:0/958031369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 msgr2=0x557262197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x557262197260 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7b880cce70 tx=0x7f7b880d7690 comp rx=0 tx=0).stop 2024-05-21T15:26:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7b96d9d640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7b840a2d00 0x7f7b840a5210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x557262193e90 tx=0x7f7b90005f30 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 -- 172.21.15.78:0/958031369 shutdown_connections 2024-05-21T15:26:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 0x5572621977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x557262197260 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 --2- 172.21.15.78:0/958031369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x557262196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 -- 172.21.15.78:0/958031369 >> 172.21.15.78:0/958031369 conn(0x557261dec4e0 msgr2=0x5572621934d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 -- 172.21.15.78:0/958031369 shutdown_connections 2024-05-21T15:26:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 -- 172.21.15.78:0/958031369 wait complete. 2024-05-21T15:26:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 -- start start 2024-05-21T15:26:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.102+0000 7f7ba3835740 1 Processor -- start 2024-05-21T15:26:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x5572621a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 0x5572621a3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x5572621a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7ba3835740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572621a3fb0 con 0x557261dcf4d0 2024-05-21T15:26:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7ba3835740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5572620b7c40 con 0x557261dd76b0 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7ba3835740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5572621a4130 con 0x557261df8390 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7ba3835740 10 client.0 _ll_register_callbacks cb 0x557261e0fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x5572621a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x5572621a2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49150/0 (socket says 172.21.15.78:49150) 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 -- 172.21.15.78:0/2055048463 learned_addr learned my addr 172.21.15.78:0/2055048463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 0x5572621a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b96d9d640 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x5572621a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 -- 172.21.15.78:0/2055048463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 msgr2=0x5572621a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x5572621a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 -- 172.21.15.78:0/2055048463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 msgr2=0x5572621a3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 0x5572621a3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 -- 172.21.15.78:0/2055048463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557261eda7d0 con 0x557261dcf4d0 2024-05-21T15:26:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b9659c640 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x5572621a2d40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800ea20 tx=0x7f7b880d5320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c4804622734c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b7dffb640 1 -- 172.21.15.78:0/2055048463 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88051400 con 0x557261dcf4d0 2024-05-21T15:26:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7b7dffb640 1 -- 172.21.15.78:0/2055048463 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f7b88099fd0 con 0x557261dcf4d0 2024-05-21T15:26:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.103+0000 7f7ba3835740 1 -- 172.21.15.78:0/2055048463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8809c3d0 con 0x557261dcf4d0 2024-05-21T15:26:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7b7dffb640 1 -- 172.21.15.78:0/2055048463 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7b88051750 con 0x557261dcf4d0 2024-05-21T15:26:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7ba3835740 10 client.18381 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7ba3835740 1 -- 172.21.15.78:0/2055048463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5572621b0790 con 0x557261dcf4d0 2024-05-21T15:26:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7b7dffb640 1 -- 172.21.15.78:0/2055048463 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8809c3d0 con 0x557261dcf4d0 2024-05-21T15:26:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7b7dffb640 1 -- 172.21.15.78:0/2055048463 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7b8800f040 con 0x557261dcf4d0 2024-05-21T15:26:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7ba3835740 10 client.18381 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7ba3835740 10 client.18381 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7ba3835740 20 client.18381 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7ba3835740 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x557261db6f60 0x5572621b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7ba3835740 4 client.18381 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7ba3835740 1 -- 172.21.15.78:0/2055048463 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55726210a370 con 0x557261db6f60 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.104+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x557261db6f60 0x5572621b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.105+0000 7f7b7dffb640 10 client.18381 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.105+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x557261db6f60 0x5572621b3af0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=437b58da202088d5 server_cookie=a83f7c609f020f2a in_seq=0 out_seq=0 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.105+0000 7f7b7dffb640 1 -- 172.21.15.78:0/2055048463 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55726210a370 con 0x557261db6f60 2024-05-21T15:26:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.105+0000 7f7b7dffb640 10 client.18381 handle_command_reply: tid=0 2024-05-21T15:26:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.105+0000 7f7ba3835740 1 client.18381 shutdown 2024-05-21T15:26:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.105+0000 7f7ba3835740 1 -- 172.21.15.78:0/2055048463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 msgr2=0x5572621a2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.105+0000 7f7ba3835740 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x5572621a2d40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800ea20 tx=0x7f7b880d5320 comp rx=0 tx=0).stop 2024-05-21T15:26:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 1 -- 172.21.15.78:0/2055048463 shutdown_connections 2024-05-21T15:26:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x557261db6f60 0x5572621b3af0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dd76b0 0x5572621a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x5572621a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 1 --2- 172.21.15.78:0/2055048463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dcf4d0 0x5572621a2d40 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 1 -- 172.21.15.78:0/2055048463 >> 172.21.15.78:0/2055048463 conn(0x557261dec4e0 msgr2=0x557262196cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7b7dffb640 0 client.18381 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:26:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 1 -- 172.21.15.78:0/2055048463 shutdown_connections 2024-05-21T15:26:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 1 -- 172.21.15.78:0/2055048463 wait complete. 2024-05-21T15:26:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.106+0000 7f7ba3835740 20 client.18381 trim_cache size 0 max 16384 2024-05-21T15:26:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.119+0000 7f7ba3835740 1 Processor -- start 2024-05-21T15:26:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.119+0000 7f7ba3835740 1 -- start start 2024-05-21T15:26:05.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.119+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dcf4d0 0x55726219b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.119+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x557262192b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.119+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x557262193090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7ba3835740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981aed20 con 0x557261dd76b0 2024-05-21T15:26:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7ba3835740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981b0860 con 0x557261dcf4d0 2024-05-21T15:26:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7ba3835740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981af3f0 con 0x557261df8390 2024-05-21T15:26:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x557262192b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x557262192b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49160/0 (socket says 172.21.15.78:49160) 2024-05-21T15:26:05.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 -- 172.21.15.78:0/1074412221 learned_addr learned my addr 172.21.15.78:0/1074412221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b9659c640 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dcf4d0 0x55726219b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 -- 172.21.15.78:0/1074412221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 msgr2=0x557262193090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x557262193090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 -- 172.21.15.78:0/1074412221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dcf4d0 msgr2=0x55726219b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dcf4d0 0x55726219b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 -- 172.21.15.78:0/1074412221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726218ed10 con 0x557261dd76b0 2024-05-21T15:26:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b9659c640 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dcf4d0 0x55726219b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:05.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x557262192b50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005940 tx=0x7f7b800058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f75c6e2dcdfe19 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b7ffff640 1 -- 172.21.15.78:0/1074412221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80049070 con 0x557261dd76b0 2024-05-21T15:26:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b7ffff640 1 -- 172.21.15.78:0/1074412221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b80005520 con 0x557261dd76b0 2024-05-21T15:26:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.120+0000 7f7b7ffff640 1 -- 172.21.15.78:0/1074412221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80035040 con 0x557261dd76b0 2024-05-21T15:26:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 -- 172.21.15.78:0/1074412221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 msgr2=0x557262192b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x557262192b50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005940 tx=0x7f7b800058b0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 -- 172.21.15.78:0/1074412221 shutdown_connections 2024-05-21T15:26:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261df8390 0x557262193090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x557262192b50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 --2- 172.21.15.78:0/1074412221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261dcf4d0 0x55726219b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 -- 172.21.15.78:0/1074412221 >> 172.21.15.78:0/1074412221 conn(0x557261dec4e0 msgr2=0x55726219a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 -- 172.21.15.78:0/1074412221 shutdown_connections 2024-05-21T15:26:05.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 -- 172.21.15.78:0/1074412221 wait complete. 2024-05-21T15:26:05.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 -- start start 2024-05-21T15:26:05.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.121+0000 7f7ba3835740 1 Processor -- start 2024-05-21T15:26:05.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261dcf4d0 0x5572621b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x5572621b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7ba3835740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261df8390 0x5572621b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7ba3835740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981b0190 con 0x557261dd76b0 2024-05-21T15:26:05.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7ba3835740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981afac0 con 0x557261df8390 2024-05-21T15:26:05.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7ba3835740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981acd20 con 0x557261dcf4d0 2024-05-21T15:26:05.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7ba3835740 10 client.0 _ll_register_callbacks cb 0x7f7b880d5a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x5572621b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x5572621b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49166/0 (socket says 172.21.15.78:49166) 2024-05-21T15:26:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 -- 172.21.15.78:0/4048767569 learned_addr learned my addr 172.21.15.78:0/4048767569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b96d9d640 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261df8390 0x5572621b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 -- 172.21.15.78:0/4048767569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261dcf4d0 msgr2=0x5572621b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261dcf4d0 0x5572621b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 -- 172.21.15.78:0/4048767569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261df8390 msgr2=0x5572621b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261df8390 0x5572621b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 -- 172.21.15.78:0/4048767569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557262219c20 con 0x557261dd76b0 2024-05-21T15:26:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b95d9b640 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x5572621b5ca0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009230 tx=0x7f7b80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6652a22fdd1e5fe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b7dffb640 1 -- 172.21.15.78:0/4048767569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80049040 con 0x557261dd76b0 2024-05-21T15:26:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7b7dffb640 1 -- 172.21.15.78:0/4048767569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f7b80005300 con 0x557261dd76b0 2024-05-21T15:26:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.122+0000 7f7ba3835740 1 -- 172.21.15.78:0/4048767569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b80005520 con 0x557261dd76b0 2024-05-21T15:26:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7b7dffb640 1 -- 172.21.15.78:0/4048767569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7b8005b050 con 0x557261dd76b0 2024-05-21T15:26:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7ba3835740 10 client.18387 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7ba3835740 1 -- 172.21.15.78:0/4048767569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55726221ae10 con 0x557261dd76b0 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7b7dffb640 1 -- 172.21.15.78:0/4048767569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b80005520 con 0x557261dd76b0 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7b7dffb640 1 -- 172.21.15.78:0/4048767569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7b800445b0 con 0x557261dd76b0 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7ba3835740 10 client.18387 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7ba3835740 10 client.18387 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7ba3835740 20 client.18387 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7ba3835740 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x557261d8c3a0 0x55726221d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7ba3835740 4 client.18387 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7ba3835740 1 -- 172.21.15.78:0/4048767569 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55726219cdc0 con 0x557261d8c3a0 2024-05-21T15:26:05.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7b9659c640 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x557261d8c3a0 0x55726221d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7b7dffb640 10 client.18387 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:26:05.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.123+0000 7f7b9659c640 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x557261d8c3a0 0x55726221d4f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89385907ffdb2c9c server_cookie=a05c826e317eaf7a in_seq=0 out_seq=0 2024-05-21T15:26:05.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7b7dffb640 1 -- 172.21.15.78:0/4048767569 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55726219cdc0 con 0x557261d8c3a0 2024-05-21T15:26:05.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7b7dffb640 10 client.18387 handle_command_reply: tid=0 2024-05-21T15:26:05.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 client.18387 shutdown 2024-05-21T15:26:05.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 -- 172.21.15.78:0/4048767569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 msgr2=0x5572621b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x5572621b5ca0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009230 tx=0x7f7b80004770 comp rx=0 tx=0).stop 2024-05-21T15:26:05.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 -- 172.21.15.78:0/4048767569 shutdown_connections 2024-05-21T15:26:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x557261d8c3a0 0x55726221d4f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557261df8390 0x5572621b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7b7dffb640 0 client.18387 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:26:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557261dd76b0 0x5572621b5ca0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 --2- 172.21.15.78:0/4048767569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557261dcf4d0 0x5572621b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 -- 172.21.15.78:0/4048767569 >> 172.21.15.78:0/4048767569 conn(0x557261dec4e0 msgr2=0x5572621a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 -- 172.21.15.78:0/4048767569 shutdown_connections 2024-05-21T15:26:05.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 1 -- 172.21.15.78:0/4048767569 wait complete. 2024-05-21T15:26:05.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7ba3835740 20 client.18387 trim_cache size 0 max 16384 2024-05-21T15:26:05.139 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:05.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7b840a2d00 msgr2=0x7f7b840a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7b840a2d00 0x7f7b840a5210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x557262193e90 tx=0x7f7b90005f30 comp rx=0 tx=0).stop 2024-05-21T15:26:05.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 msgr2=0x7f7b981997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.124+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 0x7f7b981997b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800ea50 tx=0x7f7b8800ef10 comp rx=0 tx=0).stop 2024-05-21T15:26:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.125+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1398672053 shutdown_connections 2024-05-21T15:26:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.125+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b981079d0 0x7f7b98199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.125+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7b840a2d00 0x7f7b840a5210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.125+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b9810c330 0x7f7b9819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.125+0000 7f7b9c8b8640 1 --2- 172.21.15.78:0/1398672053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b98101aa0 0x7f7b981997b0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.125+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1398672053 >> 172.21.15.78:0/1398672053 conn(0x7f7b980770b0 msgr2=0x7f7b98068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.125+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1398672053 shutdown_connections 2024-05-21T15:26:05.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.125+0000 7f7b9c8b8640 1 -- 172.21.15.78:0/1398672053 wait complete. 2024-05-21T15:26:05.142 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:26:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/975139288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd41079d0 msgr2=0x7fbbd4101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/975139288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd41079d0 0x7fbbd4101560 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009900 tx=0x7fbbc402a410 comp rx=0 tx=0).stop 2024-05-21T15:26:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/975139288 shutdown_connections 2024-05-21T15:26:05.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/975139288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbd410c330 0x7fbbd410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/975139288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbd4101aa0 0x7fbbd4101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/975139288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd41079d0 0x7fbbd4101560 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/975139288 >> 172.21.15.78:0/975139288 conn(0x7fbbd40770b0 msgr2=0x7fbbd40774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/975139288 shutdown_connections 2024-05-21T15:26:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.252+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/975139288 wait complete. 2024-05-21T15:26:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd9ba8640 1 Processor -- start 2024-05-21T15:26:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd9ba8640 1 -- start start 2024-05-21T15:26:05.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd9ba8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbd4101aa0 0x7fbbd4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd9ba8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbd41079d0 0x7fbbd4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd9ba8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd410c330 0x7fbbd419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd9ba8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4111ce0 con 0x7fbbd410c330 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd9ba8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4111b60 con 0x7fbbd41079d0 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd9ba8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4111e60 con 0x7fbbd4101aa0 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd410c330 0x7fbbd419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd410c330 0x7fbbd419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49188/0 (socket says 172.21.15.78:49188) 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd3fff640 1 -- 172.21.15.78:0/3350494263 learned_addr learned my addr 172.21.15.78:0/3350494263 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbd41079d0 0x7fbbd4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbd4101aa0 0x7fbbd4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd3fff640 1 -- 172.21.15.78:0/3350494263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbd4101aa0 msgr2=0x7fbbd4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.253+0000 7fbbd3fff640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbd4101aa0 0x7fbbd4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd3fff640 1 -- 172.21.15.78:0/3350494263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbd41079d0 msgr2=0x7fbbd4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd3fff640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbd41079d0 0x7fbbd4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd3fff640 1 -- 172.21.15.78:0/3350494263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd419d580 con 0x7fbbd410c330 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbd4101aa0 0x7fbbd4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd3fff640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd410c330 0x7fbbd419cf40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00e970 tx=0x7fbbcc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd0ff9640 1 -- 172.21.15.78:0/3350494263 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc00ccc0 con 0x7fbbd410c330 2024-05-21T15:26:05.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/3350494263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd419d870 con 0x7fbbd410c330 2024-05-21T15:26:05.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd0ff9640 1 -- 172.21.15.78:0/3350494263 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbcc00ce60 con 0x7fbbd410c330 2024-05-21T15:26:05.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd0ff9640 1 -- 172.21.15.78:0/3350494263 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc010490 con 0x7fbbd410c330 2024-05-21T15:26:05.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.254+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/3350494263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd41a51f0 con 0x7fbbd410c330 2024-05-21T15:26:05.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.256+0000 7fbbe0a4d740 1 Processor -- start 2024-05-21T15:26:05.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd0ff9640 1 -- 172.21.15.78:0/3350494263 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbbcc0106b0 con 0x7fbbd410c330 2024-05-21T15:26:05.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbe0a4d740 1 -- start start 2024-05-21T15:26:05.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63bd390 0x55eba675bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63944d0 0x55eba675c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 0x55eba675c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eba66c1c40 con 0x55eba63bd390 2024-05-21T15:26:05.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55eba6669580 con 0x55eba63944d0 2024-05-21T15:26:05.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eba66bdc00 con 0x55eba639c6b0 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63bd390 0x55eba675bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63bd390 0x55eba675bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49192/0 (socket says 172.21.15.78:49192) 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 -- 172.21.15.78:0/11731975 learned_addr learned my addr 172.21.15.78:0/11731975 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63944d0 0x55eba675c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 0x55eba675c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 -- 172.21.15.78:0/11731975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 msgr2=0x55eba675c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 0x55eba675c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 -- 172.21.15.78:0/11731975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63944d0 msgr2=0x55eba675c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63944d0 0x55eba675c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd2ffd640 1 -- 172.21.15.78:0/11731975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba63d53f0 con 0x55eba63bd390 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.257+0000 7fbbd0ff9640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbbbc0a2d00 0x7fbbbc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.258+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbbbc0a2d00 0x7fbbbc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.258+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63bd390 0x55eba675bb90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8014890 tx=0x7fbbc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cab29369f91d06e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.258+0000 7fbbd0ff9640 1 -- 172.21.15.78:0/3350494263 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fbbcc0cf9a0 con 0x7fbbd410c330 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.258+0000 7fbbb9ffb640 1 -- 172.21.15.78:0/11731975 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc801a030 con 0x55eba63bd390 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.258+0000 7fbbb9ffb640 1 -- 172.21.15.78:0/11731975 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbc8003e50 con 0x55eba63bd390 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.258+0000 7fbbb9ffb640 1 -- 172.21.15.78:0/11731975 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8012cd0 con 0x55eba63bd390 2024-05-21T15:26:05.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/11731975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63bd390 msgr2=0x55eba675bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbbbc0a2d00 0x7fbbbc0a5210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4000c00 tx=0x7fbbc4031400 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63bd390 0x55eba675bb90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8014890 tx=0x7fbbc8009b50 comp rx=0 tx=0).stop 2024-05-21T15:26:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/11731975 shutdown_connections 2024-05-21T15:26:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63944d0 0x55eba675c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 0x55eba675c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/11731975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63bd390 0x55eba675bb90 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/11731975 >> 172.21.15.78:0/11731975 conn(0x55eba63b14e0 msgr2=0x55eba67584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/11731975 shutdown_connections 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/11731975 wait complete. 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 -- start start 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.259+0000 7fbbe0a4d740 1 Processor -- start 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba67684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba639c6b0 0x55eba67689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63bd390 0x55eba67746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eba676b220 con 0x55eba63944d0 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55eba667cc40 con 0x55eba639c6b0 2024-05-21T15:26:05.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eba676b3a0 con 0x55eba63bd390 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbe0a4d740 10 client.0 _ll_register_callbacks cb 0x55eba63d4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba67684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba67684a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49194/0 (socket says 172.21.15.78:49194) 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 -- 172.21.15.78:0/2135143678 learned_addr learned my addr 172.21.15.78:0/2135143678 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba639c6b0 0x55eba67689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 -- 172.21.15.78:0/2135143678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63bd390 msgr2=0x55eba67746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd3fff640 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63bd390 0x55eba67746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63bd390 0x55eba67746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 -- 172.21.15.78:0/2135143678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba639c6b0 msgr2=0x55eba67689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba639c6b0 0x55eba67689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 -- 172.21.15.78:0/2135143678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba649f7d0 con 0x55eba63944d0 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba67684a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4033000 tx=0x7fbbc4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def25d4a77cf8c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/2135143678 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc404c070 con 0x55eba63944d0 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/2135143678 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fbbc4005460 con 0x55eba63944d0 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.260+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/2135143678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc8003e50 con 0x55eba63944d0 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/2135143678 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbbc4058050 con 0x55eba63944d0 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbe0a4d740 10 client.18399 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/2135143678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eba6777860 con 0x55eba63944d0 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/2135143678 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbc8003e50 con 0x55eba63944d0 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/2135143678 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fbbc40036f0 con 0x55eba63944d0 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbe0a4d740 10 client.18399 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbe0a4d740 10 client.18399 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbe0a4d740 20 client.18399 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55eba637bf60 0x55eba677ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbe0a4d740 4 client.18399 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/2135143678 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eba66cf370 con 0x55eba637bf60 2024-05-21T15:26:05.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.261+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55eba637bf60 0x55eba677ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.262+0000 7fbbb2ffd640 10 client.18399 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:26:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.262+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55eba637bf60 0x55eba677ad30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b6a9479d6caad39 server_cookie=fb3190f07982cad1 in_seq=0 out_seq=0 2024-05-21T15:26:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/2135143678 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eba66cf370 con 0x55eba637bf60 2024-05-21T15:26:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbb2ffd640 10 client.18399 handle_command_reply: tid=0 2024-05-21T15:26:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 client.18399 shutdown 2024-05-21T15:26:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/2135143678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 msgr2=0x55eba67684a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba67684a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4033000 tx=0x7fbbc4009260 comp rx=0 tx=0).stop 2024-05-21T15:26:05.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/2135143678 shutdown_connections 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55eba637bf60 0x55eba677ad30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba639c6b0 0x55eba67689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63bd390 0x55eba67746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbb2ffd640 0 client.18399 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/2135143678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba67684a0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/2135143678 >> 172.21.15.78:0/2135143678 conn(0x55eba63b14e0 msgr2=0x55eba675d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/2135143678 shutdown_connections 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/2135143678 wait complete. 2024-05-21T15:26:05.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.263+0000 7fbbe0a4d740 20 client.18399 trim_cache size 0 max 16384 2024-05-21T15:26:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbe0a4d740 1 Processor -- start 2024-05-21T15:26:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbe0a4d740 1 -- start start 2024-05-21T15:26:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63944d0 0x55eba6757570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba639c6b0 0x55eba67742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 0x55eba675df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41acb90 con 0x55eba639c6b0 2024-05-21T15:26:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41ae6d0 con 0x55eba63bd390 2024-05-21T15:26:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41ad260 con 0x55eba63944d0 2024-05-21T15:26:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbd2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba639c6b0 0x55eba67742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbd2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba639c6b0 0x55eba67742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49206/0 (socket says 172.21.15.78:49206) 2024-05-21T15:26:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbd2ffd640 1 -- 172.21.15.78:0/1033073454 learned_addr learned my addr 172.21.15.78:0/1033073454 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbd2ffd640 1 -- 172.21.15.78:0/1033073454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63944d0 msgr2=0x55eba6757570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/1033073454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63944d0 0x55eba6757570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbd2ffd640 1 -- 172.21.15.78:0/1033073454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 msgr2=0x55eba675df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/1033073454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 0x55eba675df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.277+0000 7fbbd2ffd640 1 -- 172.21.15.78:0/1033073454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba6752870 con 0x55eba639c6b0 2024-05-21T15:26:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.278+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/1033073454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba639c6b0 0x55eba67742b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80143a0 tx=0x7fbbc80140f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920f3410000dece server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.278+0000 7fbbb97fa640 1 -- 172.21.15.78:0/1033073454 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8014160 con 0x55eba639c6b0 2024-05-21T15:26:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.278+0000 7fbbb97fa640 1 -- 172.21.15.78:0/1033073454 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbc8003e50 con 0x55eba639c6b0 2024-05-21T15:26:05.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.278+0000 7fbbb97fa640 1 -- 172.21.15.78:0/1033073454 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc802a620 con 0x55eba639c6b0 2024-05-21T15:26:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.278+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/1033073454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba639c6b0 msgr2=0x55eba67742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.278+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/1033073454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba639c6b0 0x55eba67742b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80143a0 tx=0x7fbbc80140f0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/1033073454 shutdown_connections 2024-05-21T15:26:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/1033073454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 0x55eba675df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/1033073454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba639c6b0 0x55eba67742b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/1033073454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba63944d0 0x55eba6757570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/1033073454 >> 172.21.15.78:0/1033073454 conn(0x55eba63b14e0 msgr2=0x55eba675d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/1033073454 shutdown_connections 2024-05-21T15:26:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/1033073454 wait complete. 2024-05-21T15:26:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 -- start start 2024-05-21T15:26:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 Processor -- start 2024-05-21T15:26:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba6770b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 0x55eba676beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 0x55eba676c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41ae000 con 0x55eba63944d0 2024-05-21T15:26:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41ad930 con 0x55eba63bd390 2024-05-21T15:26:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41aab90 con 0x55eba639c6b0 2024-05-21T15:26:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.279+0000 7fbbe0a4d740 10 client.0 _ll_register_callbacks cb 0x7fbbc4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba6770b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba6770b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49220/0 (socket says 172.21.15.78:49220) 2024-05-21T15:26:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 -- 172.21.15.78:0/3804414576 learned_addr learned my addr 172.21.15.78:0/3804414576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd3fff640 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 0x55eba676c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 -- 172.21.15.78:0/3804414576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 msgr2=0x55eba676beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 0x55eba676beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 -- 172.21.15.78:0/3804414576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 msgr2=0x55eba676c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 0x55eba676c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 -- 172.21.15.78:0/3804414576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba676cae0 con 0x55eba63944d0 2024-05-21T15:26:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd3fff640 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 0x55eba676c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbd37fe640 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba6770b60 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4006010 tx=0x7fbbc4032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37c8debfea485b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/3804414576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc404c070 con 0x55eba63944d0 2024-05-21T15:26:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/3804414576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fbbc40037f0 con 0x55eba63944d0 2024-05-21T15:26:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/3804414576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc8003e50 con 0x55eba63944d0 2024-05-21T15:26:05.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/3804414576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbbc406a050 con 0x55eba63944d0 2024-05-21T15:26:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbe0a4d740 10 client.18405 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/3804414576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eba67e2fa0 con 0x55eba63944d0 2024-05-21T15:26:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.280+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/3804414576 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbc8003e50 con 0x55eba63944d0 2024-05-21T15:26:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/3804414576 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fbbc4048560 con 0x55eba63944d0 2024-05-21T15:26:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbe0a4d740 10 client.18405 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbe0a4d740 10 client.18405 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:26:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbe0a4d740 20 client.18405 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55eba63513a0 0x55eba67e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbe0a4d740 4 client.18405 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/3804414576 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eba66c96c0 con 0x55eba63513a0 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55eba63513a0 0x55eba67e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbb2ffd640 10 client.18405 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.281+0000 7fbbd2ffd640 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55eba63513a0 0x55eba67e57e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b1684655a429347 server_cookie=9810c94d0bdefec8 in_seq=0 out_seq=0 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbb2ffd640 1 -- 172.21.15.78:0/3804414576 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eba66c96c0 con 0x55eba63513a0 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbb2ffd640 10 client.18405 handle_command_reply: tid=0 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 client.18405 shutdown 2024-05-21T15:26:05.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/3804414576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 msgr2=0x55eba6770b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba6770b60 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4006010 tx=0x7fbbc4032df0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/3804414576 shutdown_connections 2024-05-21T15:26:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x55eba63513a0 0x55eba67e57e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55eba63bd390 0x55eba676c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eba639c6b0 0x55eba676beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbb2ffd640 0 client.18405 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:26:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 --2- 172.21.15.78:0/3804414576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eba63944d0 0x55eba6770b60 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/3804414576 >> 172.21.15.78:0/3804414576 conn(0x55eba63b14e0 msgr2=0x55eba675e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/3804414576 shutdown_connections 2024-05-21T15:26:05.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 1 -- 172.21.15.78:0/3804414576 wait complete. 2024-05-21T15:26:05.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.282+0000 7fbbe0a4d740 20 client.18405 trim_cache size 0 max 16384 2024-05-21T15:26:05.295 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbbbc0a2d00 msgr2=0x7fbbbc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbbbc0a2d00 0x7fbbbc0a5210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4000c00 tx=0x7fbbc4031400 comp rx=0 tx=0).stop 2024-05-21T15:26:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd410c330 msgr2=0x7fbbd419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd410c330 0x7fbbd419cf40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00e970 tx=0x7fbbcc00ed70 comp rx=0 tx=0).stop 2024-05-21T15:26:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/3350494263 shutdown_connections 2024-05-21T15:26:05.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbd41079d0 0x7fbbd4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbbbc0a2d00 0x7fbbbc0a5210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd410c330 0x7fbbd419cf40 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 --2- 172.21.15.78:0/3350494263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbd4101aa0 0x7fbbd4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/3350494263 >> 172.21.15.78:0/3350494263 conn(0x7fbbd40770b0 msgr2=0x7fbbd4103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/3350494263 shutdown_connections 2024-05-21T15:26:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.283+0000 7fbbd9ba8640 1 -- 172.21.15.78:0/3350494263 wait complete. 2024-05-21T15:26:05.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:26:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 -- 172.21.15.78:0/161694117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 msgr2=0x7fa09010f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 --2- 172.21.15.78:0/161694117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 0x7fa09010f510 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa088009ca0 tx=0x7fa08802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 -- 172.21.15.78:0/161694117 shutdown_connections 2024-05-21T15:26:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 --2- 172.21.15.78:0/161694117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa09010a610 0x7fa09010ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 --2- 172.21.15.78:0/161694117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 0x7fa09010f510 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 --2- 172.21.15.78:0/161694117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa090103bb0 0x7fa090103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 -- 172.21.15.78:0/161694117 >> 172.21.15.78:0/161694117 conn(0x7fa0900fd210 msgr2=0x7fa0900ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 -- 172.21.15.78:0/161694117 shutdown_connections 2024-05-21T15:26:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.407+0000 7fa0967cc640 1 -- 172.21.15.78:0/161694117 wait complete. 2024-05-21T15:26:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa0967cc640 1 Processor -- start 2024-05-21T15:26:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa0967cc640 1 -- start start 2024-05-21T15:26:05.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa0967cc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa090103bb0 0x7fa0901997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa0967cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa09010a610 0x7fa090199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa0967cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 0x7fa09019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa0967cc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa090112dd0 con 0x7fa09010d000 2024-05-21T15:26:05.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa0967cc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa090112c50 con 0x7fa090103bb0 2024-05-21T15:26:05.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa0967cc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa090112f50 con 0x7fa09010a610 2024-05-21T15:26:05.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa094d69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 0x7fa09019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa094d69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 0x7fa09019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49240/0 (socket says 172.21.15.78:49240) 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.408+0000 7fa094d69640 1 -- 172.21.15.78:0/1754007044 learned_addr learned my addr 172.21.15.78:0/1754007044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa08ffff640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa090103bb0 0x7fa0901997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa09010a610 0x7fa090199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa094d69640 1 -- 172.21.15.78:0/1754007044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa09010a610 msgr2=0x7fa090199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa094d69640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa09010a610 0x7fa090199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa094d69640 1 -- 172.21.15.78:0/1754007044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa090103bb0 msgr2=0x7fa0901997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa094d69640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa090103bb0 0x7fa0901997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa094d69640 1 -- 172.21.15.78:0/1754007044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09019e610 con 0x7fa09010d000 2024-05-21T15:26:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa09010a610 0x7fa090199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa08ffff640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa090103bb0 0x7fa0901997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa094d69640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 0x7fa09019dfd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa088031a20 tx=0x7fa088031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa08d7fa640 1 -- 172.21.15.78:0/1754007044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088003c30 con 0x7fa09010d000 2024-05-21T15:26:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa08d7fa640 1 -- 172.21.15.78:0/1754007044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa088003dd0 con 0x7fa09010d000 2024-05-21T15:26:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa0967cc640 1 -- 172.21.15.78:0/1754007044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09019e8a0 con 0x7fa09010d000 2024-05-21T15:26:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa08d7fa640 1 -- 172.21.15.78:0/1754007044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08802fcf0 con 0x7fa09010d000 2024-05-21T15:26:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.409+0000 7fa0967cc640 1 -- 172.21.15.78:0/1754007044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09019edf0 con 0x7fa09010d000 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa08d7fa640 1 -- 172.21.15.78:0/1754007044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa088004050 con 0x7fa09010d000 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa09d670740 1 Processor -- start 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa09d670740 1 -- start start 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa09d670740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e509d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa09d670740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e50a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa09d670740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 0x55a35e50a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa09d670740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a35e46fc40 con 0x55a35e1424d0 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa09d670740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a35e417580 con 0x55a35e16b390 2024-05-21T15:26:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa09d670740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a35e46bc00 con 0x55a35e14a6b0 2024-05-21T15:26:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e50a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e50a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49252/0 (socket says 172.21.15.78:49252) 2024-05-21T15:26:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa08d7fa640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa07c0a2d00 0x7fa07c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa08ffff640 1 -- 172.21.15.78:0/3068163442 learned_addr learned my addr 172.21.15.78:0/3068163442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e509d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa094d69640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa07c0a2d00 0x7fa07c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.412+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 0x55a35e50a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa08d7fa640 1 -- 172.21.15.78:0/1754007044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fa0880f0850 con 0x7fa09010d000 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa08f7fe640 1 -- 172.21.15.78:0/3068163442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 msgr2=0x55a35e50a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 0x55a35e50a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa08f7fe640 1 -- 172.21.15.78:0/3068163442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 msgr2=0x55a35e50a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e50a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa08f7fe640 1 -- 172.21.15.78:0/3068163442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a35e1833f0 con 0x55a35e16b390 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa08ffff640 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e50a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e509d20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa090068540 tx=0x7fa07800e560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=734d17f5478d4bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa094d69640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa07c0a2d00 0x7fa07c0a5210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0880ee510 tx=0x7fa08808ee40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa0867fc640 1 -- 172.21.15.78:0/3068163442 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078018030 con 0x55a35e16b390 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa0867fc640 1 -- 172.21.15.78:0/3068163442 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0780035f0 con 0x55a35e16b390 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.413+0000 7fa0867fc640 1 -- 172.21.15.78:0/3068163442 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0780126b0 con 0x55a35e16b390 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 -- 172.21.15.78:0/3068163442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 msgr2=0x55a35e509d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e509d20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa090068540 tx=0x7fa07800e560 comp rx=0 tx=0).stop 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 -- 172.21.15.78:0/3068163442 shutdown_connections 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 0x55a35e50a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e50a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 --2- 172.21.15.78:0/3068163442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e509d20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 -- 172.21.15.78:0/3068163442 >> 172.21.15.78:0/3068163442 conn(0x55a35e15f4e0 msgr2=0x55a35e5064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 -- 172.21.15.78:0/3068163442 shutdown_connections 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 -- 172.21.15.78:0/3068163442 wait complete. 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.414+0000 7fa09d670740 1 -- start start 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa09d670740 1 Processor -- start 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa09d670740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e515d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa09d670740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 0x55a35e5162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa09d670740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e514280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa09d670740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a35e516fb0 con 0x55a35e1424d0 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa09d670740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a35e42ac40 con 0x55a35e16b390 2024-05-21T15:26:05.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa09d670740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a35e517130 con 0x55a35e14a6b0 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa09d670740 10 client.0 _ll_register_callbacks cb 0x55a35e182be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e515d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e515d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49262/0 (socket says 172.21.15.78:49262) 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 -- 172.21.15.78:0/3448429703 learned_addr learned my addr 172.21.15.78:0/3448429703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 -- 172.21.15.78:0/3448429703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 msgr2=0x55a35e5162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 0x55a35e5162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 -- 172.21.15.78:0/3448429703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 msgr2=0x55a35e514280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e514280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 -- 172.21.15.78:0/3448429703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a35e24d7d0 con 0x55a35e1424d0 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e515d70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa078004760 tx=0x7fa078003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b2f3aaa46d016b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa06f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.415+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3448429703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078018040 con 0x55a35e1424d0 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 1 -- 172.21.15.78:0/3448429703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0780035f0 con 0x55a35e1424d0 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3448429703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fa07801d460 con 0x55a35e1424d0 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3448429703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa07801d660 con 0x55a35e1424d0 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 10 client.18417 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 1 -- 172.21.15.78:0/3448429703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a35e5237c0 con 0x55a35e1424d0 2024-05-21T15:26:05.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3448429703 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0780035f0 con 0x55a35e1424d0 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3448429703 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa07800f040 con 0x55a35e1424d0 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 10 client.18417 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 10 client.18417 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 20 client.18417 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55a35e129f60 0x55a35e526b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 4 client.18417 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.416+0000 7fa09d670740 1 -- 172.21.15.78:0/3448429703 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a35e47d370 con 0x55a35e129f60 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.417+0000 7fa08ffff640 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55a35e129f60 0x55a35e526b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.417+0000 7fa06f7fe640 10 client.18417 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.417+0000 7fa08ffff640 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55a35e129f60 0x55a35e526b20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9ba905167cadbe2 server_cookie=a1c5a2a7655f2c66 in_seq=0 out_seq=0 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3448429703 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a35e47d370 con 0x55a35e129f60 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa06f7fe640 10 client.18417 handle_command_reply: tid=0 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 client.18417 shutdown 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 -- 172.21.15.78:0/3448429703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 msgr2=0x55a35e515d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e515d70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fa078004760 tx=0x7fa078003c80 comp rx=0 tx=0).stop 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 -- 172.21.15.78:0/3448429703 shutdown_connections 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55a35e129f60 0x55a35e526b20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e514280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e14a6b0 0x55a35e5162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa06f7fe640 0 client.18417 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:26:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 --2- 172.21.15.78:0/3448429703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e1424d0 0x55a35e515d70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 -- 172.21.15.78:0/3448429703 >> 172.21.15.78:0/3448429703 conn(0x55a35e15f4e0 msgr2=0x55a35e50ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 -- 172.21.15.78:0/3448429703 shutdown_connections 2024-05-21T15:26:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 1 -- 172.21.15.78:0/3448429703 wait complete. 2024-05-21T15:26:05.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.418+0000 7fa09d670740 20 client.18417 trim_cache size 0 max 16384 2024-05-21T15:26:05.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa09d670740 1 Processor -- start 2024-05-21T15:26:05.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa09d670740 1 -- start start 2024-05-21T15:26:05.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa09d670740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e1424d0 0x55a35e513260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa09d670740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa09d670740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e52a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa09d670740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901aed20 con 0x55a35e14a6b0 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa09d670740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901b0860 con 0x55a35e16b390 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa09d670740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901af3f0 con 0x55a35e1424d0 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49272/0 (socket says 172.21.15.78:49272) 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08ffff640 1 -- 172.21.15.78:0/1005361459 learned_addr learned my addr 172.21.15.78:0/1005361459 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e1424d0 0x55a35e513260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e52a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08ffff640 1 -- 172.21.15.78:0/1005361459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e1424d0 msgr2=0x55a35e513260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08ffff640 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e1424d0 0x55a35e513260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08ffff640 1 -- 172.21.15.78:0/1005361459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 msgr2=0x55a35e52a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08ffff640 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e52a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.432+0000 7fa08ffff640 1 -- 172.21.15.78:0/1005361459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a35e50b310 con 0x55a35e14a6b0 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.433+0000 7fa08ffff640 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529b50 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x55a35e50b2e0 tx=0x7fa0800098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565db210e9358b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.433+0000 7fa085ffb640 1 -- 172.21.15.78:0/1005361459 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08001f030 con 0x55a35e14a6b0 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.433+0000 7fa085ffb640 1 -- 172.21.15.78:0/1005361459 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa080005500 con 0x55a35e14a6b0 2024-05-21T15:26:05.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.433+0000 7fa085ffb640 1 -- 172.21.15.78:0/1005361459 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08001b600 con 0x55a35e14a6b0 2024-05-21T15:26:05.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 -- 172.21.15.78:0/1005361459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 msgr2=0x55a35e529b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529b50 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x55a35e50b2e0 tx=0x7fa0800098d0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 -- 172.21.15.78:0/1005361459 shutdown_connections 2024-05-21T15:26:05.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e16b390 0x55a35e52a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529b50 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 --2- 172.21.15.78:0/1005361459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e1424d0 0x55a35e513260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 -- 172.21.15.78:0/1005361459 >> 172.21.15.78:0/1005361459 conn(0x55a35e15f4e0 msgr2=0x55a35e512b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 -- 172.21.15.78:0/1005361459 shutdown_connections 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 -- 172.21.15.78:0/1005361459 wait complete. 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 -- start start 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 Processor -- start 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e1424d0 0x55a35e528ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.434+0000 7fa09d670740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa09d670740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e16b390 0x55a35e58cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa09d670740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901b0190 con 0x55a35e14a6b0 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa09d670740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901afac0 con 0x55a35e1424d0 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa09d670740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0901acd20 con 0x55a35e16b390 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa09d670740 10 client.0 _ll_register_callbacks cb 0x7fa0800093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49274/0 (socket says 172.21.15.78:49274) 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 -- 172.21.15.78:0/3832024493 learned_addr learned my addr 172.21.15.78:0/3832024493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e1424d0 0x55a35e528ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e16b390 0x55a35e58cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 -- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e16b390 msgr2=0x55a35e58cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e16b390 0x55a35e58cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 -- 172.21.15.78:0/3832024493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e1424d0 msgr2=0x55a35e528ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e1424d0 0x55a35e528ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 -- 172.21.15.78:0/3832024493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a35e58d190 con 0x55a35e14a6b0 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e16b390 0x55a35e58cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa08ffff640 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529410 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa080004e30 tx=0x7fa080004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96326bca37be2da5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa06f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3832024493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08001f070 con 0x55a35e14a6b0 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3832024493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fa080004730 con 0x55a35e14a6b0 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa09d670740 1 -- 172.21.15.78:0/3832024493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa080005500 con 0x55a35e14a6b0 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3832024493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa080004930 con 0x55a35e14a6b0 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa09d670740 10 client.18423 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.435+0000 7fa09d670740 1 -- 172.21.15.78:0/3832024493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a35e58e3a0 con 0x55a35e14a6b0 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3832024493 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa080005500 con 0x55a35e14a6b0 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3832024493 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa08000f040 con 0x55a35e14a6b0 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa09d670740 10 client.18423 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa09d670740 10 client.18423 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa09d670740 20 client.18423 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa09d670740 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55a35e0ff3a0 0x55a35e590a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa09d670740 4 client.18423 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa09d670740 1 -- 172.21.15.78:0/3832024493 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a35e4776c0 con 0x55a35e0ff3a0 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.436+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55a35e0ff3a0 0x55a35e590a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa08f7fe640 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55a35e0ff3a0 0x55a35e590a20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a466926e82979544 server_cookie=4c4b23f4011f9563 in_seq=0 out_seq=0 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa06f7fe640 10 client.18423 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa06f7fe640 1 -- 172.21.15.78:0/3832024493 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a35e4776c0 con 0x55a35e0ff3a0 2024-05-21T15:26:05.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa06f7fe640 10 client.18423 handle_command_reply: tid=0 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 client.18423 shutdown 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 -- 172.21.15.78:0/3832024493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 msgr2=0x55a35e529410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529410 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa080004e30 tx=0x7fa080004f60 comp rx=0 tx=0).stop 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 -- 172.21.15.78:0/3832024493 shutdown_connections 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55a35e0ff3a0 0x55a35e590a20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a35e16b390 0x55a35e58cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a35e14a6b0 0x55a35e529410 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa06f7fe640 0 client.18423 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:26:05.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 --2- 172.21.15.78:0/3832024493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a35e1424d0 0x55a35e528ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 -- 172.21.15.78:0/3832024493 >> 172.21.15.78:0/3832024493 conn(0x55a35e15f4e0 msgr2=0x55a35e516170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 -- 172.21.15.78:0/3832024493 shutdown_connections 2024-05-21T15:26:05.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 1 -- 172.21.15.78:0/3832024493 wait complete. 2024-05-21T15:26:05.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.437+0000 7fa09d670740 20 client.18423 trim_cache size 0 max 16384 2024-05-21T15:26:05.449 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 -- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa07c0a2d00 msgr2=0x7fa07c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa07c0a2d00 0x7fa07c0a5210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0880ee510 tx=0x7fa08808ee40 comp rx=0 tx=0).stop 2024-05-21T15:26:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 -- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 msgr2=0x7fa09019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 0x7fa09019dfd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa088031a20 tx=0x7fa088031a50 comp rx=0 tx=0).stop 2024-05-21T15:26:05.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 -- 172.21.15.78:0/1754007044 shutdown_connections 2024-05-21T15:26:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa07c0a2d00 0x7fa07c0a5210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa09010d000 0x7fa09019dfd0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa09010a610 0x7fa090199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 --2- 172.21.15.78:0/1754007044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa090103bb0 0x7fa0901997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 -- 172.21.15.78:0/1754007044 >> 172.21.15.78:0/1754007044 conn(0x7fa0900fd210 msgr2=0x7fa09010d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 -- 172.21.15.78:0/1754007044 shutdown_connections 2024-05-21T15:26:05.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.438+0000 7fa0967cc640 1 -- 172.21.15.78:0/1754007044 wait complete. 2024-05-21T15:26:05.452 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:26:05.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 -- 172.21.15.78:0/1042618573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef0101ae0 msgr2=0x7f7ef0101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 --2- 172.21.15.78:0/1042618573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef0101ae0 0x7f7ef0101ef0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009900 tx=0x7f7ed802a410 comp rx=0 tx=0).stop 2024-05-21T15:26:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 -- 172.21.15.78:0/1042618573 shutdown_connections 2024-05-21T15:26:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 --2- 172.21.15.78:0/1042618573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef010c370 0x7f7ef010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 --2- 172.21.15.78:0/1042618573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef0101ae0 0x7f7ef0101ef0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 --2- 172.21.15.78:0/1042618573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef0107a10 0x7f7ef01015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 -- 172.21.15.78:0/1042618573 >> 172.21.15.78:0/1042618573 conn(0x7f7ef00770c0 msgr2=0x7f7ef0077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 -- 172.21.15.78:0/1042618573 shutdown_connections 2024-05-21T15:26:05.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.563+0000 7f7ef5851640 1 -- 172.21.15.78:0/1042618573 wait complete. 2024-05-21T15:26:05.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ef5851640 1 Processor -- start 2024-05-21T15:26:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ef5851640 1 -- start start 2024-05-21T15:26:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ef5851640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef0101ae0 0x7f7ef0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ef5851640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0107a10 0x7f7ef0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ef5851640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef010c370 0x7f7ef019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ef5851640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0111d70 con 0x7f7ef010c370 2024-05-21T15:26:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ef5851640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0111bf0 con 0x7f7ef0107a10 2024-05-21T15:26:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ef5851640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0111ef0 con 0x7f7ef0101ae0 2024-05-21T15:26:05.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7eef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef010c370 0x7f7ef019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7eef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef010c370 0x7f7ef019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49292/0 (socket says 172.21.15.78:49292) 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7eef7fe640 1 -- 172.21.15.78:0/352540 learned_addr learned my addr 172.21.15.78:0/352540 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef0101ae0 0x7f7ef0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.564+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0107a10 0x7f7ef0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7eef7fe640 1 -- 172.21.15.78:0/352540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef0101ae0 msgr2=0x7f7ef0198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7eef7fe640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef0101ae0 0x7f7ef0198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7eef7fe640 1 -- 172.21.15.78:0/352540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0107a10 msgr2=0x7f7ef0198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7eef7fe640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0107a10 0x7f7ef0198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7eef7fe640 1 -- 172.21.15.78:0/352540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef019d5f0 con 0x7f7ef010c370 2024-05-21T15:26:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0107a10 0x7f7ef0198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7eef7fe640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef010c370 0x7f7ef019cfb0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800eb10 tx=0x7f7ee800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7ef484f640 1 -- 172.21.15.78:0/352540 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8004b20 con 0x7f7ef010c370 2024-05-21T15:26:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7ef484f640 1 -- 172.21.15.78:0/352540 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee8004cc0 con 0x7f7ef010c370 2024-05-21T15:26:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7ef5851640 1 -- 172.21.15.78:0/352540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef019d8e0 con 0x7f7ef010c370 2024-05-21T15:26:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7ef484f640 1 -- 172.21.15.78:0/352540 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8010760 con 0x7f7ef010c370 2024-05-21T15:26:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.565+0000 7f7ef5851640 1 -- 172.21.15.78:0/352540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01a5260 con 0x7f7ef010c370 2024-05-21T15:26:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7efc7d0740 1 Processor -- start 2024-05-21T15:26:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7ef484f640 1 -- 172.21.15.78:0/352540 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7ee8010900 con 0x7f7ef010c370 2024-05-21T15:26:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7efc7d0740 1 -- start start 2024-05-21T15:26:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0bf9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0bfa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0bfa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0b5dc40 con 0x55a0c08304d0 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0b05580 con 0x55a0c08386b0 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0b59c00 con 0x55a0c0859390 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7eeeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0bfa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7eeeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0bfa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49308/0 (socket says 172.21.15.78:49308) 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7eeeffd640 1 -- 172.21.15.78:0/3631563986 learned_addr learned my addr 172.21.15.78:0/3631563986 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0bf9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0bfa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7eeeffd640 1 -- 172.21.15.78:0/3631563986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 msgr2=0x55a0c0bf9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0bf9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7eeeffd640 1 -- 172.21.15.78:0/3631563986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 msgr2=0x55a0c0bfa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0bfa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7eeeffd640 1 -- 172.21.15.78:0/3631563986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c08713f0 con 0x55a0c08304d0 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7ef484f640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7edc0a2d80 0x7f7edc0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.568+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0bfa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.569+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0bfa300 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000b740 tx=0x7f7ee000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a894f0b3723c2880 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.569+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7edc0a2d80 0x7f7edc0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.569+0000 7f7ef484f640 1 -- 172.21.15.78:0/352540 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f7ee80cfa70 con 0x7f7ef010c370 2024-05-21T15:26:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.569+0000 7f7ee5ffb640 1 -- 172.21.15.78:0/3631563986 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0003dd0 con 0x55a0c08304d0 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.569+0000 7f7ee5ffb640 1 -- 172.21.15.78:0/3631563986 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee0015070 con 0x55a0c08304d0 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.569+0000 7f7ee5ffb640 1 -- 172.21.15.78:0/3631563986 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0011440 con 0x55a0c08304d0 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3631563986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 msgr2=0x55a0c0bfa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0bfa300 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000b740 tx=0x7f7ee000bc00 comp rx=0 tx=0).stop 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7edc0a2d80 0x7f7edc0a5290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ed802aa00 tx=0x7f7ed8000bc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3631563986 shutdown_connections 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0bfa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0bfa300 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3631563986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0bf9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3631563986 >> 172.21.15.78:0/3631563986 conn(0x55a0c084d4e0 msgr2=0x55a0c0bf66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3631563986 shutdown_connections 2024-05-21T15:26:05.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3631563986 wait complete. 2024-05-21T15:26:05.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 -- start start 2024-05-21T15:26:05.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.570+0000 7f7efc7d0740 1 Processor -- start 2024-05-21T15:26:05.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08304d0 0x55a0c0c06900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c06e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c12920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0c094b0 con 0x55a0c08386b0 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0b18c40 con 0x55a0c08304d0 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0c0c09630 con 0x55a0c0859390 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7efc7d0740 10 client.0 _ll_register_callbacks cb 0x55a0c0870be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c06e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c06e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49322/0 (socket says 172.21.15.78:49322) 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 -- 172.21.15.78:0/3058385839 learned_addr learned my addr 172.21.15.78:0/3058385839 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08304d0 0x55a0c0c06900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7eef7fe640 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c12920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 -- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 msgr2=0x55a0c0c12920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c12920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 -- 172.21.15.78:0/3058385839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08304d0 msgr2=0x55a0c0c06900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08304d0 0x55a0c0c06900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 -- 172.21.15.78:0/3058385839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c093b7d0 con 0x55a0c08386b0 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7eef7fe640 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c12920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c06e40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80040d0 tx=0x7f7ed8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd970ca5993928a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.571+0000 7f7eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7eceffd640 1 -- 172.21.15.78:0/3058385839 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed804f070 con 0x55a0c08386b0 2024-05-21T15:26:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7eceffd640 1 -- 172.21.15.78:0/3058385839 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f7ed8004c50 con 0x55a0c08386b0 2024-05-21T15:26:05.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3058385839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee0015070 con 0x55a0c08386b0 2024-05-21T15:26:05.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7eceffd640 1 -- 172.21.15.78:0/3058385839 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7ed805b050 con 0x55a0c08386b0 2024-05-21T15:26:05.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 10 client.18435 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3058385839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a0c0c15b60 con 0x55a0c08386b0 2024-05-21T15:26:05.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7eceffd640 1 -- 172.21.15.78:0/3058385839 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee0015070 con 0x55a0c08386b0 2024-05-21T15:26:05.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7eceffd640 1 -- 172.21.15.78:0/3058385839 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed8048050 con 0x55a0c08386b0 2024-05-21T15:26:05.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 10 client.18435 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 10 client.18435 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 20 client.18435 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55a0c0817f60 0x55a0c0c19050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 4 client.18435 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3058385839 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0c0b6b370 con 0x55a0c0817f60 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.572+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55a0c0817f60 0x55a0c0c19050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.573+0000 7f7eceffd640 10 client.18435 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.573+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55a0c0817f60 0x55a0c0c19050 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=682fc1788dd04972 server_cookie=547e6315193e43b in_seq=0 out_seq=0 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7eceffd640 1 -- 172.21.15.78:0/3058385839 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a0c0b6b370 con 0x55a0c0817f60 2024-05-21T15:26:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7eceffd640 10 client.18435 handle_command_reply: tid=0 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 client.18435 shutdown 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3058385839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 msgr2=0x55a0c0c06e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c06e40 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80040d0 tx=0x7f7ed8009260 comp rx=0 tx=0).stop 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3058385839 shutdown_connections 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55a0c0817f60 0x55a0c0c19050 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c12920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c06e40 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7eceffd640 0 client.18435 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:26:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/3058385839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08304d0 0x55a0c0c06900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3058385839 >> 172.21.15.78:0/3058385839 conn(0x55a0c084d4e0 msgr2=0x55a0c0bfa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3058385839 shutdown_connections 2024-05-21T15:26:05.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 1 -- 172.21.15.78:0/3058385839 wait complete. 2024-05-21T15:26:05.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.574+0000 7f7efc7d0740 20 client.18435 trim_cache size 0 max 16384 2024-05-21T15:26:05.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.587+0000 7f7efc7d0740 1 Processor -- start 2024-05-21T15:26:05.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.587+0000 7f7efc7d0740 1 -- start start 2024-05-21T15:26:05.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c08304d0 0x55a0c0c08900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c12500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c0859390 0x55a0c0c1e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01acb50 con 0x55a0c08386b0 2024-05-21T15:26:05.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01ae690 con 0x55a0c0859390 2024-05-21T15:26:05.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01ad220 con 0x55a0c08304d0 2024-05-21T15:26:05.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c12500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c12500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49332/0 (socket says 172.21.15.78:49332) 2024-05-21T15:26:05.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 -- 172.21.15.78:0/844005728 learned_addr learned my addr 172.21.15.78:0/844005728 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7eef7fe640 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c0859390 0x55a0c0c1e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c08304d0 0x55a0c0c08900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 -- 172.21.15.78:0/844005728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c08304d0 msgr2=0x55a0c0c08900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c08304d0 0x55a0c0c08900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 -- 172.21.15.78:0/844005728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c0859390 msgr2=0x55a0c0c1e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c0859390 0x55a0c0c1e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 -- 172.21.15.78:0/844005728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c0c04c60 con 0x55a0c08386b0 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7eef7fe640 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c0859390 0x55a0c0c1e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c08304d0 0x55a0c0c08900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c12500 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009900 tx=0x7f7ed8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60351c1a7cbd1871 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee57fa640 1 -- 172.21.15.78:0/844005728 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8064070 con 0x55a0c08386b0 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.588+0000 7f7ee57fa640 1 -- 172.21.15.78:0/844005728 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed80035f0 con 0x55a0c08386b0 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7ee57fa640 1 -- 172.21.15.78:0/844005728 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed804f040 con 0x55a0c08386b0 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 -- 172.21.15.78:0/844005728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 msgr2=0x55a0c0c12500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c12500 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009900 tx=0x7f7ed8005e50 comp rx=0 tx=0).stop 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 -- 172.21.15.78:0/844005728 shutdown_connections 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c0859390 0x55a0c0c1e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08386b0 0x55a0c0c12500 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/844005728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c08304d0 0x55a0c0c08900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 -- 172.21.15.78:0/844005728 >> 172.21.15.78:0/844005728 conn(0x55a0c084d4e0 msgr2=0x55a0c0bfa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 -- 172.21.15.78:0/844005728 shutdown_connections 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.589+0000 7f7efc7d0740 1 -- 172.21.15.78:0/844005728 wait complete. 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 1 -- start start 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 1 Processor -- start 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0c0a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0c0aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c0b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01adfc0 con 0x55a0c08304d0 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01ad8f0 con 0x55a0c08386b0 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01aab50 con 0x55a0c0859390 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7efc7d0740 10 client.0 _ll_register_callbacks cb 0x7f7ed8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7eeeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0c0a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7eeeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0c0a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49346/0 (socket says 172.21.15.78:49346) 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7eeeffd640 1 -- 172.21.15.78:0/1039039264 learned_addr learned my addr 172.21.15.78:0/1039039264 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7eef7fe640 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c0b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.590+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0c0aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eeeffd640 1 -- 172.21.15.78:0/1039039264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 msgr2=0x55a0c0c0b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c0b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eeeffd640 1 -- 172.21.15.78:0/1039039264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 msgr2=0x55a0c0c0aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0c0aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eeeffd640 1 -- 172.21.15.78:0/1039039264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c0c80470 con 0x55a0c08304d0 2024-05-21T15:26:05.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0c0aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:05.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eeeffd640 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0c0a580 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x55a0c0bf7790 tx=0x7f7ee000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f0ce97af55443d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eceffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eceffd640 1 -- 172.21.15.78:0/1039039264 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0004960 con 0x55a0c08304d0 2024-05-21T15:26:05.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eceffd640 1 -- 172.21.15.78:0/1039039264 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f7ee002a0b0 con 0x55a0c08304d0 2024-05-21T15:26:05.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 1 -- 172.21.15.78:0/1039039264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0c0b656c0 con 0x55a0c08304d0 2024-05-21T15:26:05.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eceffd640 1 -- 172.21.15.78:0/1039039264 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7ee003a050 con 0x55a0c08304d0 2024-05-21T15:26:05.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 10 client.18441 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 1 -- 172.21.15.78:0/1039039264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a0c0c81610 con 0x55a0c08304d0 2024-05-21T15:26:05.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eceffd640 1 -- 172.21.15.78:0/1039039264 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a0c0b656c0 con 0x55a0c08304d0 2024-05-21T15:26:05.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7eceffd640 1 -- 172.21.15.78:0/1039039264 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7ee0011560 con 0x55a0c08304d0 2024-05-21T15:26:05.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 10 client.18441 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 10 client.18441 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 20 client.18441 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55a0c07ed3a0 0x55a0c0c83ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 4 client.18441 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.591+0000 7f7efc7d0740 1 -- 172.21.15.78:0/1039039264 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0c0ae2820 con 0x55a0c07ed3a0 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.592+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55a0c07ed3a0 0x55a0c0c83ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.592+0000 7f7eceffd640 10 client.18441 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.592+0000 7f7ee7fff640 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55a0c07ed3a0 0x55a0c0c83ac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ccf95d105ab7bb03 server_cookie=29fb62864615bb18 in_seq=0 out_seq=0 2024-05-21T15:26:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.592+0000 7f7eceffd640 1 -- 172.21.15.78:0/1039039264 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0c0ae2820 con 0x55a0c07ed3a0 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.592+0000 7f7eceffd640 10 client.18441 handle_command_reply: tid=0 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.592+0000 7f7efc7d0740 1 client.18441 shutdown 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.592+0000 7f7efc7d0740 1 -- 172.21.15.78:0/1039039264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 msgr2=0x55a0c0c0a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0c0a580 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x55a0c0bf7790 tx=0x7f7ee000b390 comp rx=0 tx=0).stop 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 -- 172.21.15.78:0/1039039264 shutdown_connections 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x55a0c07ed3a0 0x55a0c0c83ac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0c08386b0 0x55a0c0c0aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c0859390 0x55a0c0c0b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7eceffd640 0 client.18441 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:26:05.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 --2- 172.21.15.78:0/1039039264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a0c08304d0 0x55a0c0c0a580 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 -- 172.21.15.78:0/1039039264 >> 172.21.15.78:0/1039039264 conn(0x55a0c084d4e0 msgr2=0x55a0c0bfba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 -- 172.21.15.78:0/1039039264 shutdown_connections 2024-05-21T15:26:05.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 1 -- 172.21.15.78:0/1039039264 wait complete. 2024-05-21T15:26:05.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7efc7d0740 20 client.18441 trim_cache size 0 max 16384 2024-05-21T15:26:05.606 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 -- 172.21.15.78:0/352540 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7edc0a2d80 msgr2=0x7f7edc0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7edc0a2d80 0x7f7edc0a5290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ed802aa00 tx=0x7f7ed8000bc0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 -- 172.21.15.78:0/352540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef010c370 msgr2=0x7f7ef019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef010c370 0x7f7ef019cfb0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800eb10 tx=0x7f7ee800c2d0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 -- 172.21.15.78:0/352540 shutdown_connections 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0107a10 0x7f7ef0198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7edc0a2d80 0x7f7edc0a5290 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ef010c370 0x7f7ef019cfb0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 --2- 172.21.15.78:0/352540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef0101ae0 0x7f7ef0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 -- 172.21.15.78:0/352540 >> 172.21.15.78:0/352540 conn(0x7f7ef00770c0 msgr2=0x7f7ef01030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 -- 172.21.15.78:0/352540 shutdown_connections 2024-05-21T15:26:05.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.593+0000 7f7ef5851640 1 -- 172.21.15.78:0/352540 wait complete. 2024-05-21T15:26:05.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:26:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 -- 172.21.15.78:0/3645724962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 msgr2=0x7f0140102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 --2- 172.21.15.78:0/3645724962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 0x7f0140102ba0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0134009900 tx=0x7f013402a410 comp rx=0 tx=0).stop 2024-05-21T15:26:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 -- 172.21.15.78:0/3645724962 shutdown_connections 2024-05-21T15:26:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 --2- 172.21.15.78:0/3645724962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0140103210 0x7f014010f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 --2- 172.21.15.78:0/3645724962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 0x7f0140102ba0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 --2- 172.21.15.78:0/3645724962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140108ac0 0x7f0140102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 -- 172.21.15.78:0/3645724962 >> 172.21.15.78:0/3645724962 conn(0x7f01400fe1c0 msgr2=0x7f0140100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 -- 172.21.15.78:0/3645724962 shutdown_connections 2024-05-21T15:26:05.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.719+0000 7f0144b8b640 1 -- 172.21.15.78:0/3645724962 wait complete. 2024-05-21T15:26:05.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f0144b8b640 1 Processor -- start 2024-05-21T15:26:05.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f0144b8b640 1 -- start start 2024-05-21T15:26:05.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f0144b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0140103210 0x7f014019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f0144b8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140108ac0 0x7f014019fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f0144b8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 0x7f0140199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f0144b8b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0140112e00 con 0x7f0140108f60 2024-05-21T15:26:05.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f0144b8b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0140112c80 con 0x7f0140108ac0 2024-05-21T15:26:05.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f0144b8b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0140112f80 con 0x7f0140103210 2024-05-21T15:26:05.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f013ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 0x7f0140199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f013ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 0x7f0140199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49374/0 (socket says 172.21.15.78:49374) 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.720+0000 7f013ed9d640 1 -- 172.21.15.78:0/600585262 learned_addr learned my addr 172.21.15.78:0/600585262 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f013dd9b640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140108ac0 0x7f014019fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f013e59c640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0140103210 0x7f014019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f013ed9d640 1 -- 172.21.15.78:0/600585262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0140103210 msgr2=0x7f014019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f013ed9d640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0140103210 0x7f014019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f013ed9d640 1 -- 172.21.15.78:0/600585262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140108ac0 msgr2=0x7f014019fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f013ed9d640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140108ac0 0x7f014019fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f013ed9d640 1 -- 172.21.15.78:0/600585262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0140199d90 con 0x7f0140108f60 2024-05-21T15:26:05.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f013ed9d640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 0x7f0140199660 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f013800eaa0 tx=0x7f013800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f012f7fe640 1 -- 172.21.15.78:0/600585262 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013800cdb0 con 0x7f0140108f60 2024-05-21T15:26:05.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f0144b8b640 1 -- 172.21.15.78:0/600585262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014019a020 con 0x7f0140108f60 2024-05-21T15:26:05.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f012f7fe640 1 -- 172.21.15.78:0/600585262 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0138005020 con 0x7f0140108f60 2024-05-21T15:26:05.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f012f7fe640 1 -- 172.21.15.78:0/600585262 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380116b0 con 0x7f0140108f60 2024-05-21T15:26:05.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.721+0000 7f0144b8b640 1 -- 172.21.15.78:0/600585262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01401a6280 con 0x7f0140108f60 2024-05-21T15:26:05.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f012f7fe640 1 -- 172.21.15.78:0/600585262 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0138005560 con 0x7f0140108f60 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f014ba30740 1 Processor -- start 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f014ba30740 1 -- start start 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f014ba30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 0x55589f123d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f014ba30740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f124260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f014ba30740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f1247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f014ba30740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55589f089c40 con 0x55589ed5c4d0 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f014ba30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55589f031580 con 0x55589ed85390 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f014ba30740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55589f085c00 con 0x55589ed646b0 2024-05-21T15:26:05.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.724+0000 7f012f7fe640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f01280a2d00 0x7f01280a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f124260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f124260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49376/0 (socket says 172.21.15.78:49376) 2024-05-21T15:26:05.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 -- 172.21.15.78:0/1179783907 learned_addr learned my addr 172.21.15.78:0/1179783907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f012f7fe640 1 -- 172.21.15.78:0/600585262 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f01380cf800 con 0x7f0140108f60 2024-05-21T15:26:05.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013ed9d640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f01280a2d00 0x7f01280a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013dd9b640 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 0x55589f123d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013dd9b640 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f1247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 -- 172.21.15.78:0/1179783907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 msgr2=0x55589f1247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f1247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 -- 172.21.15.78:0/1179783907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 msgr2=0x55589f123d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 0x55589f123d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 -- 172.21.15.78:0/1179783907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589ed9d3f0 con 0x55589ed5c4d0 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013dd9b640 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 0x55589f123d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013ed9d640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f01280a2d00 0x7f01280a5210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01380ccf20 tx=0x7f013806dc50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f013e59c640 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f124260 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f013000cb40 tx=0x7f013000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b27daaebe6e27b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f012cff9640 1 -- 172.21.15.78:0/1179783907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130010430 con 0x55589ed5c4d0 2024-05-21T15:26:05.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f012cff9640 1 -- 172.21.15.78:0/1179783907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0130010ab0 con 0x55589ed5c4d0 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.725+0000 7f012cff9640 1 -- 172.21.15.78:0/1179783907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013001ac30 con 0x55589ed5c4d0 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 -- 172.21.15.78:0/1179783907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 msgr2=0x55589f124260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f124260 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f013000cb40 tx=0x7f013000e350 comp rx=0 tx=0).stop 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 -- 172.21.15.78:0/1179783907 shutdown_connections 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f1247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f124260 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 --2- 172.21.15.78:0/1179783907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 0x55589f123d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 -- 172.21.15.78:0/1179783907 >> 172.21.15.78:0/1179783907 conn(0x55589ed794e0 msgr2=0x55589f1204d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 -- 172.21.15.78:0/1179783907 shutdown_connections 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.726+0000 7f014ba30740 1 -- 172.21.15.78:0/1179783907 wait complete. 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 1 -- start start 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 1 Processor -- start 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f12e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f12e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 0x55589f134500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55589f131010 con 0x55589ed5c4d0 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55589f044c40 con 0x55589ed85390 2024-05-21T15:26:05.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55589f131190 con 0x55589ed646b0 2024-05-21T15:26:05.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f014ba30740 10 client.0 _ll_register_callbacks cb 0x55589ed9cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f12e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f12e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49388/0 (socket says 172.21.15.78:49388) 2024-05-21T15:26:05.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013dd9b640 1 -- 172.21.15.78:0/2049932203 learned_addr learned my addr 172.21.15.78:0/2049932203 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013e59c640 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f12e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013dd9b640 1 -- 172.21.15.78:0/2049932203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 msgr2=0x55589f12e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013dd9b640 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f12e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013dd9b640 1 -- 172.21.15.78:0/2049932203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 msgr2=0x55589f134500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013dd9b640 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 0x55589f134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013dd9b640 1 -- 172.21.15.78:0/2049932203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589ee677d0 con 0x55589ed5c4d0 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.727+0000 7f013e59c640 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f12e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f013dd9b640 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f12e3d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0134005940 tx=0x7f01340319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ac01286651d33b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f0125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f0125ffb640 1 -- 172.21.15.78:0/2049932203 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134045020 con 0x55589ed5c4d0 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f0125ffb640 1 -- 172.21.15.78:0/2049932203 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f0134004450 con 0x55589ed5c4d0 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 1 -- 172.21.15.78:0/2049932203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0130010ab0 con 0x55589ed5c4d0 2024-05-21T15:26:05.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f0125ffb640 1 -- 172.21.15.78:0/2049932203 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0134050050 con 0x55589ed5c4d0 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 10 client.18453 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 1 -- 172.21.15.78:0/2049932203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55589f13d810 con 0x55589ed5c4d0 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f0125ffb640 1 -- 172.21.15.78:0/2049932203 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0130010ab0 con 0x55589ed5c4d0 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f0125ffb640 1 -- 172.21.15.78:0/2049932203 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f013403a040 con 0x55589ed5c4d0 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 10 client.18453 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 20 client.18453 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589ed43f60 0x55589f140d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 4 client.18453 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.728+0000 7f014ba30740 1 -- 172.21.15.78:0/2049932203 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55589f097370 con 0x55589ed43f60 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.729+0000 7f013e59c640 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589ed43f60 0x55589f140d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.729+0000 7f0125ffb640 10 client.18453 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.729+0000 7f013e59c640 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589ed43f60 0x55589f140d90 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51efe20d7005dcc server_cookie=c78646a162f86c56 in_seq=0 out_seq=0 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f0125ffb640 1 -- 172.21.15.78:0/2049932203 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55589f097370 con 0x55589ed43f60 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f0125ffb640 10 client.18453 handle_command_reply: tid=0 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 client.18453 shutdown 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 -- 172.21.15.78:0/2049932203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 msgr2=0x55589f12e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f12e3d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0134005940 tx=0x7f01340319b0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 -- 172.21.15.78:0/2049932203 shutdown_connections 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589ed43f60 0x55589f140d90 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed85390 0x55589f134500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f12e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 --2- 172.21.15.78:0/2049932203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f12e3d0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f0125ffb640 0 client.18453 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 -- 172.21.15.78:0/2049932203 >> 172.21.15.78:0/2049932203 conn(0x55589ed794e0 msgr2=0x55589f125a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 -- 172.21.15.78:0/2049932203 shutdown_connections 2024-05-21T15:26:05.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 1 -- 172.21.15.78:0/2049932203 wait complete. 2024-05-21T15:26:05.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.730+0000 7f014ba30740 20 client.18453 trim_cache size 0 max 16384 2024-05-21T15:26:05.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f014ba30740 1 Processor -- start 2024-05-21T15:26:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f014ba30740 1 -- start start 2024-05-21T15:26:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f014ba30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed5c4d0 0x55589f1264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f014ba30740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f1343c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f014ba30740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed85390 0x55589f127a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f014ba30740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01401aede0 con 0x55589ed85390 2024-05-21T15:26:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f014ba30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01401b0920 con 0x55589ed5c4d0 2024-05-21T15:26:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f014ba30740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01401af4b0 con 0x55589ed646b0 2024-05-21T15:26:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f013dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed85390 0x55589f127a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f013dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed85390 0x55589f127a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49400/0 (socket says 172.21.15.78:49400) 2024-05-21T15:26:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f013dd9b640 1 -- 172.21.15.78:0/2463839670 learned_addr learned my addr 172.21.15.78:0/2463839670 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.744+0000 7f013dd9b640 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed5c4d0 0x55589f1264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f013e59c640 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f1343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f013dd9b640 1 -- 172.21.15.78:0/2463839670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 msgr2=0x55589f1343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f013dd9b640 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f1343c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f013dd9b640 1 -- 172.21.15.78:0/2463839670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed5c4d0 msgr2=0x55589f1264a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f013dd9b640 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed5c4d0 0x55589f1264a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f013dd9b640 1 -- 172.21.15.78:0/2463839670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589f134900 con 0x55589ed85390 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f013dd9b640 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed85390 0x55589f127a80 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0134001eb0 tx=0x7f01340058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49050720ae1b0264 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f0127fff640 1 -- 172.21.15.78:0/2463839670 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134045040 con 0x55589ed85390 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f0127fff640 1 -- 172.21.15.78:0/2463839670 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0134031bd0 con 0x55589ed85390 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.745+0000 7f0127fff640 1 -- 172.21.15.78:0/2463839670 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134035040 con 0x55589ed85390 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- 172.21.15.78:0/2463839670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed85390 msgr2=0x55589f127a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed85390 0x55589f127a80 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0134001eb0 tx=0x7f01340058b0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- 172.21.15.78:0/2463839670 shutdown_connections 2024-05-21T15:26:05.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed85390 0x55589f127a80 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed646b0 0x55589f1343c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 --2- 172.21.15.78:0/2463839670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed5c4d0 0x55589f1264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- 172.21.15.78:0/2463839670 >> 172.21.15.78:0/2463839670 conn(0x55589ed794e0 msgr2=0x55589f1244e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- 172.21.15.78:0/2463839670 shutdown_connections 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- 172.21.15.78:0/2463839670 wait complete. 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- start start 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 Processor -- start 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f13ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed646b0 0x55589f11f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed85390 0x55589f11f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01401b0250 con 0x55589ed5c4d0 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01401afb80 con 0x55589ed646b0 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.746+0000 7f014ba30740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01401acde0 con 0x55589ed85390 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f014ba30740 10 client.0 _ll_register_callbacks cb 0x7f0134009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f13ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f13ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49416/0 (socket says 172.21.15.78:49416) 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 -- 172.21.15.78:0/281752274 learned_addr learned my addr 172.21.15.78:0/281752274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed85390 0x55589f11f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013e59c640 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed646b0 0x55589f11f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 -- 172.21.15.78:0/281752274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed85390 msgr2=0x55589f11f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed85390 0x55589f11f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 -- 172.21.15.78:0/281752274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed646b0 msgr2=0x55589f11f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed646b0 0x55589f11f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 -- 172.21.15.78:0/281752274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589f11fee0 con 0x55589ed5c4d0 2024-05-21T15:26:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f013dd9b640 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f13ab80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f01340096c0 tx=0x7f013403df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ccf689e56a4611 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f0125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f0125ffb640 1 -- 172.21.15.78:0/281752274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134035030 con 0x55589ed5c4d0 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f0125ffb640 1 -- 172.21.15.78:0/281752274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7f0134004250 con 0x55589ed5c4d0 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f014ba30740 1 -- 172.21.15.78:0/281752274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0134031bd0 con 0x55589ed5c4d0 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f0125ffb640 1 -- 172.21.15.78:0/281752274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0134059040 con 0x55589ed5c4d0 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f014ba30740 10 client.18459 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.747+0000 7f014ba30740 1 -- 172.21.15.78:0/281752274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55589f1391a0 con 0x55589ed5c4d0 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f0125ffb640 1 -- 172.21.15.78:0/281752274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0134031bd0 con 0x55589ed5c4d0 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f0125ffb640 1 -- 172.21.15.78:0/281752274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f013405f070 con 0x55589ed5c4d0 2024-05-21T15:26:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f014ba30740 10 client.18459 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f014ba30740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f014ba30740 20 client.18459 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f014ba30740 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589ed193a0 0x55589f1ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f014ba30740 4 client.18459 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f014ba30740 1 -- 172.21.15.78:0/281752274 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55589f0916c0 con 0x55589ed193a0 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f013e59c640 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589ed193a0 0x55589f1ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f0125ffb640 10 client.18459 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.748+0000 7f013e59c640 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589ed193a0 0x55589f1ab3f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77792dc80b76569b server_cookie=601245851edcd23c in_seq=0 out_seq=0 2024-05-21T15:26:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0125ffb640 1 -- 172.21.15.78:0/281752274 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55589f0916c0 con 0x55589ed193a0 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0125ffb640 10 client.18459 handle_command_reply: tid=0 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 client.18459 shutdown 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 -- 172.21.15.78:0/281752274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 msgr2=0x55589f13ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f13ab80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f01340096c0 tx=0x7f013403df60 comp rx=0 tx=0).stop 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 -- 172.21.15.78:0/281752274 shutdown_connections 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589ed193a0 0x55589f1ab3f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589ed646b0 0x55589f11f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0125ffb640 0 client.18459 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589ed85390 0x55589f11f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 --2- 172.21.15.78:0/281752274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589ed5c4d0 0x55589f13ab80 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 -- 172.21.15.78:0/281752274 >> 172.21.15.78:0/281752274 conn(0x55589ed794e0 msgr2=0x55589f12f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 -- 172.21.15.78:0/281752274 shutdown_connections 2024-05-21T15:26:05.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 1 -- 172.21.15.78:0/281752274 wait complete. 2024-05-21T15:26:05.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f014ba30740 20 client.18459 trim_cache size 0 max 16384 2024-05-21T15:26:05.762 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:05.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0144b8b640 1 -- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f01280a2d00 msgr2=0x7f01280a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0144b8b640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f01280a2d00 0x7f01280a5210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01380ccf20 tx=0x7f013806dc50 comp rx=0 tx=0).stop 2024-05-21T15:26:05.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0144b8b640 1 -- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 msgr2=0x7f0140199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:05.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0144b8b640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 0x7f0140199660 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f013800eaa0 tx=0x7f013800ef60 comp rx=0 tx=0).stop 2024-05-21T15:26:05.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0144b8b640 1 -- 172.21.15.78:0/600585262 shutdown_connections 2024-05-21T15:26:05.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.749+0000 7f0144b8b640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0140108ac0 0x7f014019fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.750+0000 7f0144b8b640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f01280a2d00 0x7f01280a5210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.750+0000 7f0144b8b640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0140108f60 0x7f0140199660 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.750+0000 7f0144b8b640 1 --2- 172.21.15.78:0/600585262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0140103210 0x7f014019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:05.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.750+0000 7f0144b8b640 1 -- 172.21.15.78:0/600585262 >> 172.21.15.78:0/600585262 conn(0x7f01400fe1c0 msgr2=0x7f014010e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:05.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.750+0000 7f0144b8b640 1 -- 172.21.15.78:0/600585262 shutdown_connections 2024-05-21T15:26:05.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:05.750+0000 7f0144b8b640 1 -- 172.21.15.78:0/600585262 wait complete. 2024-05-21T15:26:05.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:05 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1198527573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:26:05.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:05 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1198527573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:26:05.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:05 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1198527573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:26:06.766 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T15:26:06.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:26:06.767 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T15:26:06.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:06 smithi078 ceph-mon[31997]: pgmap v309: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T15:26:06.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:06 smithi084 ceph-mon[36334]: pgmap v309: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T15:26:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 -- 172.21.15.78:0/2975906705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 msgr2=0x7ffa7010e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 --2- 172.21.15.78:0/2975906705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 0x7ffa7010e760 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009ca0 tx=0x7ffa6802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 -- 172.21.15.78:0/2975906705 shutdown_connections 2024-05-21T15:26:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 --2- 172.21.15.78:0/2975906705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 0x7ffa7010e760 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 --2- 172.21.15.78:0/2975906705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa70101ab0 0x7ffa70101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 --2- 172.21.15.78:0/2975906705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa701079e0 0x7ffa70101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 -- 172.21.15.78:0/2975906705 >> 172.21.15.78:0/2975906705 conn(0x7ffa700fd210 msgr2=0x7ffa700ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 -- 172.21.15.78:0/2975906705 shutdown_connections 2024-05-21T15:26:06.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.894+0000 7ffa75d38640 1 -- 172.21.15.78:0/2975906705 wait complete. 2024-05-21T15:26:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa75d38640 1 Processor -- start 2024-05-21T15:26:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa75d38640 1 -- start start 2024-05-21T15:26:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa75d38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa70101ab0 0x7ffa7019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa75d38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa701079e0 0x7ffa7019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa75d38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 0x7ffa7019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa75d38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70111d10 con 0x7ffa7010c340 2024-05-21T15:26:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa75d38640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70111b90 con 0x7ffa701079e0 2024-05-21T15:26:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa75d38640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70111e90 con 0x7ffa70101ab0 2024-05-21T15:26:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 0x7ffa7019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 0x7ffa7019b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49440/0 (socket says 172.21.15.78:49440) 2024-05-21T15:26:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa6ffff640 1 -- 172.21.15.78:0/3455391658 learned_addr learned my addr 172.21.15.78:0/3455391658 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.895+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa70101ab0 0x7ffa7019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa701079e0 0x7ffa7019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6ffff640 1 -- 172.21.15.78:0/3455391658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa70101ab0 msgr2=0x7ffa7019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6ffff640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa70101ab0 0x7ffa7019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6ffff640 1 -- 172.21.15.78:0/3455391658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa701079e0 msgr2=0x7ffa7019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6ffff640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa701079e0 0x7ffa7019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6ffff640 1 -- 172.21.15.78:0/3455391658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa701a58a0 con 0x7ffa7010c340 2024-05-21T15:26:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa701079e0 0x7ffa7019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6ffff640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 0x7ffa7019b010 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffa68031a20 tx=0x7ffa68031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6d7fa640 1 -- 172.21.15.78:0/3455391658 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68003c30 con 0x7ffa7010c340 2024-05-21T15:26:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6d7fa640 1 -- 172.21.15.78:0/3455391658 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68003dd0 con 0x7ffa7010c340 2024-05-21T15:26:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa75d38640 1 -- 172.21.15.78:0/3455391658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa701a5b30 con 0x7ffa7010c340 2024-05-21T15:26:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa6d7fa640 1 -- 172.21.15.78:0/3455391658 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6802fcf0 con 0x7ffa7010c340 2024-05-21T15:26:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.896+0000 7ffa75d38640 1 -- 172.21.15.78:0/3455391658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701a6080 con 0x7ffa7010c340 2024-05-21T15:26:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.898+0000 7ffa6d7fa640 1 -- 172.21.15.78:0/3455391658 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa68004050 con 0x7ffa7010c340 2024-05-21T15:26:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa7cbe4740 1 Processor -- start 2024-05-21T15:26:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa7cbe4740 1 -- start start 2024-05-21T15:26:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 0x5642d39d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5642d3938990 con 0x5642d360b550 2024-05-21T15:26:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5642d38e05b0 con 0x5642d3634410 2024-05-21T15:26:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5642d3934950 con 0x5642d3613730 2024-05-21T15:26:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6d7fa640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ffa5c0a2d00 0x7ffa5c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49452/0 (socket says 172.21.15.78:49452) 2024-05-21T15:26:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6f7fe640 1 -- 172.21.15.78:0/1862726955 learned_addr learned my addr 172.21.15.78:0/1862726955 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 0x5642d39d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6ffff640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ffa5c0a2d00 0x7ffa5c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6d7fa640 1 -- 172.21.15.78:0/3455391658 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7ffa680f0830 con 0x7ffa7010c340 2024-05-21T15:26:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6f7fe640 1 -- 172.21.15.78:0/1862726955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 msgr2=0x5642d39d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39d40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6f7fe640 1 -- 172.21.15.78:0/1862726955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 msgr2=0x5642d39d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 0x5642d39d3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa6f7fe640 1 -- 172.21.15.78:0/1862726955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642d37147e0 con 0x5642d360b550 2024-05-21T15:26:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.899+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.900+0000 7ffa6ffff640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ffa5c0a2d00 0x7ffa5c0a5210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa680ee4f0 tx=0x7ffa68079040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.900+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39d3b60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000bc10 tx=0x7ffa6000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e010f8077477c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.900+0000 7ffa65ffb640 1 -- 172.21.15.78:0/1862726955 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60015070 con 0x5642d360b550 2024-05-21T15:26:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.900+0000 7ffa65ffb640 1 -- 172.21.15.78:0/1862726955 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa60011920 con 0x5642d360b550 2024-05-21T15:26:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.900+0000 7ffa65ffb640 1 -- 172.21.15.78:0/1862726955 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60010840 con 0x5642d360b550 2024-05-21T15:26:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1862726955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 msgr2=0x5642d39d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39d3b60 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000bc10 tx=0x7ffa6000f730 comp rx=0 tx=0).stop 2024-05-21T15:26:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1862726955 shutdown_connections 2024-05-21T15:26:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39d3b60 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1862726955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 0x5642d39d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1862726955 >> 172.21.15.78:0/1862726955 conn(0x5642d3628560 msgr2=0x5642d39cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1862726955 shutdown_connections 2024-05-21T15:26:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1862726955 wait complete. 2024-05-21T15:26:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.901+0000 7ffa7cbe4740 1 -- start start 2024-05-21T15:26:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa7cbe4740 1 Processor -- start 2024-05-21T15:26:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 0x5642d39ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5642d39e0970 con 0x5642d360b550 2024-05-21T15:26:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5642d38f3b40 con 0x5642d3634410 2024-05-21T15:26:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5642d39e0af0 con 0x5642d3613730 2024-05-21T15:26:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa7cbe4740 10 client.0 _ll_register_callbacks cb 0x5642d38e80f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39df660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49456/0 (socket says 172.21.15.78:49456) 2024-05-21T15:26:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa66ffd640 1 -- 172.21.15.78:0/1037100418 learned_addr learned my addr 172.21.15.78:0/1037100418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa66ffd640 1 -- 172.21.15.78:0/1037100418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 msgr2=0x5642d39dfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39dfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa66ffd640 1 -- 172.21.15.78:0/1037100418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 msgr2=0x5642d39ddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 0x5642d39ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.902+0000 7ffa66ffd640 1 -- 172.21.15.78:0/1037100418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642d3774f70 con 0x5642d360b550 2024-05-21T15:26:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39df660 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ffa70068540 tx=0x7ffa58013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc73ee8e4cd42da server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa4effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa4effd640 1 -- 172.21.15.78:0/1037100418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58020060 con 0x5642d360b550 2024-05-21T15:26:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa4effd640 1 -- 172.21.15.78:0/1037100418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7ffa58011dc0 con 0x5642d360b550 2024-05-21T15:26:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1037100418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa60011920 con 0x5642d360b550 2024-05-21T15:26:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa4effd640 1 -- 172.21.15.78:0/1037100418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffa58027050 con 0x5642d360b550 2024-05-21T15:26:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 10 client.18471 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1037100418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5642d360c170 con 0x5642d360b550 2024-05-21T15:26:06.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa4effd640 1 -- 172.21.15.78:0/1037100418 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa60011920 con 0x5642d360b550 2024-05-21T15:26:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa4effd640 1 -- 172.21.15.78:0/1037100418 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ffa5800f040 con 0x5642d360b550 2024-05-21T15:26:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 10 client.18471 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 10 client.18471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 20 client.18471 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5642d35f31f0 0x5642d39f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 4 client.18471 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:06.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.903+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1037100418 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642d39460c0 con 0x5642d35f31f0 2024-05-21T15:26:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.904+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5642d35f31f0 0x5642d39f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.904+0000 7ffa4effd640 10 client.18471 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.904+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5642d35f31f0 0x5642d39f01f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51703693765e9b45 server_cookie=9919b0f4e6fcee68 in_seq=0 out_seq=0 2024-05-21T15:26:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa4effd640 1 -- 172.21.15.78:0/1037100418 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642d39460c0 con 0x5642d35f31f0 2024-05-21T15:26:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa4effd640 10 client.18471 handle_command_reply: tid=0 2024-05-21T15:26:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 client.18471 shutdown 2024-05-21T15:26:06.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1037100418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 msgr2=0x5642d39df660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39df660 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ffa70068540 tx=0x7ffa58013f60 comp rx=0 tx=0).stop 2024-05-21T15:26:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1037100418 shutdown_connections 2024-05-21T15:26:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5642d35f31f0 0x5642d39f01f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3634410 0x5642d39ddb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39dfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa4effd640 0 client.18471 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:06.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1037100418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d360b550 0x5642d39df660 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1037100418 >> 172.21.15.78:0/1037100418 conn(0x5642d3628560 msgr2=0x5642d39d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:06.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1037100418 shutdown_connections 2024-05-21T15:26:06.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1037100418 wait complete. 2024-05-21T15:26:06.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.905+0000 7ffa7cbe4740 20 client.18471 trim_cache size 0 max 16384 2024-05-21T15:26:06.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.923+0000 7ffa7cbe4740 1 Processor -- start 2024-05-21T15:26:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.923+0000 7ffa7cbe4740 1 -- start start 2024-05-21T15:26:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.923+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.923+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d360b550 0x5642d39f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.923+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.923+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701b0f20 con 0x5642d35f31f0 2024-05-21T15:26:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.923+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701b2a60 con 0x5642d360b550 2024-05-21T15:26:06.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.923+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701b15f0 con 0x5642d3613730 2024-05-21T15:26:06.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49458/0 (socket says 172.21.15.78:49458) 2024-05-21T15:26:06.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 -- 172.21.15.78:0/172776482 learned_addr learned my addr 172.21.15.78:0/172776482 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:06.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d360b550 0x5642d39f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 -- 172.21.15.78:0/172776482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 msgr2=0x5642d39f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:06.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 -- 172.21.15.78:0/172776482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d360b550 msgr2=0x5642d39f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d360b550 0x5642d39f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 -- 172.21.15.78:0/172776482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642d39dc080 con 0x5642d35f31f0 2024-05-21T15:26:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d360b550 0x5642d39f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39cfea0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800e6e0 tx=0x7ffa5800b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e2dfd39a21a673 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa657fa640 1 -- 172.21.15.78:0/172776482 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58030030 con 0x5642d35f31f0 2024-05-21T15:26:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa657fa640 1 -- 172.21.15.78:0/172776482 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa58005500 con 0x5642d35f31f0 2024-05-21T15:26:06.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.924+0000 7ffa657fa640 1 -- 172.21.15.78:0/172776482 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58011e20 con 0x5642d35f31f0 2024-05-21T15:26:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/172776482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 msgr2=0x5642d39cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39cfea0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800e6e0 tx=0x7ffa5800b340 comp rx=0 tx=0).stop 2024-05-21T15:26:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/172776482 shutdown_connections 2024-05-21T15:26:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d360b550 0x5642d39f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d3613730 0x5642d39f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/172776482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39cfea0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/172776482 >> 172.21.15.78:0/172776482 conn(0x5642d3628560 msgr2=0x5642d39cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/172776482 shutdown_connections 2024-05-21T15:26:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/172776482 wait complete. 2024-05-21T15:26:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.925+0000 7ffa7cbe4740 1 -- start start 2024-05-21T15:26:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa7cbe4740 1 Processor -- start 2024-05-21T15:26:06.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d360b550 0x5642d39e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa7cbe4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3613730 0x5642d39e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701b2390 con 0x5642d35f31f0 2024-05-21T15:26:06.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701b1cc0 con 0x5642d3613730 2024-05-21T15:26:06.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa7cbe4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701aef20 con 0x5642d360b550 2024-05-21T15:26:06.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa7cbe4740 10 client.0 _ll_register_callbacks cb 0x7ffa5800bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49472/0 (socket says 172.21.15.78:49472) 2024-05-21T15:26:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 -- 172.21.15.78:0/1178724697 learned_addr learned my addr 172.21.15.78:0/1178724697 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d360b550 0x5642d39e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3613730 0x5642d39e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 -- 172.21.15.78:0/1178724697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d360b550 msgr2=0x5642d39e15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d360b550 0x5642d39e15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 -- 172.21.15.78:0/1178724697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3613730 msgr2=0x5642d39e1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3613730 0x5642d39e1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 -- 172.21.15.78:0/1178724697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642d39e2130 con 0x5642d35f31f0 2024-05-21T15:26:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa66ffd640 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39e6260 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ffa58004700 tx=0x7ffa58005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503157816e0be8ca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.926+0000 7ffa4effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa4effd640 1 -- 172.21.15.78:0/1178724697 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58030040 con 0x5642d35f31f0 2024-05-21T15:26:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1178724697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa58005500 con 0x5642d35f31f0 2024-05-21T15:26:06.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa4effd640 1 -- 172.21.15.78:0/1178724697 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7ffa5801b400 con 0x5642d35f31f0 2024-05-21T15:26:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa4effd640 1 -- 172.21.15.78:0/1178724697 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffa58020040 con 0x5642d35f31f0 2024-05-21T15:26:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 10 client.18477 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1178724697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5642d3a58300 con 0x5642d35f31f0 2024-05-21T15:26:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa4effd640 1 -- 172.21.15.78:0/1178724697 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa58005500 con 0x5642d35f31f0 2024-05-21T15:26:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa4effd640 1 -- 172.21.15.78:0/1178724697 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ffa5802b660 con 0x5642d35f31f0 2024-05-21T15:26:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 10 client.18477 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 20 client.18477 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5642d3634410 0x5642d3a5a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 4 client.18477 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T15:26:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.927+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1178724697 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5642d3940410 con 0x5642d3634410 2024-05-21T15:26:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.928+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5642d3634410 0x5642d3a5a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:06.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.928+0000 7ffa6f7fe640 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5642d3634410 0x5642d3a5a980 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf323c0b79420f22 server_cookie=faf1f092580e557 in_seq=0 out_seq=0 2024-05-21T15:26:06.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.928+0000 7ffa4effd640 10 client.18477 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:06.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa4effd640 1 -- 172.21.15.78:0/1178724697 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5642d3940410 con 0x5642d3634410 2024-05-21T15:26:06.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa4effd640 10 client.18477 handle_command_reply: tid=0 2024-05-21T15:26:06.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa7cbe4740 1 client.18477 shutdown 2024-05-21T15:26:06.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1178724697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 msgr2=0x5642d39e6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39e6260 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ffa58004700 tx=0x7ffa58005db0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1178724697 shutdown_connections 2024-05-21T15:26:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x5642d3634410 0x5642d3a5a980 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5642d3613730 0x5642d39e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa4effd640 0 client.18477 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642d360b550 0x5642d39e15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.929+0000 7ffa7cbe4740 1 --2- 172.21.15.78:0/1178724697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642d35f31f0 0x5642d39e6260 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1178724697 >> 172.21.15.78:0/1178724697 conn(0x5642d3628560 msgr2=0x5642d39d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1178724697 shutdown_connections 2024-05-21T15:26:06.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa7cbe4740 1 -- 172.21.15.78:0/1178724697 wait complete. 2024-05-21T15:26:06.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa7cbe4740 20 client.18477 trim_cache size 0 max 16384 2024-05-21T15:26:06.945 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:06.945 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-05-21T15:26:06.945 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "004751fe-2925-4a89-92e6-c655436ac557", 2024-05-21T15:26:06.946 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-05-21T15:26:06.946 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:06.947 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:06 smithi172 ceph-mon[36402]: pgmap v309: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T15:26:06.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 -- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ffa5c0a2d00 msgr2=0x7ffa5c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ffa5c0a2d00 0x7ffa5c0a5210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa680ee4f0 tx=0x7ffa68079040 comp rx=0 tx=0).stop 2024-05-21T15:26:06.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 -- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 msgr2=0x7ffa7019b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:06.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 0x7ffa7019b010 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffa68031a20 tx=0x7ffa68031a50 comp rx=0 tx=0).stop 2024-05-21T15:26:06.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 -- 172.21.15.78:0/3455391658 shutdown_connections 2024-05-21T15:26:06.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa701079e0 0x7ffa7019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ffa5c0a2d00 0x7ffa5c0a5210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa7010c340 0x7ffa7019b010 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 --2- 172.21.15.78:0/3455391658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa70101ab0 0x7ffa7019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:06.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 -- 172.21.15.78:0/3455391658 >> 172.21.15.78:0/3455391658 conn(0x7ffa700fd210 msgr2=0x7ffa700fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:06.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 -- 172.21.15.78:0/3455391658 shutdown_connections 2024-05-21T15:26:06.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:06.930+0000 7ffa75d38640 1 -- 172.21.15.78:0/3455391658 wait complete. 2024-05-21T15:26:06.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:26:06.950 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:26:07.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 -- 172.21.15.78:0/4176624708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0341079e0 msgr2=0x7fc034101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 --2- 172.21.15.78:0/4176624708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0341079e0 0x7fc034101570 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc028009900 tx=0x7fc02802a410 comp rx=0 tx=0).stop 2024-05-21T15:26:07.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 -- 172.21.15.78:0/4176624708 shutdown_connections 2024-05-21T15:26:07.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 --2- 172.21.15.78:0/4176624708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc03410c340 0x7fc03410e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 --2- 172.21.15.78:0/4176624708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc034101ab0 0x7fc034101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 --2- 172.21.15.78:0/4176624708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0341079e0 0x7fc034101570 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 -- 172.21.15.78:0/4176624708 >> 172.21.15.78:0/4176624708 conn(0x7fc0340fd210 msgr2=0x7fc0340ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:07.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 -- 172.21.15.78:0/4176624708 shutdown_connections 2024-05-21T15:26:07.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.063+0000 7fc03c16c640 1 -- 172.21.15.78:0/4176624708 wait complete. 2024-05-21T15:26:07.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03c16c640 1 Processor -- start 2024-05-21T15:26:07.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03c16c640 1 -- start start 2024-05-21T15:26:07.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03c16c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc034101ab0 0x7fc0341986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03c16c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0341079e0 0x7fc034198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03c16c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc03410c340 0x7fc03419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03c16c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc034111cf0 con 0x7fc034101ab0 2024-05-21T15:26:07.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03c16c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc034111b70 con 0x7fc03410c340 2024-05-21T15:26:07.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03c16c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc034111e70 con 0x7fc0341079e0 2024-05-21T15:26:07.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc039f08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc034101ab0 0x7fc0341986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc039f08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc034101ab0 0x7fc0341986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49490/0 (socket says 172.21.15.78:49490) 2024-05-21T15:26:07.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc039f08640 1 -- 172.21.15.78:0/852678552 learned_addr learned my addr 172.21.15.78:0/852678552 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:07.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.064+0000 7fc03a709640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc03410c340 0x7fc03419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc039707640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0341079e0 0x7fc034198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc039f08640 1 -- 172.21.15.78:0/852678552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0341079e0 msgr2=0x7fc034198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc039f08640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0341079e0 0x7fc034198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc039f08640 1 -- 172.21.15.78:0/852678552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc03410c340 msgr2=0x7fc03419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc039f08640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc03410c340 0x7fc03419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc039f08640 1 -- 172.21.15.78:0/852678552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03419d530 con 0x7fc034101ab0 2024-05-21T15:26:07.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc039707640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0341079e0 0x7fc034198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:07.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc039f08640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc034101ab0 0x7fc0341986d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc028005940 tx=0x7fc028004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:07.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc032ffd640 1 -- 172.21.15.78:0/852678552 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028046070 con 0x7fc034101ab0 2024-05-21T15:26:07.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc032ffd640 1 -- 172.21.15.78:0/852678552 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc028003ad0 con 0x7fc034101ab0 2024-05-21T15:26:07.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc03c16c640 1 -- 172.21.15.78:0/852678552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03419d7c0 con 0x7fc034101ab0 2024-05-21T15:26:07.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc032ffd640 1 -- 172.21.15.78:0/852678552 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028035050 con 0x7fc034101ab0 2024-05-21T15:26:07.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.065+0000 7fc03c16c640 1 -- 172.21.15.78:0/852678552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03419dd00 con 0x7fc034101ab0 2024-05-21T15:26:07.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.067+0000 7fc032ffd640 1 -- 172.21.15.78:0/852678552 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc028049070 con 0x7fc034101ab0 2024-05-21T15:26:07.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc043018740 1 Processor -- start 2024-05-21T15:26:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc043018740 1 -- start start 2024-05-21T15:26:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc032ffd640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc01c0a2d80 0x7fc01c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc043018740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c81333d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc043018740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81334270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc043018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc043018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c81299d00 con 0x557c80f746b0 2024-05-21T15:26:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc043018740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557c812414c0 con 0x557c80f6c4d0 2024-05-21T15:26:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc043018740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557c81295cc0 con 0x557c80f95390 2024-05-21T15:26:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc03a709640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc01c0a2d80 0x7fc01c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.068+0000 7fc032ffd640 1 -- 172.21.15.78:0/852678552 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fc0280f1040 con 0x7fc034101ab0 2024-05-21T15:26:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81334270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc03a709640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81334270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.78:55128/0 (socket says 172.21.15.78:55128) 2024-05-21T15:26:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 -- 172.21.15.78:0/2040130529 learned_addr learned my addr 172.21.15.78:0/2040130529 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039f08640 1 --2- 172.21.15.78:0/2040130529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c81333d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc03a709640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc01c0a2d80 0x7fc01c0a5290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc02c005fd0 tx=0x7fc02c013b60 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 -- 172.21.15.78:0/2040130529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 msgr2=0x557c81333d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 --2- 172.21.15.78:0/2040130529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c81333d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 -- 172.21.15.78:0/2040130529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 msgr2=0x557c813347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 --2- 172.21.15.78:0/2040130529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813347b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 -- 172.21.15.78:0/2040130529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c810777d0 con 0x557c80f6c4d0 2024-05-21T15:26:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc039707640 1 --2- 172.21.15.78:0/2040130529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81334270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc024009be0 tx=0x7fc02400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2423920e26060a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc01bfff640 1 -- 172.21.15.78:0/2040130529 <== mon.2 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024016070 con 0x557c80f6c4d0 2024-05-21T15:26:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc01bfff640 1 -- 172.21.15.78:0/2040130529 <== mon.2 v2:172.21.15.84:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc02400ee70 con 0x557c80f6c4d0 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.069+0000 7fc01bfff640 1 -- 172.21.15.78:0/2040130529 <== mon.2 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024011470 con 0x557c80f6c4d0 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 -- 172.21.15.78:0/2040130529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 msgr2=0x557c81334270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 --2- 172.21.15.78:0/2040130529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81334270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc024009be0 tx=0x7fc02400e6c0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 -- 172.21.15.78:0/2040130529 shutdown_connections 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 --2- 172.21.15.78:0/2040130529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81334270 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 --2- 172.21.15.78:0/2040130529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 --2- 172.21.15.78:0/2040130529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c81333d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 -- 172.21.15.78:0/2040130529 >> 172.21.15.78:0/2040130529 conn(0x557c80f894e0 msgr2=0x557c813304e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 -- 172.21.15.78:0/2040130529 shutdown_connections 2024-05-21T15:26:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.070+0000 7fc043018740 1 -- 172.21.15.78:0/2040130529 wait complete. 2024-05-21T15:26:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 1 -- start start 2024-05-21T15:26:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 1 Processor -- start 2024-05-21T15:26:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c8133fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f95390 0x557c8133e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c81340fc0 con 0x557c80f746b0 2024-05-21T15:26:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557c81254b90 con 0x557c80f95390 2024-05-21T15:26:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557c81341140 con 0x557c80f6c4d0 2024-05-21T15:26:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc043018740 10 client.0 _ll_register_callbacks cb 0x557c81290da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc039707640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc039707640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49510/0 (socket says 172.21.15.78:49510) 2024-05-21T15:26:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc039707640 1 -- 172.21.15.78:0/1219523780 learned_addr learned my addr 172.21.15.78:0/1219523780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc039f08640 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c8133fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.071+0000 7fc03a709640 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f95390 0x557c8133e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc039707640 1 -- 172.21.15.78:0/1219523780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 msgr2=0x557c8133fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc039707640 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c8133fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc039707640 1 -- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f95390 msgr2=0x557c8133e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc039707640 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f95390 0x557c8133e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc039707640 1 -- 172.21.15.78:0/1219523780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c80fad3f0 con 0x557c80f746b0 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc03a709640 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f95390 0x557c8133e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc039707640 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813403c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc024017000 tx=0x7fc0240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6e991a64513207 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc0197fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc0197fa640 1 -- 172.21.15.78:0/1219523780 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024028060 con 0x557c80f746b0 2024-05-21T15:26:07.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc0197fa640 1 -- 172.21.15.78:0/1219523780 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fc024004e50 con 0x557c80f746b0 2024-05-21T15:26:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc043018740 1 -- 172.21.15.78:0/1219523780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc02400ee70 con 0x557c80f746b0 2024-05-21T15:26:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc0197fa640 1 -- 172.21.15.78:0/1219523780 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc024016040 con 0x557c80f746b0 2024-05-21T15:26:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc043018740 10 client.18489 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc043018740 1 -- 172.21.15.78:0/1219523780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557c80f6d0f0 con 0x557c80f746b0 2024-05-21T15:26:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc0197fa640 1 -- 172.21.15.78:0/1219523780 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc02400ee70 con 0x557c80f746b0 2024-05-21T15:26:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.072+0000 7fc0197fa640 1 -- 172.21.15.78:0/1219523780 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fc02400f040 con 0x557c80f746b0 2024-05-21T15:26:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc043018740 10 client.18489 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc043018740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc043018740 20 client.18489 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc043018740 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x557c80f53f60 0x557c81350b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc043018740 4 client.18489 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc043018740 1 -- 172.21.15.78:0/1219523780 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c812a7430 con 0x557c80f53f60 2024-05-21T15:26:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc039f08640 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x557c80f53f60 0x557c81350b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc0197fa640 10 client.18489 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.073+0000 7fc039f08640 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x557c80f53f60 0x557c81350b00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6199dcfb728ffde9 server_cookie=e54a2d4f3ca4432a in_seq=0 out_seq=0 2024-05-21T15:26:07.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc0197fa640 1 -- 172.21.15.78:0/1219523780 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557c812a7430 con 0x557c80f53f60 2024-05-21T15:26:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc0197fa640 10 client.18489 handle_command_reply: tid=0 2024-05-21T15:26:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 client.18489 shutdown 2024-05-21T15:26:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 -- 172.21.15.78:0/1219523780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 msgr2=0x557c813403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813403c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fc024017000 tx=0x7fc0240048f0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 -- 172.21.15.78:0/1219523780 shutdown_connections 2024-05-21T15:26:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x557c80f53f60 0x557c81350b00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f95390 0x557c8133e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c813403c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc0197fa640 0 client.18489 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:07.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 --2- 172.21.15.78:0/1219523780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c8133fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 -- 172.21.15.78:0/1219523780 >> 172.21.15.78:0/1219523780 conn(0x557c80f894e0 msgr2=0x557c81335a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:07.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 -- 172.21.15.78:0/1219523780 shutdown_connections 2024-05-21T15:26:07.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 1 -- 172.21.15.78:0/1219523780 wait complete. 2024-05-21T15:26:07.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.074+0000 7fc043018740 20 client.18489 trim_cache size 0 max 16384 2024-05-21T15:26:07.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc043018740 1 Processor -- start 2024-05-21T15:26:07.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc043018740 1 -- start start 2024-05-21T15:26:07.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc043018740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81337ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc043018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c81337ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc043018740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c8132fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc043018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341aca50 con 0x557c80f746b0 2024-05-21T15:26:07.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc043018740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341ae590 con 0x557c80f6c4d0 2024-05-21T15:26:07.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc043018740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341ad120 con 0x557c80f95390 2024-05-21T15:26:07.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc039707640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c81337ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc039707640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c81337ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49516/0 (socket says 172.21.15.78:49516) 2024-05-21T15:26:07.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc039707640 1 -- 172.21.15.78:0/2481098667 learned_addr learned my addr 172.21.15.78:0/2481098667 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:07.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc03a709640 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c8132fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.092+0000 7fc039f08640 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81337ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc039707640 1 -- 172.21.15.78:0/2481098667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 msgr2=0x557c8132fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:07.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc039707640 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c8132fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc039707640 1 -- 172.21.15.78:0/2481098667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 msgr2=0x557c81337ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:07.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc039707640 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81337ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc039707640 1 -- 172.21.15.78:0/2481098667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8132a8c0 con 0x557c80f746b0 2024-05-21T15:26:07.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc03a709640 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c8132fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc039707640 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c81337ff0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fc024009be0 tx=0x7fc024009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46146dc6b6159d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc01b7fe640 1 -- 172.21.15.78:0/2481098667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024028070 con 0x557c80f746b0 2024-05-21T15:26:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc01b7fe640 1 -- 172.21.15.78:0/2481098667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc024011950 con 0x557c80f746b0 2024-05-21T15:26:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.093+0000 7fc01b7fe640 1 -- 172.21.15.78:0/2481098667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024023410 con 0x557c80f746b0 2024-05-21T15:26:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 -- 172.21.15.78:0/2481098667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 msgr2=0x557c81337ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c81337ff0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fc024009be0 tx=0x7fc024009940 comp rx=0 tx=0).stop 2024-05-21T15:26:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 -- 172.21.15.78:0/2481098667 shutdown_connections 2024-05-21T15:26:07.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f95390 0x557c8132fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f746b0 0x557c81337ff0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 --2- 172.21.15.78:0/2481098667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f6c4d0 0x557c81337ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 -- 172.21.15.78:0/2481098667 >> 172.21.15.78:0/2481098667 conn(0x557c80f894e0 msgr2=0x557c8133cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:07.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 -- 172.21.15.78:0/2481098667 shutdown_connections 2024-05-21T15:26:07.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 -- 172.21.15.78:0/2481098667 wait complete. 2024-05-21T15:26:07.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 -- start start 2024-05-21T15:26:07.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.094+0000 7fc043018740 1 Processor -- start 2024-05-21T15:26:07.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc043018740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c81350df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc043018740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f746b0 0x557c81351330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc043018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f95390 0x557c813b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc043018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341adec0 con 0x557c80f95390 2024-05-21T15:26:07.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc043018740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341ad7f0 con 0x557c80f746b0 2024-05-21T15:26:07.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc043018740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341aaa50 con 0x557c80f6c4d0 2024-05-21T15:26:07.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc043018740 10 client.0 _ll_register_callbacks cb 0x7fc02c009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:07.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f95390 0x557c813b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f95390 0x557c813b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49518/0 (socket says 172.21.15.78:49518) 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 -- 172.21.15.78:0/831001442 learned_addr learned my addr 172.21.15.78:0/831001442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc039f08640 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c81350df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc039707640 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f746b0 0x557c81351330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 -- 172.21.15.78:0/831001442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 msgr2=0x557c81350df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c81350df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 -- 172.21.15.78:0/831001442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f746b0 msgr2=0x557c81351330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f746b0 0x557c81351330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 -- 172.21.15.78:0/831001442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c813b7160 con 0x557c80f95390 2024-05-21T15:26:07.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc039f08640 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c81350df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:07.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc03a709640 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f95390 0x557c813b6c20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc02c013570 tx=0x7fc02c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2062cdfcebf86501 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:07.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc0197fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:07.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc0197fa640 1 -- 172.21.15.78:0/831001442 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c004b20 con 0x557c80f95390 2024-05-21T15:26:07.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc0197fa640 1 -- 172.21.15.78:0/831001442 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8658+0+0 (secure 0 0 0) 0x7fc02c004cc0 con 0x557c80f95390 2024-05-21T15:26:07.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.095+0000 7fc043018740 1 -- 172.21.15.78:0/831001442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc024011950 con 0x557c80f95390 2024-05-21T15:26:07.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc0197fa640 1 -- 172.21.15.78:0/831001442 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc02c005020 con 0x557c80f95390 2024-05-21T15:26:07.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc043018740 10 client.18495 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc043018740 1 -- 172.21.15.78:0/831001442 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557c813b82f0 con 0x557c80f95390 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc0197fa640 1 -- 172.21.15.78:0/831001442 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc024011950 con 0x557c80f95390 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc0197fa640 1 -- 172.21.15.78:0/831001442 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fc02c01f070 con 0x557c80f95390 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc043018740 10 client.18495 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc043018740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc043018740 20 client.18495 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc043018740 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x557c80f293a0 0x557c813baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc043018740 4 client.18495 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:26:07.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc043018740 1 -- 172.21.15.78:0/831001442 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c812a1780 con 0x557c80f293a0 2024-05-21T15:26:07.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.096+0000 7fc039f08640 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x557c80f293a0 0x557c813baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:07.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc0197fa640 10 client.18495 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:07.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc039f08640 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x557c80f293a0 0x557c813baab0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedc9860edc06f98 server_cookie=42b0d32bc32ee06 in_seq=0 out_seq=0 2024-05-21T15:26:07.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc0197fa640 1 -- 172.21.15.78:0/831001442 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557c812a1780 con 0x557c80f293a0 2024-05-21T15:26:07.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc0197fa640 10 client.18495 handle_command_reply: tid=0 2024-05-21T15:26:07.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 client.18495 shutdown 2024-05-21T15:26:07.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 -- 172.21.15.78:0/831001442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f95390 msgr2=0x557c813b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f95390 0x557c813b6c20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc02c013570 tx=0x7fc02c01b000 comp rx=0 tx=0).stop 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 -- 172.21.15.78:0/831001442 shutdown_connections 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x557c80f293a0 0x557c813baab0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557c80f746b0 0x557c81351330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c80f95390 0x557c813b6c20 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 --2- 172.21.15.78:0/831001442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c80f6c4d0 0x557c81350df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc0197fa640 0 client.18495 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 -- 172.21.15.78:0/831001442 >> 172.21.15.78:0/831001442 conn(0x557c80f894e0 msgr2=0x557c81335780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 -- 172.21.15.78:0/831001442 shutdown_connections 2024-05-21T15:26:07.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 1 -- 172.21.15.78:0/831001442 wait complete. 2024-05-21T15:26:07.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.097+0000 7fc043018740 20 client.18495 trim_cache size 0 max 16384 2024-05-21T15:26:07.112 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:07.112 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T15:26:07.112 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-05-21T15:26:07.112 INFO:teuthology.orchestra.run.smithi078.stdout: "004751fe-2925-4a89-92e6-c655436ac557": { 2024-05-21T15:26:07.112 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-05-21T15:26:07.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "004751fe-2925-4a89-92e6-c655436ac557", 2024-05-21T15:26:07.113 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-05-21T15:26:07.113 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:07.113 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:07.113 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:07.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 -- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc01c0a2d80 msgr2=0x7fc01c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc01c0a2d80 0x7fc01c0a5290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc02c005fd0 tx=0x7fc02c013b60 comp rx=0 tx=0).stop 2024-05-21T15:26:07.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 -- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc034101ab0 msgr2=0x7fc0341986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:07.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc034101ab0 0x7fc0341986d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc028005940 tx=0x7fc028004270 comp rx=0 tx=0).stop 2024-05-21T15:26:07.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 -- 172.21.15.78:0/852678552 shutdown_connections 2024-05-21T15:26:07.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc03410c340 0x7fc03419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc01c0a2d80 0x7fc01c0a5290 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0341079e0 0x7fc034198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 --2- 172.21.15.78:0/852678552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc034101ab0 0x7fc0341986d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:07.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 -- 172.21.15.78:0/852678552 >> 172.21.15.78:0/852678552 conn(0x7fc0340fd210 msgr2=0x7fc0340fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:07.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 -- 172.21.15.78:0/852678552 shutdown_connections 2024-05-21T15:26:07.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:07.098+0000 7fc03c16c640 1 -- 172.21.15.78:0/852678552 wait complete. 2024-05-21T15:26:07.116 INFO:tasks.cephfs.filesystem:scrub status for tag:004751fe-2925-4a89-92e6-c655436ac557 - {'path': '/', 'tag': '004751fe-2925-4a89-92e6-c655436ac557', 'options': 'recursive,force'} 2024-05-21T15:26:07.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub queued for path: / 2024-05-21T15:26:07.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:26:07.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:26:07.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:26:07.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:26:07.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:26:07.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:07 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub queued for path: / 2024-05-21T15:26:07.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:26:07.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:26:07.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:26:07.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:26:07.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:26:07.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:07 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub queued for path: / 2024-05-21T15:26:07.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:26:07.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:26:07.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:26:07.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:26:07.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:26:07.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:07.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:07 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:08 smithi078 ceph-mon[31997]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:26:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:08 smithi078 ceph-mon[31997]: scrub summary: active paths [/] 2024-05-21T15:26:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:08 smithi078 ceph-mon[31997]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:08.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:08 smithi078 ceph-mon[31997]: osdmap e97: 12 total, 12 up, 12 in 2024-05-21T15:26:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:08 smithi084 ceph-mon[36334]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:26:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:08 smithi084 ceph-mon[36334]: scrub summary: active paths [/] 2024-05-21T15:26:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:08 smithi084 ceph-mon[36334]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:08.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:08 smithi084 ceph-mon[36334]: osdmap e97: 12 total, 12 up, 12 in 2024-05-21T15:26:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:08 smithi172 ceph-mon[36402]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:26:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:08 smithi172 ceph-mon[36402]: scrub summary: active paths [/] 2024-05-21T15:26:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:08 smithi172 ceph-mon[36402]: scrub summary: idle+waiting paths [/] 2024-05-21T15:26:08.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:08 smithi172 ceph-mon[36402]: osdmap e97: 12 total, 12 up, 12 in 2024-05-21T15:26:09.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:09 smithi078 ceph-mon[31997]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 35 op/s 2024-05-21T15:26:09.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:09 smithi084 ceph-mon[36334]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 35 op/s 2024-05-21T15:26:09.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:09 smithi172 ceph-mon[36402]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 35 op/s 2024-05-21T15:26:12.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:11 smithi078 ceph-mon[31997]: pgmap v313: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-05-21T15:26:12.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:11 smithi084 ceph-mon[36334]: pgmap v313: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-05-21T15:26:12.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:11 smithi172 ceph-mon[36402]: pgmap v313: 97 pgs: 97 active+clean; 13 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-05-21T15:26:13.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:12 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:26:13.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:12 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:26:13.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:12 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:26:13.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:12 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:26:13.349 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:12 smithi078 ceph-mon[31997]: scrub summary: idle 2024-05-21T15:26:13.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:12 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:26:13.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:12 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:26:13.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:12 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:26:13.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:12 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:26:13.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:12 smithi084 ceph-mon[36334]: scrub summary: idle 2024-05-21T15:26:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:12 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:26:13.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:12 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:26:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:12 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:26:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:12 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:26:13.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:12 smithi172 ceph-mon[36402]: scrub summary: idle 2024-05-21T15:26:14.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:13 smithi078 ceph-mon[31997]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 204 B/s wr, 49 op/s 2024-05-21T15:26:14.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:13 smithi084 ceph-mon[36334]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 204 B/s wr, 49 op/s 2024-05-21T15:26:14.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:13 smithi172 ceph-mon[36402]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 204 B/s wr, 49 op/s 2024-05-21T15:26:15.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-21T15:26:15.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 -- 172.21.15.78:0/2580207958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930107ed0 msgr2=0x7fe930101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:15.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 --2- 172.21.15.78:0/2580207958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930107ed0 0x7fe930101b20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe918009900 tx=0x7fe91802a410 comp rx=0 tx=0).stop 2024-05-21T15:26:15.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 -- 172.21.15.78:0/2580207958 shutdown_connections 2024-05-21T15:26:15.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 --2- 172.21.15.78:0/2580207958 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe930102190 0x7fe93010e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 --2- 172.21.15.78:0/2580207958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930107ed0 0x7fe930101b20 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 --2- 172.21.15.78:0/2580207958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe930107a30 0x7fe9301015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 -- 172.21.15.78:0/2580207958 >> 172.21.15.78:0/2580207958 conn(0x7fe9300fd210 msgr2=0x7fe9300ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:15.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 -- 172.21.15.78:0/2580207958 shutdown_connections 2024-05-21T15:26:15.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.667+0000 7fe9364eb640 1 -- 172.21.15.78:0/2580207958 wait complete. 2024-05-21T15:26:15.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe9364eb640 1 Processor -- start 2024-05-21T15:26:15.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe9364eb640 1 -- start start 2024-05-21T15:26:15.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe9364eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930102190 0x7fe930198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:15.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe9364eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe930107a30 0x7fe930198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:15.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe9364eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe930107ed0 0x7fe93019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe9364eb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe930111dd0 con 0x7fe930102190 2024-05-21T15:26:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe9364eb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe930111c50 con 0x7fe930107ed0 2024-05-21T15:26:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe9364eb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe930111f50 con 0x7fe930107a30 2024-05-21T15:26:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930102190 0x7fe930198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930102190 0x7fe930198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35534/0 (socket says 172.21.15.78:35534) 2024-05-21T15:26:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe92ffff640 1 -- 172.21.15.78:0/4034532376 learned_addr learned my addr 172.21.15.78:0/4034532376 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe927fff640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe930107a30 0x7fe930198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.668+0000 7fe934a88640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe930107ed0 0x7fe93019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92ffff640 1 -- 172.21.15.78:0/4034532376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe930107a30 msgr2=0x7fe930198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92ffff640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe930107a30 0x7fe930198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92ffff640 1 -- 172.21.15.78:0/4034532376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe930107ed0 msgr2=0x7fe93019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92ffff640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe930107ed0 0x7fe93019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92ffff640 1 -- 172.21.15.78:0/4034532376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93019d570 con 0x7fe930102190 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92ffff640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930102190 0x7fe930198710 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d740 tx=0x7fe92000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92dffb640 1 -- 172.21.15.78:0/4034532376 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920017070 con 0x7fe930102190 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93019d860 con 0x7fe930102190 2024-05-21T15:26:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92dffb640 1 -- 172.21.15.78:0/4034532376 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe92000de50 con 0x7fe930102190 2024-05-21T15:26:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe92dffb640 1 -- 172.21.15.78:0/4034532376 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92000f6b0 con 0x7fe930102190 2024-05-21T15:26:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.669+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9301a51e0 con 0x7fe930102190 2024-05-21T15:26:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.670+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe930102d20 con 0x7fe930102190 2024-05-21T15:26:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.672+0000 7fe92dffb640 1 -- 172.21.15.78:0/4034532376 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe92001a070 con 0x7fe930102190 2024-05-21T15:26:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.673+0000 7fe92dffb640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe91c0a2cf0 0x7fe91c0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.673+0000 7fe927fff640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe91c0a2cf0 0x7fe91c0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.673+0000 7fe92dffb640 1 -- 172.21.15.78:0/4034532376 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8702+0+0 (secure 0 0 0) 0x7fe9200cee40 con 0x7fe930102190 2024-05-21T15:26:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.674+0000 7fe927fff640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe91c0a2cf0 0x7fe91c0a5200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe91802aa00 tx=0x7fe918000bc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:15.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.677+0000 7fe92dffb640 1 -- 172.21.15.78:0/4034532376 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe92009b360 con 0x7fe930102190 2024-05-21T15:26:15.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.873+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe9300fdd30 con 0x7fe91c0a2cf0 2024-05-21T15:26:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.882+0000 7fe92dffb640 1 -- 172.21.15.78:0/4034532376 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fe9300fdd30 con 0x7fe91c0a2cf0 2024-05-21T15:26:15.885 INFO:teuthology.orchestra.run.smithi078.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-21T00:00:00", "created": "2024-05-21T15:16:14", "first": "2024-05-21T15:17:00", "last": "2024-05-21T15:26:00", "last_pruned": "2024-05-21T15:26:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-21T15:26:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe91c0a2cf0 msgr2=0x7fe91c0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe91c0a2cf0 0x7fe91c0a5200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe91802aa00 tx=0x7fe918000bc0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930102190 msgr2=0x7fe930198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930102190 0x7fe930198710 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d740 tx=0x7fe92000dc00 comp rx=0 tx=0).stop 2024-05-21T15:26:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 shutdown_connections 2024-05-21T15:26:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe930107ed0 0x7fe93019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe91c0a2cf0 0x7fe91c0a5200 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe930107a30 0x7fe930198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 --2- 172.21.15.78:0/4034532376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe930102190 0x7fe930198710 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 >> 172.21.15.78:0/4034532376 conn(0x7fe9300fd210 msgr2=0x7fe9300fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 shutdown_connections 2024-05-21T15:26:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:15.884+0000 7fe9364eb640 1 -- 172.21.15.78:0/4034532376 wait complete. 2024-05-21T15:26:15.898 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-21T15:26:16.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 -- 172.21.15.78:0/602396284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f010c3a0 msgr2=0x7fe2f010e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:16.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 --2- 172.21.15.78:0/602396284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f010c3a0 0x7fe2f010e7c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8009ca0 tx=0x7fe2e802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 -- 172.21.15.78:0/602396284 shutdown_connections 2024-05-21T15:26:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 --2- 172.21.15.78:0/602396284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f0101b10 0x7fe2f0101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 --2- 172.21.15.78:0/602396284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f010c3a0 0x7fe2f010e7c0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 --2- 172.21.15.78:0/602396284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2f0107a40 0x7fe2f01015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 -- 172.21.15.78:0/602396284 >> 172.21.15.78:0/602396284 conn(0x7fe2f00fd210 msgr2=0x7fe2f00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 -- 172.21.15.78:0/602396284 shutdown_connections 2024-05-21T15:26:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.009+0000 7fe2f500d640 1 -- 172.21.15.78:0/602396284 wait complete. 2024-05-21T15:26:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.010+0000 7fe2f500d640 1 Processor -- start 2024-05-21T15:26:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.010+0000 7fe2f500d640 1 -- start start 2024-05-21T15:26:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2f500d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f0101b10 0x7fe2f01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2f500d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2f0107a40 0x7fe2f0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2f500d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f010c3a0 0x7fe2f019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2f500d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f0112e70 con 0x7fe2f0101b10 2024-05-21T15:26:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2f500d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f0112cf0 con 0x7fe2f010c3a0 2024-05-21T15:26:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2f500d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f0112ff0 con 0x7fe2f0107a40 2024-05-21T15:26:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f0101b10 0x7fe2f01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f0101b10 0x7fe2f01997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35552/0 (socket says 172.21.15.78:35552) 2024-05-21T15:26:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 -- 172.21.15.78:0/1465520642 learned_addr learned my addr 172.21.15.78:0/1465520642 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2ef59e640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f010c3a0 0x7fe2f019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2ee59c640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2f0107a40 0x7fe2f0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 -- 172.21.15.78:0/1465520642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2f0107a40 msgr2=0x7fe2f0199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2f0107a40 0x7fe2f0199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 -- 172.21.15.78:0/1465520642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f010c3a0 msgr2=0x7fe2f019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f010c3a0 0x7fe2f019e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 -- 172.21.15.78:0/1465520642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f019e5d0 con 0x7fe2f0101b10 2024-05-21T15:26:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2eed9d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f0101b10 0x7fe2f01997f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c9a0 tx=0x7fe2e000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2d7fff640 1 -- 172.21.15.78:0/1465520642 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0014070 con 0x7fe2f0101b10 2024-05-21T15:26:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.011+0000 7fe2d7fff640 1 -- 172.21.15.78:0/1465520642 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e0004440 con 0x7fe2f0101b10 2024-05-21T15:26:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.012+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f019e8c0 con 0x7fe2f0101b10 2024-05-21T15:26:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.012+0000 7fe2d7fff640 1 -- 172.21.15.78:0/1465520642 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000f450 con 0x7fe2f0101b10 2024-05-21T15:26:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.012+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f019ee10 con 0x7fe2f0101b10 2024-05-21T15:26:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.013+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f004f720 con 0x7fe2f0101b10 2024-05-21T15:26:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.014+0000 7fe2d7fff640 1 -- 172.21.15.78:0/1465520642 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe2e0017070 con 0x7fe2f0101b10 2024-05-21T15:26:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.015+0000 7fe2d7fff640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe2d80a2cf0 0x7fe2d80a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.015+0000 7fe2ee59c640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe2d80a2cf0 0x7fe2d80a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.015+0000 7fe2d7fff640 1 -- 172.21.15.78:0/1465520642 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8702+0+0 (secure 0 0 0) 0x7fe2e00cf600 con 0x7fe2f0101b10 2024-05-21T15:26:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.015+0000 7fe2ee59c640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe2d80a2cf0 0x7fe2d80a5200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00fecd0 tx=0x7fe2e400b390 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.018+0000 7fe2d7fff640 1 -- 172.21.15.78:0/1465520642 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe2e001e020 con 0x7fe2f0101b10 2024-05-21T15:26:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.214+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 --> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe2f00ff480 con 0x7fe2d80a2cf0 2024-05-21T15:26:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.223+0000 7fe2d7fff640 1 -- 172.21.15.78:0/1465520642 <== mgr.15399 v2:172.21.15.78:6800/1100997097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe2f00ff480 con 0x7fe2d80a2cf0 2024-05-21T15:26:16.226 INFO:teuthology.orchestra.run.smithi078.stdout:/ 1m 6m3h 2024-05-21T15:26:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe2d80a2cf0 msgr2=0x7fe2d80a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe2d80a2cf0 0x7fe2d80a5200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00fecd0 tx=0x7fe2e400b390 comp rx=0 tx=0).stop 2024-05-21T15:26:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f0101b10 msgr2=0x7fe2f01997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f0101b10 0x7fe2f01997f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000c9a0 tx=0x7fe2e000ce60 comp rx=0 tx=0).stop 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 shutdown_connections 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2f010c3a0 0x7fe2f019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe2d80a2cf0 0x7fe2d80a5200 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2f0107a40 0x7fe2f0199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 --2- 172.21.15.78:0/1465520642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2f0101b10 0x7fe2f01997f0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 >> 172.21.15.78:0/1465520642 conn(0x7fe2f00fd210 msgr2=0x7fe2f00fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 shutdown_connections 2024-05-21T15:26:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:16.225+0000 7fe2f500d640 1 -- 172.21.15.78:0/1465520642 wait complete. 2024-05-21T15:26:16.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:15 smithi078 ceph-mon[31997]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 204 B/s wr, 49 op/s 2024-05-21T15:26:16.240 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-21T15:26:16.251 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-21T15:26:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:15 smithi084 ceph-mon[36334]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 204 B/s wr, 49 op/s 2024-05-21T15:26:16.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:15 smithi172 ceph-mon[36402]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 204 B/s wr, 49 op/s 2024-05-21T15:26:17.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:17 smithi078 ceph-mon[31997]: from='client.18501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:17.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:17 smithi078 ceph-mon[31997]: from='client.18507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:17.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:17 smithi078 ceph-mon[31997]: osdmap e98: 12 total, 12 up, 12 in 2024-05-21T15:26:17.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:17 smithi084 ceph-mon[36334]: from='client.18501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:17.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:17 smithi084 ceph-mon[36334]: from='client.18507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:17.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:17 smithi084 ceph-mon[36334]: osdmap e98: 12 total, 12 up, 12 in 2024-05-21T15:26:17.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:17 smithi172 ceph-mon[36402]: from='client.18501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:17.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:17 smithi172 ceph-mon[36402]: from='client.18507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T15:26:17.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:17 smithi172 ceph-mon[36402]: osdmap e98: 12 total, 12 up, 12 in 2024-05-21T15:26:18.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:18 smithi078 ceph-mon[31997]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 219 B/s wr, 51 op/s 2024-05-21T15:26:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:18 smithi084 ceph-mon[36334]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 219 B/s wr, 51 op/s 2024-05-21T15:26:18.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:18 smithi172 ceph-mon[36402]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 219 B/s wr, 51 op/s 2024-05-21T15:26:20.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:20 smithi078 ceph-mon[31997]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 204 B/s wr, 48 op/s 2024-05-21T15:26:20.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:20 smithi084 ceph-mon[36334]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 204 B/s wr, 48 op/s 2024-05-21T15:26:20.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:20 smithi172 ceph-mon[36402]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 204 B/s wr, 48 op/s 2024-05-21T15:26:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:22 smithi078 ceph-mon[31997]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 204 B/s wr, 47 op/s 2024-05-21T15:26:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:22 smithi084 ceph-mon[36334]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 204 B/s wr, 47 op/s 2024-05-21T15:26:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:22 smithi172 ceph-mon[36402]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 204 B/s wr, 47 op/s 2024-05-21T15:26:24.736 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:24 smithi172 ceph-mon[36402]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-05-21T15:26:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:24 smithi078 ceph-mon[31997]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-05-21T15:26:24.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:24 smithi084 ceph-mon[36334]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-05-21T15:26:25.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:25 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:25.823 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:25 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:25.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:25 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T15:26:26.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:26 smithi078 ceph-mon[31997]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-05-21T15:26:26.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:26 smithi084 ceph-mon[36334]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-05-21T15:26:26.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:26 smithi172 ceph-mon[36402]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-05-21T15:26:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:27.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:27.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:27.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:27.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:27.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 97 B/s wr, 16 op/s 2024-05-21T15:26:27.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-21T15:26:27.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: Unable to set osd_memory_target on smithi084 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-21T15:26:27.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi078 to 176.1M 2024-05-21T15:26:27.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: Unable to set osd_memory_target on smithi078 to 184744448: error parsing value: Value '184744448' is below minimum 939524096 2024-05-21T15:26:27.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:27.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-05-21T15:26:27.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-21T15:26:27.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:27.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:27.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:27 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 97 B/s wr, 16 op/s 2024-05-21T15:26:28.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.099 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-21T15:26:28.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: Unable to set osd_memory_target on smithi084 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-21T15:26:28.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi078 to 176.1M 2024-05-21T15:26:28.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: Unable to set osd_memory_target on smithi078 to 184744448: error parsing value: Value '184744448' is below minimum 939524096 2024-05-21T15:26:28.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-05-21T15:26:28.101 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-21T15:26:28.102 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:28.102 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:28.102 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:27 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:28.147 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 97 B/s wr, 16 op/s 2024-05-21T15:26:28.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-05-21T15:26:28.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: Unable to set osd_memory_target on smithi084 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-21T15:26:28.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi078 to 176.1M 2024-05-21T15:26:28.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: Unable to set osd_memory_target on smithi078 to 184744448: error parsing value: Value '184744448' is below minimum 939524096 2024-05-21T15:26:28.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T15:26:28.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: Adjusting osd_memory_target on smithi172 to 176.2M 2024-05-21T15:26:28.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: Unable to set osd_memory_target on smithi172 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-05-21T15:26:28.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T15:26:28.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T15:26:28.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:27 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' 2024-05-21T15:26:30.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:29 smithi078 ceph-mon[31997]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:30.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:29 smithi084 ceph-mon[36334]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:30.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:29 smithi172 ceph-mon[36402]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:32.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:31 smithi078 ceph-mon[31997]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:32.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:31 smithi084 ceph-mon[36334]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:32.435 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:31 smithi172 ceph-mon[36402]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:34.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:33 smithi078 ceph-mon[31997]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:34.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:33 smithi084 ceph-mon[36334]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:34.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:33 smithi172 ceph-mon[36402]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:36.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:35 smithi078 ceph-mon[31997]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:36.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:35 smithi084 ceph-mon[36334]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:36.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:35 smithi172 ceph-mon[36402]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:37.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-05-21T15:26:37.117 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-21T15:26:37.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 -- 172.21.15.78:0/451036289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe378101b30 msgr2=0x7fe378101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/451036289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe378101b30 0x7fe378101f40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe360009900 tx=0x7fe36002a410 comp rx=0 tx=0).stop 2024-05-21T15:26:37.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 -- 172.21.15.78:0/451036289 shutdown_connections 2024-05-21T15:26:37.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/451036289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe37810c370 0x7fe37810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/451036289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe378101b30 0x7fe378101f40 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/451036289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe378107a10 0x7fe3781015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 -- 172.21.15.78:0/451036289 >> 172.21.15.78:0/451036289 conn(0x7fe378077130 msgr2=0x7fe378077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 -- 172.21.15.78:0/451036289 shutdown_connections 2024-05-21T15:26:37.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.241+0000 7fe37d0c3640 1 -- 172.21.15.78:0/451036289 wait complete. 2024-05-21T15:26:37.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.242+0000 7fe37d0c3640 1 Processor -- start 2024-05-21T15:26:37.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.242+0000 7fe37d0c3640 1 -- start start 2024-05-21T15:26:37.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37d0c3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe378101b30 0x7fe37819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37d0c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe378107a10 0x7fe37819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37d0c3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe37810c370 0x7fe37819b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37d0c3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe378111d20 con 0x7fe37810c370 2024-05-21T15:26:37.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37d0c3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe378111ba0 con 0x7fe378107a10 2024-05-21T15:26:37.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37d0c3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe378111ea0 con 0x7fe378101b30 2024-05-21T15:26:37.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe37810c370 0x7fe37819b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe37810c370 0x7fe37819b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58586/0 (socket says 172.21.15.78:58586) 2024-05-21T15:26:37.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 -- 172.21.15.78:0/1965900806 learned_addr learned my addr 172.21.15.78:0/1965900806 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37659c640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe378107a10 0x7fe37819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe376d9d640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe378101b30 0x7fe37819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 -- 172.21.15.78:0/1965900806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe378101b30 msgr2=0x7fe37819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe378101b30 0x7fe37819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 -- 172.21.15.78:0/1965900806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe378107a10 msgr2=0x7fe37819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe378107a10 0x7fe37819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 -- 172.21.15.78:0/1965900806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3781a5910 con 0x7fe37810c370 2024-05-21T15:26:37.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.243+0000 7fe37759e640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe37810c370 0x7fe37819b080 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe37000cad0 tx=0x7fe37000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.244+0000 7fe36ffff640 1 -- 172.21.15.78:0/1965900806 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37000ae20 con 0x7fe37810c370 2024-05-21T15:26:37.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.244+0000 7fe37d0c3640 1 -- 172.21.15.78:0/1965900806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3781a5c00 con 0x7fe37810c370 2024-05-21T15:26:37.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.244+0000 7fe36ffff640 1 -- 172.21.15.78:0/1965900806 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe370005020 con 0x7fe37810c370 2024-05-21T15:26:37.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.244+0000 7fe36ffff640 1 -- 172.21.15.78:0/1965900806 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37000f6f0 con 0x7fe37810c370 2024-05-21T15:26:37.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.244+0000 7fe37d0c3640 1 -- 172.21.15.78:0/1965900806 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3781a6120 con 0x7fe37810c370 2024-05-21T15:26:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.246+0000 7fe36ffff640 1 -- 172.21.15.78:0/1965900806 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe37000f8f0 con 0x7fe37810c370 2024-05-21T15:26:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.246+0000 7fe383f65740 1 Processor -- start 2024-05-21T15:26:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.246+0000 7fe383f65740 1 -- start start 2024-05-21T15:26:37.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe36ffff640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe3640a2d00 0x7fe3640a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37659c640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe3640a2d00 0x7fe3640a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe36ffff640 1 -- 172.21.15.78:0/1965900806 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7fe3700cf400 con 0x7fe37810c370 2024-05-21T15:26:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe383f65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b76abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe383f65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 0x55589b76b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe383f65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b76b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe383f65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55589b6d0d00 con 0x55589b3ab6b0 2024-05-21T15:26:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe383f65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55589b6784c0 con 0x55589b3cc390 2024-05-21T15:26:37.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe383f65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55589b6cccc0 con 0x55589b3a34d0 2024-05-21T15:26:37.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b76b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b76b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58588/0 (socket says 172.21.15.78:58588) 2024-05-21T15:26:37.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37759e640 1 -- 172.21.15.78:0/2210519937 learned_addr learned my addr 172.21.15.78:0/2210519937 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe376d9d640 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b76abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37659c640 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 0x55589b76b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37759e640 1 -- 172.21.15.78:0/2210519937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 msgr2=0x55589b76b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37759e640 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 0x55589b76b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37759e640 1 -- 172.21.15.78:0/2210519937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 msgr2=0x55589b76abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37759e640 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b76abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37759e640 1 -- 172.21.15.78:0/2210519937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589b4ae7d0 con 0x55589b3ab6b0 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.247+0000 7fe37659c640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe3640a2d00 0x7fe3640a5210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe360005940 tx=0x7fe360031690 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.248+0000 7fe37759e640 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b76b640 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe370098310 tx=0x7fe37009fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c6cb3d9d79fbd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.248+0000 7fe36d7fa640 1 -- 172.21.15.78:0/2210519937 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3700b4070 con 0x55589b3ab6b0 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.248+0000 7fe36d7fa640 1 -- 172.21.15.78:0/2210519937 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe370005560 con 0x55589b3ab6b0 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.248+0000 7fe36d7fa640 1 -- 172.21.15.78:0/2210519937 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37001d040 con 0x55589b3ab6b0 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 -- 172.21.15.78:0/2210519937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 msgr2=0x55589b76b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b76b640 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe370098310 tx=0x7fe37009fd00 comp rx=0 tx=0).stop 2024-05-21T15:26:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 -- 172.21.15.78:0/2210519937 shutdown_connections 2024-05-21T15:26:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b76b640 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 0x55589b76b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 --2- 172.21.15.78:0/2210519937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b76abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 -- 172.21.15.78:0/2210519937 >> 172.21.15.78:0/2210519937 conn(0x55589b3c04e0 msgr2=0x55589b7674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 -- 172.21.15.78:0/2210519937 shutdown_connections 2024-05-21T15:26:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 -- 172.21.15.78:0/2210519937 wait complete. 2024-05-21T15:26:37.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 -- start start 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 Processor -- start 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.249+0000 7fe383f65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 0x55589b77b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b77b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b7796d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55589b77c490 con 0x55589b3ab6b0 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55589b68bb90 con 0x55589b3cc390 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55589b77c610 con 0x55589b3a34d0 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 10 client.0 _ll_register_callbacks cb 0x55589b6c7da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b77b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b77b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58594/0 (socket says 172.21.15.78:58594) 2024-05-21T15:26:37.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 -- 172.21.15.78:0/1512298588 learned_addr learned my addr 172.21.15.78:0/1512298588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37759e640 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b7796d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe376d9d640 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 0x55589b77b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 -- 172.21.15.78:0/1512298588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 msgr2=0x55589b77b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 0x55589b77b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 -- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 msgr2=0x55589b7796d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b7796d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 -- 172.21.15.78:0/1512298588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589b3e43f0 con 0x55589b3ab6b0 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37759e640 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b7796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe37659c640 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b77b770 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe3600096c0 tx=0x7fe36002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa28994d16679f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe35e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe35e7fc640 1 -- 172.21.15.78:0/1512298588 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360035040 con 0x55589b3ab6b0 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 1 -- 172.21.15.78:0/1512298588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe370005560 con 0x55589b3ab6b0 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe35e7fc640 1 -- 172.21.15.78:0/1512298588 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7fe360003ac0 con 0x55589b3ab6b0 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe35e7fc640 1 -- 172.21.15.78:0/1512298588 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe360003cc0 con 0x55589b3ab6b0 2024-05-21T15:26:37.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 10 client.18519 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:37.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.250+0000 7fe383f65740 1 -- 172.21.15.78:0/1512298588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55589b3a40f0 con 0x55589b3ab6b0 2024-05-21T15:26:37.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe35e7fc640 1 -- 172.21.15.78:0/1512298588 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe370005560 con 0x55589b3ab6b0 2024-05-21T15:26:37.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe35e7fc640 1 -- 172.21.15.78:0/1512298588 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe36003c450 con 0x55589b3ab6b0 2024-05-21T15:26:37.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe383f65740 10 client.18519 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:37.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe383f65740 10 client.18519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:37.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe383f65740 20 client.18519 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:37.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe383f65740 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589b38af60 0x55589b78c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe383f65740 4 client.18519 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe383f65740 1 -- 172.21.15.78:0/1512298588 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55589b6de430 con 0x55589b38af60 2024-05-21T15:26:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.251+0000 7fe376d9d640 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589b38af60 0x55589b78c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.252+0000 7fe35e7fc640 10 client.18519 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.252+0000 7fe376d9d640 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589b38af60 0x55589b78c070 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd31794e21635d83 server_cookie=c7f7b948916f0610 in_seq=0 out_seq=0 2024-05-21T15:26:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.252+0000 7fe35e7fc640 1 -- 172.21.15.78:0/1512298588 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55589b6de430 con 0x55589b38af60 2024-05-21T15:26:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.252+0000 7fe35e7fc640 10 client.18519 handle_command_reply: tid=0 2024-05-21T15:26:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.252+0000 7fe383f65740 1 client.18519 shutdown 2024-05-21T15:26:37.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 -- 172.21.15.78:0/1512298588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 msgr2=0x55589b77b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b77b770 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe3600096c0 tx=0x7fe36002a410 comp rx=0 tx=0).stop 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 -- 172.21.15.78:0/1512298588 shutdown_connections 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589b38af60 0x55589b78c070 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b7796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe35e7fc640 0 client.18519 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3ab6b0 0x55589b77b770 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 --2- 172.21.15.78:0/1512298588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3a34d0 0x55589b77b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 -- 172.21.15.78:0/1512298588 >> 172.21.15.78:0/1512298588 conn(0x55589b3c04e0 msgr2=0x55589b76ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 -- 172.21.15.78:0/1512298588 shutdown_connections 2024-05-21T15:26:37.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 1 -- 172.21.15.78:0/1512298588 wait complete. 2024-05-21T15:26:37.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.253+0000 7fe383f65740 20 client.18519 trim_cache size 0 max 16384 2024-05-21T15:26:37.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.270+0000 7fe383f65740 1 Processor -- start 2024-05-21T15:26:37.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.270+0000 7fe383f65740 1 -- start start 2024-05-21T15:26:37.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe383f65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b76b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe383f65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3ab6b0 0x55589b772f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe383f65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b7734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe383f65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781acb70 con 0x55589b3a34d0 2024-05-21T15:26:37.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe383f65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781ae6b0 con 0x55589b3cc390 2024-05-21T15:26:37.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe383f65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781ad240 con 0x55589b3ab6b0 2024-05-21T15:26:37.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b76b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b76b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58602/0 (socket says 172.21.15.78:58602) 2024-05-21T15:26:37.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 -- 172.21.15.78:0/707860359 learned_addr learned my addr 172.21.15.78:0/707860359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 -- 172.21.15.78:0/707860359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3ab6b0 msgr2=0x55589b772f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 --2- 172.21.15.78:0/707860359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3ab6b0 0x55589b772f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 -- 172.21.15.78:0/707860359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 msgr2=0x55589b7734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 --2- 172.21.15.78:0/707860359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b7734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 -- 172.21.15.78:0/707860359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589b77b020 con 0x55589b3a34d0 2024-05-21T15:26:37.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe376d9d640 1 --2- 172.21.15.78:0/707860359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b76b4f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe3680058c0 tx=0x7fe368011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7466cb7442b95 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe36cff9640 1 -- 172.21.15.78:0/707860359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368021030 con 0x55589b3a34d0 2024-05-21T15:26:37.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe36cff9640 1 -- 172.21.15.78:0/707860359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe368005500 con 0x55589b3a34d0 2024-05-21T15:26:37.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.271+0000 7fe36cff9640 1 -- 172.21.15.78:0/707860359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368004a00 con 0x55589b3a34d0 2024-05-21T15:26:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 -- 172.21.15.78:0/707860359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 msgr2=0x55589b76b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 --2- 172.21.15.78:0/707860359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b76b4f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe3680058c0 tx=0x7fe368011810 comp rx=0 tx=0).stop 2024-05-21T15:26:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 -- 172.21.15.78:0/707860359 shutdown_connections 2024-05-21T15:26:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 --2- 172.21.15.78:0/707860359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3cc390 0x55589b7734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 --2- 172.21.15.78:0/707860359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3ab6b0 0x55589b772f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 --2- 172.21.15.78:0/707860359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b76b4f0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 -- 172.21.15.78:0/707860359 >> 172.21.15.78:0/707860359 conn(0x55589b3c04e0 msgr2=0x55589b76cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 -- 172.21.15.78:0/707860359 shutdown_connections 2024-05-21T15:26:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 -- 172.21.15.78:0/707860359 wait complete. 2024-05-21T15:26:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.272+0000 7fe383f65740 1 -- start start 2024-05-21T15:26:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe383f65740 1 Processor -- start 2024-05-21T15:26:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe383f65740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b781ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe383f65740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3ab6b0 0x55589b77f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe383f65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3cc390 0x55589b77f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe383f65740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781adfe0 con 0x55589b3a34d0 2024-05-21T15:26:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe383f65740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781ad910 con 0x55589b3ab6b0 2024-05-21T15:26:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe383f65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781aab70 con 0x55589b3cc390 2024-05-21T15:26:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe383f65740 10 client.0 _ll_register_callbacks cb 0x7fe360031920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b781ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b781ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58606/0 (socket says 172.21.15.78:58606) 2024-05-21T15:26:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 -- 172.21.15.78:0/4027995505 learned_addr learned my addr 172.21.15.78:0/4027995505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe37759e640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3cc390 0x55589b77f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe37659c640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3ab6b0 0x55589b77f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 -- 172.21.15.78:0/4027995505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3cc390 msgr2=0x55589b77f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3cc390 0x55589b77f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 -- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3ab6b0 msgr2=0x55589b77f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3ab6b0 0x55589b77f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 -- 172.21.15.78:0/4027995505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589b77ff70 con 0x55589b3a34d0 2024-05-21T15:26:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe37759e640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3cc390 0x55589b77f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe37659c640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3ab6b0 0x55589b77f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:37.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe376d9d640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b781ee0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe368000c00 tx=0x7fe36801d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea567c0dfe9180f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe35e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.273+0000 7fe35e7fc640 1 -- 172.21.15.78:0/4027995505 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368021040 con 0x55589b3a34d0 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe35e7fc640 1 -- 172.21.15.78:0/4027995505 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7fe3680046c0 con 0x55589b3a34d0 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 1 -- 172.21.15.78:0/4027995505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe368005500 con 0x55589b3a34d0 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe35e7fc640 1 -- 172.21.15.78:0/4027995505 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe3680048e0 con 0x55589b3a34d0 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 10 client.18525 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 1 -- 172.21.15.78:0/4027995505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55589b7f4390 con 0x55589b3a34d0 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe35e7fc640 1 -- 172.21.15.78:0/4027995505 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe368005500 con 0x55589b3a34d0 2024-05-21T15:26:37.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe35e7fc640 1 -- 172.21.15.78:0/4027995505 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe368034030 con 0x55589b3a34d0 2024-05-21T15:26:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 10 client.18525 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 20 client.18525 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589b3603a0 0x55589b7f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 4 client.18525 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T15:26:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe383f65740 1 -- 172.21.15.78:0/4027995505 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55589b6d8780 con 0x55589b3603a0 2024-05-21T15:26:37.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.274+0000 7fe37659c640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589b3603a0 0x55589b7f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe35e7fc640 10 client.18525 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe37659c640 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589b3603a0 0x55589b7f6ac0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4aeb1899f7d901 server_cookie=99a145cc6f583bbe in_seq=0 out_seq=0 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe35e7fc640 1 -- 172.21.15.78:0/4027995505 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55589b6d8780 con 0x55589b3603a0 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe35e7fc640 10 client.18525 handle_command_reply: tid=0 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 client.18525 shutdown 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 -- 172.21.15.78:0/4027995505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 msgr2=0x55589b781ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b781ee0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe368000c00 tx=0x7fe36801d740 comp rx=0 tx=0).stop 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 -- 172.21.15.78:0/4027995505 shutdown_connections 2024-05-21T15:26:37.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x55589b3603a0 0x55589b7f6ac0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55589b3ab6b0 0x55589b77f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe35e7fc640 0 client.18525 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55589b3cc390 0x55589b77f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 --2- 172.21.15.78:0/4027995505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55589b3a34d0 0x55589b781ee0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 -- 172.21.15.78:0/4027995505 >> 172.21.15.78:0/4027995505 conn(0x55589b3c04e0 msgr2=0x55589b771f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 -- 172.21.15.78:0/4027995505 shutdown_connections 2024-05-21T15:26:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 1 -- 172.21.15.78:0/4027995505 wait complete. 2024-05-21T15:26:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.275+0000 7fe383f65740 20 client.18525 trim_cache size 0 max 16384 2024-05-21T15:26:37.289 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:37.289 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-05-21T15:26:37.289 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-05-21T15:26:37.289 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:37.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 -- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe3640a2d00 msgr2=0x7fe3640a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe3640a2d00 0x7fe3640a5210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe360005940 tx=0x7fe360031690 comp rx=0 tx=0).stop 2024-05-21T15:26:37.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 -- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe37810c370 msgr2=0x7fe37819b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe37810c370 0x7fe37819b080 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe37000cad0 tx=0x7fe37000cf90 comp rx=0 tx=0).stop 2024-05-21T15:26:37.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 -- 172.21.15.78:0/1965900806 shutdown_connections 2024-05-21T15:26:37.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe378107a10 0x7fe37819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fe3640a2d00 0x7fe3640a5210 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe37810c370 0x7fe37819b080 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 --2- 172.21.15.78:0/1965900806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe378101b30 0x7fe37819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 -- 172.21.15.78:0/1965900806 >> 172.21.15.78:0/1965900806 conn(0x7fe378077130 msgr2=0x7fe3781030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 -- 172.21.15.78:0/1965900806 shutdown_connections 2024-05-21T15:26:37.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.276+0000 7fe37d0c3640 1 -- 172.21.15.78:0/1965900806 wait complete. 2024-05-21T15:26:37.292 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T15:26:37.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:26:37.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 -- 172.21.15.78:0/159558121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88108090 msgr2=0x7fbd88102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 --2- 172.21.15.78:0/159558121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88108090 0x7fbd88102200 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009900 tx=0x7fbd7802a410 comp rx=0 tx=0).stop 2024-05-21T15:26:37.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 -- 172.21.15.78:0/159558121 shutdown_connections 2024-05-21T15:26:37.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 --2- 172.21.15.78:0/159558121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88102870 0x7fbd8810e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 --2- 172.21.15.78:0/159558121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88108090 0x7fbd88102200 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 --2- 172.21.15.78:0/159558121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88107bf0 0x7fbd88101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 -- 172.21.15.78:0/159558121 >> 172.21.15.78:0/159558121 conn(0x7fbd880fc800 msgr2=0x7fbd880fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 -- 172.21.15.78:0/159558121 shutdown_connections 2024-05-21T15:26:37.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 -- 172.21.15.78:0/159558121 wait complete. 2024-05-21T15:26:37.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 Processor -- start 2024-05-21T15:26:37.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.401+0000 7fbd9041a640 1 -- start start 2024-05-21T15:26:37.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd9041a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88102870 0x7fbd8819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd9041a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88107bf0 0x7fbd8819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd9041a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88108090 0x7fbd88199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd9041a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881130b0 con 0x7fbd88107bf0 2024-05-21T15:26:37.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd9041a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88112f30 con 0x7fbd88108090 2024-05-21T15:26:37.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd9041a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88113230 con 0x7fbd88102870 2024-05-21T15:26:37.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88107bf0 0x7fbd8819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88107bf0 0x7fbd8819fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58622/0 (socket says 172.21.15.78:58622) 2024-05-21T15:26:37.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 -- 172.21.15.78:0/1441893054 learned_addr learned my addr 172.21.15.78:0/1441893054 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8e1b6640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88102870 0x7fbd8819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8e9b7640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88108090 0x7fbd88199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 -- 172.21.15.78:0/1441893054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88102870 msgr2=0x7fbd8819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88102870 0x7fbd8819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 -- 172.21.15.78:0/1441893054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88108090 msgr2=0x7fbd88199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88108090 0x7fbd88199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 -- 172.21.15.78:0/1441893054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd88199f10 con 0x7fbd88107bf0 2024-05-21T15:26:37.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd8d9b5640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88107bf0 0x7fbd8819fba0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fbd7802a930 tx=0x7fbd78003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.402+0000 7fbd7f7fe640 1 -- 172.21.15.78:0/1441893054 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7803f070 con 0x7fbd88107bf0 2024-05-21T15:26:37.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.403+0000 7fbd7f7fe640 1 -- 172.21.15.78:0/1441893054 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd78031c90 con 0x7fbd88107bf0 2024-05-21T15:26:37.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.403+0000 7fbd7f7fe640 1 -- 172.21.15.78:0/1441893054 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78035030 con 0x7fbd88107bf0 2024-05-21T15:26:37.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.403+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8819a1a0 con 0x7fbd88107bf0 2024-05-21T15:26:37.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.403+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd881a64e0 con 0x7fbd88107bf0 2024-05-21T15:26:37.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.403+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd88069140 con 0x7fbd88107bf0 2024-05-21T15:26:37.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.405+0000 7fbd7f7fe640 1 -- 172.21.15.78:0/1441893054 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbd78042070 con 0x7fbd88107bf0 2024-05-21T15:26:37.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.405+0000 7fbd7f7fe640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd740a2d80 0x7fbd740a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.405+0000 7fbd8e1b6640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd740a2d80 0x7fbd740a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.405+0000 7fbd7f7fe640 1 -- 172.21.15.78:0/1441893054 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7fbd780f01e0 con 0x7fbd88107bf0 2024-05-21T15:26:37.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.406+0000 7fbd8e1b6640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd740a2d80 0x7fbd740a5290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbd8819a9b0 tx=0x7fbd84009500 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.409+0000 7fbd7f7fe640 1 -- 172.21.15.78:0/1441893054 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd780bc720 con 0x7fbd88107bf0 2024-05-21T15:26:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.622+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd8806a350 con 0x7fbd88107bf0 2024-05-21T15:26:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.622+0000 7fbd7f7fe640 1 -- 172.21.15.78:0/1441893054 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7fbd780c0a20 con 0x7fbd88107bf0 2024-05-21T15:26:37.626 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:26:37.626 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:26:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:26:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd740a2d80 msgr2=0x7fbd740a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd740a2d80 0x7fbd740a5290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbd8819a9b0 tx=0x7fbd84009500 comp rx=0 tx=0).stop 2024-05-21T15:26:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88107bf0 msgr2=0x7fbd8819fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88107bf0 0x7fbd8819fba0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fbd7802a930 tx=0x7fbd78003b20 comp rx=0 tx=0).stop 2024-05-21T15:26:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 shutdown_connections 2024-05-21T15:26:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd88108090 0x7fbd88199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fbd740a2d80 0x7fbd740a5290 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd88107bf0 0x7fbd8819fba0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 --2- 172.21.15.78:0/1441893054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd88102870 0x7fbd8819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.624+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 >> 172.21.15.78:0/1441893054 conn(0x7fbd880fc800 msgr2=0x7fbd880fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.625+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 shutdown_connections 2024-05-21T15:26:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.625+0000 7fbd9041a640 1 -- 172.21.15.78:0/1441893054 wait complete. 2024-05-21T15:26:37.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-21T15:26:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.748+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1558044027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e74102880 msgr2=0x7f0e7410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.748+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1558044027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e74102880 0x7f0e7410e690 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009900 tx=0x7f0e6c02a440 comp rx=0 tx=0).stop 2024-05-21T15:26:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.748+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1558044027 shutdown_connections 2024-05-21T15:26:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.748+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1558044027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e74102880 0x7f0e7410e690 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.748+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1558044027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e741080a0 0x7f0e74102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.748+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1558044027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e74107c00 0x7f0e74101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.748+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1558044027 >> 172.21.15.78:0/1558044027 conn(0x7f0e740768d0 msgr2=0x7f0e74076d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.749+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1558044027 shutdown_connections 2024-05-21T15:26:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.749+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1558044027 wait complete. 2024-05-21T15:26:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.749+0000 7f0e79dcc640 1 Processor -- start 2024-05-21T15:26:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.749+0000 7f0e79dcc640 1 -- start start 2024-05-21T15:26:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e79dcc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e74102880 0x7f0e7419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e79dcc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e74107c00 0x7f0e7419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e79dcc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e741080a0 0x7f0e741988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e79dcc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e74112050 con 0x7f0e741080a0 2024-05-21T15:26:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e79dcc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e74111ed0 con 0x7f0e74107c00 2024-05-21T15:26:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e79dcc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e741121d0 con 0x7f0e74102880 2024-05-21T15:26:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e741080a0 0x7f0e741988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e741080a0 0x7f0e741988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58650/0 (socket says 172.21.15.78:58650) 2024-05-21T15:26:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 -- 172.21.15.78:0/1559384078 learned_addr learned my addr 172.21.15.78:0/1559384078 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e74107c00 0x7f0e7419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e737fe640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e74102880 0x7f0e7419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 -- 172.21.15.78:0/1559384078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e74102880 msgr2=0x7f0e7419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e74102880 0x7f0e7419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 -- 172.21.15.78:0/1559384078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e74107c00 msgr2=0x7f0e7419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e74107c00 0x7f0e7419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 -- 172.21.15.78:0/1559384078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e74198f50 con 0x7f0e741080a0 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e73fff640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e741080a0 0x7f0e741988b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009230 tx=0x7f0e6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e70ff9640 1 -- 172.21.15.78:0/1559384078 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c03f030 con 0x7f0e741080a0 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.750+0000 7f0e70ff9640 1 -- 172.21.15.78:0/1559384078 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6c003ce0 con 0x7f0e741080a0 2024-05-21T15:26:37.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.751+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1559384078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e741991e0 con 0x7f0e741080a0 2024-05-21T15:26:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.751+0000 7f0e70ff9640 1 -- 172.21.15.78:0/1559384078 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c035070 con 0x7f0e741080a0 2024-05-21T15:26:37.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.751+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1559384078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e741996d0 con 0x7f0e741080a0 2024-05-21T15:26:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.753+0000 7f0e70ff9640 1 -- 172.21.15.78:0/1559384078 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0e6c042030 con 0x7f0e741080a0 2024-05-21T15:26:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.753+0000 7f0e80c71740 1 Processor -- start 2024-05-21T15:26:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.753+0000 7f0e80c71740 1 -- start start 2024-05-21T15:26:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.753+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606eae390 0x56360724ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.753+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e854d0 0x56360724f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.753+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e8d6b0 0x56360724f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.753+0000 7f0e80c71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5636071b2c40 con 0x563606eae390 2024-05-21T15:26:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e80c71740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56360715a580 con 0x563606e854d0 2024-05-21T15:26:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e80c71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5636071aec00 con 0x563606e8d6b0 2024-05-21T15:26:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606eae390 0x56360724ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606eae390 0x56360724ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58656/0 (socket says 172.21.15.78:58656) 2024-05-21T15:26:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1506111742 learned_addr learned my addr 172.21.15.78:0/1506111742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e737fe640 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e854d0 0x56360724f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1506111742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e8d6b0 msgr2=0x56360724f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e8d6b0 0x56360724f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1506111742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e854d0 msgr2=0x56360724f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e854d0 0x56360724f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e70ff9640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e5c0a2d00 0x7f0e5c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1506111742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563606ec63f0 con 0x563606eae390 2024-05-21T15:26:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e737fe640 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e854d0 0x56360724f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e73fff640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e5c0a2d00 0x7f0e5c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606eae390 0x56360724ef30 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009a50 tx=0x7f0e68009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e35b466b6d9489a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e70ff9640 1 -- 172.21.15.78:0/1559384078 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f0e6c0efd70 con 0x7f0e741080a0 2024-05-21T15:26:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e59ffb640 1 -- 172.21.15.78:0/1506111742 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68018020 con 0x563606eae390 2024-05-21T15:26:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e59ffb640 1 -- 172.21.15.78:0/1506111742 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e68010d40 con 0x563606eae390 2024-05-21T15:26:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.754+0000 7f0e59ffb640 1 -- 172.21.15.78:0/1506111742 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68013760 con 0x563606eae390 2024-05-21T15:26:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 -- 172.21.15.78:0/1506111742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606eae390 msgr2=0x56360724ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e73fff640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e5c0a2d00 0x7f0e5c0a5210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0ed4e0 tx=0x7f0e6c08f270 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606eae390 0x56360724ef30 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009a50 tx=0x7f0e68009f10 comp rx=0 tx=0).stop 2024-05-21T15:26:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 -- 172.21.15.78:0/1506111742 shutdown_connections 2024-05-21T15:26:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e854d0 0x56360724f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e8d6b0 0x56360724f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1506111742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606eae390 0x56360724ef30 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 -- 172.21.15.78:0/1506111742 >> 172.21.15.78:0/1506111742 conn(0x563606ea24e0 msgr2=0x56360724b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 -- 172.21.15.78:0/1506111742 shutdown_connections 2024-05-21T15:26:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.755+0000 7f0e80c71740 1 -- 172.21.15.78:0/1506111742 wait complete. 2024-05-21T15:26:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 1 -- start start 2024-05-21T15:26:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 1 Processor -- start 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e854d0 0x563607259500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e8d6b0 0x563607259a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606eae390 0x563607265730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56360725c280 con 0x563606e854d0 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56360716dc40 con 0x563606e8d6b0 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56360725c400 con 0x563606eae390 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e854d0 0x563607259500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e80c71740 10 client.0 _ll_register_callbacks cb 0x563606ec5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e854d0 0x563607259500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58672/0 (socket says 172.21.15.78:58672) 2024-05-21T15:26:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1212087259 learned_addr learned my addr 172.21.15.78:0/1212087259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.756+0000 7f0e737fe640 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e8d6b0 0x563607259a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1212087259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606eae390 msgr2=0x563607265730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606eae390 0x563607265730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e8d6b0 msgr2=0x563607259a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e8d6b0 0x563607259a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1212087259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563606f907d0 con 0x563606e854d0 2024-05-21T15:26:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e737fe640 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e8d6b0 0x563607259a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e854d0 0x563607259500 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009a20 tx=0x7f0e680047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98badc95c4e6ed91 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e52ffd640 1 -- 172.21.15.78:0/1212087259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68018030 con 0x563606e854d0 2024-05-21T15:26:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e52ffd640 1 -- 172.21.15.78:0/1212087259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f0e68010b40 con 0x563606e854d0 2024-05-21T15:26:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e80c71740 1 -- 172.21.15.78:0/1212087259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e68010d40 con 0x563606e854d0 2024-05-21T15:26:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e52ffd640 1 -- 172.21.15.78:0/1212087259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0e6802f050 con 0x563606e854d0 2024-05-21T15:26:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e80c71740 10 client.18543 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e80c71740 1 -- 172.21.15.78:0/1212087259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563607268940 con 0x563606e854d0 2024-05-21T15:26:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e52ffd640 1 -- 172.21.15.78:0/1212087259 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e68010d40 con 0x563606e854d0 2024-05-21T15:26:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.757+0000 7f0e52ffd640 1 -- 172.21.15.78:0/1212087259 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0e6801d680 con 0x563606e854d0 2024-05-21T15:26:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e80c71740 10 client.18543 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e80c71740 10 client.18543 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:26:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e80c71740 20 client.18543 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563606e6cf60 0x56360726bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e80c71740 4 client.18543 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e80c71740 1 -- 172.21.15.78:0/1212087259 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636071c0370 con 0x563606e6cf60 2024-05-21T15:26:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e737fe640 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563606e6cf60 0x56360726bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e52ffd640 10 client.18543 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:26:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.758+0000 7f0e737fe640 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563606e6cf60 0x56360726bdb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bce45928f3a48fcc server_cookie=ceb8a940c00052ad in_seq=0 out_seq=0 2024-05-21T15:26:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e52ffd640 1 -- 172.21.15.78:0/1212087259 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5636071c0370 con 0x563606e6cf60 2024-05-21T15:26:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e52ffd640 10 client.18543 handle_command_reply: tid=0 2024-05-21T15:26:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e80c71740 1 client.18543 shutdown 2024-05-21T15:26:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e80c71740 1 -- 172.21.15.78:0/1212087259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e854d0 msgr2=0x563607259500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e854d0 0x563607259500 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009a20 tx=0x7f0e680047d0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e80c71740 1 -- 172.21.15.78:0/1212087259 shutdown_connections 2024-05-21T15:26:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563606e6cf60 0x56360726bdb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e8d6b0 0x563607259a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e52ffd640 0 client.18543 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:26:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.759+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606eae390 0x563607265730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.760+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1212087259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e854d0 0x563607259500 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.760+0000 7f0e80c71740 1 -- 172.21.15.78:0/1212087259 >> 172.21.15.78:0/1212087259 conn(0x563606ea24e0 msgr2=0x56360724ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.760+0000 7f0e80c71740 1 -- 172.21.15.78:0/1212087259 shutdown_connections 2024-05-21T15:26:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.760+0000 7f0e80c71740 1 -- 172.21.15.78:0/1212087259 wait complete. 2024-05-21T15:26:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.760+0000 7f0e80c71740 20 client.18543 trim_cache size 0 max 16384 2024-05-21T15:26:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e80c71740 1 Processor -- start 2024-05-21T15:26:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e80c71740 1 -- start start 2024-05-21T15:26:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 0x563607248570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 0x5636072656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x56360726f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e80c71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e741acc10 con 0x563606e8d6b0 2024-05-21T15:26:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e80c71740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e741ae750 con 0x563606e6cf60 2024-05-21T15:26:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e80c71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e741ad2e0 con 0x563606e854d0 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x56360726f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x56360726f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58682/0 (socket says 172.21.15.78:58682) 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.773+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1870627545 learned_addr learned my addr 172.21.15.78:0/1870627545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1870627545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 msgr2=0x5636072656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1870627545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 0x5636072656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1870627545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 msgr2=0x563607248570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1870627545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 0x563607248570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e72ffd640 1 -- 172.21.15.78:0/1870627545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56360724bb10 con 0x563606e8d6b0 2024-05-21T15:26:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/1870627545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x56360726f1c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0e68001eb0 tx=0x7f0e68004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973e2dcb096a845c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e597fa640 1 -- 172.21.15.78:0/1870627545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68018040 con 0x563606e8d6b0 2024-05-21T15:26:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e597fa640 1 -- 172.21.15.78:0/1870627545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e68024020 con 0x563606e8d6b0 2024-05-21T15:26:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.774+0000 7f0e597fa640 1 -- 172.21.15.78:0/1870627545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68014d60 con 0x563606e8d6b0 2024-05-21T15:26:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 -- 172.21.15.78:0/1870627545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 msgr2=0x56360726f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1870627545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x56360726f1c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0e68001eb0 tx=0x7f0e68004680 comp rx=0 tx=0).stop 2024-05-21T15:26:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 -- 172.21.15.78:0/1870627545 shutdown_connections 2024-05-21T15:26:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1870627545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x56360726f1c0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1870627545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 0x5636072656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 --2- 172.21.15.78:0/1870627545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 0x563607248570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 -- 172.21.15.78:0/1870627545 >> 172.21.15.78:0/1870627545 conn(0x563606ea24e0 msgr2=0x563607258230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 -- 172.21.15.78:0/1870627545 shutdown_connections 2024-05-21T15:26:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 -- 172.21.15.78:0/1870627545 wait complete. 2024-05-21T15:26:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 -- start start 2024-05-21T15:26:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.775+0000 7f0e80c71740 1 Processor -- start 2024-05-21T15:26:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 0x563607249390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 0x5636072498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e80c71740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x5636072d3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e80c71740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e741ae080 con 0x563606e8d6b0 2024-05-21T15:26:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e80c71740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e741ad9b0 con 0x563606e6cf60 2024-05-21T15:26:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e80c71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e741aac10 con 0x563606e854d0 2024-05-21T15:26:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e80c71740 10 client.0 _ll_register_callbacks cb 0x7f0e68009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x5636072d3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x5636072d3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58694/0 (socket says 172.21.15.78:58694) 2024-05-21T15:26:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 -- 172.21.15.78:0/2819639772 learned_addr learned my addr 172.21.15.78:0/2819639772 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 0x563607249390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e737fe640 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 0x5636072498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 -- 172.21.15.78:0/2819639772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 msgr2=0x5636072498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 0x5636072498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 -- 172.21.15.78:0/2819639772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 msgr2=0x563607249390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 0x563607249390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 -- 172.21.15.78:0/2819639772 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56360724a170 con 0x563606e8d6b0 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e72ffd640 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x5636072d3ef0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f0e68010ec0 tx=0x7f0e68010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685d4f1f6a77b0e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e52ffd640 1 -- 172.21.15.78:0/2819639772 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e680133a0 con 0x563606e8d6b0 2024-05-21T15:26:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e52ffd640 1 -- 172.21.15.78:0/2819639772 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f0e6801d420 con 0x563606e8d6b0 2024-05-21T15:26:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e80c71740 1 -- 172.21.15.78:0/2819639772 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e68024020 con 0x563606e8d6b0 2024-05-21T15:26:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.776+0000 7f0e52ffd640 1 -- 172.21.15.78:0/2819639772 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0e6801d620 con 0x563606e8d6b0 2024-05-21T15:26:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e80c71740 10 client.18549 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e80c71740 1 -- 172.21.15.78:0/2819639772 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5636072d5480 con 0x563606e8d6b0 2024-05-21T15:26:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e52ffd640 1 -- 172.21.15.78:0/2819639772 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e68024020 con 0x563606e8d6b0 2024-05-21T15:26:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e52ffd640 1 -- 172.21.15.78:0/2819639772 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0e68045070 con 0x563606e8d6b0 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e80c71740 10 client.18549 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e80c71740 10 client.18549 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e80c71740 20 client.18549 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e80c71740 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563606eae390 0x5636072d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e80c71740 4 client.18549 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e80c71740 1 -- 172.21.15.78:0/2819639772 --> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636071ba6c0 con 0x563606eae390 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.777+0000 7f0e737fe640 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563606eae390 0x5636072d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e737fe640 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563606eae390 0x5636072d79f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=375f2390b0b43703 server_cookie=b923d001d7cdf1ad in_seq=0 out_seq=0 2024-05-21T15:26:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e52ffd640 10 client.18549 ms_handle_connect on v2:172.21.15.84:6832/502215426 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e52ffd640 1 -- 172.21.15.78:0/2819639772 <== mds.4 v2:172.21.15.84:6832/502215426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636071ba6c0 con 0x563606eae390 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e52ffd640 10 client.18549 handle_command_reply: tid=0 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 client.18549 shutdown 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 -- 172.21.15.78:0/2819639772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 msgr2=0x5636072d3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x5636072d3ef0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f0e68010ec0 tx=0x7f0e68010c80 comp rx=0 tx=0).stop 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 -- 172.21.15.78:0/2819639772 shutdown_connections 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.84:6832/502215426,v1:172.21.15.84:6833/502215426] conn(0x563606eae390 0x5636072d79f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563606e8d6b0 0x5636072d3ef0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563606e854d0 0x5636072498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e52ffd640 0 client.18549 ms_handle_reset on v2:172.21.15.84:6832/502215426 2024-05-21T15:26:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 --2- 172.21.15.78:0/2819639772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563606e6cf60 0x563607249390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 -- 172.21.15.78:0/2819639772 >> 172.21.15.78:0/2819639772 conn(0x563606ea24e0 msgr2=0x5636072482d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 -- 172.21.15.78:0/2819639772 shutdown_connections 2024-05-21T15:26:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 1 -- 172.21.15.78:0/2819639772 wait complete. 2024-05-21T15:26:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.778+0000 7f0e80c71740 20 client.18549 trim_cache size 0 max 16384 2024-05-21T15:26:37.794 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e5c0a2d00 msgr2=0x7f0e5c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e5c0a2d00 0x7f0e5c0a5210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0ed4e0 tx=0x7f0e6c08f270 comp rx=0 tx=0).stop 2024-05-21T15:26:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e741080a0 msgr2=0x7f0e741988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e741080a0 0x7f0e741988b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009230 tx=0x7f0e6c0042f0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1559384078 shutdown_connections 2024-05-21T15:26:37.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e74107c00 0x7f0e7419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f0e5c0a2d00 0x7f0e5c0a5210 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e741080a0 0x7f0e741988b0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 --2- 172.21.15.78:0/1559384078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e74102880 0x7f0e7419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1559384078 >> 172.21.15.78:0/1559384078 conn(0x7f0e740768d0 msgr2=0x7f0e740ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1559384078 shutdown_connections 2024-05-21T15:26:37.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.779+0000 7f0e79dcc640 1 -- 172.21.15.78:0/1559384078 wait complete. 2024-05-21T15:26:37.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-21T15:26:37.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2736808795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8103210 msgr2=0x7f61e810f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2736808795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8103210 0x7f61e810f5e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009900 tx=0x7f61e002a410 comp rx=0 tx=0).stop 2024-05-21T15:26:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2736808795 shutdown_connections 2024-05-21T15:26:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2736808795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8108f60 0x7f61e8102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2736808795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8103210 0x7f61e810f5e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2736808795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61e8108ac0 0x7f61e8102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2736808795 >> 172.21.15.78:0/2736808795 conn(0x7f61e80fe180 msgr2=0x7f61e81005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2736808795 shutdown_connections 2024-05-21T15:26:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.907+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2736808795 wait complete. 2024-05-21T15:26:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ef7f5640 1 Processor -- start 2024-05-21T15:26:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ef7f5640 1 -- start start 2024-05-21T15:26:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ef7f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8103210 0x7f61e8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ef7f5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61e8108ac0 0x7f61e8199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ef7f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8108f60 0x7f61e819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ef7f5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8112e00 con 0x7f61e8108f60 2024-05-21T15:26:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ef7f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8112c80 con 0x7f61e8103210 2024-05-21T15:26:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ef7f5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8112f80 con 0x7f61e8108ac0 2024-05-21T15:26:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61edd92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8108f60 0x7f61e819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61edd92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8108f60 0x7f61e819dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58710/0 (socket says 172.21.15.78:58710) 2024-05-21T15:26:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61edd92640 1 -- 172.21.15.78:0/2716377359 learned_addr learned my addr 172.21.15.78:0/2716377359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ed591640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8103210 0x7f61e8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ecd90640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61e8108ac0 0x7f61e8199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61edd92640 1 -- 172.21.15.78:0/2716377359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61e8108ac0 msgr2=0x7f61e8199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61edd92640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61e8108ac0 0x7f61e8199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61edd92640 1 -- 172.21.15.78:0/2716377359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8103210 msgr2=0x7f61e8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61edd92640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8103210 0x7f61e8199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61edd92640 1 -- 172.21.15.78:0/2716377359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e819e5e0 con 0x7f61e8108f60 2024-05-21T15:26:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.908+0000 7f61ed591640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8103210 0x7f61e8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.909+0000 7f61edd92640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8108f60 0x7f61e819dfa0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005ab0 tx=0x7f61e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.909+0000 7f61e67fc640 1 -- 172.21.15.78:0/2716377359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e003f030 con 0x7f61e8108f60 2024-05-21T15:26:37.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.909+0000 7f61e67fc640 1 -- 172.21.15.78:0/2716377359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e0003ce0 con 0x7f61e8108f60 2024-05-21T15:26:37.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.909+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2716377359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e819e870 con 0x7f61e8108f60 2024-05-21T15:26:37.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.909+0000 7f61e67fc640 1 -- 172.21.15.78:0/2716377359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0035070 con 0x7f61e8108f60 2024-05-21T15:26:37.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.909+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2716377359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e819ed60 con 0x7f61e8108f60 2024-05-21T15:26:37.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.911+0000 7f61e67fc640 1 -- 172.21.15.78:0/2716377359 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f61e0042030 con 0x7f61e8108f60 2024-05-21T15:26:37.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.911+0000 7f61f6774740 1 Processor -- start 2024-05-21T15:26:37.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.911+0000 7f61f6774740 1 -- start start 2024-05-21T15:26:37.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61f6774740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 0x5653c0b3fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61f6774740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61f6774740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61f6774740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5653c0aa5c40 con 0x5653c07784d0 2024-05-21T15:26:37.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61f6774740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5653c0a4d580 con 0x5653c07806b0 2024-05-21T15:26:37.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61f6774740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5653c0aa1c00 con 0x5653c07a1390 2024-05-21T15:26:37.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b40260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58712/0 (socket says 172.21.15.78:58712) 2024-05-21T15:26:37.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 -- 172.21.15.78:0/620069443 learned_addr learned my addr 172.21.15.78:0/620069443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61e67fc640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f61d00a2d00 0x7f61d00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ecd90640 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ecd90640 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 0x5653c0b3fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61edd92640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f61d00a2d00 0x7f61d00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 -- 172.21.15.78:0/620069443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 msgr2=0x5653c0b3fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 0x5653c0b3fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 -- 172.21.15.78:0/620069443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 msgr2=0x5653c0b407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 -- 172.21.15.78:0/620069443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c07b93f0 con 0x5653c07784d0 2024-05-21T15:26:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61e67fc640 1 -- 172.21.15.78:0/2716377359 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f61e00efde0 con 0x7f61e8108f60 2024-05-21T15:26:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61ed591640 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b40260 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f61d800f410 tx=0x7f61d800f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82680e11f14956dd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.912+0000 7f61cf7fe640 1 -- 172.21.15.78:0/620069443 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8004d20 con 0x5653c07784d0 2024-05-21T15:26:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61edd92640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f61d00a2d00 0x7f61d00a5210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61e00ed550 tx=0x7f61e00b7f10 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61cf7fe640 1 -- 172.21.15.78:0/620069443 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d80119e0 con 0x5653c07784d0 2024-05-21T15:26:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61cf7fe640 1 -- 172.21.15.78:0/620069443 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8005580 con 0x5653c07784d0 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61f6774740 1 -- 172.21.15.78:0/620069443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 msgr2=0x5653c0b40260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61f6774740 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b40260 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f61d800f410 tx=0x7f61d800f8d0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61f6774740 1 -- 172.21.15.78:0/620069443 shutdown_connections 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61f6774740 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61f6774740 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b40260 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61f6774740 1 --2- 172.21.15.78:0/620069443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 0x5653c0b3fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.913+0000 7f61f6774740 1 -- 172.21.15.78:0/620069443 >> 172.21.15.78:0/620069443 conn(0x5653c07954e0 msgr2=0x5653c0b3c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 -- 172.21.15.78:0/620069443 shutdown_connections 2024-05-21T15:26:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 -- 172.21.15.78:0/620069443 wait complete. 2024-05-21T15:26:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 -- start start 2024-05-21T15:26:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 Processor -- start 2024-05-21T15:26:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b4c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 0x5653c0b4a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5653c0b4cfb0 con 0x5653c07784d0 2024-05-21T15:26:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5653c0a60c40 con 0x5653c07806b0 2024-05-21T15:26:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.914+0000 7f61f6774740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5653c0b4d130 con 0x5653c07a1390 2024-05-21T15:26:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61f6774740 10 client.0 _ll_register_callbacks cb 0x5653c07b8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58722/0 (socket says 172.21.15.78:58722) 2024-05-21T15:26:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 -- 172.21.15.78:0/1514526949 learned_addr learned my addr 172.21.15.78:0/1514526949 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ed591640 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b4c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 -- 172.21.15.78:0/1514526949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 msgr2=0x5653c0b4a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 0x5653c0b4a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 -- 172.21.15.78:0/1514526949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 msgr2=0x5653c0b4c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b4c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 -- 172.21.15.78:0/1514526949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c08837d0 con 0x5653c07784d0 2024-05-21T15:26:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ed591640 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b4c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ecd90640 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4be10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f61dc012950 tx=0x7f61dc012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2eacfc2603fa99 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ccff9640 1 -- 172.21.15.78:0/1514526949 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc011d90 con 0x5653c07784d0 2024-05-21T15:26:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ccff9640 1 -- 172.21.15.78:0/1514526949 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f61dc01d420 con 0x5653c07784d0 2024-05-21T15:26:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61f6774740 1 -- 172.21.15.78:0/1514526949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d80119e0 con 0x5653c07784d0 2024-05-21T15:26:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ccff9640 1 -- 172.21.15.78:0/1514526949 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f61dc01d620 con 0x5653c07784d0 2024-05-21T15:26:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61f6774740 10 client.18561 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61f6774740 1 -- 172.21.15.78:0/1514526949 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5653c0b59890 con 0x5653c07784d0 2024-05-21T15:26:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.915+0000 7f61ccff9640 1 -- 172.21.15.78:0/1514526949 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d80119e0 con 0x5653c07784d0 2024-05-21T15:26:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61ccff9640 1 -- 172.21.15.78:0/1514526949 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f61dc024850 con 0x5653c07784d0 2024-05-21T15:26:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61f6774740 10 client.18561 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61f6774740 10 client.18561 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:26:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61f6774740 20 client.18561 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61f6774740 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5653c075ff60 0x5653c0b5ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61f6774740 4 client.18561 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61f6774740 1 -- 172.21.15.78:0/1514526949 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653c0ab3370 con 0x5653c075ff60 2024-05-21T15:26:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61ed591640 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5653c075ff60 0x5653c0b5ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61ccff9640 10 client.18561 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:26:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.916+0000 7f61ed591640 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5653c075ff60 0x5653c0b5ce60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5635a6b65ff7a0 server_cookie=b8ef078c45f7df00 in_seq=0 out_seq=0 2024-05-21T15:26:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61ccff9640 1 -- 172.21.15.78:0/1514526949 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653c0ab3370 con 0x5653c075ff60 2024-05-21T15:26:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61ccff9640 10 client.18561 handle_command_reply: tid=0 2024-05-21T15:26:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 client.18561 shutdown 2024-05-21T15:26:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 -- 172.21.15.78:0/1514526949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 msgr2=0x5653c0b4be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4be10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f61dc012950 tx=0x7f61dc012e10 comp rx=0 tx=0).stop 2024-05-21T15:26:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 -- 172.21.15.78:0/1514526949 shutdown_connections 2024-05-21T15:26:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5653c075ff60 0x5653c0b5ce60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0b4c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07a1390 0x5653c0b4a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61ccff9640 0 client.18561 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:26:37.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 --2- 172.21.15.78:0/1514526949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4be10 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.917+0000 7f61f6774740 1 -- 172.21.15.78:0/1514526949 >> 172.21.15.78:0/1514526949 conn(0x5653c07954e0 msgr2=0x5653c0b41a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.918+0000 7f61f6774740 1 -- 172.21.15.78:0/1514526949 shutdown_connections 2024-05-21T15:26:37.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.918+0000 7f61f6774740 1 -- 172.21.15.78:0/1514526949 wait complete. 2024-05-21T15:26:37.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.918+0000 7f61f6774740 20 client.18561 trim_cache size 0 max 16384 2024-05-21T15:26:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61f6774740 1 Processor -- start 2024-05-21T15:26:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61f6774740 1 -- start start 2024-05-21T15:26:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61f6774740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61f6774740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07806b0 0x5653c0b62620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61f6774740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07a1390 0x5653c0b62b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61f6774740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81aed80 con 0x5653c07784d0 2024-05-21T15:26:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61f6774740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81b08c0 con 0x5653c07a1390 2024-05-21T15:26:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61f6774740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81af450 con 0x5653c07806b0 2024-05-21T15:26:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61ecd90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61ecd90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58738/0 (socket says 172.21.15.78:58738) 2024-05-21T15:26:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.931+0000 7f61ecd90640 1 -- 172.21.15.78:0/2302626616 learned_addr learned my addr 172.21.15.78:0/2302626616 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ed591640 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07806b0 0x5653c0b62620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ecd90640 1 -- 172.21.15.78:0/2302626616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07806b0 msgr2=0x5653c0b62620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ecd90640 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07806b0 0x5653c0b62620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ecd90640 1 -- 172.21.15.78:0/2302626616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07a1390 msgr2=0x5653c0b62b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ecd90640 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07a1390 0x5653c0b62b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ecd90640 1 -- 172.21.15.78:0/2302626616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c0b37970 con 0x5653c07784d0 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ed591640 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07806b0 0x5653c0b62620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ecd90640 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4c0e0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c00 tx=0x7f61dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318d9d104a4f8e66 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ceffd640 1 -- 172.21.15.78:0/2302626616 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc031070 con 0x5653c07784d0 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ceffd640 1 -- 172.21.15.78:0/2302626616 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61dc018030 con 0x5653c07784d0 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.932+0000 7f61ceffd640 1 -- 172.21.15.78:0/2302626616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0249b0 con 0x5653c07784d0 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 -- 172.21.15.78:0/2302626616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 msgr2=0x5653c0b4c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4c0e0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f61dc000c00 tx=0x7f61dc005830 comp rx=0 tx=0).stop 2024-05-21T15:26:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 -- 172.21.15.78:0/2302626616 shutdown_connections 2024-05-21T15:26:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07a1390 0x5653c0b62b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07806b0 0x5653c0b62620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 --2- 172.21.15.78:0/2302626616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07784d0 0x5653c0b4c0e0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 -- 172.21.15.78:0/2302626616 >> 172.21.15.78:0/2302626616 conn(0x5653c07954e0 msgr2=0x5653c0b4b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 -- 172.21.15.78:0/2302626616 shutdown_connections 2024-05-21T15:26:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 -- 172.21.15.78:0/2302626616 wait complete. 2024-05-21T15:26:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 -- start start 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.933+0000 7f61f6774740 1 Processor -- start 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61f6774740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07784d0 0x5653c0bc1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61f6774740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0bc1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61f6774740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07a1390 0x5653c0bbbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61f6774740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81b01f0 con 0x5653c07a1390 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61f6774740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81afb20 con 0x5653c07806b0 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61f6774740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81acd80 con 0x5653c07784d0 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61f6774740 10 client.0 _ll_register_callbacks cb 0x7f61dc00c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07a1390 0x5653c0bbbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07a1390 0x5653c0bbbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58750/0 (socket says 172.21.15.78:58750) 2024-05-21T15:26:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 -- 172.21.15.78:0/1566184906 learned_addr learned my addr 172.21.15.78:0/1566184906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ed591640 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0bc1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07784d0 0x5653c0bc1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 -- 172.21.15.78:0/1566184906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07784d0 msgr2=0x5653c0bc1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07784d0 0x5653c0bc1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 -- 172.21.15.78:0/1566184906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 msgr2=0x5653c0bc1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0bc1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 -- 172.21.15.78:0/1566184906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c0bbc350 con 0x5653c07a1390 2024-05-21T15:26:37.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ecd90640 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07a1390 0x5653c0bbbd40 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f61dc024b60 tx=0x7f61dc024590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6852c59b030d230f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:37.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:37.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ccff9640 1 -- 172.21.15.78:0/1566184906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc031070 con 0x5653c07a1390 2024-05-21T15:26:37.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61ccff9640 1 -- 172.21.15.78:0/1566184906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f61dc004020 con 0x5653c07a1390 2024-05-21T15:26:37.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.934+0000 7f61f6774740 1 -- 172.21.15.78:0/1566184906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61dc018030 con 0x5653c07a1390 2024-05-21T15:26:37.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61ccff9640 1 -- 172.21.15.78:0/1566184906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f61dc01d420 con 0x5653c07a1390 2024-05-21T15:26:37.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61f6774740 10 client.18567 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:37.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61f6774740 1 -- 172.21.15.78:0/1566184906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5653c0bc8f60 con 0x5653c07a1390 2024-05-21T15:26:37.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61ccff9640 1 -- 172.21.15.78:0/1566184906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61dc018030 con 0x5653c07a1390 2024-05-21T15:26:37.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61ccff9640 1 -- 172.21.15.78:0/1566184906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f61dc003680 con 0x5653c07a1390 2024-05-21T15:26:37.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61f6774740 10 client.18567 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:37.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61f6774740 10 client.18567 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-05-21T15:26:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61f6774740 20 client.18567 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61f6774740 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5653c07353a0 0x5653c0bcb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61f6774740 4 client.18567 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61f6774740 1 -- 172.21.15.78:0/1566184906 --> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653c0aad6c0 con 0x5653c07353a0 2024-05-21T15:26:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.935+0000 7f61ed591640 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5653c07353a0 0x5653c0bcb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61ccff9640 10 client.18567 ms_handle_connect on v2:172.21.15.172:6832/1961121180 2024-05-21T15:26:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61ed591640 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5653c07353a0 0x5653c0bcb5d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f30700cad571e71a server_cookie=f16644f71cde1e26 in_seq=0 out_seq=0 2024-05-21T15:26:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61ccff9640 1 -- 172.21.15.78:0/1566184906 <== mds.3 v2:172.21.15.172:6832/1961121180 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653c0aad6c0 con 0x5653c07353a0 2024-05-21T15:26:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61ccff9640 10 client.18567 handle_command_reply: tid=0 2024-05-21T15:26:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 client.18567 shutdown 2024-05-21T15:26:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 -- 172.21.15.78:0/1566184906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07a1390 msgr2=0x5653c0bbbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07a1390 0x5653c0bbbd40 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f61dc024b60 tx=0x7f61dc024590 comp rx=0 tx=0).stop 2024-05-21T15:26:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 -- 172.21.15.78:0/1566184906 shutdown_connections 2024-05-21T15:26:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.172:6832/1961121180,v1:172.21.15.172:6833/1961121180] conn(0x5653c07353a0 0x5653c0bcb5d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5653c07806b0 0x5653c0bc1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61ccff9640 0 client.18567 ms_handle_reset on v2:172.21.15.172:6832/1961121180 2024-05-21T15:26:37.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653c07a1390 0x5653c0bbbd40 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 --2- 172.21.15.78:0/1566184906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5653c07784d0 0x5653c0bc1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 -- 172.21.15.78:0/1566184906 >> 172.21.15.78:0/1566184906 conn(0x5653c07954e0 msgr2=0x5653c0b3fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 -- 172.21.15.78:0/1566184906 shutdown_connections 2024-05-21T15:26:37.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 1 -- 172.21.15.78:0/1566184906 wait complete. 2024-05-21T15:26:37.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.936+0000 7f61f6774740 20 client.18567 trim_cache size 0 max 16384 2024-05-21T15:26:37.953 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f61d00a2d00 msgr2=0x7f61d00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f61d00a2d00 0x7f61d00a5210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61e00ed550 tx=0x7f61e00b7f10 comp rx=0 tx=0).stop 2024-05-21T15:26:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8108f60 msgr2=0x7f61e819dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8108f60 0x7f61e819dfa0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005ab0 tx=0x7f61e00042f0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2716377359 shutdown_connections 2024-05-21T15:26:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f61d00a2d00 0x7f61d00a5210 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61e8108f60 0x7f61e819dfa0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61e8108ac0 0x7f61e8199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 --2- 172.21.15.78:0/2716377359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e8103210 0x7f61e8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2716377359 >> 172.21.15.78:0/2716377359 conn(0x7f61e80fe180 msgr2=0x7f61e81053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2716377359 shutdown_connections 2024-05-21T15:26:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:37.937+0000 7f61ef7f5640 1 -- 172.21.15.78:0/2716377359 wait complete. 2024-05-21T15:26:37.957 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-21T15:26:38.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 -- 172.21.15.78:0/941010673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c108f90 msgr2=0x7f552c101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 --2- 172.21.15.78:0/941010673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c108f90 0x7f552c101100 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5520009900 tx=0x7f552002a410 comp rx=0 tx=0).stop 2024-05-21T15:26:38.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 -- 172.21.15.78:0/941010673 shutdown_connections 2024-05-21T15:26:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 --2- 172.21.15.78:0/941010673 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f552c101790 0x7f552c10f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 --2- 172.21.15.78:0/941010673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c108f90 0x7f552c101100 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 --2- 172.21.15.78:0/941010673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c108af0 0x7f552c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 -- 172.21.15.78:0/941010673 >> 172.21.15.78:0/941010673 conn(0x7f552c0768d0 msgr2=0x7f552c076d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 -- 172.21.15.78:0/941010673 shutdown_connections 2024-05-21T15:26:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.066+0000 7f5532707640 1 -- 172.21.15.78:0/941010673 wait complete. 2024-05-21T15:26:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5532707640 1 Processor -- start 2024-05-21T15:26:38.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5532707640 1 -- start start 2024-05-21T15:26:38.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5532707640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c108af0 0x7f552c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5532707640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f552c1022c0 0x7f552c1026d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5532707640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c1a51b0 0x7f552c1a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5532707640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f552c112e30 con 0x7f552c1a51b0 2024-05-21T15:26:38.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5532707640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f552c112cb0 con 0x7f552c1022c0 2024-05-21T15:26:38.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5532707640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f552c112fb0 con 0x7f552c108af0 2024-05-21T15:26:38.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.067+0000 7f5530ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c1a51b0 0x7f552c1a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5530ca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c1a51b0 0x7f552c1a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58756/0 (socket says 172.21.15.78:58756) 2024-05-21T15:26:38.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5530ca4640 1 -- 172.21.15.78:0/965232719 learned_addr learned my addr 172.21.15.78:0/965232719 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f552b7fe640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f552c1022c0 0x7f552c1026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f552bfff640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c108af0 0x7f552c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5530ca4640 1 -- 172.21.15.78:0/965232719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c108af0 msgr2=0x7f552c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5530ca4640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c108af0 0x7f552c101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5530ca4640 1 -- 172.21.15.78:0/965232719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f552c1022c0 msgr2=0x7f552c1026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5530ca4640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f552c1022c0 0x7f552c1026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5530ca4640 1 -- 172.21.15.78:0/965232719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c1a7b90 con 0x7f552c1a51b0 2024-05-21T15:26:38.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f552bfff640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c108af0 0x7f552c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:38.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f552b7fe640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f552c1022c0 0x7f552c1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:38.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5530ca4640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c1a51b0 0x7f552c1a75d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f552400eaa0 tx=0x7f552400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f55297fa640 1 -- 172.21.15.78:0/965232719 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400cdb0 con 0x7f552c1a51b0 2024-05-21T15:26:38.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f55297fa640 1 -- 172.21.15.78:0/965232719 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5524005020 con 0x7f552c1a51b0 2024-05-21T15:26:38.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5532707640 1 -- 172.21.15.78:0/965232719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c1a7e80 con 0x7f552c1a51b0 2024-05-21T15:26:38.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f55297fa640 1 -- 172.21.15.78:0/965232719 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240116b0 con 0x7f552c1a51b0 2024-05-21T15:26:38.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.068+0000 7f5532707640 1 -- 172.21.15.78:0/965232719 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a82c0 con 0x7f552c1a51b0 2024-05-21T15:26:38.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55395a8740 1 Processor -- start 2024-05-21T15:26:38.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55297fa640 1 -- 172.21.15.78:0/965232719 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5524011850 con 0x7f552c1a51b0 2024-05-21T15:26:38.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55395a8740 1 -- start start 2024-05-21T15:26:38.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55395a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdf9390 0x55f29c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55395a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c198260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55395a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd86b0 0x55f29c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55395a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f29c0fdc40 con 0x55f29bdd04d0 2024-05-21T15:26:38.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55395a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f29c0a5580 con 0x55f29bdf9390 2024-05-21T15:26:38.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f55395a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f29c0f9c00 con 0x55f29bdd86b0 2024-05-21T15:26:38.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f552bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f552bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58764/0 (socket says 172.21.15.78:58764) 2024-05-21T15:26:38.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.071+0000 7f552bfff640 1 -- 172.21.15.78:0/3325818718 learned_addr learned my addr 172.21.15.78:0/3325818718 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552b7fe640 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd86b0 0x55f29c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552b7fe640 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdf9390 0x55f29c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f55297fa640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55140a2d80 0x7f55140a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552bfff640 1 -- 172.21.15.78:0/3325818718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd86b0 msgr2=0x55f29c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552bfff640 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd86b0 0x55f29c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552bfff640 1 -- 172.21.15.78:0/3325818718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdf9390 msgr2=0x55f29c197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552bfff640 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdf9390 0x55f29c197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552bfff640 1 -- 172.21.15.78:0/3325818718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f29be113f0 con 0x55f29bdd04d0 2024-05-21T15:26:38.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552b7fe640 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd86b0 0x55f29c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:38.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f5530ca4640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55140a2d80 0x7f55140a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f552bfff640 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c198260 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f551c009c50 tx=0x7f551c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d62e8201b9c1c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f55297fa640 1 -- 172.21.15.78:0/965232719 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f55240cf760 con 0x7f552c1a51b0 2024-05-21T15:26:38.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f55127fc640 1 -- 172.21.15.78:0/3325818718 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c004020 con 0x55f29bdd04d0 2024-05-21T15:26:38.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f55127fc640 1 -- 172.21.15.78:0/3325818718 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551c0041c0 con 0x55f29bdd04d0 2024-05-21T15:26:38.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.072+0000 7f55127fc640 1 -- 172.21.15.78:0/3325818718 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c0126b0 con 0x55f29bdd04d0 2024-05-21T15:26:38.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 -- 172.21.15.78:0/3325818718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 msgr2=0x55f29c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f5530ca4640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55140a2d80 0x7f55140a5290 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55240985c0 tx=0x7f55240c6780 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c198260 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f551c009c50 tx=0x7f551c010740 comp rx=0 tx=0).stop 2024-05-21T15:26:38.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 -- 172.21.15.78:0/3325818718 shutdown_connections 2024-05-21T15:26:38.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd86b0 0x55f29c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c198260 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 --2- 172.21.15.78:0/3325818718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdf9390 0x55f29c197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 -- 172.21.15.78:0/3325818718 >> 172.21.15.78:0/3325818718 conn(0x55f29bded4e0 msgr2=0x55f29c1944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 -- 172.21.15.78:0/3325818718 shutdown_connections 2024-05-21T15:26:38.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 -- 172.21.15.78:0/3325818718 wait complete. 2024-05-21T15:26:38.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.073+0000 7f55395a8740 1 -- start start 2024-05-21T15:26:38.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f55395a8740 1 Processor -- start 2024-05-21T15:26:38.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f55395a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd04d0 0x55f29c1a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f55395a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd86b0 0x55f29c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f55395a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 0x55f29c1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f55395a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f29c1a5010 con 0x55f29bdd86b0 2024-05-21T15:26:38.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f55395a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f29c0b8c40 con 0x55f29bdd04d0 2024-05-21T15:26:38.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f55395a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f29c1a5190 con 0x55f29bdf9390 2024-05-21T15:26:38.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f55395a8740 10 client.0 _ll_register_callbacks cb 0x55f29be10be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:38.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd86b0 0x55f29c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd86b0 0x55f29c1a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58778/0 (socket says 172.21.15.78:58778) 2024-05-21T15:26:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552bfff640 1 -- 172.21.15.78:0/3098884233 learned_addr learned my addr 172.21.15.78:0/3098884233 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552b7fe640 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd04d0 0x55f29c1a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552b7fe640 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 0x55f29c1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552bfff640 1 -- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 msgr2=0x55f29c1a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552bfff640 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 0x55f29c1a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552bfff640 1 -- 172.21.15.78:0/3098884233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd04d0 msgr2=0x55f29c1a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552bfff640 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd04d0 0x55f29c1a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552bfff640 1 -- 172.21.15.78:0/3098884233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f29bedb7d0 con 0x55f29bdd86b0 2024-05-21T15:26:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.074+0000 7f552b7fe640 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 0x55f29c1a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f552bfff640 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd86b0 0x55f29c1a4310 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f551c005870 tx=0x7f551c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71148a6bb2c6c1cf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f550b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f550b7fe640 1 -- 172.21.15.78:0/3098884233 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c009790 con 0x55f29bdd86b0 2024-05-21T15:26:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f550b7fe640 1 -- 172.21.15.78:0/3098884233 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f551c003f00 con 0x55f29bdd86b0 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 1 -- 172.21.15.78:0/3098884233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f551c0041c0 con 0x55f29bdd86b0 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f550b7fe640 1 -- 172.21.15.78:0/3098884233 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f551c012450 con 0x55f29bdd86b0 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 10 client.18579 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 1 -- 172.21.15.78:0/3098884233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f29c1b1780 con 0x55f29bdd86b0 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f550b7fe640 1 -- 172.21.15.78:0/3098884233 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551c0041c0 con 0x55f29bdd86b0 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f550b7fe640 1 -- 172.21.15.78:0/3098884233 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f551c00f040 con 0x55f29bdd86b0 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 10 client.18579 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 10 client.18579 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:26:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 20 client.18579 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55f29bdb7f60 0x55f29c1b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 4 client.18579 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.075+0000 7f55395a8740 1 -- 172.21.15.78:0/3098884233 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f29c10b370 con 0x55f29bdb7f60 2024-05-21T15:26:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.076+0000 7f552b7fe640 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55f29bdb7f60 0x55f29c1b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.076+0000 7f550b7fe640 10 client.18579 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:26:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.076+0000 7f552b7fe640 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55f29bdb7f60 0x55f29c1b4ae0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aea614a6f8962d9c server_cookie=565a98efd274648d in_seq=0 out_seq=0 2024-05-21T15:26:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f550b7fe640 1 -- 172.21.15.78:0/3098884233 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f29c10b370 con 0x55f29bdb7f60 2024-05-21T15:26:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f550b7fe640 10 client.18579 handle_command_reply: tid=0 2024-05-21T15:26:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 client.18579 shutdown 2024-05-21T15:26:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 -- 172.21.15.78:0/3098884233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd86b0 msgr2=0x55f29c1a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd86b0 0x55f29c1a4310 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f551c005870 tx=0x7f551c0056d0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 -- 172.21.15.78:0/3098884233 shutdown_connections 2024-05-21T15:26:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55f29bdb7f60 0x55f29c1b4ae0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 0x55f29c1a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f550b7fe640 0 client.18579 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:26:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd86b0 0x55f29c1a4310 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 --2- 172.21.15.78:0/3098884233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd04d0 0x55f29c1a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 -- 172.21.15.78:0/3098884233 >> 172.21.15.78:0/3098884233 conn(0x55f29bded4e0 msgr2=0x55f29c199a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 -- 172.21.15.78:0/3098884233 shutdown_connections 2024-05-21T15:26:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.077+0000 7f55395a8740 1 -- 172.21.15.78:0/3098884233 wait complete. 2024-05-21T15:26:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.078+0000 7f55395a8740 20 client.18579 trim_cache size 0 max 16384 2024-05-21T15:26:38.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f55395a8740 1 Processor -- start 2024-05-21T15:26:38.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f55395a8740 1 -- start start 2024-05-21T15:26:38.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f55395a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd04d0 0x55f29c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f55395a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 0x55f29c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f55395a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdf9390 0x55f29c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f55395a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b0f10 con 0x55f29bdf9390 2024-05-21T15:26:38.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f55395a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b2a50 con 0x55f29bdd86b0 2024-05-21T15:26:38.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f55395a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b15e0 con 0x55f29bdd04d0 2024-05-21T15:26:38.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f552b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdf9390 0x55f29c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f552b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdf9390 0x55f29c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58784/0 (socket says 172.21.15.78:58784) 2024-05-21T15:26:38.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f552b7fe640 1 -- 172.21.15.78:0/2742846028 learned_addr learned my addr 172.21.15.78:0/2742846028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f552b7fe640 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd04d0 0x55f29c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.091+0000 7f552bfff640 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 0x55f29c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f552b7fe640 1 -- 172.21.15.78:0/2742846028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd04d0 msgr2=0x55f29c19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f552b7fe640 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd04d0 0x55f29c19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f552b7fe640 1 -- 172.21.15.78:0/2742846028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 msgr2=0x55f29c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f552b7fe640 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 0x55f29c1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f552b7fe640 1 -- 172.21.15.78:0/2742846028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f29c1a07a0 con 0x55f29bdf9390 2024-05-21T15:26:38.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f552b7fe640 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdf9390 0x55f29c1987f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x55f29c19c9f0 tx=0x7f5520009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f599726303e787ad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f5511ffb640 1 -- 172.21.15.78:0/2742846028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520047040 con 0x55f29bdf9390 2024-05-21T15:26:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f5511ffb640 1 -- 172.21.15.78:0/2742846028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55200046c0 con 0x55f29bdf9390 2024-05-21T15:26:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.092+0000 7f5511ffb640 1 -- 172.21.15.78:0/2742846028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520035040 con 0x55f29bdf9390 2024-05-21T15:26:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 -- 172.21.15.78:0/2742846028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdf9390 msgr2=0x55f29c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdf9390 0x55f29c1987f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x55f29c19c9f0 tx=0x7f5520009260 comp rx=0 tx=0).stop 2024-05-21T15:26:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 -- 172.21.15.78:0/2742846028 shutdown_connections 2024-05-21T15:26:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 0x55f29c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdf9390 0x55f29c1987f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 --2- 172.21.15.78:0/2742846028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdd04d0 0x55f29c19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 -- 172.21.15.78:0/2742846028 >> 172.21.15.78:0/2742846028 conn(0x55f29bded4e0 msgr2=0x55f29c197ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 -- 172.21.15.78:0/2742846028 shutdown_connections 2024-05-21T15:26:38.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 -- 172.21.15.78:0/2742846028 wait complete. 2024-05-21T15:26:38.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 -- start start 2024-05-21T15:26:38.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.093+0000 7f55395a8740 1 Processor -- start 2024-05-21T15:26:38.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f55395a8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c1a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f55395a8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 0x55f29c1a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f55395a8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 0x55f29c21acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f55395a8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b2380 con 0x55f29bdd04d0 2024-05-21T15:26:38.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f55395a8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b1cb0 con 0x55f29bdd86b0 2024-05-21T15:26:38.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f55395a8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1aef10 con 0x55f29bdf9390 2024-05-21T15:26:38.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f55395a8740 10 client.0 _ll_register_callbacks cb 0x7f551c009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:38.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c1a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c1a73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58792/0 (socket says 172.21.15.78:58792) 2024-05-21T15:26:38.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 -- 172.21.15.78:0/873271192 learned_addr learned my addr 172.21.15.78:0/873271192 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552bfff640 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 0x55f29c1a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 -- 172.21.15.78:0/873271192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 msgr2=0x55f29c21acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 0x55f29c21acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 -- 172.21.15.78:0/873271192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 msgr2=0x55f29c1a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 0x55f29c1a7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 -- 172.21.15.78:0/873271192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f29c1a81e0 con 0x55f29bdd04d0 2024-05-21T15:26:38.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f552b7fe640 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c1a73d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5520005a60 tx=0x7f5520004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8816a2aced5aa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f550b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:38.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f550b7fe640 1 -- 172.21.15.78:0/873271192 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520035050 con 0x55f29bdd04d0 2024-05-21T15:26:38.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f550b7fe640 1 -- 172.21.15.78:0/873271192 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f5520005000 con 0x55f29bdd04d0 2024-05-21T15:26:38.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.094+0000 7f55395a8740 1 -- 172.21.15.78:0/873271192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55200046c0 con 0x55f29bdd04d0 2024-05-21T15:26:38.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f550b7fe640 1 -- 172.21.15.78:0/873271192 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5520005200 con 0x55f29bdd04d0 2024-05-21T15:26:38.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f55395a8740 10 client.18585 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:38.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f55395a8740 1 -- 172.21.15.78:0/873271192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f29c21c2b0 con 0x55f29bdd04d0 2024-05-21T15:26:38.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f550b7fe640 1 -- 172.21.15.78:0/873271192 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55200046c0 con 0x55f29bdd04d0 2024-05-21T15:26:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f550b7fe640 1 -- 172.21.15.78:0/873271192 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f552004a630 con 0x55f29bdd04d0 2024-05-21T15:26:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f55395a8740 10 client.18585 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f55395a8740 10 client.18585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-21T15:26:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f55395a8740 20 client.18585 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f55395a8740 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55f29bd8d3a0 0x55f29c21e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f55395a8740 4 client.18585 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f55395a8740 1 -- 172.21.15.78:0/873271192 --> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f29c1056c0 con 0x55f29bd8d3a0 2024-05-21T15:26:38.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.095+0000 7f552bfff640 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55f29bd8d3a0 0x55f29c21e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f550b7fe640 10 client.18585 ms_handle_connect on v2:172.21.15.172:6836/3093965276 2024-05-21T15:26:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f552bfff640 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55f29bd8d3a0 0x55f29c21e900 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75445e72e219e6c6 server_cookie=67cc19f7724adac2 in_seq=0 out_seq=0 2024-05-21T15:26:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f550b7fe640 1 -- 172.21.15.78:0/873271192 <== mds.2 v2:172.21.15.172:6836/3093965276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f29c1056c0 con 0x55f29bd8d3a0 2024-05-21T15:26:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f550b7fe640 10 client.18585 handle_command_reply: tid=0 2024-05-21T15:26:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 client.18585 shutdown 2024-05-21T15:26:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 -- 172.21.15.78:0/873271192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 msgr2=0x55f29c1a73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c1a73d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5520005a60 tx=0x7f5520004230 comp rx=0 tx=0).stop 2024-05-21T15:26:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 -- 172.21.15.78:0/873271192 shutdown_connections 2024-05-21T15:26:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.172:6836/3093965276,v1:172.21.15.172:6837/3093965276] conn(0x55f29bd8d3a0 0x55f29c21e900 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f29bdd86b0 0x55f29c1a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f550b7fe640 0 client.18585 ms_handle_reset on v2:172.21.15.172:6836/3093965276 2024-05-21T15:26:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f29bdf9390 0x55f29c21acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 --2- 172.21.15.78:0/873271192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f29bdd04d0 0x55f29c1a73d0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 -- 172.21.15.78:0/873271192 >> 172.21.15.78:0/873271192 conn(0x55f29bded4e0 msgr2=0x55f29c1985d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 -- 172.21.15.78:0/873271192 shutdown_connections 2024-05-21T15:26:38.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 1 -- 172.21.15.78:0/873271192 wait complete. 2024-05-21T15:26:38.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.096+0000 7f55395a8740 20 client.18585 trim_cache size 0 max 16384 2024-05-21T15:26:38.112 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:38.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 -- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55140a2d80 msgr2=0x7f55140a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55140a2d80 0x7f55140a5290 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55240985c0 tx=0x7f55240c6780 comp rx=0 tx=0).stop 2024-05-21T15:26:38.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 -- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c1a51b0 msgr2=0x7f552c1a75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c1a51b0 0x7f552c1a75d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f552400eaa0 tx=0x7f552400ef60 comp rx=0 tx=0).stop 2024-05-21T15:26:38.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 -- 172.21.15.78:0/965232719 shutdown_connections 2024-05-21T15:26:38.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f552c1022c0 0x7f552c1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f55140a2d80 0x7f55140a5290 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f552c1a51b0 0x7f552c1a75d0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 --2- 172.21.15.78:0/965232719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c108af0 0x7f552c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 -- 172.21.15.78:0/965232719 >> 172.21.15.78:0/965232719 conn(0x7f552c0768d0 msgr2=0x7f552c10ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 -- 172.21.15.78:0/965232719 shutdown_connections 2024-05-21T15:26:38.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.097+0000 7f5532707640 1 -- 172.21.15.78:0/965232719 wait complete. 2024-05-21T15:26:38.115 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-21T15:26:38.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2532970170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a64101020 msgr2=0x7f5a64101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2532970170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a64101020 0x7f5a64101430 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009900 tx=0x7f5a5802a410 comp rx=0 tx=0).stop 2024-05-21T15:26:38.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2532970170 shutdown_connections 2024-05-21T15:26:38.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2532970170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a6410d520 0x7f5a6410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2532970170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a64101020 0x7f5a64101430 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2532970170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a64108b70 0x7f5a64100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2532970170 >> 172.21.15.78:0/2532970170 conn(0x7f5a640768d0 msgr2=0x7f5a64076d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2532970170 shutdown_connections 2024-05-21T15:26:38.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.225+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2532970170 wait complete. 2024-05-21T15:26:38.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a68a2b640 1 Processor -- start 2024-05-21T15:26:38.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a68a2b640 1 -- start start 2024-05-21T15:26:38.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a68a2b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a64101020 0x7f5a641997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a68a2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a64108b70 0x7f5a64199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a68a2b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6410d520 0x7f5a6419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a68a2b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a64112e80 con 0x7f5a6410d520 2024-05-21T15:26:38.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a68a2b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5a64112d00 con 0x7f5a64108b70 2024-05-21T15:26:38.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a68a2b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a64113000 con 0x7f5a64101020 2024-05-21T15:26:38.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a62d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6410d520 0x7f5a6419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a62d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6410d520 0x7f5a6419e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58814/0 (socket says 172.21.15.78:58814) 2024-05-21T15:26:38.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a62d9d640 1 -- 172.21.15.78:0/2413816360 learned_addr learned my addr 172.21.15.78:0/2413816360 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a6259c640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a64101020 0x7f5a641997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a64108b70 0x7f5a64199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a62d9d640 1 -- 172.21.15.78:0/2413816360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a64101020 msgr2=0x7f5a641997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a62d9d640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a64101020 0x7f5a641997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a62d9d640 1 -- 172.21.15.78:0/2413816360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a64108b70 msgr2=0x7f5a64199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a62d9d640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a64108b70 0x7f5a64199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.226+0000 7f5a62d9d640 1 -- 172.21.15.78:0/2413816360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6419e640 con 0x7f5a6410d520 2024-05-21T15:26:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.227+0000 7f5a62d9d640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6410d520 0x7f5a6419e000 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00d950 tx=0x7f5a5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.227+0000 7f5a537fe640 1 -- 172.21.15.78:0/2413816360 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c017070 con 0x7f5a6410d520 2024-05-21T15:26:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.227+0000 7f5a537fe640 1 -- 172.21.15.78:0/2413816360 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a5c009c30 con 0x7f5a6410d520 2024-05-21T15:26:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.227+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2413816360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6419e930 con 0x7f5a6410d520 2024-05-21T15:26:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.227+0000 7f5a537fe640 1 -- 172.21.15.78:0/2413816360 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c003d60 con 0x7f5a6410d520 2024-05-21T15:26:38.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.227+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2413816360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6419ee00 con 0x7f5a6410d520 2024-05-21T15:26:38.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.229+0000 7f5a537fe640 1 -- 172.21.15.78:0/2413816360 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5a5c003f00 con 0x7f5a6410d520 2024-05-21T15:26:38.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6f9af740 1 Processor -- start 2024-05-21T15:26:38.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6f9af740 1 -- start start 2024-05-21T15:26:38.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 0x556a751d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a75137c40 con 0x556a74e126b0 2024-05-21T15:26:38.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556a750df580 con 0x556a74e0a4d0 2024-05-21T15:26:38.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556a75133c00 con 0x556a74e33390 2024-05-21T15:26:38.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a537fe640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a4c0a2d00 0x7f5a4c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a61d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a61d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58816/0 (socket says 172.21.15.78:58816) 2024-05-21T15:26:38.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a61d9b640 1 -- 172.21.15.78:0/3380802200 learned_addr learned my addr 172.21.15.78:0/3380802200 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 0x556a751d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a6259c640 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.230+0000 7f5a62d9d640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a4c0a2d00 0x7f5a4c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a537fe640 1 -- 172.21.15.78:0/2413816360 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f5a5c0ced60 con 0x7f5a6410d520 2024-05-21T15:26:38.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a61d9b640 1 -- 172.21.15.78:0/3380802200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 msgr2=0x556a751d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 0x556a751d1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a61d9b640 1 -- 172.21.15.78:0/3380802200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 msgr2=0x556a751d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a61d9b640 1 -- 172.21.15.78:0/3380802200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a74e4b3f0 con 0x556a74e126b0 2024-05-21T15:26:38.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a62d9d640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a4c0a2d00 0x7f5a4c0a5210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c09b210 tx=0x7f5a5c06d240 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d27a0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x556a751cee90 tx=0x7f5a5802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3eac9b52718b30f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a50ff9640 1 -- 172.21.15.78:0/3380802200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58035030 con 0x556a74e126b0 2024-05-21T15:26:38.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a50ff9640 1 -- 172.21.15.78:0/3380802200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a580038c0 con 0x556a74e126b0 2024-05-21T15:26:38.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.231+0000 7f5a50ff9640 1 -- 172.21.15.78:0/3380802200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58004a60 con 0x556a74e126b0 2024-05-21T15:26:38.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 -- 172.21.15.78:0/3380802200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 msgr2=0x556a751d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d27a0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x556a751cee90 tx=0x7f5a5802a410 comp rx=0 tx=0).stop 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 -- 172.21.15.78:0/3380802200 shutdown_connections 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d27a0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/3380802200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 0x556a751d1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 -- 172.21.15.78:0/3380802200 >> 172.21.15.78:0/3380802200 conn(0x556a74e274e0 msgr2=0x556a751ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 -- 172.21.15.78:0/3380802200 shutdown_connections 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 -- 172.21.15.78:0/3380802200 wait complete. 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.232+0000 7f5a6f9af740 1 -- start start 2024-05-21T15:26:38.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6f9af740 1 Processor -- start 2024-05-21T15:26:38.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 0x556a751dc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a751defb0 con 0x556a74e126b0 2024-05-21T15:26:38.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556a750f2c40 con 0x556a74e0a4d0 2024-05-21T15:26:38.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556a751df130 con 0x556a74e33390 2024-05-21T15:26:38.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6f9af740 10 client.0 _ll_register_callbacks cb 0x556a74e4abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751de360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58832/0 (socket says 172.21.15.78:58832) 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6259c640 1 -- 172.21.15.78:0/2881624979 learned_addr learned my addr 172.21.15.78:0/2881624979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 0x556a751dc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6259c640 1 -- 172.21.15.78:0/2881624979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 msgr2=0x556a751dc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6259c640 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 0x556a751dc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6259c640 1 -- 172.21.15.78:0/2881624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 msgr2=0x556a751dde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6259c640 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751dde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a6259c640 1 -- 172.21.15.78:0/2881624979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a74f157d0 con 0x556a74e126b0 2024-05-21T15:26:38.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.233+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751dde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T15:26:38.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6259c640 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751de360 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5a640744d0 tx=0x7f5a5400c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198959e5aab50a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:38.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2881624979 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54003b70 con 0x556a74e126b0 2024-05-21T15:26:38.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2881624979 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f5a54003d10 con 0x556a74e126b0 2024-05-21T15:26:38.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2881624979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a580038c0 con 0x556a74e126b0 2024-05-21T15:26:38.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2881624979 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5a54029050 con 0x556a74e126b0 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 10 client.18597 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2881624979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556a751eb840 con 0x556a74e126b0 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2881624979 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a580038c0 con 0x556a74e126b0 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2881624979 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5a5401b870 con 0x556a74e126b0 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 10 client.18597 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 10 client.18597 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 20 client.18597 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x556a74df1f60 0x556a751eed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 4 client.18597 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:38.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.234+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2881624979 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a75145370 con 0x556a74df1f60 2024-05-21T15:26:38.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.235+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x556a74df1f60 0x556a751eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.235+0000 7f5a49ffb640 10 client.18597 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:26:38.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.235+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x556a74df1f60 0x556a751eed70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c4e541b23af75ac server_cookie=2a83006b53158ed5 in_seq=0 out_seq=0 2024-05-21T15:26:38.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2881624979 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a75145370 con 0x556a74df1f60 2024-05-21T15:26:38.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a49ffb640 10 client.18597 handle_command_reply: tid=0 2024-05-21T15:26:38.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 client.18597 shutdown 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2881624979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 msgr2=0x556a751de360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751de360 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5a640744d0 tx=0x7f5a5400c700 comp rx=0 tx=0).stop 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2881624979 shutdown_connections 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x556a74df1f60 0x556a751eed70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e33390 0x556a751dc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a49ffb640 0 client.18597 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751de360 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2881624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a751dde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2881624979 >> 172.21.15.78:0/2881624979 conn(0x556a74e274e0 msgr2=0x556a751d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2881624979 shutdown_connections 2024-05-21T15:26:38.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2881624979 wait complete. 2024-05-21T15:26:38.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.236+0000 7f5a6f9af740 20 client.18597 trim_cache size 0 max 16384 2024-05-21T15:26:38.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.249+0000 7f5a6f9af740 1 Processor -- start 2024-05-21T15:26:38.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.249+0000 7f5a6f9af740 1 -- start start 2024-05-21T15:26:38.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e0a4d0 0x556a751dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e33390 0x556a751d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641acaa0 con 0x556a74e126b0 2024-05-21T15:26:38.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641ae5e0 con 0x556a74e33390 2024-05-21T15:26:38.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641ad170 con 0x556a74e0a4d0 2024-05-21T15:26:38.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58834/0 (socket says 172.21.15.78:58834) 2024-05-21T15:26:38.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 -- 172.21.15.78:0/410899980 learned_addr learned my addr 172.21.15.78:0/410899980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 -- 172.21.15.78:0/410899980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e0a4d0 msgr2=0x556a751dbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/410899980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e0a4d0 0x556a751dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 --2- 172.21.15.78:0/410899980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e0a4d0 0x556a751dbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 -- 172.21.15.78:0/410899980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e33390 msgr2=0x556a751d5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 --2- 172.21.15.78:0/410899980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e33390 0x556a751d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 -- 172.21.15.78:0/410899980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a751ddbf0 con 0x556a74e126b0 2024-05-21T15:26:38.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a6259c640 1 --2- 172.21.15.78:0/410899980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d59d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5a54011fd0 tx=0x7f5a5400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71967af8566e7217 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a4bfff640 1 -- 172.21.15.78:0/410899980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54004c00 con 0x556a74e126b0 2024-05-21T15:26:38.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a4bfff640 1 -- 172.21.15.78:0/410899980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a54020070 con 0x556a74e126b0 2024-05-21T15:26:38.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.250+0000 7f5a4bfff640 1 -- 172.21.15.78:0/410899980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54004030 con 0x556a74e126b0 2024-05-21T15:26:38.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 -- 172.21.15.78:0/410899980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 msgr2=0x556a751d59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/410899980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d59d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5a54011fd0 tx=0x7f5a5400cef0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 -- 172.21.15.78:0/410899980 shutdown_connections 2024-05-21T15:26:38.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/410899980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e33390 0x556a751d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/410899980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e126b0 0x556a751d59d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/410899980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e0a4d0 0x556a751dbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 -- 172.21.15.78:0/410899980 >> 172.21.15.78:0/410899980 conn(0x556a74e274e0 msgr2=0x556a751db7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 -- 172.21.15.78:0/410899980 shutdown_connections 2024-05-21T15:26:38.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.251+0000 7f5a6f9af740 1 -- 172.21.15.78:0/410899980 wait complete. 2024-05-21T15:26:38.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 1 -- start start 2024-05-21T15:26:38.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 1 Processor -- start 2024-05-21T15:26:38.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a7524e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e126b0 0x556a7524ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e33390 0x556a751e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641adf10 con 0x556a74e33390 2024-05-21T15:26:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641ad840 con 0x556a74e0a4d0 2024-05-21T15:26:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641aaaa0 con 0x556a74e126b0 2024-05-21T15:26:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6f9af740 10 client.0 _ll_register_callbacks cb 0x7f5a5400aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e33390 0x556a751e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e33390 0x556a751e17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58848/0 (socket says 172.21.15.78:58848) 2024-05-21T15:26:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 -- 172.21.15.78:0/2720050179 learned_addr learned my addr 172.21.15.78:0/2720050179 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a7524e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a6259c640 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e126b0 0x556a7524ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 -- 172.21.15.78:0/2720050179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e126b0 msgr2=0x556a7524ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e126b0 0x556a7524ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 -- 172.21.15.78:0/2720050179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 msgr2=0x556a7524e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a7524e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 -- 172.21.15.78:0/2720050179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a751e1ef0 con 0x556a74e33390 2024-05-21T15:26:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a61d9b640 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e33390 0x556a751e17e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009900 tx=0x7f5a580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75349dd5cdfbe254 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.252+0000 7f5a49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2720050179 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58035040 con 0x556a74e33390 2024-05-21T15:26:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2720050179 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f5a580574c0 con 0x556a74e33390 2024-05-21T15:26:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2720050179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a54020070 con 0x556a74e33390 2024-05-21T15:26:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2720050179 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5a58003e80 con 0x556a74e33390 2024-05-21T15:26:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 10 client.18603 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2720050179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556a75256b00 con 0x556a74e33390 2024-05-21T15:26:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2720050179 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a54020070 con 0x556a74e33390 2024-05-21T15:26:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2720050179 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5a58031b00 con 0x556a74e33390 2024-05-21T15:26:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 10 client.18603 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 10 client.18603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T15:26:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 20 client.18603 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x556a74dc73a0 0x556a75258d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 4 client.18603 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.253+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2720050179 --> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a7513f6c0 con 0x556a74dc73a0 2024-05-21T15:26:38.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.254+0000 7f5a6259c640 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x556a74dc73a0 0x556a75258d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6259c640 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x556a74dc73a0 0x556a75258d90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=426fbd8462489ee6 server_cookie=7aa72c8c80208c2b in_seq=0 out_seq=0 2024-05-21T15:26:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a49ffb640 10 client.18603 ms_handle_connect on v2:172.21.15.172:6838/3459432480 2024-05-21T15:26:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a49ffb640 1 -- 172.21.15.78:0/2720050179 <== mds.1 v2:172.21.15.172:6838/3459432480 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a7513f6c0 con 0x556a74dc73a0 2024-05-21T15:26:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a49ffb640 10 client.18603 handle_command_reply: tid=0 2024-05-21T15:26:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 client.18603 shutdown 2024-05-21T15:26:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2720050179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e33390 msgr2=0x556a751e17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e33390 0x556a751e17e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009900 tx=0x7f5a580046c0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2720050179 shutdown_connections 2024-05-21T15:26:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.172:6838/3459432480,v1:172.21.15.172:6839/3459432480] conn(0x556a74dc73a0 0x556a75258d90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a74e33390 0x556a751e17e0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a74e126b0 0x556a7524ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a49ffb640 0 client.18603 ms_handle_reset on v2:172.21.15.172:6838/3459432480 2024-05-21T15:26:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 --2- 172.21.15.78:0/2720050179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a74e0a4d0 0x556a7524e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.255+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2720050179 >> 172.21.15.78:0/2720050179 conn(0x556a74e274e0 msgr2=0x556a751dc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2720050179 shutdown_connections 2024-05-21T15:26:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a6f9af740 1 -- 172.21.15.78:0/2720050179 wait complete. 2024-05-21T15:26:38.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a6f9af740 20 client.18603 trim_cache size 0 max 16384 2024-05-21T15:26:38.269 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a4c0a2d00 msgr2=0x7f5a4c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a4c0a2d00 0x7f5a4c0a5210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c09b210 tx=0x7f5a5c06d240 comp rx=0 tx=0).stop 2024-05-21T15:26:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6410d520 msgr2=0x7f5a6419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6410d520 0x7f5a6419e000 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00d950 tx=0x7f5a5c00de10 comp rx=0 tx=0).stop 2024-05-21T15:26:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2413816360 shutdown_connections 2024-05-21T15:26:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a64108b70 0x7f5a64199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5a4c0a2d00 0x7f5a4c0a5210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a6410d520 0x7f5a6419e000 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 --2- 172.21.15.78:0/2413816360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a64101020 0x7f5a641997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2413816360 >> 172.21.15.78:0/2413816360 conn(0x7f5a640768d0 msgr2=0x7f5a641094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2413816360 shutdown_connections 2024-05-21T15:26:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.256+0000 7f5a68a2b640 1 -- 172.21.15.78:0/2413816360 wait complete. 2024-05-21T15:26:38.272 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:37 smithi078 ceph-mon[31997]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:38.272 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:37 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1441893054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:26:38.273 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-21T15:26:38.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.382+0000 7f1c7b59d640 1 -- 172.21.15.78:0/3473729404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 msgr2=0x7f1c741015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.382+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/3473729404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 0x7f1c741015b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009900 tx=0x7f1c7002a410 comp rx=0 tx=0).stop 2024-05-21T15:26:38.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.382+0000 7f1c7b59d640 1 -- 172.21.15.78:0/3473729404 shutdown_connections 2024-05-21T15:26:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.382+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/3473729404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c74101af0 0x7f1c74101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.382+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/3473729404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c7410c380 0x7f1c7410e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.382+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/3473729404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 0x7f1c741015b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.382+0000 7f1c7b59d640 1 -- 172.21.15.78:0/3473729404 >> 172.21.15.78:0/3473729404 conn(0x7f1c740fd210 msgr2=0x7f1c740ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.383+0000 7f1c7b59d640 1 -- 172.21.15.78:0/3473729404 shutdown_connections 2024-05-21T15:26:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.383+0000 7f1c7b59d640 1 -- 172.21.15.78:0/3473729404 wait complete. 2024-05-21T15:26:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.383+0000 7f1c7b59d640 1 Processor -- start 2024-05-21T15:26:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.383+0000 7f1c7b59d640 1 -- start start 2024-05-21T15:26:38.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c7b59d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c74101af0 0x7f1c7419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c7b59d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 0x7f1c7419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c7b59d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7410c380 0x7f1c7419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c7b59d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74111d30 con 0x7f1c74107a20 2024-05-21T15:26:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c7b59d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74111bb0 con 0x7f1c7410c380 2024-05-21T15:26:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c7b59d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74111eb0 con 0x7f1c74101af0 2024-05-21T15:26:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 0x7f1c7419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 0x7f1c7419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58872/0 (socket says 172.21.15.78:58872) 2024-05-21T15:26:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 -- 172.21.15.78:0/2707901242 learned_addr learned my addr 172.21.15.78:0/2707901242 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c79339640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c74101af0 0x7f1c7419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7410c380 0x7f1c7419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 -- 172.21.15.78:0/2707901242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c74101af0 msgr2=0x7f1c7419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c74101af0 0x7f1c7419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 -- 172.21.15.78:0/2707901242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7410c380 msgr2=0x7f1c7419b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7410c380 0x7f1c7419b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 -- 172.21.15.78:0/2707901242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c741a5880 con 0x7f1c74107a20 2024-05-21T15:26:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c78b38640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 0x7f1c7419ab10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c970 tx=0x7f1c6400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c6a7fc640 1 -- 172.21.15.78:0/2707901242 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64007c70 con 0x7f1c74107a20 2024-05-21T15:26:38.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c6a7fc640 1 -- 172.21.15.78:0/2707901242 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c64007e10 con 0x7f1c74107a20 2024-05-21T15:26:38.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c6a7fc640 1 -- 172.21.15.78:0/2707901242 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640188c0 con 0x7f1c74107a20 2024-05-21T15:26:38.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c7b59d640 1 -- 172.21.15.78:0/2707901242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c741a5b70 con 0x7f1c74107a20 2024-05-21T15:26:38.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.384+0000 7f1c7b59d640 1 -- 172.21.15.78:0/2707901242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c741a6110 con 0x7f1c74107a20 2024-05-21T15:26:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c6a7fc640 1 -- 172.21.15.78:0/2707901242 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1c6400f450 con 0x7f1c74107a20 2024-05-21T15:26:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c82446740 1 Processor -- start 2024-05-21T15:26:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c82446740 1 -- start start 2024-05-21T15:26:38.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c82446740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a0d1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c82446740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 0x56423a0d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c82446740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c82446740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56423a037c40 con 0x564239d126b0 2024-05-21T15:26:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c82446740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564239fdf580 con 0x564239d0a4d0 2024-05-21T15:26:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c82446740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56423a033c00 con 0x564239d33390 2024-05-21T15:26:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.387+0000 7f1c6a7fc640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1c600a2d80 0x7f1c600a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58876/0 (socket says 172.21.15.78:58876) 2024-05-21T15:26:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 -- 172.21.15.78:0/3672657279 learned_addr learned my addr 172.21.15.78:0/3672657279 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1c600a2d80 0x7f1c600a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a0d1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c79339640 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 0x56423a0d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c6a7fc640 1 -- 172.21.15.78:0/2707901242 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f1c640cf5b0 con 0x7f1c74107a20 2024-05-21T15:26:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 -- 172.21.15.78:0/3672657279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 msgr2=0x56423a0d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a0d1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 -- 172.21.15.78:0/3672657279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 msgr2=0x56423a0d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 0x56423a0d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 -- 172.21.15.78:0/3672657279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564239d4b3f0 con 0x564239d126b0 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1c600a2d80 0x7f1c600a5290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009270 tx=0x7f1c6c009db0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c78b38640 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0d27a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f1c64007590 tx=0x7f1c64020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768cee6662bd1efa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c5f7fe640 1 -- 172.21.15.78:0/3672657279 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6409c140 con 0x564239d126b0 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c5f7fe640 1 -- 172.21.15.78:0/3672657279 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c640a46b0 con 0x564239d126b0 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.388+0000 7f1c5f7fe640 1 -- 172.21.15.78:0/3672657279 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400f740 con 0x564239d126b0 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 -- 172.21.15.78:0/3672657279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 msgr2=0x56423a0d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0d27a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f1c64007590 tx=0x7f1c64020000 comp rx=0 tx=0).stop 2024-05-21T15:26:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 -- 172.21.15.78:0/3672657279 shutdown_connections 2024-05-21T15:26:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 0x56423a0d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0d27a0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 --2- 172.21.15.78:0/3672657279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a0d1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 -- 172.21.15.78:0/3672657279 >> 172.21.15.78:0/3672657279 conn(0x564239d274e0 msgr2=0x56423a0ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 -- 172.21.15.78:0/3672657279 shutdown_connections 2024-05-21T15:26:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.389+0000 7f1c82446740 1 -- 172.21.15.78:0/3672657279 wait complete. 2024-05-21T15:26:38.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 1 -- start start 2024-05-21T15:26:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 1 Processor -- start 2024-05-21T15:26:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 0x56423a0dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d126b0 0x56423a0dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d33390 0x56423a0e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56423a0defb0 con 0x564239d33390 2024-05-21T15:26:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564239ff2c40 con 0x564239d126b0 2024-05-21T15:26:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56423a0df130 con 0x564239d0a4d0 2024-05-21T15:26:38.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c82446740 10 client.0 _ll_register_callbacks cb 0x564239d4abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c79b3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d33390 0x56423a0e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c79b3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d33390 0x56423a0e24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58882/0 (socket says 172.21.15.78:58882) 2024-05-21T15:26:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c79b3a640 1 -- 172.21.15.78:0/2789732651 learned_addr learned my addr 172.21.15.78:0/2789732651 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.390+0000 7f1c78b38640 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d126b0 0x56423a0dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c79339640 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 0x56423a0dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c79b3a640 1 -- 172.21.15.78:0/2789732651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 msgr2=0x56423a0dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 0x56423a0dc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c79b3a640 1 -- 172.21.15.78:0/2789732651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d126b0 msgr2=0x56423a0dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d126b0 0x56423a0dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c79b3a640 1 -- 172.21.15.78:0/2789732651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564239e157d0 con 0x564239d33390 2024-05-21T15:26:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d33390 0x56423a0e24a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c010d80 tx=0x7f1c6c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f879e8e8394ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2789732651 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c024070 con 0x564239d33390 2024-05-21T15:26:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2789732651 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f1c6c0045c0 con 0x564239d33390 2024-05-21T15:26:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c82446740 1 -- 172.21.15.78:0/2789732651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c640a46b0 con 0x564239d33390 2024-05-21T15:26:38.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2789732651 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1c6c0047c0 con 0x564239d33390 2024-05-21T15:26:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c82446740 10 client.18615 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c82446740 1 -- 172.21.15.78:0/2789732651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56423a0eb7b0 con 0x564239d33390 2024-05-21T15:26:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2789732651 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c640a46b0 con 0x564239d33390 2024-05-21T15:26:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2789732651 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1c6c01f050 con 0x564239d33390 2024-05-21T15:26:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c82446740 10 client.18615 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.391+0000 7f1c82446740 10 client.18615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.392+0000 7f1c82446740 20 client.18615 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:38.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.392+0000 7f1c82446740 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564239cf1f60 0x56423a0eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.392+0000 7f1c82446740 4 client.18615 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.392+0000 7f1c82446740 1 -- 172.21.15.78:0/2789732651 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56423a045370 con 0x564239cf1f60 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.392+0000 7f1c79339640 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564239cf1f60 0x56423a0eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.392+0000 7f1c5cff9640 10 client.18615 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.392+0000 7f1c79339640 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564239cf1f60 0x56423a0eed30 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c638343f8efef4ab server_cookie=b06155ae4afdd20e in_seq=0 out_seq=0 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2789732651 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56423a045370 con 0x564239cf1f60 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c5cff9640 10 client.18615 handle_command_reply: tid=0 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 client.18615 shutdown 2024-05-21T15:26:38.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 -- 172.21.15.78:0/2789732651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d33390 msgr2=0x56423a0e24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d33390 0x56423a0e24a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c010d80 tx=0x7f1c6c009500 comp rx=0 tx=0).stop 2024-05-21T15:26:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 -- 172.21.15.78:0/2789732651 shutdown_connections 2024-05-21T15:26:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564239cf1f60 0x56423a0eed30 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d126b0 0x56423a0dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c5cff9640 0 client.18615 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d33390 0x56423a0e24a0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 --2- 172.21.15.78:0/2789732651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 0x56423a0dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 -- 172.21.15.78:0/2789732651 >> 172.21.15.78:0/2789732651 conn(0x564239d274e0 msgr2=0x56423a0d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 -- 172.21.15.78:0/2789732651 shutdown_connections 2024-05-21T15:26:38.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.393+0000 7f1c82446740 1 -- 172.21.15.78:0/2789732651 wait complete. 2024-05-21T15:26:38.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.394+0000 7f1c82446740 20 client.18615 trim_cache size 0 max 16384 2024-05-21T15:26:38.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:37 smithi084 ceph-mon[36334]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:38.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:37 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1441893054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:26:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c82446740 1 Processor -- start 2024-05-21T15:26:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c82446740 1 -- start start 2024-05-21T15:26:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c82446740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 0x56423a0e2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c82446740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0db1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c82446740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d33390 0x56423a0d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c82446740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741aed10 con 0x564239d126b0 2024-05-21T15:26:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c82446740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741b0850 con 0x564239d33390 2024-05-21T15:26:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c82446740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741af3e0 con 0x564239d0a4d0 2024-05-21T15:26:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c78b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0db1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c78b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0db1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58894/0 (socket says 172.21.15.78:58894) 2024-05-21T15:26:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.407+0000 7f1c78b38640 1 -- 172.21.15.78:0/135779502 learned_addr learned my addr 172.21.15.78:0/135779502 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c79339640 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 0x56423a0e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d33390 0x56423a0d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c78b38640 1 -- 172.21.15.78:0/135779502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 msgr2=0x56423a0e2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c78b38640 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 0x56423a0e2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c78b38640 1 -- 172.21.15.78:0/135779502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d33390 msgr2=0x56423a0d59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c78b38640 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d33390 0x56423a0d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c78b38640 1 -- 172.21.15.78:0/135779502 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423a0d32b0 con 0x564239d126b0 2024-05-21T15:26:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c78b38640 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0db1a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f1c64097ea0 tx=0x7f1c640ccfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9e0ebd5b3cf4fd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c5effd640 1 -- 172.21.15.78:0/135779502 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b4020 con 0x564239d126b0 2024-05-21T15:26:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c5effd640 1 -- 172.21.15.78:0/135779502 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c640a46b0 con 0x564239d126b0 2024-05-21T15:26:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.408+0000 7f1c5effd640 1 -- 172.21.15.78:0/135779502 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640980b0 con 0x564239d126b0 2024-05-21T15:26:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 -- 172.21.15.78:0/135779502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 msgr2=0x56423a0db1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0db1a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f1c64097ea0 tx=0x7f1c640ccfb0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 -- 172.21.15.78:0/135779502 shutdown_connections 2024-05-21T15:26:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d33390 0x56423a0d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0db1a0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 --2- 172.21.15.78:0/135779502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d0a4d0 0x56423a0e2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 -- 172.21.15.78:0/135779502 >> 172.21.15.78:0/135779502 conn(0x564239d274e0 msgr2=0x56423a0daa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 -- 172.21.15.78:0/135779502 shutdown_connections 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 -- 172.21.15.78:0/135779502 wait complete. 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 -- start start 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.409+0000 7f1c82446740 1 Processor -- start 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c82446740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 0x56423a0e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c82446740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c82446740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c82446740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741b0180 con 0x564239d126b0 2024-05-21T15:26:38.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c82446740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741afab0 con 0x564239d0a4d0 2024-05-21T15:26:38.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c82446740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c741acd10 con 0x564239d33390 2024-05-21T15:26:38.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c82446740 10 client.0 _ll_register_callbacks cb 0x7f1c6c0125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T15:26:38.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0e3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58902/0 (socket says 172.21.15.78:58902) 2024-05-21T15:26:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 -- 172.21.15.78:0/2194967372 learned_addr learned my addr 172.21.15.78:0/2194967372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c79339640 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 0x56423a0e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 -- 172.21.15.78:0/2194967372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 msgr2=0x56423a1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a1550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 -- 172.21.15.78:0/2194967372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 msgr2=0x56423a0e38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T15:26:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 0x56423a0e38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 -- 172.21.15.78:0/2194967372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423a0e4690 con 0x564239d126b0 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c79b3a640 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c78b38640 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0e3e10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f1c640cd040 tx=0x7f1c6400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2f83cfd630dedf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2194967372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640b4070 con 0x564239d126b0 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2194967372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f1c640a4430 con 0x564239d126b0 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.410+0000 7f1c82446740 1 -- 172.21.15.78:0/2194967372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c640a46b0 con 0x564239d126b0 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2194967372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1c6400fb50 con 0x564239d126b0 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c82446740 10 client.18621 fetch_fsmap learned FSMap version 37 2024-05-21T15:26:38.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c82446740 1 -- 172.21.15.78:0/2194967372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56423a156660 con 0x564239d126b0 2024-05-21T15:26:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2194967372 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c640a46b0 con 0x564239d126b0 2024-05-21T15:26:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2194967372 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1c640cc030 con 0x564239d126b0 2024-05-21T15:26:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c82446740 10 client.18621 fetch_fsmap finished waiting for FSMap version 37 2024-05-21T15:26:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c82446740 10 client.18621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T15:26:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c82446740 20 client.18621 populate_metadata read hostname 'smithi078' 2024-05-21T15:26:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c82446740 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564239cc73a0 0x56423a1588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c82446740 4 client.18621 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T15:26:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c82446740 1 -- 172.21.15.78:0/2194967372 --> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56423a03f6c0 con 0x564239cc73a0 2024-05-21T15:26:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.411+0000 7f1c79339640 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564239cc73a0 0x56423a1588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c5cff9640 10 client.18621 ms_handle_connect on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c79339640 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564239cc73a0 0x56423a1588f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6d135c3e735fce server_cookie=54d2d5cae85a0b49 in_seq=0 out_seq=0 2024-05-21T15:26:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c5cff9640 1 -- 172.21.15.78:0/2194967372 <== mds.0 v2:172.21.15.84:6838/652490614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56423a03f6c0 con 0x564239cc73a0 2024-05-21T15:26:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c5cff9640 10 client.18621 handle_command_reply: tid=0 2024-05-21T15:26:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 client.18621 shutdown 2024-05-21T15:26:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 -- 172.21.15.78:0/2194967372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 msgr2=0x56423a0e3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0e3e10 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f1c640cd040 tx=0x7f1c6400c8e0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 -- 172.21.15.78:0/2194967372 shutdown_connections 2024-05-21T15:26:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.84:6838/652490614,v1:172.21.15.84:6839/652490614] conn(0x564239cc73a0 0x56423a1588f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564239d33390 0x56423a1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c5cff9640 0 client.18621 ms_handle_reset on v2:172.21.15.84:6838/652490614 2024-05-21T15:26:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564239d126b0 0x56423a0e3e10 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 --2- 172.21.15.78:0/2194967372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564239d0a4d0 0x56423a0e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 -- 172.21.15.78:0/2194967372 >> 172.21.15.78:0/2194967372 conn(0x564239d274e0 msgr2=0x56423a0d4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 -- 172.21.15.78:0/2194967372 shutdown_connections 2024-05-21T15:26:38.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 1 -- 172.21.15.78:0/2194967372 wait complete. 2024-05-21T15:26:38.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.412+0000 7f1c82446740 20 client.18621 trim_cache size 0 max 16384 2024-05-21T15:26:38.427 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-05-21T15:26:38.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 -- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1c600a2d80 msgr2=0x7f1c600a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1c600a2d80 0x7f1c600a5290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009270 tx=0x7f1c6c009db0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 -- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 msgr2=0x7f1c7419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:38.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 0x7f1c7419ab10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c970 tx=0x7f1c6400ce30 comp rx=0 tx=0).stop 2024-05-21T15:26:38.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 -- 172.21.15.78:0/2707901242 shutdown_connections 2024-05-21T15:26:38.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1c7410c380 0x7f1c7419b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f1c600a2d80 0x7f1c600a5290 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c74107a20 0x7f1c7419ab10 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 --2- 172.21.15.78:0/2707901242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c74101af0 0x7f1c7419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:38.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 -- 172.21.15.78:0/2707901242 >> 172.21.15.78:0/2707901242 conn(0x7f1c740fd210 msgr2=0x7f1c740fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:38.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 -- 172.21.15.78:0/2707901242 shutdown_connections 2024-05-21T15:26:38.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:38.413+0000 7f1c7b59d640 1 -- 172.21.15.78:0/2707901242 wait complete. 2024-05-21T15:26:38.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:37 smithi172 ceph-mon[36402]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:38.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:37 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1441893054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:26:39.430 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-21T15:26:39.431 INFO:tasks.fwd_scrub:done joining 2024-05-21T15:26:39.431 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-21T15:26:39.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:26:39.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 -- 172.21.15.78:0/732413640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 msgr2=0x7fc2f0100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:39.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/732413640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 0x7fc2f0100b80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009ca0 tx=0x7fc2e002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 -- 172.21.15.78:0/732413640 shutdown_connections 2024-05-21T15:26:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/732413640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f0108f70 0x7fc2f01010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/732413640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2f0101770 0x7fc2f010f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/732413640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 0x7fc2f0100b80 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 -- 172.21.15.78:0/732413640 >> 172.21.15.78:0/732413640 conn(0x7fc2f0076840 msgr2=0x7fc2f0076c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 -- 172.21.15.78:0/732413640 shutdown_connections 2024-05-21T15:26:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 -- 172.21.15.78:0/732413640 wait complete. 2024-05-21T15:26:39.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.564+0000 7fc2f5bff640 1 Processor -- start 2024-05-21T15:26:39.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2f5bff640 1 -- start start 2024-05-21T15:26:39.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2f5bff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 0x7fc2f0198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:39.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2f5bff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2f01994d0 0x7fc2f019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:39.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2f5bff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f019dcf0 0x7fc2f019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:39.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2f5bff640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0112e10 con 0x7fc2f0108ad0 2024-05-21T15:26:39.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2f5bff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0112c90 con 0x7fc2f019dcf0 2024-05-21T15:26:39.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2f5bff640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0112f90 con 0x7fc2f01994d0 2024-05-21T15:26:39.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 0x7fc2f0198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:39.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 0x7fc2f0198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38676/0 (socket says 172.21.15.78:38676) 2024-05-21T15:26:39.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 -- 172.21.15.78:0/1881590509 learned_addr learned my addr 172.21.15.78:0/1881590509 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:39.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2eeffd640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2f01994d0 0x7fc2f019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:39.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 -- 172.21.15.78:0/1881590509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2f01994d0 msgr2=0x7fc2f019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:39.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2f01994d0 0x7fc2f019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2effff640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f019dcf0 0x7fc2f019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:39.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 -- 172.21.15.78:0/1881590509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f019dcf0 msgr2=0x7fc2f019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:39.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f019dcf0 0x7fc2f019e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 -- 172.21.15.78:0/1881590509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f0199b80 con 0x7fc2f0108ad0 2024-05-21T15:26:39.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2eeffd640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2f01994d0 0x7fc2f019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:39.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.565+0000 7fc2ef7fe640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 0x7fc2f0198f90 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000f80 tx=0x7fc2e00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:39.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.566+0000 7fc2ecff9640 1 -- 172.21.15.78:0/1881590509 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e002fa20 con 0x7fc2f0108ad0 2024-05-21T15:26:39.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.566+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f01a5b20 con 0x7fc2f0108ad0 2024-05-21T15:26:39.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.566+0000 7fc2ecff9640 1 -- 172.21.15.78:0/1881590509 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e002fbc0 con 0x7fc2f0108ad0 2024-05-21T15:26:39.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.566+0000 7fc2ecff9640 1 -- 172.21.15.78:0/1881590509 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0046800 con 0x7fc2f0108ad0 2024-05-21T15:26:39.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.566+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01a60c0 con 0x7fc2f0108ad0 2024-05-21T15:26:39.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.567+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f0069140 con 0x7fc2f0108ad0 2024-05-21T15:26:39.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.568+0000 7fc2ecff9640 1 -- 172.21.15.78:0/1881590509 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc2e0004b40 con 0x7fc2f0108ad0 2024-05-21T15:26:39.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.569+0000 7fc2ecff9640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2d80a2d00 0x7fc2d80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:39.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.569+0000 7fc2eeffd640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2d80a2d00 0x7fc2d80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.569+0000 7fc2ecff9640 1 -- 172.21.15.78:0/1881590509 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7fc2e00f0a50 con 0x7fc2f0108ad0 2024-05-21T15:26:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.569+0000 7fc2eeffd640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2d80a2d00 0x7fc2d80a5210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4005e10 tx=0x7fc2e4005da0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.572+0000 7fc2ecff9640 1 -- 172.21.15.78:0/1881590509 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc2e00bd780 con 0x7fc2f0108ad0 2024-05-21T15:26:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.751+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2f002d2a0 con 0x7fc2f0108ad0 2024-05-21T15:26:39.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.753+0000 7fc2ecff9640 1 -- 172.21.15.78:0/1881590509 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23224 (secure 0 0 0) 0x7fc2e00bd080 con 0x7fc2f0108ad0 2024-05-21T15:26:39.756 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:26:39.756 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":98,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:26:16.428629+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6801/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2012910515":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2194549734":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/3526992601":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2604499013":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:26:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2d80a2d00 msgr2=0x7fc2d80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2d80a2d00 0x7fc2d80a5210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4005e10 tx=0x7fc2e4005da0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 msgr2=0x7fc2f0198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 0x7fc2f0198f90 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0000f80 tx=0x7fc2e00319f0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 shutdown_connections 2024-05-21T15:26:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2f019dcf0 0x7fc2f019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc2d80a2d00 0x7fc2d80a5210 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2f01994d0 0x7fc2f019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 --2- 172.21.15.78:0/1881590509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2f0108ad0 0x7fc2f0198f90 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 >> 172.21.15.78:0/1881590509 conn(0x7fc2f0076840 msgr2=0x7fc2f010ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 shutdown_connections 2024-05-21T15:26:39.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.755+0000 7fc2f5bff640 1 -- 172.21.15.78:0/1881590509 wait complete. 2024-05-21T15:26:39.770 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:26:39.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:26:39.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/4068728012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108b50 msgr2=0x7f4c08100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/4068728012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108b50 0x7f4c08100be0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009ca0 tx=0x7f4bf802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/4068728012 shutdown_connections 2024-05-21T15:26:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/4068728012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c08108ff0 0x7f4c08101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/4068728012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c08101790 0x7f4c0810f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/4068728012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108b50 0x7f4c08100be0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/4068728012 >> 172.21.15.78:0/4068728012 conn(0x7f4c08076830 msgr2=0x7f4c08076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/4068728012 shutdown_connections 2024-05-21T15:26:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.881+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/4068728012 wait complete. 2024-05-21T15:26:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0eeb8640 1 Processor -- start 2024-05-21T15:26:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0eeb8640 1 -- start start 2024-05-21T15:26:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0eeb8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c08101790 0x7f4c0819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0eeb8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c08108b50 0x7f4c0819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0eeb8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108ff0 0x7f4c081996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:39.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0eeb8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08112e90 con 0x7f4c08108ff0 2024-05-21T15:26:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0eeb8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08112d10 con 0x7f4c08108b50 2024-05-21T15:26:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0eeb8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08113010 con 0x7f4c08101790 2024-05-21T15:26:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108ff0 0x7f4c081996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108ff0 0x7f4c081996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38682/0 (socket says 172.21.15.78:38682) 2024-05-21T15:26:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 -- 172.21.15.78:0/733452537 learned_addr learned my addr 172.21.15.78:0/733452537 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0cc54640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c08101790 0x7f4c0819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c07fff640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c08108b50 0x7f4c0819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:39.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 -- 172.21.15.78:0/733452537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c08101790 msgr2=0x7f4c0819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c08101790 0x7f4c0819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 -- 172.21.15.78:0/733452537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c08108b50 msgr2=0x7f4c0819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c08108b50 0x7f4c0819fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 -- 172.21.15.78:0/733452537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c08199de0 con 0x7f4c08108ff0 2024-05-21T15:26:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.882+0000 7f4c0d455640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108ff0 0x7f4c081996b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000d750 tx=0x7f4c0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.883+0000 7f4c05ffb640 1 -- 172.21.15.78:0/733452537 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00017070 con 0x7f4c08108ff0 2024-05-21T15:26:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.883+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0819a070 con 0x7f4c08108ff0 2024-05-21T15:26:39.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.883+0000 7f4c05ffb640 1 -- 172.21.15.78:0/733452537 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c00005500 con 0x7f4c08108ff0 2024-05-21T15:26:39.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.883+0000 7f4c05ffb640 1 -- 172.21.15.78:0/733452537 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00004d90 con 0x7f4c08108ff0 2024-05-21T15:26:39.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.883+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081a62d0 con 0x7f4c08108ff0 2024-05-21T15:26:39.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.884+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c08103f00 con 0x7f4c08108ff0 2024-05-21T15:26:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.885+0000 7f4c05ffb640 1 -- 172.21.15.78:0/733452537 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4c00004f30 con 0x7f4c08108ff0 2024-05-21T15:26:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.886+0000 7f4c05ffb640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4bf40a2d00 0x7f4bf40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.886+0000 7f4c05ffb640 1 -- 172.21.15.78:0/733452537 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f4c000cf1d0 con 0x7f4c08108ff0 2024-05-21T15:26:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.886+0000 7f4c0cc54640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4bf40a2d00 0x7f4bf40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:39.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.886+0000 7f4c0cc54640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4bf40a2d00 0x7f4bf40a5210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8005e30 tx=0x7f4bf8005fd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:39.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:39.890+0000 7f4c05ffb640 1 -- 172.21.15.78:0/733452537 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c0009b710 con 0x7f4c08108ff0 2024-05-21T15:26:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.066+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c0810c380 con 0x7f4c08108ff0 2024-05-21T15:26:40.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.066+0000 7f4c05ffb640 1 -- 172.21.15.78:0/733452537 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f4c0009fa10 con 0x7f4c08108ff0 2024-05-21T15:26:40.070 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:26:40.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4bf40a2d00 msgr2=0x7f4bf40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4bf40a2d00 0x7f4bf40a5210 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8005e30 tx=0x7f4bf8005fd0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108ff0 msgr2=0x7f4c081996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108ff0 0x7f4c081996b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000d750 tx=0x7f4c0000dc10 comp rx=0 tx=0).stop 2024-05-21T15:26:40.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 shutdown_connections 2024-05-21T15:26:40.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c08108b50 0x7f4c0819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f4bf40a2d00 0x7f4bf40a5210 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c08108ff0 0x7f4c081996b0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 --2- 172.21.15.78:0/733452537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c08101790 0x7f4c0819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 >> 172.21.15.78:0/733452537 conn(0x7f4c08076830 msgr2=0x7f4c08109550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:40.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 shutdown_connections 2024-05-21T15:26:40.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.068+0000 7f4c0eeb8640 1 -- 172.21.15.78:0/733452537 wait complete. 2024-05-21T15:26:40.082 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:39 smithi078 ceph-mon[31997]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:40.082 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:39 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1881590509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:26:40.083 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:26:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 -- 172.21.15.78:0/677226176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108aa0 msgr2=0x7fa480100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 --2- 172.21.15.78:0/677226176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108aa0 0x7fa480100b20 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa478009ca0 tx=0x7fa47802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 -- 172.21.15.78:0/677226176 shutdown_connections 2024-05-21T15:26:40.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 --2- 172.21.15.78:0/677226176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa480108f40 0x7fa480101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 --2- 172.21.15.78:0/677226176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4801016d0 0x7fa48010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 --2- 172.21.15.78:0/677226176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108aa0 0x7fa480100b20 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 -- 172.21.15.78:0/677226176 >> 172.21.15.78:0/677226176 conn(0x7fa480076830 msgr2=0x7fa480076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 -- 172.21.15.78:0/677226176 shutdown_connections 2024-05-21T15:26:40.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.193+0000 7fa484d77640 1 -- 172.21.15.78:0/677226176 wait complete. 2024-05-21T15:26:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.194+0000 7fa484d77640 1 Processor -- start 2024-05-21T15:26:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.194+0000 7fa484d77640 1 -- start start 2024-05-21T15:26:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.194+0000 7fa484d77640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4801016d0 0x7fa4801997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa484d77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa480108aa0 0x7fa480199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa484d77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108f40 0x7fa48019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa484d77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa480112de0 con 0x7fa480108f40 2024-05-21T15:26:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa484d77640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa480112c60 con 0x7fa4801016d0 2024-05-21T15:26:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa484d77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa480112f60 con 0x7fa480108aa0 2024-05-21T15:26:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108f40 0x7fa48019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108f40 0x7fa48019e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38704/0 (socket says 172.21.15.78:38704) 2024-05-21T15:26:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 -- 172.21.15.78:0/3015452812 learned_addr learned my addr 172.21.15.78:0/3015452812 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47e59c640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4801016d0 0x7fa4801997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47dd9b640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa480108aa0 0x7fa480199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 -- 172.21.15.78:0/3015452812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa480108aa0 msgr2=0x7fa480199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa480108aa0 0x7fa480199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 -- 172.21.15.78:0/3015452812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4801016d0 msgr2=0x7fa4801997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4801016d0 0x7fa4801997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 -- 172.21.15.78:0/3015452812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48019e640 con 0x7fa480108f40 2024-05-21T15:26:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa47ed9d640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108f40 0x7fa48019e000 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa47000d950 tx=0x7fa47000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:40.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa4777fe640 1 -- 172.21.15.78:0/3015452812 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470017070 con 0x7fa480108f40 2024-05-21T15:26:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48019e930 con 0x7fa480108f40 2024-05-21T15:26:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa4777fe640 1 -- 172.21.15.78:0/3015452812 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa470009c30 con 0x7fa480108f40 2024-05-21T15:26:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa4777fe640 1 -- 172.21.15.78:0/3015452812 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470003d60 con 0x7fa480108f40 2024-05-21T15:26:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.195+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4801a62b0 con 0x7fa480108f40 2024-05-21T15:26:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.196+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa480103e40 con 0x7fa480108f40 2024-05-21T15:26:40.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.197+0000 7fa4777fe640 1 -- 172.21.15.78:0/3015452812 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa470003f00 con 0x7fa480108f40 2024-05-21T15:26:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.198+0000 7fa4777fe640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa46c0a2d00 0x7fa46c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.198+0000 7fa47e59c640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa46c0a2d00 0x7fa46c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.198+0000 7fa4777fe640 1 -- 172.21.15.78:0/3015452812 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7fa4700ceeb0 con 0x7fa480108f40 2024-05-21T15:26:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.199+0000 7fa47e59c640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa46c0a2d00 0x7fa46c0a5210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa478004af0 tx=0x7fa478000f40 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.202+0000 7fa4777fe640 1 -- 172.21.15.78:0/3015452812 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa4700fd850 con 0x7fa480108f40 2024-05-21T15:26:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.376+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa480106fd0 con 0x7fa480108f40 2024-05-21T15:26:40.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.377+0000 7fa4777fe640 1 -- 172.21.15.78:0/3015452812 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fa47009ba40 con 0x7fa480108f40 2024-05-21T15:26:40.380 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:26:40.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa46c0a2d00 msgr2=0x7fa46c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa46c0a2d00 0x7fa46c0a5210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa478004af0 tx=0x7fa478000f40 comp rx=0 tx=0).stop 2024-05-21T15:26:40.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108f40 msgr2=0x7fa48019e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108f40 0x7fa48019e000 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa47000d950 tx=0x7fa47000de10 comp rx=0 tx=0).stop 2024-05-21T15:26:40.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 shutdown_connections 2024-05-21T15:26:40.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fa46c0a2d00 0x7fa46c0a5210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa480108f40 0x7fa48019e000 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa480108aa0 0x7fa480199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.378+0000 7fa484d77640 1 --2- 172.21.15.78:0/3015452812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4801016d0 0x7fa4801997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.379+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 >> 172.21.15.78:0/3015452812 conn(0x7fa480076830 msgr2=0x7fa4801041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:40.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.379+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 shutdown_connections 2024-05-21T15:26:40.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.379+0000 7fa484d77640 1 -- 172.21.15.78:0/3015452812 wait complete. 2024-05-21T15:26:40.393 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:26:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:39 smithi084 ceph-mon[36334]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:40.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:39 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1881590509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:26:40.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:39 smithi172 ceph-mon[36402]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:40.471 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:39 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1881590509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:26:40.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 -- 172.21.15.78:0/899480172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601079d0 msgr2=0x7fc960101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 --2- 172.21.15.78:0/899480172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601079d0 0x7fc960101560 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc948009900 tx=0x7fc94802a410 comp rx=0 tx=0).stop 2024-05-21T15:26:40.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 -- 172.21.15.78:0/899480172 shutdown_connections 2024-05-21T15:26:40.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 --2- 172.21.15.78:0/899480172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc96010c330 0x7fc96010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 --2- 172.21.15.78:0/899480172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc960101aa0 0x7fc960101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 --2- 172.21.15.78:0/899480172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9601079d0 0x7fc960101560 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 -- 172.21.15.78:0/899480172 >> 172.21.15.78:0/899480172 conn(0x7fc9600770b0 msgr2=0x7fc9600774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:40.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 -- 172.21.15.78:0/899480172 shutdown_connections 2024-05-21T15:26:40.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.504+0000 7fc9656e9640 1 -- 172.21.15.78:0/899480172 wait complete. 2024-05-21T15:26:40.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc9656e9640 1 Processor -- start 2024-05-21T15:26:40.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc9656e9640 1 -- start start 2024-05-21T15:26:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc9656e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc960101aa0 0x7fc960198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc9656e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9601079d0 0x7fc960198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc9656e9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96010c330 0x7fc96019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc9656e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc960111ce0 con 0x7fc960101aa0 2024-05-21T15:26:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc9656e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc960111b60 con 0x7fc9601079d0 2024-05-21T15:26:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc9656e9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc960111e60 con 0x7fc96010c330 2024-05-21T15:26:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc95effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc960101aa0 0x7fc960198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc95effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc960101aa0 0x7fc960198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38712/0 (socket says 172.21.15.78:38712) 2024-05-21T15:26:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc95effd640 1 -- 172.21.15.78:0/3995275996 learned_addr learned my addr 172.21.15.78:0/3995275996 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:40.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc95e7fc640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9601079d0 0x7fc960198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.505+0000 7fc95f7fe640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96010c330 0x7fc96019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc95effd640 1 -- 172.21.15.78:0/3995275996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96010c330 msgr2=0x7fc96019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc95effd640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96010c330 0x7fc96019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc95effd640 1 -- 172.21.15.78:0/3995275996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9601079d0 msgr2=0x7fc960198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc95effd640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9601079d0 0x7fc960198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc95effd640 1 -- 172.21.15.78:0/3995275996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96019d580 con 0x7fc960101aa0 2024-05-21T15:26:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc95f7fe640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96010c330 0x7fc96019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:26:40.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc95effd640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc960101aa0 0x7fc960198720 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc948005940 tx=0x7fc9480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:40.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc957fff640 1 -- 172.21.15.78:0/3995275996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94803f030 con 0x7fc960101aa0 2024-05-21T15:26:40.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc957fff640 1 -- 172.21.15.78:0/3995275996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc948003ce0 con 0x7fc960101aa0 2024-05-21T15:26:40.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96019d810 con 0x7fc960101aa0 2024-05-21T15:26:40.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc957fff640 1 -- 172.21.15.78:0/3995275996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948035070 con 0x7fc960101aa0 2024-05-21T15:26:40.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.506+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96019dd00 con 0x7fc960101aa0 2024-05-21T15:26:40.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.507+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc960069f50 con 0x7fc960101aa0 2024-05-21T15:26:40.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.509+0000 7fc957fff640 1 -- 172.21.15.78:0/3995275996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc948042030 con 0x7fc960101aa0 2024-05-21T15:26:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.509+0000 7fc957fff640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc94c0a2d00 0x7fc94c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.509+0000 7fc95e7fc640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc94c0a2d00 0x7fc94c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.509+0000 7fc957fff640 1 -- 172.21.15.78:0/3995275996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7fc9480efd10 con 0x7fc960101aa0 2024-05-21T15:26:40.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.510+0000 7fc95e7fc640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc94c0a2d00 0x7fc94c0a5210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc96010d680 tx=0x7fc95000b390 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:40.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.513+0000 7fc957fff640 1 -- 172.21.15.78:0/3995275996 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc9480bc8a0 con 0x7fc960101aa0 2024-05-21T15:26:40.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.690+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc960105e90 con 0x7fc960101aa0 2024-05-21T15:26:40.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.691+0000 7fc957fff640 1 -- 172.21.15.78:0/3995275996 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7fc9480bc1a0 con 0x7fc960101aa0 2024-05-21T15:26:40.695 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:26:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.692+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc94c0a2d00 msgr2=0x7fc94c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.692+0000 7fc9656e9640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc94c0a2d00 0x7fc94c0a5210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc96010d680 tx=0x7fc95000b390 comp rx=0 tx=0).stop 2024-05-21T15:26:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc960101aa0 msgr2=0x7fc960198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc960101aa0 0x7fc960198720 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc948005940 tx=0x7fc9480042f0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 shutdown_connections 2024-05-21T15:26:40.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9601079d0 0x7fc960198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fc94c0a2d00 0x7fc94c0a5210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc96010c330 0x7fc96019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 --2- 172.21.15.78:0/3995275996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc960101aa0 0x7fc960198720 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 >> 172.21.15.78:0/3995275996 conn(0x7fc9600770b0 msgr2=0x7fc960103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 shutdown_connections 2024-05-21T15:26:40.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.693+0000 7fc9656e9640 1 -- 172.21.15.78:0/3995275996 wait complete. 2024-05-21T15:26:40.707 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-21T15:26:40.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 -- 172.21.15.78:0/3447768730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e810c330 msgr2=0x7f35e810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3447768730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e810c330 0x7f35e810e750 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009900 tx=0x7f35e002a410 comp rx=0 tx=0).stop 2024-05-21T15:26:40.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 -- 172.21.15.78:0/3447768730 shutdown_connections 2024-05-21T15:26:40.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3447768730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35e8101aa0 0x7f35e8101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3447768730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e810c330 0x7f35e810e750 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3447768730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e81079d0 0x7f35e8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 -- 172.21.15.78:0/3447768730 >> 172.21.15.78:0/3447768730 conn(0x7f35e80770c0 msgr2=0x7f35e8077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:40.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 -- 172.21.15.78:0/3447768730 shutdown_connections 2024-05-21T15:26:40.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.815+0000 7f35eebd5640 1 -- 172.21.15.78:0/3447768730 wait complete. 2024-05-21T15:26:40.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35eebd5640 1 Processor -- start 2024-05-21T15:26:40.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35eebd5640 1 -- start start 2024-05-21T15:26:40.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35eebd5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35e8101aa0 0x7f35e81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35eebd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81079d0 0x7f35e8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35eebd5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e810c330 0x7f35e819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35eebd5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111d30 con 0x7f35e81079d0 2024-05-21T15:26:40.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35eebd5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111bb0 con 0x7f35e8101aa0 2024-05-21T15:26:40.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35eebd5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8111eb0 con 0x7f35e810c330 2024-05-21T15:26:40.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81079d0 0x7f35e8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81079d0 0x7f35e8198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38740/0 (socket says 172.21.15.78:38740) 2024-05-21T15:26:40.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35e7fff640 1 -- 172.21.15.78:0/3284699295 learned_addr learned my addr 172.21.15.78:0/3284699295 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:26:40.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35ed172640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e810c330 0x7f35e819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35ec971640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35e8101aa0 0x7f35e81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35e7fff640 1 -- 172.21.15.78:0/3284699295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e810c330 msgr2=0x7f35e819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35e7fff640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e810c330 0x7f35e819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35e7fff640 1 -- 172.21.15.78:0/3284699295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35e8101aa0 msgr2=0x7f35e81985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:40.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35e7fff640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35e8101aa0 0x7f35e81985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:40.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.816+0000 7f35e7fff640 1 -- 172.21.15.78:0/3284699295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e819d430 con 0x7f35e81079d0 2024-05-21T15:26:40.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.817+0000 7f35e7fff640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81079d0 0x7f35e8198b30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f35d800d740 tx=0x7f35d800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:40.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.817+0000 7f35e5ffb640 1 -- 172.21.15.78:0/3284699295 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8017070 con 0x7f35e81079d0 2024-05-21T15:26:40.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.817+0000 7f35e5ffb640 1 -- 172.21.15.78:0/3284699295 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d800de50 con 0x7f35e81079d0 2024-05-21T15:26:40.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.817+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819d720 con 0x7f35e81079d0 2024-05-21T15:26:40.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.817+0000 7f35e5ffb640 1 -- 172.21.15.78:0/3284699295 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8003e00 con 0x7f35e81079d0 2024-05-21T15:26:40.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.817+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a50a0 con 0x7f35e81079d0 2024-05-21T15:26:40.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.818+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e8102c80 con 0x7f35e81079d0 2024-05-21T15:26:40.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.819+0000 7f35e5ffb640 1 -- 172.21.15.78:0/3284699295 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f35d8003fa0 con 0x7f35e81079d0 2024-05-21T15:26:40.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.820+0000 7f35e5ffb640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f35d40a2d80 0x7f35d40a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:26:40.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.820+0000 7f35e5ffb640 1 -- 172.21.15.78:0/3284699295 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8658+0+0 (secure 0 0 0) 0x7f35d80cf2d0 con 0x7f35e81079d0 2024-05-21T15:26:40.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.820+0000 7f35ec971640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f35d40a2d80 0x7f35d40a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:26:40.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.820+0000 7f35ec971640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f35d40a2d80 0x7f35d40a5290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35d0005c40 tx=0x7f35d0005bd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:26:40.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:40.823+0000 7f35e5ffb640 1 -- 172.21.15.78:0/3284699295 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35d809bee0 con 0x7f35e81079d0 2024-05-21T15:26:41.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.051+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f35e8105e90 con 0x7f35e81079d0 2024-05-21T15:26:41.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.059+0000 7f35e5ffb640 1 -- 172.21.15.78:0/3284699295 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740899 (secure 0 0 0) 0x7f35d809b7e0 con 0x7f35e81079d0 2024-05-21T15:26:41.073 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:26:41.073 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:41.074 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 25, 2024-05-21T15:26:41.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.074 INFO:teuthology.orchestra.run.smithi078.stdout: "active_gid": 15399, 2024-05-21T15:26:41.074 INFO:teuthology.orchestra.run.smithi078.stdout: "active_name": "x", 2024-05-21T15:26:41.074 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addrs": { 2024-05-21T15:26:41.074 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-05-21T15:26:41.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-05-21T15:26:41.075 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6800", 2024-05-21T15:26:41.075 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1100997097 2024-05-21T15:26:41.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-05-21T15:26:41.075 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6801", 2024-05-21T15:26:41.076 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1100997097 2024-05-21T15:26:41.076 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.076 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:41.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.076 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addr": "172.21.15.78:6801/1100997097", 2024-05-21T15:26:41.076 INFO:teuthology.orchestra.run.smithi078.stdout: "active_change": "2024-05-21T15:16:08.699221+0000", 2024-05-21T15:26:41.076 INFO:teuthology.orchestra.run.smithi078.stdout: "active_mgr_features": 4540701547738038271, 2024-05-21T15:26:41.077 INFO:teuthology.orchestra.run.smithi078.stdout: "available": true, 2024-05-21T15:26:41.077 INFO:teuthology.orchestra.run.smithi078.stdout: "standbys": [ 2024-05-21T15:26:41.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.077 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24973, 2024-05-21T15:26:41.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "z", 2024-05-21T15:26:41.077 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-05-21T15:26:41.077 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-05-21T15:26:41.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-05-21T15:26:41.078 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.078 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.078 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.078 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:41.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:41.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.079 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:41.079 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.080 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T15:26:41.080 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.080 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.080 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.080 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.080 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.081 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.081 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.081 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.081 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.082 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.082 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.082 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.082 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.082 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.082 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.082 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.083 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.083 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.083 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.083 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.083 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.084 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.084 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.084 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.084 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.084 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.085 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.085 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.085 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.086 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.086 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.086 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.086 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.086 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.086 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.087 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.087 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.087 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.087 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.087 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.087 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.087 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.088 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.088 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.088 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.088 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.089 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.089 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.090 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.090 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-05-21T15:26:41.090 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-05-21T15:26:41.090 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.090 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.091 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.091 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.091 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-05-21T15:26:41.091 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.092 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.092 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-05-21T15:26:41.092 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-05-21T15:26:41.092 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.092 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.092 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-05-21T15:26:41.093 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.093 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.093 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-05-21T15:26:41.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.094 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.094 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-05-21T15:26:41.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-05-21T15:26:41.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.094 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.094 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.095 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.095 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.095 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.095 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-05-21T15:26:41.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.096 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.096 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-05-21T15:26:41.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-05-21T15:26:41.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.097 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.097 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.097 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.097 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.097 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-05-21T15:26:41.097 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.098 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.098 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-05-21T15:26:41.098 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-05-21T15:26:41.098 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.098 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.098 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.099 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-05-21T15:26:41.099 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.099 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.099 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.099 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-05-21T15:26:41.099 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.099 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.100 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.100 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-05-21T15:26:41.100 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-05-21T15:26:41.100 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.100 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.100 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.101 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.101 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.101 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.101 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.101 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-05-21T15:26:41.101 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.102 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-05-21T15:26:41.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-05-21T15:26:41.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.102 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.103 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.103 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.103 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.103 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.103 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.103 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T15:26:41.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.104 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.104 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-05-21T15:26:41.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-05-21T15:26:41.104 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.105 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.105 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.105 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.105 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.105 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-05-21T15:26:41.105 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.106 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.106 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.106 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-05-21T15:26:41.106 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.107 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.107 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.107 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-05-21T15:26:41.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-05-21T15:26:41.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.107 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.107 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.108 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T15:26:41.108 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.108 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.109 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-05-21T15:26:41.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-05-21T15:26:41.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.109 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.109 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-05-21T15:26:41.110 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.110 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.110 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.110 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T15:26:41.110 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:26:41.110 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.110 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.111 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-05-21T15:26:41.111 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-05-21T15:26:41.111 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.111 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:41.112 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-05-21T15:26:41.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-05-21T15:26:41.112 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.112 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T15:26:41.112 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:26:41.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.113 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-05-21T15:26:41.113 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-05-21T15:26:41.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.113 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.113 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-05-21T15:26:41.113 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:41.114 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-05-21T15:26:41.114 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T15:26:41.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.115 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-05-21T15:26:41.115 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-05-21T15:26:41.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.115 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.116 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T15:26:41.116 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.116 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.116 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.116 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T15:26:41.116 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T15:26:41.116 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.117 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.117 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-05-21T15:26:41.117 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-05-21T15:26:41.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.117 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.118 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-05-21T15:26:41.118 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-05-21T15:26:41.118 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-05-21T15:26:41.118 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.118 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-05-21T15:26:41.118 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T15:26:41.118 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.119 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.119 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-05-21T15:26:41.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-05-21T15:26:41.119 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.119 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.119 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-05-21T15:26:41.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.120 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.120 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T15:26:41.120 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:26:41.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.121 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.121 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-05-21T15:26:41.121 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-05-21T15:26:41.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.121 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.121 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:41.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-05-21T15:26:41.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-05-21T15:26:41.122 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.122 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T15:26:41.122 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:26:41.122 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.122 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.123 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.123 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.123 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.123 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.123 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.123 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.124 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.124 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.124 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.124 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.124 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.124 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.125 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.125 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.125 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.125 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.126 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.126 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.126 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.126 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.128 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.129 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.129 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.129 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.129 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.129 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.129 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.129 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.130 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.130 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.130 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.130 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.130 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.130 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.131 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.131 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.131 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.131 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.131 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.132 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.132 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.132 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.132 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-05-21T15:26:41.132 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-05-21T15:26:41.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.133 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.133 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.133 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:41.133 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.133 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.133 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.134 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T15:26:41.134 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.134 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.134 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.134 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-05-21T15:26:41.134 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-05-21T15:26:41.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.135 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.135 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-05-21T15:26:41.135 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.135 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.135 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.136 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-05-21T15:26:41.136 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-05-21T15:26:41.136 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-05-21T15:26:41.136 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-05-21T15:26:41.136 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-05-21T15:26:41.136 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.136 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-05-21T15:26:41.137 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.137 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.137 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-05-21T15:26:41.137 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-05-21T15:26:41.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.138 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.138 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.138 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.138 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.138 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.138 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.138 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T15:26:41.139 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.139 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.139 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.139 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:41.139 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:41.139 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.140 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.140 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:41.140 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.140 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.140 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.140 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T15:26:41.140 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.141 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.141 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-05-21T15:26:41.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-05-21T15:26:41.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.141 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.142 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:41.142 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:41.142 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.142 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.142 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T15:26:41.143 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T15:26:41.143 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.143 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.143 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-05-21T15:26:41.143 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-05-21T15:26:41.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.144 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.144 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.144 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.144 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.144 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.144 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.144 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T15:26:41.144 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.145 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.145 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.145 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-05-21T15:26:41.146 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.146 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.146 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.146 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-05-21T15:26:41.146 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-05-21T15:26:41.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.146 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.147 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-05-21T15:26:41.147 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.147 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.147 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.147 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T15:26:41.147 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.148 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.148 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-05-21T15:26:41.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-05-21T15:26:41.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.148 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.149 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-05-21T15:26:41.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.149 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.149 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T15:26:41.149 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.150 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-05-21T15:26:41.150 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-05-21T15:26:41.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.150 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.151 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-05-21T15:26:41.151 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.151 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.151 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.151 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T15:26:41.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.152 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-05-21T15:26:41.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-05-21T15:26:41.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.153 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.153 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T15:26:41.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T15:26:41.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.154 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-05-21T15:26:41.154 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-05-21T15:26:41.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:41.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T15:26:41.156 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.156 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.156 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.156 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-05-21T15:26:41.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T15:26:41.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.157 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.157 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-05-21T15:26:41.157 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.157 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.157 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.157 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T15:26:41.158 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.158 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-05-21T15:26:41.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-05-21T15:26:41.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.159 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-05-21T15:26:41.159 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-05-21T15:26:41.160 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-05-21T15:26:41.160 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.160 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T15:26:41.160 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.160 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-05-21T15:26:41.161 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-05-21T15:26:41.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.161 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.162 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T15:26:41.162 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.162 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-05-21T15:26:41.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-05-21T15:26:41.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.163 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.163 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.163 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.163 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.163 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.164 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T15:26:41.164 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.164 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-05-21T15:26:41.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-05-21T15:26:41.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.165 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.165 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T15:26:41.166 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.166 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.166 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.166 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-05-21T15:26:41.166 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-05-21T15:26:41.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.167 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.167 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T15:26:41.167 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.167 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:41.168 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.168 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.168 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-05-21T15:26:41.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-05-21T15:26:41.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.169 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T15:26:41.169 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.169 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.169 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.169 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-05-21T15:26:41.169 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.170 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.170 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-05-21T15:26:41.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-05-21T15:26:41.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.170 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.171 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.171 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T15:26:41.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.171 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-05-21T15:26:41.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.172 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.172 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-05-21T15:26:41.172 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-05-21T15:26:41.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.172 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T15:26:41.173 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:41.173 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.173 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.174 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-05-21T15:26:41.174 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-05-21T15:26:41.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.174 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.174 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.174 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T15:26:41.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.175 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-05-21T15:26:41.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.175 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.176 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-05-21T15:26:41.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-05-21T15:26:41.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T15:26:41.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.177 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-05-21T15:26:41.177 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.178 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.178 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-05-21T15:26:41.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-05-21T15:26:41.178 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.178 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.178 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.179 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T15:26:41.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.179 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-05-21T15:26:41.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.180 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.180 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-05-21T15:26:41.180 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-05-21T15:26:41.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.180 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T15:26:41.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.181 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.181 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-05-21T15:26:41.181 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.182 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-05-21T15:26:41.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-05-21T15:26:41.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.182 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.182 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T15:26:41.183 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.183 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.183 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.183 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-05-21T15:26:41.183 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.183 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.184 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-05-21T15:26:41.184 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-05-21T15:26:41.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-21T15:26:41.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.185 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.185 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.185 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-05-21T15:26:41.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.186 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-05-21T15:26:41.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-05-21T15:26:41.186 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.186 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.186 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.186 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T15:26:41.186 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.187 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.187 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:41.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.187 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-05-21T15:26:41.188 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-05-21T15:26:41.188 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.188 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-21T15:26:41.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.189 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-05-21T15:26:41.189 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.189 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.189 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-05-21T15:26:41.189 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-05-21T15:26:41.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.190 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T15:26:41.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.191 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:41.191 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.191 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.191 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-05-21T15:26:41.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-05-21T15:26:41.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.192 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.192 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.192 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-21T15:26:41.192 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-05-21T15:26:41.193 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.193 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.193 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.193 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-05-21T15:26:41.193 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-05-21T15:26:41.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.194 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.194 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T15:26:41.194 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.194 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.194 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-05-21T15:26:41.195 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.195 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.195 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-05-21T15:26:41.195 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-05-21T15:26:41.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.196 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T15:26:41.196 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.196 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.196 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.196 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-05-21T15:26:41.196 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.197 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.197 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-05-21T15:26:41.197 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-05-21T15:26:41.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.198 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.198 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.198 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.198 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.198 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.198 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.198 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T15:26:41.199 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.199 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.199 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.199 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-05-21T15:26:41.199 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-05-21T15:26:41.199 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.200 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:41.200 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.200 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.200 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T15:26:41.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.201 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-05-21T15:26:41.201 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T15:26:41.201 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.201 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.202 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-05-21T15:26:41.202 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.202 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.202 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.202 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T15:26:41.203 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.203 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.203 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.203 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-05-21T15:26:41.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-05-21T15:26:41.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.204 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-05-21T15:26:41.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.204 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T15:26:41.204 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.205 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.205 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-05-21T15:26:41.205 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-05-21T15:26:41.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.205 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.206 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-05-21T15:26:41.206 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.206 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.206 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.206 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-05-21T15:26:41.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.207 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.207 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-05-21T15:26:41.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-05-21T15:26:41.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.207 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.208 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.208 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.208 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.208 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.208 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.208 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T15:26:41.208 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.209 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.209 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-05-21T15:26:41.209 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-05-21T15:26:41.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.209 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.210 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-05-21T15:26:41.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.211 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.211 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-05-21T15:26:41.211 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-05-21T15:26:41.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.211 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T15:26:41.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.212 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T15:26:41.212 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.213 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.213 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-05-21T15:26:41.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-05-21T15:26:41.213 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.213 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.213 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:41.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.214 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.214 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.214 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-05-21T15:26:41.214 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.214 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.215 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-05-21T15:26:41.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-05-21T15:26:41.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.215 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.216 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.216 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.216 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.216 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.216 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T15:26:41.216 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.217 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.217 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-05-21T15:26:41.217 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-05-21T15:26:41.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.217 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.218 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.218 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.218 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.218 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T15:26:41.218 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.219 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.219 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.219 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.219 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.219 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.219 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.220 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.220 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.220 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.220 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.220 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.220 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.220 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.221 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.221 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.221 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.221 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.221 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.221 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.221 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.222 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.222 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-05-21T15:26:41.222 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-05-21T15:26:41.222 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.222 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.222 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.223 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.223 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.223 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.223 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T15:26:41.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.224 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.224 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.225 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.225 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.225 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.225 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T15:26:41.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.226 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.226 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.226 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.227 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.227 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.227 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.227 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.227 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.228 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.228 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.228 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.228 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.228 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.228 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.228 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.229 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.229 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.229 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.229 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.229 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.229 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.230 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.230 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.230 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.230 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.230 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.231 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.231 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T15:26:41.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T15:26:41.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.231 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.232 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T15:26:41.232 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.233 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.233 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.233 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T15:26:41.233 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T15:26:41.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.234 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-05-21T15:26:41.234 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.234 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.234 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.234 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T15:26:41.234 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.235 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.235 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-05-21T15:26:41.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-05-21T15:26:41.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.235 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.236 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.236 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.236 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-05-21T15:26:41.236 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.237 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-05-21T15:26:41.237 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-05-21T15:26:41.237 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.237 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.237 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.238 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.238 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.238 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.238 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T15:26:41.238 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.238 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.239 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.239 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-05-21T15:26:41.239 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-05-21T15:26:41.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.239 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.240 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.240 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.240 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-05-21T15:26:41.240 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.241 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.241 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-05-21T15:26:41.241 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-05-21T15:26:41.241 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.241 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-05-21T15:26:41.242 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.242 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.242 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.242 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-05-21T15:26:41.242 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-05-21T15:26:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T15:26:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-05-21T15:26:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-05-21T15:26:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-05-21T15:26:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T15:26:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-05-21T15:26:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-05-21T15:26:41.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.246 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.246 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T15:26:41.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.246 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.246 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T15:26:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-05-21T15:26:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-05-21T15:26:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.249 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.249 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T15:26:41.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.249 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-05-21T15:26:41.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-05-21T15:26:41.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.250 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.250 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.251 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.251 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T15:26:41.251 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.251 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.252 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-05-21T15:26:41.252 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-05-21T15:26:41.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.252 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.252 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.252 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.253 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.253 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.253 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T15:26:41.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.254 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-05-21T15:26:41.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-05-21T15:26:41.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.255 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.255 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.255 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T15:26:41.255 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.256 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-05-21T15:26:41.256 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-05-21T15:26:41.256 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.256 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.256 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.256 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.257 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.257 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.257 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T15:26:41.257 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.257 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.258 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-05-21T15:26:41.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-05-21T15:26:41.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.258 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-05-21T15:26:41.258 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.259 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-05-21T15:26:41.259 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.259 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.259 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-05-21T15:26:41.260 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-05-21T15:26:41.260 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.260 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.260 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.260 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.260 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.260 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.261 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T15:26:41.261 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.261 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.261 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-05-21T15:26:41.262 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-05-21T15:26:41.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.262 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.262 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.262 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.262 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.262 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.263 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T15:26:41.263 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.263 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-05-21T15:26:41.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-05-21T15:26:41.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.265 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T15:26:41.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.265 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-05-21T15:26:41.265 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-05-21T15:26:41.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.266 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.266 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.266 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.266 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T15:26:41.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.267 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.267 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-05-21T15:26:41.267 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-05-21T15:26:41.267 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.268 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.268 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T15:26:41.268 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.269 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.269 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-05-21T15:26:41.269 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-05-21T15:26:41.269 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.269 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.269 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.270 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.270 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.270 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.270 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.270 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T15:26:41.270 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.271 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-05-21T15:26:41.271 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.272 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.272 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.272 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.272 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.273 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.273 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.273 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.273 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.273 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.273 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.273 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.274 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.274 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.274 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.274 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.275 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.275 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.275 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.275 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.275 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.275 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.275 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.276 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.276 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.276 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.276 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.277 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.278 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.278 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.278 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.278 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.278 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.278 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.279 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.279 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.279 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.280 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.280 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.280 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.280 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.280 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.281 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.281 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.281 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.281 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.281 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-05-21T15:26:41.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-05-21T15:26:41.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-05-21T15:26:41.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.282 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.282 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.283 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T15:26:41.283 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.283 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-05-21T15:26:41.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-05-21T15:26:41.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.284 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.284 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-05-21T15:26:41.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.284 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.284 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.284 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T15:26:41.285 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.285 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-05-21T15:26:41.286 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.286 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.286 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.286 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T15:26:41.286 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T15:26:41.286 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.287 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.287 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.287 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.287 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.287 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.287 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.288 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.288 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.288 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T15:26:41.288 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T15:26:41.288 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.289 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.289 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.289 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.289 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.290 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T15:26:41.290 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T15:26:41.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.290 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.291 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.291 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.292 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-05-21T15:26:41.292 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T15:26:41.292 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.292 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.294 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T15:26:41.294 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T15:26:41.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.294 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.295 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.295 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.295 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.296 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.296 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T15:26:41.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T15:26:41.296 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.296 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.297 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.297 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.297 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.297 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.297 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.297 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.297 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.298 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.298 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T15:26:41.298 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T15:26:41.298 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.298 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.298 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.299 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.299 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.299 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.299 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.299 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.299 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.300 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T15:26:41.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T15:26:41.300 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.300 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.300 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.300 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.301 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.301 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.301 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.301 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.301 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.301 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.302 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T15:26:41.302 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T15:26:41.302 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.302 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.302 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.302 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.302 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.303 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.303 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.303 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.303 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.303 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.303 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.304 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T15:26:41.304 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T15:26:41.304 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.304 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.304 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.305 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.305 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.305 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.305 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T15:26:41.306 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T15:26:41.306 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.306 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.307 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.307 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.307 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.307 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.307 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T15:26:41.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T15:26:41.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.308 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.308 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.308 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.308 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.308 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.309 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.309 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.309 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.309 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T15:26:41.309 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T15:26:41.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.310 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.310 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.310 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.310 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.311 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.311 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T15:26:41.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T15:26:41.312 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.313 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.313 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.313 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.313 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T15:26:41.313 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T15:26:41.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.314 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.314 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.314 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:41.314 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.314 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.314 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.315 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.315 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.315 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T15:26:41.315 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T15:26:41.315 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.316 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.316 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.316 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.316 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.317 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.317 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.317 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.317 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.317 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-05-21T15:26:41.317 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-05-21T15:26:41.317 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.318 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.319 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.319 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.319 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.319 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.319 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T15:26:41.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T15:26:41.320 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.320 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.320 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.320 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:41.320 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.320 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.320 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.320 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.321 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.321 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T15:26:41.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T15:26:41.321 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.321 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.322 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.322 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.323 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.323 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T15:26:41.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T15:26:41.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.324 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.324 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.324 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.325 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.325 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T15:26:41.325 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T15:26:41.325 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.325 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.326 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.326 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.326 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.326 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.327 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.327 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T15:26:41.327 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T15:26:41.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.327 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.328 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.328 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.328 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.328 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.329 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.329 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.329 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T15:26:41.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T15:26:41.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.329 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.329 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.330 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.330 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.330 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.330 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.330 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.330 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.331 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T15:26:41.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T15:26:41.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.332 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.332 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.332 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.332 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.332 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.333 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T15:26:41.333 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T15:26:41.333 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.333 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.334 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.334 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.334 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T15:26:41.335 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T15:26:41.335 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.335 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.335 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.335 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.335 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.335 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.336 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.336 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.336 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T15:26:41.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T15:26:41.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.337 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.337 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.338 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.338 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.338 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T15:26:41.338 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T15:26:41.339 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.339 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.339 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.339 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.339 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.339 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.339 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.339 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.340 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.340 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.340 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T15:26:41.340 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T15:26:41.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.341 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.341 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.341 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.341 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.341 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.341 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.342 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.342 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T15:26:41.342 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T15:26:41.342 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.342 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.343 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.343 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.343 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.343 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.344 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.344 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T15:26:41.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T15:26:41.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.345 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.345 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.345 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.346 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T15:26:41.346 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T15:26:41.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.346 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.346 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.346 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.347 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.347 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.347 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.347 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.347 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.348 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T15:26:41.348 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T15:26:41.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.348 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.348 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T15:26:41.349 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.349 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.349 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.349 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.349 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.349 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.349 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.350 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T15:26:41.350 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T15:26:41.350 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.350 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.350 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.351 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.351 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.351 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.351 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.352 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.352 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T15:26:41.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T15:26:41.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.352 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-05-21T15:26:41.353 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.353 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.353 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.354 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T15:26:41.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T15:26:41.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.354 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-05-21T15:26:41.355 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.355 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.355 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.355 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.355 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.356 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.356 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T15:26:41.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T15:26:41.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.356 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.357 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.357 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.357 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.357 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.358 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T15:26:41.358 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T15:26:41.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.358 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.358 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:41.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.360 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T15:26:41.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T15:26:41.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.360 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.360 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.361 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.361 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.361 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.361 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.361 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.361 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.362 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T15:26:41.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T15:26:41.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.363 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.363 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.363 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.363 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T15:26:41.364 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T15:26:41.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.364 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.364 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:41.364 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.364 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.364 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.365 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.365 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.365 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.365 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.365 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T15:26:41.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T15:26:41.366 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.367 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.367 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.367 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T15:26:41.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T15:26:41.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.368 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-05-21T15:26:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-05-21T15:26:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.371 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-05-21T15:26:41.371 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-05-21T15:26:41.371 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.371 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.371 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.372 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.372 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.372 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.372 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.372 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.372 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.372 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.372 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.373 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-05-21T15:26:41.373 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-05-21T15:26:41.373 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.373 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.373 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.373 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.374 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-05-21T15:26:41.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.374 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.374 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.375 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-05-21T15:26:41.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-05-21T15:26:41.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.375 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-05-21T15:26:41.375 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.375 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.376 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.376 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.376 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-05-21T15:26:41.377 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-05-21T15:26:41.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.377 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.377 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.378 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.378 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.378 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.378 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.378 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.379 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.379 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.380 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.380 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.380 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.380 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.380 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.380 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.380 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.380 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.381 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.381 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.382 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.382 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.382 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.382 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.382 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.383 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.383 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.383 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.383 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.383 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.384 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.384 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.384 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.385 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.385 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.385 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.385 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.386 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.386 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.387 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.387 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.387 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.387 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.387 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.388 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-05-21T15:26:41.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-05-21T15:26:41.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.389 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.389 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-05-21T15:26:41.389 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.390 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-05-21T15:26:41.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T15:26:41.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.390 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.390 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.391 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.391 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:41.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:41.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.392 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-05-21T15:26:41.392 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.392 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.393 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.393 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.393 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.393 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:41.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:41.394 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.394 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.394 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-05-21T15:26:41.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.394 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.394 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.395 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.395 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.395 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.395 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-05-21T15:26:41.395 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-05-21T15:26:41.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.396 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.397 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-05-21T15:26:41.397 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-05-21T15:26:41.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.398 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-05-21T15:26:41.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.398 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.398 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.398 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.399 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.399 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-05-21T15:26:41.399 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-05-21T15:26:41.400 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.400 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-05-21T15:26:41.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.400 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.400 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.400 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.401 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-05-21T15:26:41.401 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.401 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.401 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.401 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.401 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.401 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-05-21T15:26:41.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-05-21T15:26:41.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.402 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.402 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-05-21T15:26:41.402 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-05-21T15:26:41.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-05-21T15:26:41.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.403 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.403 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.403 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.403 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-05-21T15:26:41.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-05-21T15:26:41.404 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.404 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.404 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.404 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.404 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.404 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.404 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.405 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.405 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.405 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.405 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.405 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.405 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.406 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.406 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-05-21T15:26:41.406 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.406 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.406 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.406 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-05-21T15:26:41.406 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-05-21T15:26:41.407 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.407 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.407 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.407 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.407 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.407 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.407 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-05-21T15:26:41.408 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.408 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.408 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.408 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.408 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.408 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.409 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.409 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.409 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.409 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.409 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.409 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.409 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.409 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.410 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.410 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.410 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.410 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.410 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.411 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.411 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.411 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.411 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.411 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.412 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.412 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.412 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.412 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.412 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.412 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.413 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.414 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.414 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.414 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.414 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.414 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.414 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.414 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.415 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.415 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.415 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.415 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.415 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.415 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.415 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.416 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.416 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.417 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.417 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.417 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.417 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.418 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-05-21T15:26:41.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-05-21T15:26:41.418 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.418 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.418 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.418 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-05-21T15:26:41.418 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.419 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T15:26:41.419 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.420 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-05-21T15:26:41.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-05-21T15:26:41.420 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.420 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.420 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-05-21T15:26:41.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.420 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T15:26:41.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.421 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-05-21T15:26:41.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-05-21T15:26:41.422 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.422 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-05-21T15:26:41.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.423 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-05-21T15:26:41.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.423 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-05-21T15:26:41.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-05-21T15:26:41.424 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.424 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.424 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-05-21T15:26:41.424 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.424 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.425 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.425 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T15:26:41.425 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.425 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.425 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-05-21T15:26:41.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-05-21T15:26:41.426 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.426 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.426 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.426 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.426 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.427 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.427 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T15:26:41.427 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.427 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.427 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.427 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:41.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:41.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.428 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.428 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.428 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:41.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.428 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T15:26:41.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.429 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.429 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-05-21T15:26:41.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-05-21T15:26:41.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.430 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-05-21T15:26:41.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.430 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T15:26:41.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.431 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-05-21T15:26:41.432 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.432 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.432 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.432 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.432 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.433 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.433 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.433 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.433 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.433 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.434 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.434 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.434 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.434 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.435 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.435 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.435 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.435 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.436 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.436 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.436 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.436 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.436 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.437 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.437 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.437 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.437 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.438 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.438 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.438 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.438 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.438 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.439 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.439 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.439 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.439 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.439 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.439 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.440 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.440 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.440 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.440 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.440 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.441 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.441 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.442 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-05-21T15:26:41.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-05-21T15:26:41.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.442 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-05-21T15:26:41.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.443 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.443 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.443 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.443 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.443 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.444 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-05-21T15:26:41.444 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-05-21T15:26:41.444 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.444 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.444 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.444 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-05-21T15:26:41.444 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.445 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.445 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.445 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.445 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.445 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:41.446 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:41.446 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.446 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.446 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.446 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:41.446 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.447 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.447 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.447 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.447 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.447 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.447 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-05-21T15:26:41.448 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-05-21T15:26:41.448 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-05-21T15:26:41.448 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.448 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-05-21T15:26:41.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-05-21T15:26:41.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-05-21T15:26:41.449 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.449 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.450 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.450 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T15:26:41.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.450 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.450 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-05-21T15:26:41.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-05-21T15:26:41.451 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.451 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.451 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.451 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-05-21T15:26:41.451 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.452 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.452 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T15:26:41.452 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.452 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.452 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-05-21T15:26:41.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-05-21T15:26:41.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.453 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.453 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.453 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.453 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.454 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.454 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-05-21T15:26:41.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.454 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-05-21T15:26:41.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-05-21T15:26:41.456 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.456 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T15:26:41.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.457 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.458 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.458 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.458 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.458 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.458 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.458 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.458 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.459 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.459 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.459 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.460 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.460 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.460 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.460 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.461 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.461 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.461 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.461 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.461 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.462 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.462 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.463 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.463 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.463 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.463 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.463 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.463 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.463 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.464 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.464 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.465 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.466 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.466 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.466 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.466 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-05-21T15:26:41.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-05-21T15:26:41.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.467 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.467 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.468 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-05-21T15:26:41.468 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.468 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-05-21T15:26:41.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-05-21T15:26:41.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-05-21T15:26:41.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.469 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-05-21T15:26:41.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.470 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-05-21T15:26:41.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-05-21T15:26:41.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.471 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.471 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.471 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-05-21T15:26:41.471 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:26:41.472 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.472 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.472 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-05-21T15:26:41.472 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-05-21T15:26:41.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.473 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:41.473 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:41.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-05-21T15:26:41.473 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.473 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T15:26:41.473 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.474 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.474 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-05-21T15:26:41.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-05-21T15:26:41.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.475 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.475 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.475 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.475 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.475 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-05-21T15:26:41.475 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.476 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.476 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-05-21T15:26:41.476 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-05-21T15:26:41.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.476 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-05-21T15:26:41.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.477 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.477 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:26:41.477 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-05-21T15:26:41.478 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.479 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.479 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.479 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.479 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.479 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.480 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.480 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.480 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.480 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.480 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.481 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.481 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.481 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.481 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.481 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.481 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.482 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.482 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.482 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.482 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.482 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.482 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.482 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.483 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.483 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.483 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.483 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.483 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.483 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.484 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.484 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.484 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.484 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.484 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.485 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.485 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.485 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.485 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.485 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.485 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.485 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.486 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.486 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.486 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.486 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.486 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.487 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.487 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.488 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.488 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-05-21T15:26:41.489 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.489 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.489 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.489 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.490 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.490 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.490 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.490 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.491 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.491 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.491 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.491 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.491 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.491 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.491 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.492 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.492 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.492 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.492 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.492 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.493 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.493 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.493 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.493 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.493 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.493 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.494 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.494 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.494 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.495 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.495 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.495 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.496 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.496 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.496 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.496 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.496 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.496 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.497 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.497 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.497 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.498 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.498 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.498 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.498 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.498 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.498 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.498 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.499 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.499 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.499 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-05-21T15:26:41.500 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.500 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.500 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.500 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-05-21T15:26:41.500 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-05-21T15:26:41.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.501 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-05-21T15:26:41.501 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.501 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.501 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.501 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T15:26:41.501 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.502 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.502 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.502 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-05-21T15:26:41.502 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-05-21T15:26:41.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.502 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.503 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.503 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-05-21T15:26:41.503 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.503 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.503 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T15:26:41.503 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.504 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.504 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.504 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.504 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.505 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.505 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.505 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.505 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.505 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.505 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.506 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.506 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.506 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.506 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.506 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.507 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.507 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.508 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.508 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.508 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.509 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.509 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.509 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.509 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.510 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.510 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.510 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.510 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.510 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.510 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.511 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.513 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.513 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.513 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.513 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.513 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.513 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.514 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-05-21T15:26:41.514 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.514 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.515 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.515 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-05-21T15:26:41.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-05-21T15:26:41.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-05-21T15:26:41.516 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.516 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-05-21T15:26:41.516 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T15:26:41.516 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.516 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.517 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.518 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.518 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.518 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.518 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.518 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.519 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.519 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.519 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.520 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.521 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.521 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.521 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.521 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.522 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.522 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.522 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.522 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.523 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.523 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.523 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.523 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.523 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.523 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.524 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.524 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.524 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.525 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.525 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.525 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.525 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.526 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.526 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.526 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-05-21T15:26:41.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-05-21T15:26:41.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.527 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.527 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.528 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-05-21T15:26:41.528 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T15:26:41.528 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.528 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.528 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-05-21T15:26:41.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-05-21T15:26:41.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.529 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.529 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.529 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-05-21T15:26:41.529 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.529 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-05-21T15:26:41.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.530 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.530 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-05-21T15:26:41.530 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-05-21T15:26:41.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.531 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.531 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-05-21T15:26:41.531 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.532 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T15:26:41.532 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.532 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.532 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.532 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-05-21T15:26:41.533 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-05-21T15:26:41.533 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.533 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.533 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.533 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.533 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.534 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-05-21T15:26:41.534 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.534 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.534 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-05-21T15:26:41.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-05-21T15:26:41.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.535 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-05-21T15:26:41.535 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.535 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.535 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.536 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T15:26:41.536 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T15:26:41.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.536 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-05-21T15:26:41.537 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.537 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.537 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.537 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.538 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.538 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.538 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.538 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.538 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.538 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.539 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.539 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.539 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.539 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.539 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.539 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.541 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.541 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.542 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.542 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.542 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.543 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.543 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.544 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.544 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.544 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.544 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.545 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.545 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.545 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.545 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.545 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.545 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.545 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.546 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.546 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.546 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.546 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.548 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.548 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.549 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-05-21T15:26:41.549 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.549 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.549 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.551 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-05-21T15:26:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-05-21T15:26:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-05-21T15:26:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-05-21T15:26:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T15:26:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.578 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.579 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.579 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.579 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.579 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.580 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.580 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.580 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.580 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.580 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.581 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.582 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.582 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.583 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-05-21T15:26:41.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-05-21T15:26:41.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.584 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.584 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.584 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.584 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-05-21T15:26:41.584 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-05-21T15:26:41.584 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-05-21T15:26:41.584 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.585 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-05-21T15:26:41.585 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.585 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.585 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-05-21T15:26:41.586 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.586 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.586 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.586 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.588 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.588 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.588 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.588 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.588 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.588 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.588 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.589 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.591 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.591 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.592 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.592 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.592 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.593 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.593 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.593 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.593 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.593 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.593 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.594 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.594 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.594 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.594 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.596 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.596 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-05-21T15:26:41.597 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.597 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.597 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.597 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.598 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.598 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.598 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.598 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.599 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.599 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.599 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.599 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.599 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.599 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.600 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.600 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.601 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.601 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.601 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.601 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.602 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.602 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.602 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.603 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.603 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.603 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.603 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.604 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.604 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.604 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.604 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.605 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.606 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.606 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.606 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.606 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.606 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-05-21T15:26:41.607 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.607 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.607 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.607 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.608 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.609 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.609 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.609 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.609 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.609 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.609 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.609 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.610 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.610 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.612 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.614 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.614 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.614 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.615 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.616 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.616 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:41.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:41.617 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.617 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.617 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.617 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:41.617 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.618 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.618 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.618 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-05-21T15:26:41.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-05-21T15:26:41.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-05-21T15:26:41.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-05-21T15:26:41.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-05-21T15:26:41.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T15:26:41.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.620 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.620 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.621 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-05-21T15:26:41.621 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.621 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.621 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.621 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-05-21T15:26:41.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-05-21T15:26:41.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.622 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.622 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T15:26:41.623 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.623 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-05-21T15:26:41.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-05-21T15:26:41.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.624 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.624 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.625 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.625 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.625 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.625 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.626 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.626 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.626 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.627 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.627 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.627 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.627 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.627 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.627 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.628 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.628 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.628 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.630 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.630 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.630 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.631 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.631 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.631 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.631 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.632 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.632 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.632 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.632 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.632 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.633 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.633 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.633 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.633 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.633 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.635 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-05-21T15:26:41.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-05-21T15:26:41.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-05-21T15:26:41.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.636 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-05-21T15:26:41.636 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.636 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.637 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:41.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:41.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:41.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-05-21T15:26:41.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.639 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.639 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-05-21T15:26:41.639 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.640 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.640 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.640 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-05-21T15:26:41.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-05-21T15:26:41.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.641 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.641 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.641 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.642 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-05-21T15:26:41.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-05-21T15:26:41.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.642 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.642 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T15:26:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T15:26:41.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.645 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.650 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.654 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-05-21T15:26:41.654 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-05-21T15:26:41.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.654 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.655 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.655 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.655 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.655 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.655 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.656 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T15:26:41.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T15:26:41.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.656 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-05-21T15:26:41.656 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.658 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-05-21T15:26:41.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-05-21T15:26:41.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.658 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-05-21T15:26:41.658 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.659 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.659 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.659 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.659 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.659 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:41.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:41.660 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.660 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-05-21T15:26:41.660 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.660 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T15:26:41.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.661 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:41.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:41.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.662 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-05-21T15:26:41.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T15:26:41.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.663 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-05-21T15:26:41.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-05-21T15:26:41.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.664 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-05-21T15:26:41.664 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.664 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.665 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.665 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-05-21T15:26:41.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-05-21T15:26:41.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-05-21T15:26:41.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.667 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-05-21T15:26:41.667 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-05-21T15:26:41.667 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.668 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-05-21T15:26:41.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-05-21T15:26:41.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-05-21T15:26:41.669 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-05-21T15:26:41.669 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-05-21T15:26:41.669 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-05-21T15:26:41.670 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.670 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.671 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.671 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.671 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.671 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.671 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.671 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.672 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.672 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.672 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.672 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.672 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.672 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.673 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.673 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.674 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.674 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.674 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.675 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.675 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.675 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.676 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.676 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.676 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.677 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.677 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.677 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.677 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.677 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.677 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.677 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.677 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.678 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.678 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.678 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.678 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.678 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.679 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.679 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.680 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-05-21T15:26:41.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-05-21T15:26:41.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.681 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.681 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.682 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.682 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.682 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-05-21T15:26:41.682 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T15:26:41.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.683 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.684 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-05-21T15:26:41.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-05-21T15:26:41.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.684 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.686 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-05-21T15:26:41.687 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.687 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.687 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.687 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-05-21T15:26:41.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-05-21T15:26:41.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.688 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.688 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-05-21T15:26:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-05-21T15:26:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.690 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.690 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.690 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.691 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.691 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.692 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.692 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.693 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.693 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.693 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.693 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.693 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.694 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.695 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.695 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.696 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.697 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.697 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.697 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.697 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.697 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.697 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.698 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.698 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.698 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.698 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.698 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.698 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.699 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.699 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.699 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.700 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.701 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:41.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:41.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.701 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.703 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:41.703 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:41.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.703 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.704 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.704 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.704 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.704 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.705 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-05-21T15:26:41.705 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.705 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.705 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.705 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.706 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.707 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.707 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.707 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.707 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.707 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.707 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.708 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.708 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.708 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.709 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.710 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.710 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.711 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.711 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.712 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.712 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.712 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.712 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.712 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.712 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.713 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.713 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-05-21T15:26:41.716 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.716 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.716 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.716 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.718 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.718 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.718 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.718 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.718 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.719 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.719 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.719 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.720 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.720 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.720 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.721 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.722 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.722 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.723 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.723 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.723 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.723 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.723 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.724 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.724 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.725 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.725 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.726 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-05-21T15:26:41.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-05-21T15:26:41.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-05-21T15:26:41.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.727 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T15:26:41.727 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.728 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.728 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-05-21T15:26:41.728 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.729 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.729 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.729 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.730 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.730 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.730 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.730 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.731 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.731 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.731 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.732 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.733 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.733 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.734 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.734 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.735 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.735 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.735 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.736 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.736 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.736 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.736 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.737 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.737 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.737 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.737 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.737 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.738 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.738 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.738 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.738 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.739 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-05-21T15:26:41.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-05-21T15:26:41.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.740 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.741 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-05-21T15:26:41.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-05-21T15:26:41.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-05-21T15:26:41.741 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.742 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.742 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-05-21T15:26:41.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-05-21T15:26:41.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.744 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-05-21T15:26:41.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-05-21T15:26:41.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.746 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.746 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.746 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-05-21T15:26:41.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-05-21T15:26:41.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.747 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.748 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.748 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.748 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.748 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.748 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-05-21T15:26:41.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-05-21T15:26:41.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.749 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.749 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.749 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-05-21T15:26:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-05-21T15:26:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.751 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.751 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.751 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.752 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-05-21T15:26:41.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-05-21T15:26:41.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.752 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.752 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.752 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.753 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.753 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.753 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.753 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.753 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.753 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.754 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-05-21T15:26:41.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-05-21T15:26:41.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.754 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:41.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-05-21T15:26:41.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.755 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.756 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-05-21T15:26:41.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-05-21T15:26:41.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.756 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.757 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.757 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.757 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.757 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.758 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-05-21T15:26:41.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-05-21T15:26:41.758 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.758 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.758 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.759 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.759 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.759 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-05-21T15:26:41.760 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-05-21T15:26:41.760 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.760 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.760 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.761 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.761 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.761 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-05-21T15:26:41.762 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.762 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.762 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.762 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-05-21T15:26:41.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-05-21T15:26:41.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.763 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.763 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.763 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T15:26:41.764 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.764 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.764 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.764 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-05-21T15:26:41.764 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-05-21T15:26:41.764 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.764 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.765 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-05-21T15:26:41.765 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.766 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.767 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.767 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.767 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.768 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.768 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.768 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.768 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.768 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.768 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.769 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.769 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.771 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.771 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.771 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.772 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.772 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.772 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.773 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.773 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.773 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.773 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.773 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.773 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.774 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.774 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.774 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.774 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.774 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.774 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.775 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.775 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.775 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.775 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.775 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.776 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.776 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-05-21T15:26:41.776 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.776 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.777 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.777 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.777 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.777 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.777 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.778 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.778 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.778 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.778 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.778 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.779 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.779 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.779 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.779 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.779 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.779 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.779 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.780 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.780 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.780 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.780 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.780 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.780 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.781 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.781 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.781 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.781 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.781 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.782 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.782 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.782 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.782 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.783 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.783 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.783 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.783 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.783 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.783 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.783 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.784 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.784 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.785 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.786 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.786 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.786 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-05-21T15:26:41.787 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.787 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.787 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.787 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.788 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.788 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.789 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.789 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.789 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.789 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.789 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.789 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.790 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.790 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.790 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.790 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.791 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.791 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.791 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.791 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.791 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.791 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.792 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.792 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.792 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.792 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.792 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.792 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.793 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.793 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.793 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.793 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.794 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.794 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.794 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.794 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.794 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.794 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.794 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.795 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.795 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.795 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.795 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.795 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.796 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.796 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.796 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.796 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.796 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.797 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-05-21T15:26:41.797 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.797 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.797 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.797 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-05-21T15:26:41.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-05-21T15:26:41.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.798 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T15:26:41.798 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.798 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.799 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:41.799 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:41.800 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.800 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.800 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-05-21T15:26:41.800 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.800 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.800 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.801 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.801 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.801 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.801 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.802 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.802 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.802 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.802 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.802 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.802 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.802 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.802 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.803 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.803 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.803 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.803 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.803 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.803 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.804 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.804 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.804 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.805 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.805 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.805 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.805 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.806 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.806 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.806 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.807 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.807 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.807 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.807 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.807 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.808 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.808 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.808 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.808 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.808 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.808 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.808 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.809 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.809 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.810 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.810 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.810 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.810 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.811 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.811 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-05-21T15:26:41.811 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.812 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.812 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.812 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-05-21T15:26:41.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-05-21T15:26:41.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.813 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.813 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.813 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T15:26:41.813 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.814 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-05-21T15:26:41.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-05-21T15:26:41.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.814 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T15:26:41.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.816 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-05-21T15:26:41.816 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-05-21T15:26:41.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T15:26:41.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.818 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.818 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-05-21T15:26:41.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-05-21T15:26:41.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.819 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.819 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T15:26:41.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.820 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-05-21T15:26:41.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-05-21T15:26:41.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.821 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.821 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.821 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T15:26:41.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.822 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-05-21T15:26:41.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-05-21T15:26:41.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.823 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.823 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.824 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-05-21T15:26:41.824 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-05-21T15:26:41.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.826 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-05-21T15:26:41.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-05-21T15:26:41.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T15:26:41.827 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.827 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.827 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.828 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-05-21T15:26:41.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-05-21T15:26:41.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.830 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:41.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:41.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.830 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-05-21T15:26:41.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-05-21T15:26:41.831 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.831 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.832 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-05-21T15:26:41.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-05-21T15:26:41.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.832 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-05-21T15:26:41.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.833 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.834 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-05-21T15:26:41.834 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-05-21T15:26:41.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.834 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.835 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-05-21T15:26:41.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-05-21T15:26:41.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.836 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.837 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.837 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.837 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.837 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.837 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.837 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.839 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.839 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.839 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.839 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.839 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.839 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.839 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.840 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.840 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.840 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.842 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.843 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.843 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.843 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.843 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.844 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.844 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.844 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.844 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.844 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.844 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.845 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.845 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.845 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.845 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.846 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.847 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-05-21T15:26:41.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-05-21T15:26:41.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.848 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.848 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.848 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.848 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.849 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-05-21T15:26:41.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-05-21T15:26:41.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.850 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.850 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.850 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.851 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-05-21T15:26:41.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-05-21T15:26:41.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T15:26:41.852 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.852 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.852 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.853 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.853 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-05-21T15:26:41.853 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.853 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.853 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.854 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.854 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.854 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.855 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.855 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.855 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.855 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.855 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.856 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.856 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.856 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.856 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.856 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.856 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.856 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.857 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.857 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.857 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.858 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.860 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.860 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.860 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.860 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.860 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.860 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.860 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.861 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.862 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.863 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.863 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.863 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-05-21T15:26:41.864 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.864 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.864 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.864 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.865 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.866 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.866 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.866 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.866 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.866 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.867 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.867 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.867 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.868 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.868 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.868 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.869 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.869 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.869 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.869 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.870 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.870 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.870 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.872 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.872 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.872 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.873 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.873 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.873 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.874 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-05-21T15:26:41.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-05-21T15:26:41.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.874 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-05-21T15:26:41.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.875 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.875 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.875 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T15:26:41.875 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.875 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.876 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-05-21T15:26:41.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-05-21T15:26:41.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.877 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.877 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.877 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-05-21T15:26:41.877 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.878 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.878 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-05-21T15:26:41.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-05-21T15:26:41.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.879 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.879 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:41.879 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.879 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.879 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T15:26:41.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.879 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.880 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-05-21T15:26:41.880 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T15:26:41.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.880 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.881 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.881 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.881 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T15:26:41.881 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.882 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.882 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.882 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-05-21T15:26:41.882 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.882 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.882 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.883 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-05-21T15:26:41.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-05-21T15:26:41.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.883 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-05-21T15:26:41.883 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.884 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-05-21T15:26:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-05-21T15:26:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.886 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.887 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:41.887 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:41.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:41.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.888 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.888 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.888 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.888 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.888 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.889 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.890 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.890 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.890 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.890 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.890 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.891 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.891 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.891 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.891 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.891 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.891 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.892 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.892 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.892 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.893 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.893 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.894 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.894 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.895 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.895 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.895 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.895 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.895 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.895 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.895 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.896 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.896 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.897 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.898 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.898 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.898 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-05-21T15:26:41.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-05-21T15:26:41.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.899 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.899 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.900 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.900 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.900 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-05-21T15:26:41.900 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-05-21T15:26:41.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-05-21T15:26:41.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.901 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.901 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.902 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.902 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.902 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.902 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-05-21T15:26:41.902 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-05-21T15:26:41.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.903 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.903 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.903 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T15:26:41.903 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.903 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.903 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.904 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.904 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.904 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.904 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:41.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.905 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 25004, 2024-05-21T15:26:41.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "y", 2024-05-21T15:26:41.905 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-05-21T15:26:41.905 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-05-21T15:26:41.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-05-21T15:26:41.906 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.906 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.906 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.906 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:41.906 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:41.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.906 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:41.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.907 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.907 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.907 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T15:26:41.907 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.908 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.908 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.908 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.909 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.909 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.909 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.910 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.910 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.910 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.910 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.910 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.910 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.910 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.910 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.911 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.912 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.913 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.913 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.914 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.914 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.914 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.914 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.914 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.915 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.915 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.915 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.916 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.916 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.916 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.916 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.918 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-05-21T15:26:41.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-05-21T15:26:41.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.919 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.919 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-05-21T15:26:41.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.920 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.920 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-05-21T15:26:41.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-05-21T15:26:41.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.920 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-05-21T15:26:41.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-05-21T15:26:41.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.922 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.922 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-05-21T15:26:41.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-05-21T15:26:41.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.923 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.923 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.923 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-05-21T15:26:41.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.924 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-05-21T15:26:41.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-05-21T15:26:41.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.925 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.925 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-05-21T15:26:41.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.926 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.926 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.926 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-05-21T15:26:41.926 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-05-21T15:26:41.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-05-21T15:26:41.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-05-21T15:26:41.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.928 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-05-21T15:26:41.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-05-21T15:26:41.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.929 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.929 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.929 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.929 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-05-21T15:26:41.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.930 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-05-21T15:26:41.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-05-21T15:26:41.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.930 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.931 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.931 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T15:26:41.931 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.932 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-05-21T15:26:41.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-05-21T15:26:41.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.933 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.933 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-05-21T15:26:41.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.934 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-05-21T15:26:41.934 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.935 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.935 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.935 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-05-21T15:26:41.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-05-21T15:26:41.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.936 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T15:26:41.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.936 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.937 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-05-21T15:26:41.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-05-21T15:26:41.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.937 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.938 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-05-21T15:26:41.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T15:26:41.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:26:41.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.939 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.939 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-05-21T15:26:41.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-05-21T15:26:41.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:41.940 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-05-21T15:26:41.940 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-05-21T15:26:41.940 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T15:26:41.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:26:41.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.941 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-05-21T15:26:41.941 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-05-21T15:26:41.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-05-21T15:26:41.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:41.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-05-21T15:26:41.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T15:26:41.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.943 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-05-21T15:26:41.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-05-21T15:26:41.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T15:26:41.944 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.944 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.944 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.944 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T15:26:41.944 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T15:26:41.944 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.945 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-05-21T15:26:41.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-05-21T15:26:41.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-05-21T15:26:41.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-05-21T15:26:41.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-05-21T15:26:41.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-05-21T15:26:41.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T15:26:41.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.947 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-05-21T15:26:41.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-05-21T15:26:41.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.947 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-05-21T15:26:41.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.948 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.948 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T15:26:41.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:26:41.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.949 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-05-21T15:26:41.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-05-21T15:26:41.949 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:41.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-05-21T15:26:41.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-05-21T15:26:41.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.950 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T15:26:41.950 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:26:41.950 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.950 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.950 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:41.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:41.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.951 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.951 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.952 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.952 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.952 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.952 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.952 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.952 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.952 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.953 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.953 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:41.953 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:41.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.954 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.954 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.955 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.955 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.955 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:41.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:41.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:41.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.956 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:41.957 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:41.957 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:41.957 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:41.957 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:41.957 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:41.957 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.958 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.958 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:41.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:41.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.958 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.959 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.959 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:41.959 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-05-21T15:26:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-05-21T15:26:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T15:26:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.962 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-05-21T15:26:41.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-05-21T15:26:41.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-05-21T15:26:41.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.963 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-05-21T15:26:41.963 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-05-21T15:26:41.963 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-05-21T15:26:41.964 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-05-21T15:26:41.964 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-05-21T15:26:41.964 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-05-21T15:26:41.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.965 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-05-21T15:26:41.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-05-21T15:26:41.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.966 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.966 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.966 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.966 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T15:26:41.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.967 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:41.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:41.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:41.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.968 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T15:26:41.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.969 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-05-21T15:26:41.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-05-21T15:26:41.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.969 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:41.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:41.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.970 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T15:26:41.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T15:26:41.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.970 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-05-21T15:26:41.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-05-21T15:26:41.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:41.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:41.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T15:26:41.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.972 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.972 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.972 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:41.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:41.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-05-21T15:26:41.973 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:41.973 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:41.973 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:41.973 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-05-21T15:26:41.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-05-21T15:26:41.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-05-21T15:26:41.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.975 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T15:26:41.975 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.975 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-05-21T15:26:41.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-05-21T15:26:41.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.976 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-05-21T15:26:41.976 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.976 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.976 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T15:26:41.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.977 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-05-21T15:26:41.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-05-21T15:26:41.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:41.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.978 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-05-21T15:26:41.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T15:26:41.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.979 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-05-21T15:26:41.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-05-21T15:26:41.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T15:26:41.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T15:26:41.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.981 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.981 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-05-21T15:26:41.981 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-05-21T15:26:41.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:41.981 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:41.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T15:26:41.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.983 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.983 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-05-21T15:26:41.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T15:26:41.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:41.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.984 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-05-21T15:26:41.984 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.984 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.984 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.984 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T15:26:41.984 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.985 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.985 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.985 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-05-21T15:26:41.985 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-05-21T15:26:41.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.985 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:41.986 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.986 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.986 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:41.986 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-05-21T15:26:41.986 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-05-21T15:26:41.987 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-05-21T15:26:41.987 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:41.987 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T15:26:41.987 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.987 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.987 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-05-21T15:26:41.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-05-21T15:26:41.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.988 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.988 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.988 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T15:26:41.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.989 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-05-21T15:26:41.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-05-21T15:26:41.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.990 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:41.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T15:26:41.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.991 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-05-21T15:26:41.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-05-21T15:26:41.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:41.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:41.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.992 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T15:26:41.993 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.993 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.993 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-05-21T15:26:41.993 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-05-21T15:26:41.993 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.993 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T15:26:41.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.994 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.994 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:41.994 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.995 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.995 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-05-21T15:26:41.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-05-21T15:26:41.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.995 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:41.996 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T15:26:41.996 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.996 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.996 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.996 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-05-21T15:26:41.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.997 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-05-21T15:26:41.997 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-05-21T15:26:41.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.997 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T15:26:41.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:41.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:41.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:41.998 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-05-21T15:26:41.998 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:41.998 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:41.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:41.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:41.999 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-05-21T15:26:41.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-05-21T15:26:41.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:41.999 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:41.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:41.999 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T15:26:42.000 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.000 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.000 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.000 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:42.000 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.000 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.001 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-05-21T15:26:42.001 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-05-21T15:26:42.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.001 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T15:26:42.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.002 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-05-21T15:26:42.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.003 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-05-21T15:26:42.003 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-05-21T15:26:42.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.003 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.003 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T15:26:42.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.004 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.004 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-05-21T15:26:42.004 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.004 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.004 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-05-21T15:26:42.005 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-05-21T15:26:42.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.005 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.005 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T15:26:42.005 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.005 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-05-21T15:26:42.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.006 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.006 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-05-21T15:26:42.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-05-21T15:26:42.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T15:26:42.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-05-21T15:26:42.008 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.008 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.008 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-05-21T15:26:42.008 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-05-21T15:26:42.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.009 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.009 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.009 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T15:26:42.009 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.009 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.009 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.009 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-05-21T15:26:42.010 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.010 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.010 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.010 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-05-21T15:26:42.010 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-05-21T15:26:42.010 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.011 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-21T15:26:42.011 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.011 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.011 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.011 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-05-21T15:26:42.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.012 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-05-21T15:26:42.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-05-21T15:26:42.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.013 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.013 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T15:26:42.013 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.013 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.013 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:42.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.014 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.014 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-05-21T15:26:42.014 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-05-21T15:26:42.014 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.014 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-21T15:26:42.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.015 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-05-21T15:26:42.015 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.016 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.016 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-05-21T15:26:42.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-05-21T15:26:42.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.016 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T15:26:42.017 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.017 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.017 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:42.017 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.018 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-05-21T15:26:42.018 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-05-21T15:26:42.018 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.018 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-21T15:26:42.019 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-05-21T15:26:42.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.020 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.020 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.020 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-05-21T15:26:42.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-05-21T15:26:42.020 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.020 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.020 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.021 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T15:26:42.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.021 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-05-21T15:26:42.021 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.022 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-05-21T15:26:42.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-05-21T15:26:42.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.022 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.023 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T15:26:42.023 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.023 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.023 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.023 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-05-21T15:26:42.023 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.024 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-05-21T15:26:42.024 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-05-21T15:26:42.024 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.024 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.024 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.024 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.025 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.025 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.025 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.025 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T15:26:42.025 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.025 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.025 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.026 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-05-21T15:26:42.026 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-05-21T15:26:42.026 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.026 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.026 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:42.027 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.027 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.027 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T15:26:42.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-05-21T15:26:42.028 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T15:26:42.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-05-21T15:26:42.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.029 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.029 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T15:26:42.029 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-05-21T15:26:42.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-05-21T15:26:42.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-05-21T15:26:42.030 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.030 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.031 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T15:26:42.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.031 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-05-21T15:26:42.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-05-21T15:26:42.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.032 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.032 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-05-21T15:26:42.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.033 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-05-21T15:26:42.033 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.033 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.033 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.033 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-05-21T15:26:42.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-05-21T15:26:42.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.034 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.034 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.034 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.034 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T15:26:42.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.035 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-05-21T15:26:42.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-05-21T15:26:42.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.036 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:42.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.036 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-05-21T15:26:42.036 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.037 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.037 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.037 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-05-21T15:26:42.037 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-05-21T15:26:42.037 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.037 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T15:26:42.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.038 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T15:26:42.038 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.039 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-05-21T15:26:42.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-05-21T15:26:42.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.039 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.039 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.040 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:42.040 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.040 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.040 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.040 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-05-21T15:26:42.040 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.041 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-05-21T15:26:42.041 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-05-21T15:26:42.041 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.041 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.041 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.041 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.042 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.042 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.042 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T15:26:42.042 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.042 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.042 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-05-21T15:26:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-05-21T15:26:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.044 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T15:26:42.044 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.044 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.044 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.045 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.045 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.045 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.047 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.047 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.047 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-05-21T15:26:42.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-05-21T15:26:42.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.049 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T15:26:42.049 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.049 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.049 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.050 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.050 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.051 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T15:26:42.051 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.051 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.051 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.051 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.052 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.052 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.052 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.053 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.053 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.053 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.053 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.053 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.054 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.054 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.055 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.055 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.055 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.055 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.056 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.056 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T15:26:42.056 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T15:26:42.056 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T15:26:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.058 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.058 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T15:26:42.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T15:26:42.058 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.059 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-05-21T15:26:42.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.059 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T15:26:42.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.060 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-05-21T15:26:42.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-05-21T15:26:42.060 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.060 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.060 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.061 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-05-21T15:26:42.061 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.061 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.061 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.062 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-05-21T15:26:42.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-05-21T15:26:42.062 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.062 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.062 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.062 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.062 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.063 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T15:26:42.063 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.063 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.063 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.063 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-05-21T15:26:42.064 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-05-21T15:26:42.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.064 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.064 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.064 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.064 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.065 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.065 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-05-21T15:26:42.065 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.065 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.065 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-05-21T15:26:42.066 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-05-21T15:26:42.066 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.066 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.066 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.066 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-05-21T15:26:42.066 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.066 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.067 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.067 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-05-21T15:26:42.067 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-05-21T15:26:42.067 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.067 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T15:26:42.067 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.067 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.068 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.068 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-05-21T15:26:42.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-05-21T15:26:42.068 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.068 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.068 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.069 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-05-21T15:26:42.069 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.069 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T15:26:42.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.069 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.069 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.070 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-05-21T15:26:42.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-05-21T15:26:42.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.070 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T15:26:42.071 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.071 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.071 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.071 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T15:26:42.071 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.071 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.071 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.072 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-05-21T15:26:42.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-05-21T15:26:42.072 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.072 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.072 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.072 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.072 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.073 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.073 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.073 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T15:26:42.073 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.073 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.073 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.074 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-05-21T15:26:42.074 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-05-21T15:26:42.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.074 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.075 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.075 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.075 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.075 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T15:26:42.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.075 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.076 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-05-21T15:26:42.076 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-05-21T15:26:42.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.076 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.076 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.076 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.077 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.077 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.077 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T15:26:42.077 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.077 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.077 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.078 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-05-21T15:26:42.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-05-21T15:26:42.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.078 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.078 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.078 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.078 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.079 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T15:26:42.079 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.079 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.080 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-05-21T15:26:42.080 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-05-21T15:26:42.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.080 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.081 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.081 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.081 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T15:26:42.081 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.081 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.081 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-05-21T15:26:42.082 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-05-21T15:26:42.082 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.082 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-05-21T15:26:42.082 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.082 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.083 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.083 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-05-21T15:26:42.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.083 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.083 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-05-21T15:26:42.084 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-05-21T15:26:42.084 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.084 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.084 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.084 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.084 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.084 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.085 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T15:26:42.085 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.085 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.085 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-05-21T15:26:42.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-05-21T15:26:42.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.086 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.086 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.086 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.086 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.087 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T15:26:42.087 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.087 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.087 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.087 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-05-21T15:26:42.087 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-05-21T15:26:42.088 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.088 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.088 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.088 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.088 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.088 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.088 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T15:26:42.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.089 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.089 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-05-21T15:26:42.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-05-21T15:26:42.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.090 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.090 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.090 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.090 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.090 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.090 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T15:26:42.091 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.091 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.091 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-05-21T15:26:42.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-05-21T15:26:42.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.092 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.092 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.092 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.092 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.092 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.092 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T15:26:42.092 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.093 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-05-21T15:26:42.093 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-05-21T15:26:42.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.093 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.094 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.094 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.094 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.094 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T15:26:42.094 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.094 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.094 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.095 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.095 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.095 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-05-21T15:26:42.095 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.095 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.095 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.096 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.096 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.096 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.097 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.097 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.097 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.097 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.097 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.097 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.097 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.098 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.098 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.098 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.098 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.098 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.098 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.099 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.099 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.099 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.099 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.099 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.099 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.099 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.099 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.100 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.100 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.100 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.100 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.100 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.100 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.100 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.101 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.101 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.101 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.101 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.101 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.101 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.102 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.102 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.102 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.102 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.102 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.102 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.102 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.103 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.103 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.103 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.104 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.104 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.104 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.104 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.105 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.105 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-05-21T15:26:42.105 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-05-21T15:26:42.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.105 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.106 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.106 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-05-21T15:26:42.106 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.106 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.106 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.106 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T15:26:42.106 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.107 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-05-21T15:26:42.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-05-21T15:26:42.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.107 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.108 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-05-21T15:26:42.108 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T15:26:42.108 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.109 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.109 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.109 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-05-21T15:26:42.110 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.110 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.110 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.110 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T15:26:42.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T15:26:42.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.110 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.111 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.111 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.111 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.111 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.111 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.112 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T15:26:42.112 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T15:26:42.112 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.112 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.112 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.113 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.113 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.113 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.113 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.113 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.114 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T15:26:42.114 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T15:26:42.114 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.114 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.114 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.114 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.115 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.115 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.115 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.115 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.115 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.115 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.116 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-05-21T15:26:42.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T15:26:42.116 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.116 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.116 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.116 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.117 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.117 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.117 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.118 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T15:26:42.118 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T15:26:42.118 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.118 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.118 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.118 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.119 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.119 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.119 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.119 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.119 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.119 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T15:26:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T15:26:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.121 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.121 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.121 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.121 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.121 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T15:26:42.122 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T15:26:42.122 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.122 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.122 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.122 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.123 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.123 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.123 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.123 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.123 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.123 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T15:26:42.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T15:26:42.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.124 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.124 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.124 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.125 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.125 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.125 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T15:26:42.125 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T15:26:42.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.126 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.126 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.126 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.126 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.127 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.127 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.127 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.127 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T15:26:42.127 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T15:26:42.127 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.128 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.128 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.128 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.128 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.128 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.128 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.128 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.129 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.129 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.129 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T15:26:42.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T15:26:42.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.130 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.130 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.130 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.130 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.130 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.130 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.131 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.131 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.131 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T15:26:42.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T15:26:42.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.132 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.132 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.132 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.132 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.132 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.132 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.133 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.133 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.133 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T15:26:42.133 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T15:26:42.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.133 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.134 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.134 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.134 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.134 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.135 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.135 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T15:26:42.135 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T15:26:42.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.135 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.136 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.136 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.136 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.136 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.136 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.136 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.136 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.137 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.137 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T15:26:42.137 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T15:26:42.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.137 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.138 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:42.138 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.138 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.138 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.138 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.138 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.139 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.139 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T15:26:42.139 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T15:26:42.139 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.139 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.139 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.139 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.140 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.140 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.140 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.140 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.140 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.140 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.140 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.141 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-05-21T15:26:42.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-05-21T15:26:42.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.141 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.141 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.141 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.141 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.142 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.142 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.142 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.142 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.142 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.142 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.143 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T15:26:42.143 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T15:26:42.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.143 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.143 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.143 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:42.143 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.144 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.144 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.144 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.144 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.144 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.145 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T15:26:42.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T15:26:42.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.145 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.145 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.146 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.146 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.146 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.146 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.146 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.146 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.147 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T15:26:42.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T15:26:42.147 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.147 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.147 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.147 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.148 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.148 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.148 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.148 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.149 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T15:26:42.149 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T15:26:42.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.149 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.149 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.149 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.150 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.150 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.150 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.151 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T15:26:42.151 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T15:26:42.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.151 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.151 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.151 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.151 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.152 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.152 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T15:26:42.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T15:26:42.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.154 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.154 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T15:26:42.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T15:26:42.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.156 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.156 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.156 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.156 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.156 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T15:26:42.157 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T15:26:42.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.157 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.157 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.157 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.157 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.157 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.158 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.158 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T15:26:42.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T15:26:42.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.159 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.160 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.160 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T15:26:42.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T15:26:42.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.160 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.161 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.162 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T15:26:42.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T15:26:42.162 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.162 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.163 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T15:26:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T15:26:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.164 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.165 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.165 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.165 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.166 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T15:26:42.166 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T15:26:42.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.166 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.166 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.167 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.167 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.167 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.167 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.168 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T15:26:42.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T15:26:42.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.169 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.169 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.169 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.169 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.169 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.169 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.170 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T15:26:42.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T15:26:42.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.170 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.170 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.170 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.171 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.171 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.172 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T15:26:42.172 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T15:26:42.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.172 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T15:26:42.173 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.173 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.173 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.174 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T15:26:42.174 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T15:26:42.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.174 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.174 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.175 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.175 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.175 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.176 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T15:26:42.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T15:26:42.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.176 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-05-21T15:26:42.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.177 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.177 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.177 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.178 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T15:26:42.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T15:26:42.178 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.178 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.178 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.178 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-05-21T15:26:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.180 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T15:26:42.180 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T15:26:42.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.180 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.180 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.181 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.181 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.181 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T15:26:42.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T15:26:42.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.182 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.182 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:42.182 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.182 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.182 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.183 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.183 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.183 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.183 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T15:26:42.183 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T15:26:42.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.184 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.184 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.185 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T15:26:42.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T15:26:42.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.185 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.186 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.186 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.186 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.186 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.186 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.186 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.187 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T15:26:42.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T15:26:42.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.187 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:42.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.189 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.189 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T15:26:42.189 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T15:26:42.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.189 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.190 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.190 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.190 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:42.191 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.192 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.192 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.192 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-05-21T15:26:42.193 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-05-21T15:26:42.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.193 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.194 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.194 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.194 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.194 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-05-21T15:26:42.194 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-05-21T15:26:42.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.195 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.195 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.195 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.195 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.195 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.195 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.196 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.196 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.196 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.196 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-05-21T15:26:42.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-05-21T15:26:42.196 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.196 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.197 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.197 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.197 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.197 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.197 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-05-21T15:26:42.197 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.198 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.198 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-05-21T15:26:42.198 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-05-21T15:26:42.198 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.198 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.199 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.199 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-05-21T15:26:42.199 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.199 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.199 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.199 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.199 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-05-21T15:26:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-05-21T15:26:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.203 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.203 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.203 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.203 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.203 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.203 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.204 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.204 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.204 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.204 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.204 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.204 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.204 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.205 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.205 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.205 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.205 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.205 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.206 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.206 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.206 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.206 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.206 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.207 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.207 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.207 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.207 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.208 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.208 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.208 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.208 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.208 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.208 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.208 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.209 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.209 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.209 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.209 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.209 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.209 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.210 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.210 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.210 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.210 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.210 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.211 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.211 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.211 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.211 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.211 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.211 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.212 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-05-21T15:26:42.212 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-05-21T15:26:42.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.213 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.213 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.213 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-05-21T15:26:42.213 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.213 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.214 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-05-21T15:26:42.214 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T15:26:42.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.214 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.215 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.215 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.216 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:42.216 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:42.216 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.216 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.216 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.216 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-05-21T15:26:42.216 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.217 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.217 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.217 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.217 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.217 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.217 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.218 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:42.218 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:42.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.218 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.218 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.218 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-05-21T15:26:42.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.219 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.219 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.219 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.219 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.219 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.220 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-05-21T15:26:42.220 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-05-21T15:26:42.220 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.220 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.220 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.220 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.220 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.220 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.221 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.221 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.221 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.221 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.221 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.221 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-05-21T15:26:42.221 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-05-21T15:26:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-05-21T15:26:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-05-21T15:26:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-05-21T15:26:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-05-21T15:26:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-05-21T15:26:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-05-21T15:26:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-05-21T15:26:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-05-21T15:26:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-05-21T15:26:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-05-21T15:26:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-05-21T15:26:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-05-21T15:26:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-05-21T15:26:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-05-21T15:26:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-05-21T15:26:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-05-21T15:26:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-05-21T15:26:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-05-21T15:26:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-05-21T15:26:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T15:26:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-05-21T15:26:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-05-21T15:26:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-05-21T15:26:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T15:26:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-05-21T15:26:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-05-21T15:26:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-05-21T15:26:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-05-21T15:26:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-05-21T15:26:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-05-21T15:26:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-05-21T15:26:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T15:26:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-05-21T15:26:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-05-21T15:26:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T15:26:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T15:26:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-05-21T15:26:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-05-21T15:26:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-05-21T15:26:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T15:26:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-05-21T15:26:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-05-21T15:26:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-05-21T15:26:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-05-21T15:26:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-05-21T15:26:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-05-21T15:26:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-05-21T15:26:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-05-21T15:26:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-05-21T15:26:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-05-21T15:26:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-05-21T15:26:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-05-21T15:26:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-05-21T15:26:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T15:26:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-05-21T15:26:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-05-21T15:26:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-05-21T15:26:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T15:26:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-05-21T15:26:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-05-21T15:26:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-05-21T15:26:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-05-21T15:26:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-05-21T15:26:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T15:26:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-05-21T15:26:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-05-21T15:26:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-05-21T15:26:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-05-21T15:26:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-05-21T15:26:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-05-21T15:26:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-05-21T15:26:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-05-21T15:26:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-05-21T15:26:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-05-21T15:26:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:26:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-05-21T15:26:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-05-21T15:26:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-05-21T15:26:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T15:26:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-05-21T15:26:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-05-21T15:26:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-05-21T15:26:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-05-21T15:26:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-05-21T15:26:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-05-21T15:26:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.302 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.302 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.302 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:26:42.302 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.302 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.302 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.303 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-05-21T15:26:42.303 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.303 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.304 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.304 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.304 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.304 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.304 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.305 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.305 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.305 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.305 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.305 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.305 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.305 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.305 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.306 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.306 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.306 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.306 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.306 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.307 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.307 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.307 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.307 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.308 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.308 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.309 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.309 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.309 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.309 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.310 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.310 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.310 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.310 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.310 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.310 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.310 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.311 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.311 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.311 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.312 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.313 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.313 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.313 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.313 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-05-21T15:26:42.314 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.314 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.314 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.314 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.315 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.316 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.316 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.316 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.316 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.316 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.316 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.316 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.316 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.317 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.317 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.317 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.317 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.317 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.317 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.318 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.319 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.319 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.319 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.319 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.319 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.320 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.320 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.320 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.320 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.320 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.320 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.320 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.321 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.321 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.321 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.321 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.321 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.321 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.322 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.322 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.322 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.323 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.323 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.323 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.323 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.323 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-05-21T15:26:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.324 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-05-21T15:26:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-05-21T15:26:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.325 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-05-21T15:26:42.326 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.326 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T15:26:42.326 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.326 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.327 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-05-21T15:26:42.327 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-05-21T15:26:42.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.327 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.328 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-05-21T15:26:42.328 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T15:26:42.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.329 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.329 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.329 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.329 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.330 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.330 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.330 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.330 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.330 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.331 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.331 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.331 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.331 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.331 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.331 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.332 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.332 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.332 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.332 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.332 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.332 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.333 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.333 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.333 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.333 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.333 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.333 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.334 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.334 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.334 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.335 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.335 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.335 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.335 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.335 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.335 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.335 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.335 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.336 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.336 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.336 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.336 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.337 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.337 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.337 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.338 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.338 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.338 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-05-21T15:26:42.339 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.339 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.339 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.339 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-05-21T15:26:42.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-05-21T15:26:42.339 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.339 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-05-21T15:26:42.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.340 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.340 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.340 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-05-21T15:26:42.340 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T15:26:42.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.341 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.341 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.342 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.342 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.342 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.342 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.342 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.342 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.342 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.343 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.344 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.345 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.345 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.345 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.346 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.346 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.346 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.346 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.346 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.347 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.347 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.347 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.347 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.347 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.348 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.348 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.348 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.348 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.348 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.348 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.349 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.349 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.349 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.350 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-05-21T15:26:42.351 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-05-21T15:26:42.351 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.351 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.351 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.352 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.352 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-05-21T15:26:42.352 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T15:26:42.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.352 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.352 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-05-21T15:26:42.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-05-21T15:26:42.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-05-21T15:26:42.353 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.354 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-05-21T15:26:42.354 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.354 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-05-21T15:26:42.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-05-21T15:26:42.355 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.355 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-05-21T15:26:42.355 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.356 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T15:26:42.356 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.356 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.357 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-05-21T15:26:42.357 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-05-21T15:26:42.357 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.357 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.358 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.358 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.358 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-05-21T15:26:42.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.359 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-05-21T15:26:42.359 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-05-21T15:26:42.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.359 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-05-21T15:26:42.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.360 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.360 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.360 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T15:26:42.360 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T15:26:42.360 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.360 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.360 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.360 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-05-21T15:26:42.361 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.361 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.361 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.362 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.363 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.363 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.363 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.363 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.363 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.364 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.364 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.364 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.365 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.365 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.365 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.365 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.365 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.366 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.366 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.366 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.366 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.367 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.367 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.367 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.367 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.367 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.367 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.368 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.368 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.368 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.368 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.368 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.368 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.368 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.369 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.369 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.369 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.370 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.370 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.370 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.370 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.370 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.370 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.370 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.371 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.371 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.371 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-05-21T15:26:42.372 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.372 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.372 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.372 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.372 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.373 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.373 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.373 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.373 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.373 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.373 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.373 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.374 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.374 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.374 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.374 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.374 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.375 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.375 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.376 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.376 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.376 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.376 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.377 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.377 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.377 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.377 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.378 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.378 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.378 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.378 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.378 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.378 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.378 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.379 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.379 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.379 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.379 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.379 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.379 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.379 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.380 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.380 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.380 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.380 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.381 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.381 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.381 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.381 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.382 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.382 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-05-21T15:26:42.382 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.382 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.382 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.383 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.383 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.383 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.383 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.383 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.383 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.384 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.384 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.384 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.384 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.384 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.384 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.385 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.385 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.385 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.385 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.386 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.386 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.386 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.387 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.387 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.387 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.387 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.387 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.387 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.388 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.388 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.388 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.388 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.389 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.389 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.389 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.389 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.389 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.389 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.390 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.390 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.390 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.390 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.391 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.391 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.391 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.391 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.392 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.392 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-05-21T15:26:42.393 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.393 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.393 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.393 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-05-21T15:26:42.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-05-21T15:26:42.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.393 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.393 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.394 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.394 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.394 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T15:26:42.394 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.394 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.395 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.395 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.395 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.395 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.395 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.396 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.396 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.396 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.396 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.396 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.397 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.397 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.397 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.398 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.398 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.399 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.399 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.399 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.399 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.400 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.400 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.400 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.400 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.401 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.401 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.401 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.401 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.401 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.401 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.402 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.402 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.402 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.402 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.402 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.402 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.403 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.403 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.403 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.404 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.404 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.404 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.404 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.404 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-05-21T15:26:42.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-05-21T15:26:42.404 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.405 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.405 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.405 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.405 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.405 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.405 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-05-21T15:26:42.406 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-05-21T15:26:42.406 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-05-21T15:26:42.406 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.406 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-05-21T15:26:42.406 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.406 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.406 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.407 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.407 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.407 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.407 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-05-21T15:26:42.407 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.407 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.407 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.408 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.408 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.408 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.408 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.408 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.408 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.409 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.409 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.409 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.409 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.409 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.409 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.409 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.410 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.410 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.410 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.410 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.411 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.411 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.411 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.411 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.411 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.411 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.412 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.412 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.412 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.412 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.412 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.412 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.413 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.413 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.413 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.414 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.414 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.414 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.414 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.414 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.414 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.414 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.415 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.415 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.415 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.415 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.415 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.416 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.416 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.416 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.416 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.416 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.417 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.417 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.417 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.417 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.417 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-05-21T15:26:42.418 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.418 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.418 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.418 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.418 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.419 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.420 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.420 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.420 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.420 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.420 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.420 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.420 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.421 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.421 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.421 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.423 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.423 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.424 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.424 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.424 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.424 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.424 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.424 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.424 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.425 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.425 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.425 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.425 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.425 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.425 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.426 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.426 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.426 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.426 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.426 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.427 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.427 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.427 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.427 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.427 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.427 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.427 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.428 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.428 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.428 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-05-21T15:26:42.428 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.428 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.429 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.429 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.429 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.429 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.429 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.430 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.430 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.430 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.430 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.431 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.431 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.431 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.432 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.432 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.433 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.433 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.433 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.433 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.434 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.434 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.434 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.434 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.434 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.434 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.435 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.435 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.435 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.435 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.435 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.435 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.435 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.436 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.436 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.436 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.436 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.437 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.437 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.437 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.437 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.437 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.438 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.438 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.438 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.438 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:42.438 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:42.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.439 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.439 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.439 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:42.439 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.440 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.440 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.440 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.440 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.440 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-05-21T15:26:42.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-05-21T15:26:42.440 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:42.441 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-05-21T15:26:42.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-05-21T15:26:42.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.441 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-05-21T15:26:42.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T15:26:42.442 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.442 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.442 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.442 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.443 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.444 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.444 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.444 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.444 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T15:26:42.444 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.444 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.444 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-05-21T15:26:42.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-05-21T15:26:42.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.445 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.446 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.446 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.446 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.446 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.447 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.447 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.447 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.447 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.447 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.447 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.447 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.448 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.448 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.448 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.448 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.448 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.448 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.448 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.449 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.449 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.449 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.449 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.449 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.449 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.449 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.450 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.450 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.450 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.450 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.450 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.451 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.451 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.451 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.451 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.451 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.452 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.452 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.452 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.452 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.452 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.452 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.453 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.453 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.453 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.453 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.453 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.453 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.453 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.453 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.454 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.454 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.454 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.454 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.455 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.455 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.455 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.456 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-05-21T15:26:42.456 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-05-21T15:26:42.456 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-05-21T15:26:42.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.457 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-05-21T15:26:42.458 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.458 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.458 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:42.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:42.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.459 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.459 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.459 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:42.459 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.459 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.459 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-05-21T15:26:42.460 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.460 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.460 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.460 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.460 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-05-21T15:26:42.461 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.461 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.461 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.461 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-05-21T15:26:42.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-05-21T15:26:42.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.462 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.462 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.463 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.463 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-05-21T15:26:42.463 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-05-21T15:26:42.463 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.464 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.464 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.464 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.464 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T15:26:42.465 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T15:26:42.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.465 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.465 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.465 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.466 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.466 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.466 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.466 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.467 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.467 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.467 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.467 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.467 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.467 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.467 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.468 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.468 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.468 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.470 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.470 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.472 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.472 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.472 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.472 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.472 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.472 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.472 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.472 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.473 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.473 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.473 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.474 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.474 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.474 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.475 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.475 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-05-21T15:26:42.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-05-21T15:26:42.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.475 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.476 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.476 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.476 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.477 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T15:26:42.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T15:26:42.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-05-21T15:26:42.478 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.478 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.478 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.479 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-05-21T15:26:42.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-05-21T15:26:42.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:42.479 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.479 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.479 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-05-21T15:26:42.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.480 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.480 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.480 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.481 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:42.481 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:42.481 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.481 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.481 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.481 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-05-21T15:26:42.482 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.482 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.482 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T15:26:42.482 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.482 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.483 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.483 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:42.483 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:42.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.483 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.483 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-05-21T15:26:42.484 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.484 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.484 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.484 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T15:26:42.484 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.484 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.485 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-05-21T15:26:42.485 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-05-21T15:26:42.485 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.485 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.485 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-05-21T15:26:42.485 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.486 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.486 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.486 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.486 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-05-21T15:26:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-05-21T15:26:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-05-21T15:26:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.488 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.488 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.488 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-05-21T15:26:42.488 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-05-21T15:26:42.488 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.489 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.489 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-05-21T15:26:42.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-05-21T15:26:42.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.489 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-05-21T15:26:42.490 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-05-21T15:26:42.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-05-21T15:26:42.490 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.490 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.490 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.490 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.491 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.491 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.491 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.491 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-05-21T15:26:42.491 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.492 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.492 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.492 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.493 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.493 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.493 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.493 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.493 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.493 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.493 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.493 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.494 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.494 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.494 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.494 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.494 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.495 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.495 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.495 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.495 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.496 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.496 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.496 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.499 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.499 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.499 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.499 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.500 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.500 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.500 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.500 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.501 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.501 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.501 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-05-21T15:26:42.501 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-05-21T15:26:42.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.502 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.502 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.502 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.502 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.502 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.502 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.502 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.502 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.503 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.503 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-05-21T15:26:42.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T15:26:42.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.503 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.504 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.504 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.504 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.504 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.504 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.504 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.505 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.505 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-05-21T15:26:42.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-05-21T15:26:42.505 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.505 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.506 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.506 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.506 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.506 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.507 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.507 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-05-21T15:26:42.507 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.508 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.508 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-05-21T15:26:42.508 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-05-21T15:26:42.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.509 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.509 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.509 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.510 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-05-21T15:26:42.510 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-05-21T15:26:42.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.510 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.511 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.511 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.511 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.511 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.512 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.512 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.513 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.513 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.513 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.513 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.513 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.513 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.514 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.514 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.514 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.514 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.514 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.514 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.514 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.515 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.516 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.516 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.516 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.516 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.517 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.517 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.518 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.518 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.518 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.518 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.518 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.519 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.519 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.521 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.521 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.521 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.521 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.521 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:42.522 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:42.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.522 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.522 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.522 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.523 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.523 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:42.524 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:42.524 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.524 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.524 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.524 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.525 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.525 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-05-21T15:26:42.526 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.526 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.526 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.526 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.527 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.527 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.527 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.527 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.528 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.528 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.528 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.528 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.528 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.528 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.528 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.529 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.529 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.529 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.529 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.530 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.530 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.531 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.531 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.531 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.531 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.532 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.532 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.532 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.532 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.532 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.532 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.534 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.534 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.535 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.535 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.535 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.535 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.535 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.536 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-05-21T15:26:42.536 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.536 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.537 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.537 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.537 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.537 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.537 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.537 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.537 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.538 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.538 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.538 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.538 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.538 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.538 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.539 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.539 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.539 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.539 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.539 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.539 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.539 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.540 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.540 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.540 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.540 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.541 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.541 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.541 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.541 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.542 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.543 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.543 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.543 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.543 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.543 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.543 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.544 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.544 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.544 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.544 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.544 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.545 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.545 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.545 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.545 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.546 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-05-21T15:26:42.546 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-05-21T15:26:42.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.547 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.547 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-05-21T15:26:42.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.547 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.547 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T15:26:42.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-05-21T15:26:42.549 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.549 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.549 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.549 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.550 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.550 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.551 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.551 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.551 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.551 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.551 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.551 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.551 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.552 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.552 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.552 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.553 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.554 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.555 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.556 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.556 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.556 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.556 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.556 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.556 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.557 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.557 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.559 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.559 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-05-21T15:26:42.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-05-21T15:26:42.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.561 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.561 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-05-21T15:26:42.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-05-21T15:26:42.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-05-21T15:26:42.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.563 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.563 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-05-21T15:26:42.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-05-21T15:26:42.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.565 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-05-21T15:26:42.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-05-21T15:26:42.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.568 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-05-21T15:26:42.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-05-21T15:26:42.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:42.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.568 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.570 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-05-21T15:26:42.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-05-21T15:26:42.570 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.570 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.570 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-05-21T15:26:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-05-21T15:26:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.574 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-05-21T15:26:42.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-05-21T15:26:42.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.576 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-05-21T15:26:42.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-05-21T15:26:42.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:42.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-05-21T15:26:42.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.578 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-05-21T15:26:42.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-05-21T15:26:42.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.580 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-05-21T15:26:42.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-05-21T15:26:42.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.581 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.581 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.581 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.581 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.582 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-05-21T15:26:42.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-05-21T15:26:42.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.582 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.582 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.583 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.583 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.583 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.583 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.584 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-05-21T15:26:42.584 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.584 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.584 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.585 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-05-21T15:26:42.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-05-21T15:26:42.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.586 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.586 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T15:26:42.586 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.586 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.587 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-05-21T15:26:42.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-05-21T15:26:42.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.588 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-05-21T15:26:42.588 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.588 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.590 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.590 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.590 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.590 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.590 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.590 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.591 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.591 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.591 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.591 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.592 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.593 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.593 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.593 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.594 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.594 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.594 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.594 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.595 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.595 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.595 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.595 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.595 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.595 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.595 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.596 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.598 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.598 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.598 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-05-21T15:26:42.599 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.599 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.599 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.599 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.600 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.600 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.600 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.600 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.600 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.600 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.601 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.601 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.601 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.601 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.601 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.601 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.601 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.602 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.602 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.602 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.605 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.605 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.606 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.606 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.606 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.606 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.606 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.607 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.607 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.608 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.609 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.609 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.609 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.609 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-05-21T15:26:42.609 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.610 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.610 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.610 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.610 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.610 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.610 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.611 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.611 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.611 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.611 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.612 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.612 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.612 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.612 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.612 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.612 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.613 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.613 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.613 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.613 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.614 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.615 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.616 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.616 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.616 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.616 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.616 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.616 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.617 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.617 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.617 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.617 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.618 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.618 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.618 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.618 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.618 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.618 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.618 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.619 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.619 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.620 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.620 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-05-21T15:26:42.620 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.620 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.620 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.621 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-05-21T15:26:42.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-05-21T15:26:42.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T15:26:42.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.622 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:42.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:42.623 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.623 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.623 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.623 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-05-21T15:26:42.623 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.623 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.624 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.624 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.624 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.625 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.625 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.625 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.625 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.626 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.626 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.626 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.626 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.626 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.626 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.627 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.627 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.627 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.627 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.627 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.628 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.628 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.629 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.630 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.630 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.630 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.630 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.630 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.631 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.631 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.631 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.631 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.631 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.631 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.632 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.633 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.634 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.634 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.634 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-05-21T15:26:42.634 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.635 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.635 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.635 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-05-21T15:26:42.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-05-21T15:26:42.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.636 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T15:26:42.636 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.636 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.637 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-05-21T15:26:42.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-05-21T15:26:42.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T15:26:42.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.639 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-05-21T15:26:42.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-05-21T15:26:42.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.639 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.639 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.639 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.640 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T15:26:42.640 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.640 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.640 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.641 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-05-21T15:26:42.641 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-05-21T15:26:42.641 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.641 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.641 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.642 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.642 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.642 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.642 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T15:26:42.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.643 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-05-21T15:26:42.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-05-21T15:26:42.643 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.643 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.644 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.644 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.644 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T15:26:42.644 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.644 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.644 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.645 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-05-21T15:26:42.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-05-21T15:26:42.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.645 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.646 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.646 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.647 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-05-21T15:26:42.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-05-21T15:26:42.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.649 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-05-21T15:26:42.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-05-21T15:26:42.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T15:26:42.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.650 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.651 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-05-21T15:26:42.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-05-21T15:26:42.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.651 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.652 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.652 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.652 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.653 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:42.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:42.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.653 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-05-21T15:26:42.653 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-05-21T15:26:42.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.655 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-05-21T15:26:42.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-05-21T15:26:42.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.655 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-05-21T15:26:42.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.656 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.656 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.656 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.656 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-05-21T15:26:42.657 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-05-21T15:26:42.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.657 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.658 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.658 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.658 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-05-21T15:26:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-05-21T15:26:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.660 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.661 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.661 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.661 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.662 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.662 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.662 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.662 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.662 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.662 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.662 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.663 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.664 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.664 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.664 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.664 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.665 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.666 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.667 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.667 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.667 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.667 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.667 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.667 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.668 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.668 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.668 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.669 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.669 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.669 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.669 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.670 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.670 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-05-21T15:26:42.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-05-21T15:26:42.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-05-21T15:26:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-05-21T15:26:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.674 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-05-21T15:26:42.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-05-21T15:26:42.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T15:26:42.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.675 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.676 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.676 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.676 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-05-21T15:26:42.676 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.676 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.677 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.677 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.677 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.677 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.677 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.677 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.677 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.678 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.678 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.678 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.678 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.678 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.678 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.678 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.679 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.679 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.679 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.679 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.680 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.681 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.681 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.682 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.683 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.683 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.683 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.683 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.683 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.683 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.683 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-05-21T15:26:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.687 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.688 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.693 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.694 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-05-21T15:26:42.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-05-21T15:26:42.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.697 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.697 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-05-21T15:26:42.697 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.698 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.698 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T15:26:42.698 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.698 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.698 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-05-21T15:26:42.699 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-05-21T15:26:42.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.699 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.699 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.699 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.699 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-05-21T15:26:42.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.700 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.700 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-05-21T15:26:42.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-05-21T15:26:42.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.701 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:42.701 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.701 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.701 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T15:26:42.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.702 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-05-21T15:26:42.702 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T15:26:42.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.703 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.704 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T15:26:42.704 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.704 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-05-21T15:26:42.705 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.705 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.705 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.705 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-05-21T15:26:42.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-05-21T15:26:42.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:42.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.706 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-05-21T15:26:42.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.706 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.707 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-05-21T15:26:42.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-05-21T15:26:42.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.709 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:42.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:42.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.711 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.711 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.712 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.712 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.712 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.712 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.712 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.713 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.713 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.713 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.714 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.714 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.716 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.716 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.717 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.717 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.717 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.717 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.717 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.717 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.718 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.718 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.718 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.719 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.720 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.720 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.721 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-05-21T15:26:42.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-05-21T15:26:42.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.723 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-05-21T15:26:42.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-05-21T15:26:42.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-05-21T15:26:42.724 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.724 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.724 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.724 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.724 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.725 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-05-21T15:26:42.725 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-05-21T15:26:42.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.725 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T15:26:42.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.727 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.727 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.727 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.727 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:42.727 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.727 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.727 INFO:teuthology.orchestra.run.smithi078.stdout: "modules": [ 2024-05-21T15:26:42.727 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-05-21T15:26:42.728 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard", 2024-05-21T15:26:42.728 INFO:teuthology.orchestra.run.smithi078.stdout: "iostat", 2024-05-21T15:26:42.728 INFO:teuthology.orchestra.run.smithi078.stdout: "nfs", 2024-05-21T15:26:42.728 INFO:teuthology.orchestra.run.smithi078.stdout: "restful", 2024-05-21T15:26:42.728 INFO:teuthology.orchestra.run.smithi078.stdout: "snap_schedule" 2024-05-21T15:26:42.728 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.728 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-05-21T15:26:42.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-05-21T15:26:42.729 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.729 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.729 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.729 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:42.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:42.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:42.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.731 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T15:26:42.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.731 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.731 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.733 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.733 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.733 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.733 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.733 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.733 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.733 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.734 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.734 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.734 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.735 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.736 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.737 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.737 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.737 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.737 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.738 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.738 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.738 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.738 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.738 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.739 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.740 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.740 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.740 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.741 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-05-21T15:26:42.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-05-21T15:26:42.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.742 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.742 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-05-21T15:26:42.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.743 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.743 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-05-21T15:26:42.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-05-21T15:26:42.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-05-21T15:26:42.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-05-21T15:26:42.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.745 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-05-21T15:26:42.745 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-05-21T15:26:42.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.746 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.746 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.746 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-05-21T15:26:42.746 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.747 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-05-21T15:26:42.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-05-21T15:26:42.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.748 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.748 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.748 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-05-21T15:26:42.748 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.748 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.748 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.749 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-05-21T15:26:42.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-05-21T15:26:42.749 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.749 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.749 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-05-21T15:26:42.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.750 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-05-21T15:26:42.750 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.750 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.750 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-05-21T15:26:42.751 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-05-21T15:26:42.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.751 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.752 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.752 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-05-21T15:26:42.752 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.752 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-05-21T15:26:42.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-05-21T15:26:42.753 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.753 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.753 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.753 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.753 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.753 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.754 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T15:26:42.754 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.754 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.754 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.754 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-05-21T15:26:42.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-05-21T15:26:42.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.755 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-05-21T15:26:42.756 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.756 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.756 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.756 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.756 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-05-21T15:26:42.757 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.757 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.757 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.757 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-05-21T15:26:42.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-05-21T15:26:42.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.757 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.758 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T15:26:42.759 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.759 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-05-21T15:26:42.759 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-05-21T15:26:42.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-05-21T15:26:42.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.760 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T15:26:42.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:26:42.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.761 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-05-21T15:26:42.761 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-05-21T15:26:42.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:42.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:42.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-05-21T15:26:42.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-05-21T15:26:42.762 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.762 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T15:26:42.762 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:26:42.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.763 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.763 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-05-21T15:26:42.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-05-21T15:26:42.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:42.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.764 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-05-21T15:26:42.764 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:42.764 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-05-21T15:26:42.764 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.764 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T15:26:42.764 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.764 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.765 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.765 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-05-21T15:26:42.765 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-05-21T15:26:42.765 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.765 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T15:26:42.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.766 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.766 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T15:26:42.766 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T15:26:42.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.767 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-05-21T15:26:42.767 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-05-21T15:26:42.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:42.767 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-05-21T15:26:42.768 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-05-21T15:26:42.768 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-05-21T15:26:42.768 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-05-21T15:26:42.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T15:26:42.768 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.768 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.769 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-05-21T15:26:42.769 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-05-21T15:26:42.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-05-21T15:26:42.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T15:26:42.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T15:26:42.770 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.770 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.771 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-05-21T15:26:42.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-05-21T15:26:42.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:42.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.771 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:42.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-05-21T15:26:42.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-05-21T15:26:42.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.772 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T15:26:42.772 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T15:26:42.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.772 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.773 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.773 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.773 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.773 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.773 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.774 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.774 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.774 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.774 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.774 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.775 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.775 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.775 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.775 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.775 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.775 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.776 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.776 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.776 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.777 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.777 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.777 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.777 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.778 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.778 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.778 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.778 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.778 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.779 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.779 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.779 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.779 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.779 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.779 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.779 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.780 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.780 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.780 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.780 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.780 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.780 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.782 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.782 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.782 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.782 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.782 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.782 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-05-21T15:26:42.783 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-05-21T15:26:42.783 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:42.783 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.783 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:42.783 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T15:26:42.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.784 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-05-21T15:26:42.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-05-21T15:26:42.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-05-21T15:26:42.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.786 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.786 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-05-21T15:26:42.786 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-05-21T15:26:42.786 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-05-21T15:26:42.786 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-05-21T15:26:42.786 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-05-21T15:26:42.786 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.787 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-05-21T15:26:42.787 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.787 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.787 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.787 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-05-21T15:26:42.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-05-21T15:26:42.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.788 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.788 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T15:26:42.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.789 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.789 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:42.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:42.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.789 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.790 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.790 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:42.790 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.790 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.790 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.790 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T15:26:42.790 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.791 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.791 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-05-21T15:26:42.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-05-21T15:26:42.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.792 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.792 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:42.792 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:42.792 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.792 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.792 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T15:26:42.792 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T15:26:42.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.793 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-05-21T15:26:42.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-05-21T15:26:42.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:42.793 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.794 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.794 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.794 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.794 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.794 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T15:26:42.794 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-05-21T15:26:42.795 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.796 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.796 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.796 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-05-21T15:26:42.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-05-21T15:26:42.796 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:42.796 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.796 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.797 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-05-21T15:26:42.797 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.797 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.797 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.797 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T15:26:42.797 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.797 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.798 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-05-21T15:26:42.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-05-21T15:26:42.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.798 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-05-21T15:26:42.799 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T15:26:42.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.800 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-05-21T15:26:42.800 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-05-21T15:26:42.800 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.800 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.800 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-05-21T15:26:42.801 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.801 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.801 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.801 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T15:26:42.801 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.801 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.802 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-05-21T15:26:42.802 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-05-21T15:26:42.802 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.802 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.802 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.802 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.802 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.803 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.803 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T15:26:42.803 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T15:26:42.803 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.803 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.804 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-05-21T15:26:42.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-05-21T15:26:42.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.804 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:42.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.805 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T15:26:42.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.805 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.805 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-05-21T15:26:42.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T15:26:42.806 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:42.806 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.806 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.806 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-05-21T15:26:42.806 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T15:26:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-05-21T15:26:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-05-21T15:26:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.809 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-05-21T15:26:42.809 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-05-21T15:26:42.809 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-05-21T15:26:42.809 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.809 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T15:26:42.809 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.810 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-05-21T15:26:42.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-05-21T15:26:42.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.810 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T15:26:42.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.811 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-05-21T15:26:42.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-05-21T15:26:42.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.812 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.812 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.812 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.813 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T15:26:42.813 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.813 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.813 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-05-21T15:26:42.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-05-21T15:26:42.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.814 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.814 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.814 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.814 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T15:26:42.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.815 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-05-21T15:26:42.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-05-21T15:26:42.815 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T15:26:42.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:42.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.817 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-05-21T15:26:42.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-05-21T15:26:42.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T15:26:42.818 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.818 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-05-21T15:26:42.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.819 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-05-21T15:26:42.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-05-21T15:26:42.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T15:26:42.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-05-21T15:26:42.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.821 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-05-21T15:26:42.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-05-21T15:26:42.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T15:26:42.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:42.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.823 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-05-21T15:26:42.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-05-21T15:26:42.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T15:26:42.824 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-05-21T15:26:42.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.825 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-05-21T15:26:42.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-05-21T15:26:42.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T15:26:42.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-05-21T15:26:42.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.827 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.827 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-05-21T15:26:42.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-05-21T15:26:42.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.828 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T15:26:42.828 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.828 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.828 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-05-21T15:26:42.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.829 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-05-21T15:26:42.829 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-05-21T15:26:42.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.830 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T15:26:42.830 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.830 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.830 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-05-21T15:26:42.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.831 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.831 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-05-21T15:26:42.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-05-21T15:26:42.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.832 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T15:26:42.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-05-21T15:26:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-05-21T15:26:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-05-21T15:26:42.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.834 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-21T15:26:42.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.834 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-05-21T15:26:42.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.835 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-05-21T15:26:42.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-05-21T15:26:42.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T15:26:42.836 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.837 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.837 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:42.837 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.837 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.837 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.837 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-05-21T15:26:42.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-05-21T15:26:42.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-21T15:26:42.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.839 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-05-21T15:26:42.839 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.839 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.839 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-05-21T15:26:42.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-05-21T15:26:42.839 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.840 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.840 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.840 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T15:26:42.840 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.840 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.840 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-05-21T15:26:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-05-21T15:26:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-05-21T15:26:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.842 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-21T15:26:42.842 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.842 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-05-21T15:26:42.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.843 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-05-21T15:26:42.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-05-21T15:26:42.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.844 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T15:26:42.844 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.844 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.844 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-05-21T15:26:42.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.845 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-05-21T15:26:42.845 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-05-21T15:26:42.845 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.845 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T15:26:42.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-05-21T15:26:42.846 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.846 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.847 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-05-21T15:26:42.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-05-21T15:26:42.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.848 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.848 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.848 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.848 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T15:26:42.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.849 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-05-21T15:26:42.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-05-21T15:26:42.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:42.850 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.850 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.850 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T15:26:42.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-05-21T15:26:42.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T15:26:42.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-05-21T15:26:42.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.852 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.852 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T15:26:42.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.852 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-05-21T15:26:42.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-05-21T15:26:42.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-05-21T15:26:42.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.854 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T15:26:42.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.854 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-05-21T15:26:42.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-05-21T15:26:42.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.855 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-05-21T15:26:42.855 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.856 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-05-21T15:26:42.856 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.856 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.856 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.856 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-05-21T15:26:42.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-05-21T15:26:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T15:26:42.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.858 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-05-21T15:26:42.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-05-21T15:26:42.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.859 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.859 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:42.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.859 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-05-21T15:26:42.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.860 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-05-21T15:26:42.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-05-21T15:26:42.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.861 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.861 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T15:26:42.861 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.861 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T15:26:42.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.862 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-05-21T15:26:42.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-05-21T15:26:42.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.863 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:42.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.863 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.863 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-05-21T15:26:42.863 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.863 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.864 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-05-21T15:26:42.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-05-21T15:26:42.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.865 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T15:26:42.865 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.865 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.865 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.866 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-05-21T15:26:42.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-05-21T15:26:42.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.866 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.866 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.866 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T15:26:42.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.870 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.871 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.871 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.871 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.871 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.871 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.871 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.871 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.872 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.872 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.872 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.873 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-05-21T15:26:42.873 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-05-21T15:26:42.873 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.873 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.873 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.874 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.874 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T15:26:42.874 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.874 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.875 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.875 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.876 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.876 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T15:26:42.876 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.876 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.876 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.876 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.877 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.877 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.877 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.877 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.878 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.878 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.878 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.878 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.878 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.878 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.878 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.879 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.879 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.879 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.879 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.879 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.879 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.880 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.880 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.880 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.880 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.881 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.881 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.881 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T15:26:42.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T15:26:42.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.882 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.882 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.882 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.882 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.882 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.882 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.882 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T15:26:42.883 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.883 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.883 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T15:26:42.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T15:26:42.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-05-21T15:26:42.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T15:26:42.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.884 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.885 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.885 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-05-21T15:26:42.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-05-21T15:26:42.885 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.885 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.885 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.886 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.886 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-05-21T15:26:42.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-05-21T15:26:42.887 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-05-21T15:26:42.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.888 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.888 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.888 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T15:26:42.888 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.888 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.888 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.889 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-05-21T15:26:42.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-05-21T15:26:42.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.890 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-05-21T15:26:42.890 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.891 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-05-21T15:26:42.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-05-21T15:26:42.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.891 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.891 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-05-21T15:26:42.891 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.891 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.892 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.892 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-05-21T15:26:42.892 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-05-21T15:26:42.892 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.892 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T15:26:42.892 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.892 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.893 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-05-21T15:26:42.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-05-21T15:26:42.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.893 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.893 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-05-21T15:26:42.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T15:26:42.894 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.894 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.895 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-05-21T15:26:42.895 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-05-21T15:26:42.895 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.895 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.895 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.895 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T15:26:42.896 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.896 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T15:26:42.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.897 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-05-21T15:26:42.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-05-21T15:26:42.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.898 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.898 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.898 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T15:26:42.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.899 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-05-21T15:26:42.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-05-21T15:26:42.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.900 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.900 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.900 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T15:26:42.900 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-05-21T15:26:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-05-21T15:26:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.902 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.902 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.902 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T15:26:42.902 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.902 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.903 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-05-21T15:26:42.903 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-05-21T15:26:42.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.903 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.903 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.903 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.903 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.904 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.904 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.904 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T15:26:42.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.905 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-05-21T15:26:42.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-05-21T15:26:42.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.905 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.906 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T15:26:42.906 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.906 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.906 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.906 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-05-21T15:26:42.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-05-21T15:26:42.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-05-21T15:26:42.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.907 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.908 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.908 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-05-21T15:26:42.908 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.908 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-05-21T15:26:42.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-05-21T15:26:42.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.910 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T15:26:42.910 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.910 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.910 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.910 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-05-21T15:26:42.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-05-21T15:26:42.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T15:26:42.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.912 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-05-21T15:26:42.912 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-05-21T15:26:42.912 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.913 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.913 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.913 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T15:26:42.914 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.914 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.914 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-05-21T15:26:42.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-05-21T15:26:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T15:26:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-05-21T15:26:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-05-21T15:26:42.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T15:26:42.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-05-21T15:26:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-05-21T15:26:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T15:26:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-05-21T15:26:42.920 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.922 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.924 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.925 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:42.927 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.929 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.930 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.930 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.930 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.931 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-05-21T15:26:42.931 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-05-21T15:26:42.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-05-21T15:26:42.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T15:26:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-05-21T15:26:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-05-21T15:26:42.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:42.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:42.933 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-05-21T15:26:42.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T15:26:42.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:42.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:42.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-05-21T15:26:42.935 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:42.935 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:42.935 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:42.936 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T15:26:42.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T15:26:42.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:42.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:42.937 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.937 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.937 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.938 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T15:26:42.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T15:26:42.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.939 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.939 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.940 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T15:26:42.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T15:26:42.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.942 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-05-21T15:26:42.942 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T15:26:42.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.943 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.943 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.944 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.944 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T15:26:42.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T15:26:42.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.945 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.946 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T15:26:42.946 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T15:26:42.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.947 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.947 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.947 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.948 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T15:26:42.948 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T15:26:42.948 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.950 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.950 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.950 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T15:26:42.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T15:26:42.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.951 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.951 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.952 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.952 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T15:26:42.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T15:26:42.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.952 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.953 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.953 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.953 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.953 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.954 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T15:26:42.954 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T15:26:42.954 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.955 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.955 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.955 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.955 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.956 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T15:26:42.956 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T15:26:42.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.958 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T15:26:42.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T15:26:42.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.958 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.958 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.959 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.959 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.960 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T15:26:42.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T15:26:42.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.961 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.962 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T15:26:42.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T15:26:42.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.963 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.964 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T15:26:42.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T15:26:42.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:42.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.966 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T15:26:42.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T15:26:42.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.967 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.968 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-05-21T15:26:42.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-05-21T15:26:42.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.970 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T15:26:42.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T15:26:42.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:42.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.972 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T15:26:42.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T15:26:42.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.974 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T15:26:42.974 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T15:26:42.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.975 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.976 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T15:26:42.976 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T15:26:42.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.978 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.978 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.978 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.978 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T15:26:42.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T15:26:42.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.980 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T15:26:42.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T15:26:42.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:42.981 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:42.981 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.981 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.981 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.982 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.982 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T15:26:42.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T15:26:42.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.983 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.984 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.984 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.984 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T15:26:42.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T15:26:42.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.984 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.985 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.985 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.985 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.986 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.986 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T15:26:42.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T15:26:42.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.986 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.987 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.987 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.987 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.987 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.988 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T15:26:42.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T15:26:42.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.990 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.990 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T15:26:42.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T15:26:42.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.991 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.991 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.991 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.991 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.991 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.992 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.992 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T15:26:42.992 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T15:26:42.992 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.993 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.993 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.993 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.993 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.993 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.993 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.994 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.994 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T15:26:42.994 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T15:26:42.994 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.994 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T15:26:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T15:26:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T15:26:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T15:26:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T15:26:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T15:26:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T15:26:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T15:26:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T15:26:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T15:26:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T15:26:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-05-21T15:26:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T15:26:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T15:26:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-05-21T15:26:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T15:26:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T15:26:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T15:26:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T15:26:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-05-21T15:26:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T15:26:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T15:26:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T15:26:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T15:26:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T15:26:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T15:26:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T15:26:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T15:26:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T15:26:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T15:26:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-05-21T15:26:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-05-21T15:26:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-05-21T15:26:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-05-21T15:26:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-05-21T15:26:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-05-21T15:26:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-05-21T15:26:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-05-21T15:26:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-05-21T15:26:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-05-21T15:26:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-05-21T15:26:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-05-21T15:26:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-05-21T15:26:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-05-21T15:26:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-05-21T15:26:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-05-21T15:26:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T15:26:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-05-21T15:26:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-05-21T15:26:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-05-21T15:26:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-05-21T15:26:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-05-21T15:26:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-05-21T15:26:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-05-21T15:26:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-05-21T15:26:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-05-21T15:26:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-05-21T15:26:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-05-21T15:26:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-05-21T15:26:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-05-21T15:26:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-05-21T15:26:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-05-21T15:26:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-05-21T15:26:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-05-21T15:26:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-05-21T15:26:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-05-21T15:26:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-05-21T15:26:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-05-21T15:26:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-05-21T15:26:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-05-21T15:26:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-05-21T15:26:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-05-21T15:26:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T15:26:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-05-21T15:26:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-05-21T15:26:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-05-21T15:26:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T15:26:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-05-21T15:26:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-05-21T15:26:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-05-21T15:26:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-05-21T15:26:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-05-21T15:26:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-05-21T15:26:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-05-21T15:26:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T15:26:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-05-21T15:26:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-05-21T15:26:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T15:26:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T15:26:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-05-21T15:26:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-05-21T15:26:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-05-21T15:26:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T15:26:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-05-21T15:26:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-05-21T15:26:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-05-21T15:26:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-05-21T15:26:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-05-21T15:26:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-05-21T15:26:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-05-21T15:26:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-05-21T15:26:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-05-21T15:26:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-05-21T15:26:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-05-21T15:26:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-05-21T15:26:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-05-21T15:26:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-05-21T15:26:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T15:26:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-05-21T15:26:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-05-21T15:26:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-05-21T15:26:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-05-21T15:26:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-05-21T15:26:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-05-21T15:26:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:43.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.110 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.110 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.110 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-05-21T15:26:43.110 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-05-21T15:26:43.110 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.110 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.111 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T15:26:43.111 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.111 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.111 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.111 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.112 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.112 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.112 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.112 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.112 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.112 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.113 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.113 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.113 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.113 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.113 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.113 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.114 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.114 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.115 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.115 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.115 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.115 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.116 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.116 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.116 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.116 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.116 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.116 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.117 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.117 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.117 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.118 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.118 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.118 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.118 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.118 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.118 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.118 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.119 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.119 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.119 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.119 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.119 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.119 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.120 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.120 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.120 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.120 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.121 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.121 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.121 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-05-21T15:26:43.121 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-05-21T15:26:43.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.122 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.122 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.122 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.122 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.122 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-05-21T15:26:43.123 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.123 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.123 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.123 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-05-21T15:26:43.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-05-21T15:26:43.123 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.123 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.124 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-05-21T15:26:43.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.124 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.124 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-05-21T15:26:43.124 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.124 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.125 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-05-21T15:26:43.125 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-05-21T15:26:43.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.125 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.125 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-05-21T15:26:43.126 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.126 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.126 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.126 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:26:43.126 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.127 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-05-21T15:26:43.127 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-05-21T15:26:43.127 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.127 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.127 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.127 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:43.128 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:43.128 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-05-21T15:26:43.128 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.128 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T15:26:43.128 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.128 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.128 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.129 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-05-21T15:26:43.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-05-21T15:26:43.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.129 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.129 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.129 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.130 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.130 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.130 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-05-21T15:26:43.130 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.130 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.130 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.131 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-05-21T15:26:43.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-05-21T15:26:43.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.131 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.131 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-05-21T15:26:43.131 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.131 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.132 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T15:26:43.132 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.132 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.132 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.132 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.132 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-05-21T15:26:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.133 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.134 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.134 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.134 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.135 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.135 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.135 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.135 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.135 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.135 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.135 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.136 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.136 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.136 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.136 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.136 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.136 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.137 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.137 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.137 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.137 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.137 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.138 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.138 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.138 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.138 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.138 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.139 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.139 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.139 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.139 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.139 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.139 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.139 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.140 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.140 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.140 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.140 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.140 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.140 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.140 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.141 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.141 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.141 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.141 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.141 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.142 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.142 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.142 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.142 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.142 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.142 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.143 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.143 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.143 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.143 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.143 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-05-21T15:26:43.144 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.144 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.144 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.144 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.145 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.146 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.146 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.146 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.146 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.146 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.146 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.147 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.147 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.147 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.147 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.147 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.148 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.148 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.148 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.148 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.148 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.149 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.149 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.149 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.149 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.150 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.150 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.150 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.150 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.150 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.151 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.151 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.151 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.151 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.151 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.151 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.151 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.152 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.152 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.153 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-05-21T15:26:43.155 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.155 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.155 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.155 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-05-21T15:26:43.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-05-21T15:26:43.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.156 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-05-21T15:26:43.156 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.156 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.156 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T15:26:43.157 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.157 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.157 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-05-21T15:26:43.157 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-05-21T15:26:43.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.158 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.158 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.158 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:43.158 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-05-21T15:26:43.158 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T15:26:43.159 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.159 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.159 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.159 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.159 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.160 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.160 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.160 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.160 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.160 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.160 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.161 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.161 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.161 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.161 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.161 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.161 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.162 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.162 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.162 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.163 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.163 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.163 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.163 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.163 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.163 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.164 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.164 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.164 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.165 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.165 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.165 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.166 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.166 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.166 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.166 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.166 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.166 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.167 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.167 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.167 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.167 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.167 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.168 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.168 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.168 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.169 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-05-21T15:26:43.169 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.169 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.170 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.170 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-05-21T15:26:43.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-05-21T15:26:43.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.170 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.170 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.170 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-05-21T15:26:43.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.171 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-05-21T15:26:43.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T15:26:43.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.171 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.172 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.172 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.172 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.172 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.173 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.173 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.173 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.173 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.173 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.173 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.174 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.174 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.174 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.174 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.174 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.175 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.175 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.175 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.176 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.176 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.176 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.176 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.176 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.177 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.177 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.178 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.178 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.178 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.178 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.178 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.178 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.178 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.179 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.179 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.179 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.180 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.180 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.180 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.181 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.181 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-05-21T15:26:43.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-05-21T15:26:43.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.182 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.182 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.182 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.182 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.182 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.183 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-05-21T15:26:43.183 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T15:26:43.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.183 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.183 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-05-21T15:26:43.183 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-05-21T15:26:43.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-05-21T15:26:43.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.184 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.184 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-05-21T15:26:43.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.185 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-05-21T15:26:43.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-05-21T15:26:43.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.186 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.186 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.186 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-05-21T15:26:43.186 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.186 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.186 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.186 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T15:26:43.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.187 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-05-21T15:26:43.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-05-21T15:26:43.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.187 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-05-21T15:26:43.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.189 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.189 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-05-21T15:26:43.189 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-05-21T15:26:43.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.189 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-05-21T15:26:43.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.190 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T15:26:43.190 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T15:26:43.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.191 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.191 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.191 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-05-21T15:26:43.192 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.192 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.192 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.192 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.192 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.192 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.193 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.193 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.194 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.194 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.194 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.194 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.194 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.194 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.194 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.195 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.195 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.195 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.196 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.196 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.196 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.196 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.196 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.196 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.197 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.197 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.197 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.197 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.198 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.198 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.198 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.198 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.198 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.198 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.198 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.199 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.199 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.199 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.199 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.199 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.199 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.199 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.200 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.201 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.202 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.202 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.202 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.202 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-05-21T15:26:43.203 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.203 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.203 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.203 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.204 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.204 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.204 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.205 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.205 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.205 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.205 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.205 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.205 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.206 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.206 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.206 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.206 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.207 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.207 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.207 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.207 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.207 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.207 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.208 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.208 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.208 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.208 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.208 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.208 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.208 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.209 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.209 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.209 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.209 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.209 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.209 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.209 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.210 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.210 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.210 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.210 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.210 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.210 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.211 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.211 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.211 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.211 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.211 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.212 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.212 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.212 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-05-21T15:26:43.213 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.213 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.213 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.214 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.214 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.214 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.215 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.215 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.215 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.215 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.215 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.216 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.216 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.216 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.216 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.216 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.217 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.217 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.217 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.217 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.217 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.217 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.218 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.218 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.218 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.218 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.218 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.219 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.219 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.219 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.219 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.219 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.219 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.219 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.219 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.220 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.220 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.220 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.220 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.220 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.220 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.220 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.221 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.221 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.221 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.221 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.221 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.221 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.222 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.222 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.222 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.222 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.222 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.222 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.222 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.223 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.223 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.223 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-05-21T15:26:43.224 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.224 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.224 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.224 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-05-21T15:26:43.225 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-05-21T15:26:43.225 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.225 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.225 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.225 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.225 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.225 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.226 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.226 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T15:26:43.226 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.226 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.226 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.226 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.227 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.227 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.228 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.228 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.228 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.228 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.228 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.228 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.228 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.228 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.229 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.229 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.229 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.229 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.229 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.229 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.231 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.231 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.231 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.232 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.232 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.233 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.233 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.233 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.233 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.233 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.233 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.233 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.234 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.234 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.234 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.234 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.235 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.235 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.235 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.235 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.235 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.235 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.235 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.236 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.236 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-05-21T15:26:43.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-05-21T15:26:43.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.236 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.237 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.237 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.237 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.237 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.237 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-05-21T15:26:43.237 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-05-21T15:26:43.238 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-05-21T15:26:43.238 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-05-21T15:26:43.238 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.238 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.238 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.239 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-05-21T15:26:43.239 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.239 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.239 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.239 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.240 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.240 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.240 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.240 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.241 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.241 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.241 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.241 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.241 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.241 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.241 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.242 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.242 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.242 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.242 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.242 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.243 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.243 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.243 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.243 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.244 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.244 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.244 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.244 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.244 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.245 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.245 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.245 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.246 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.246 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.246 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.246 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.246 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.246 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.246 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.247 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.247 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.247 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.249 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-05-21T15:26:43.250 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.250 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.250 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.250 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.251 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.251 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.251 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.251 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.251 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.252 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.252 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.252 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.252 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.252 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.252 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.252 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.252 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.253 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.253 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.253 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.254 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.254 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.254 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.254 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.255 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.255 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.255 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.255 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.256 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.256 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.256 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.256 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.256 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.256 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.256 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.258 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.258 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.258 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.259 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.259 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.259 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.259 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.259 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.260 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-05-21T15:26:43.260 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.260 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.260 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.260 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.261 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.261 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.262 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.262 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.262 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.262 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.262 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.262 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.263 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.263 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.263 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.263 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.265 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.265 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.266 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.266 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.267 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.267 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.267 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.267 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.267 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.267 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.268 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.268 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.268 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.268 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.268 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.270 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.270 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.270 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.270 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:43.270 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:43.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.271 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.271 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:43.271 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.271 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.271 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.271 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.271 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.272 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.272 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.272 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-05-21T15:26:43.272 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-05-21T15:26:43.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:43.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.273 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-05-21T15:26:43.273 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-05-21T15:26:43.273 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.273 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-05-21T15:26:43.273 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T15:26:43.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-05-21T15:26:43.274 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.274 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.275 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.275 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-05-21T15:26:43.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-05-21T15:26:43.275 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.275 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.275 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.275 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.276 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.276 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T15:26:43.276 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.276 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.277 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-05-21T15:26:43.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-05-21T15:26:43.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.278 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.278 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.278 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.279 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.279 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.279 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.279 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.279 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.279 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.279 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.279 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.280 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.280 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.280 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.280 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.280 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.280 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.280 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.281 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.281 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.281 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.281 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.281 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.281 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.282 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.282 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.282 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.282 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.283 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.283 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.283 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.284 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.284 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.284 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.284 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.284 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.284 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.284 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.285 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.285 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.285 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.285 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.285 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.285 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.286 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.286 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.286 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.286 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.286 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.286 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.286 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.287 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.287 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.287 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.287 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.287 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-05-21T15:26:43.288 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-05-21T15:26:43.288 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.288 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-05-21T15:26:43.288 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.289 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-05-21T15:26:43.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.289 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-05-21T15:26:43.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-05-21T15:26:43.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.290 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-05-21T15:26:43.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.290 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-05-21T15:26:43.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.291 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.291 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-05-21T15:26:43.292 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.292 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.292 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.292 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-05-21T15:26:43.292 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-05-21T15:26:43.292 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.293 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.293 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.293 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.293 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.293 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.293 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.294 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.294 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-05-21T15:26:43.294 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-05-21T15:26:43.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.294 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.294 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.295 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.295 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T15:26:43.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T15:26:43.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.296 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.296 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.296 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.296 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.296 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.297 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.297 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.297 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.297 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.297 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.297 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.297 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.297 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.298 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.298 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.298 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.298 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.299 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.299 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.299 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.299 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.300 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.300 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.300 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.300 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.301 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.301 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.301 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.301 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.301 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.301 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.302 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.302 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.302 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.302 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.302 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.302 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.302 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.302 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.303 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.303 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.303 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.303 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.303 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.303 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.304 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.304 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.304 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.304 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.304 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.305 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-05-21T15:26:43.305 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-05-21T15:26:43.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.305 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.306 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.306 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.306 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.307 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.307 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T15:26:43.307 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T15:26:43.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.307 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-05-21T15:26:43.308 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.308 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.308 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.309 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-05-21T15:26:43.309 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-05-21T15:26:43.309 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:43.309 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-05-21T15:26:43.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.310 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.310 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.310 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.310 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.310 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:43.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:43.311 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.311 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.311 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.311 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-05-21T15:26:43.311 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.311 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.312 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T15:26:43.312 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.312 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.312 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.312 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:43.312 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:43.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.313 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.313 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-05-21T15:26:43.313 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.313 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.313 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.313 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T15:26:43.314 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.314 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.314 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-05-21T15:26:43.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-05-21T15:26:43.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.315 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.315 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.315 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-05-21T15:26:43.315 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.315 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.315 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.315 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.316 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.316 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-05-21T15:26:43.316 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-05-21T15:26:43.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.316 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.317 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.317 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-05-21T15:26:43.317 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.317 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.317 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.317 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-05-21T15:26:43.317 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-05-21T15:26:43.318 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.318 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.318 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.318 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-05-21T15:26:43.318 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-05-21T15:26:43.319 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.319 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.319 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.319 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-05-21T15:26:43.319 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-05-21T15:26:43.319 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-05-21T15:26:43.319 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.320 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.320 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.320 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.320 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.320 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.320 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-05-21T15:26:43.321 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.321 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.321 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.321 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.321 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.322 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.322 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.323 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.323 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.323 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.323 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.323 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.323 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.323 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.324 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.324 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.324 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.324 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.325 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.325 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.325 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.325 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.325 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.326 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.326 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.326 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.327 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.327 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.327 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.327 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.327 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.327 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.329 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.329 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.329 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.329 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.329 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.329 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.330 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.330 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.330 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.330 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-05-21T15:26:43.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-05-21T15:26:43.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-05-21T15:26:43.331 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.331 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.331 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.332 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.332 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.332 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.332 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-05-21T15:26:43.332 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T15:26:43.333 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.333 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.333 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.333 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.334 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-05-21T15:26:43.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-05-21T15:26:43.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.335 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.335 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.335 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.335 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.335 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.335 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.335 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.336 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.336 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-05-21T15:26:43.337 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.337 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.337 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-05-21T15:26:43.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-05-21T15:26:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.339 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.339 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.339 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.339 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-05-21T15:26:43.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-05-21T15:26:43.339 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.340 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.340 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.340 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.340 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.341 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.341 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.341 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.342 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.342 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.342 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.342 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.342 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.342 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.342 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.342 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.343 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.343 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.343 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.343 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.343 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.344 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.345 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.345 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.345 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.346 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.346 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.346 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.346 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.346 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.347 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.347 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.347 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.347 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.347 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.348 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.348 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.348 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.348 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.348 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.348 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.349 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.349 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.349 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.350 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-05-21T15:26:43.351 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-05-21T15:26:43.351 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.351 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.351 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-05-21T15:26:43.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.353 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.353 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.354 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.354 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.354 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-05-21T15:26:43.355 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.355 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.355 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.355 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.355 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.355 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.355 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.356 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.356 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.356 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.356 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.356 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.357 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.357 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.357 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.357 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.357 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.357 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.357 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.357 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.358 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.358 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.358 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.358 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.360 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.361 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.361 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.361 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.361 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.361 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.361 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.361 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.361 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.362 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.362 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.362 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.362 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.362 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.363 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.363 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.363 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.363 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.364 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.364 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.364 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-05-21T15:26:43.365 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.365 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.365 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.365 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.366 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.367 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.367 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.367 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.367 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.367 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.367 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.368 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.368 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.369 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.369 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.369 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.369 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.369 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.369 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.370 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.370 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.370 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.370 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.370 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.371 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.371 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.371 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.371 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.371 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.371 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.371 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.372 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.372 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.372 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.372 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.372 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.372 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.372 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.373 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.373 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.373 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.373 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.373 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.374 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.374 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.374 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.374 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.375 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.375 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.375 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-05-21T15:26:43.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-05-21T15:26:43.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.376 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.376 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-05-21T15:26:43.376 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.376 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.376 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T15:26:43.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.377 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.377 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.377 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.377 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-05-21T15:26:43.377 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.378 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.378 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.378 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.378 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.378 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.378 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.378 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.379 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.379 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.379 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.380 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.380 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.380 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.380 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.380 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.380 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.380 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.381 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.381 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.381 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.381 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.382 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.382 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.382 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.382 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.383 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.383 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.383 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.384 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.384 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.384 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.384 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.385 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.385 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.385 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.385 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.386 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.387 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.387 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.387 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.387 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.388 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-05-21T15:26:43.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-05-21T15:26:43.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.389 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.389 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.389 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.390 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-05-21T15:26:43.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-05-21T15:26:43.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.390 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-05-21T15:26:43.390 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.391 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.392 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-05-21T15:26:43.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-05-21T15:26:43.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.392 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.392 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.392 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.393 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.393 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.393 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.393 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-05-21T15:26:43.394 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-05-21T15:26:43.394 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.394 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.394 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.395 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.395 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.395 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.395 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.395 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.395 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-05-21T15:26:43.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-05-21T15:26:43.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-05-21T15:26:43.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.397 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.398 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-05-21T15:26:43.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-05-21T15:26:43.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.398 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.399 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.399 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.399 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.399 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.400 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-05-21T15:26:43.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-05-21T15:26:43.400 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.400 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.401 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.401 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.401 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.401 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.401 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.402 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-05-21T15:26:43.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-05-21T15:26:43.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.402 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.402 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.402 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.403 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.403 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.403 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-05-21T15:26:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-05-21T15:26:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-05-21T15:26:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-05-21T15:26:43.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.405 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.405 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.405 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.405 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.405 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-05-21T15:26:43.406 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-05-21T15:26:43.406 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.406 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.406 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.406 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.406 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.406 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.407 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.407 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.407 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.407 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.407 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.407 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-05-21T15:26:43.407 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-05-21T15:26:43.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.408 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.408 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.408 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.408 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.408 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.408 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.408 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.409 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.409 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.409 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.409 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-05-21T15:26:43.409 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-05-21T15:26:43.409 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.410 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.410 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.410 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.410 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.410 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.410 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.411 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.411 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.411 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.411 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.411 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.412 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-05-21T15:26:43.412 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.412 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.412 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.412 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-05-21T15:26:43.412 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-05-21T15:26:43.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.412 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.413 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.413 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.413 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.413 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T15:26:43.413 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.414 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.414 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.414 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-05-21T15:26:43.414 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-05-21T15:26:43.414 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.414 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.415 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.415 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.415 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.415 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.415 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.415 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-05-21T15:26:43.415 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.416 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.416 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.416 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.417 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.417 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.417 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.418 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.418 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.418 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.418 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.418 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.419 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.420 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.420 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.420 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.420 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.421 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.421 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.421 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.421 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.422 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.422 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.422 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.423 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.423 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.423 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.423 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.424 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.424 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.424 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.424 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.424 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.425 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.425 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.425 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.425 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.425 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.425 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.426 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.426 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.426 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.426 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-05-21T15:26:43.427 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.427 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.427 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.427 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.427 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.427 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.427 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.427 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.428 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.428 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.428 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.428 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.428 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.429 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.429 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.429 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.429 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.429 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.430 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.430 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.430 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.431 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.431 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.431 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.432 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.432 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.433 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.433 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.433 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.433 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.433 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.433 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.434 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.434 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.434 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.434 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.435 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.435 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.435 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.435 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.436 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.436 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.436 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.436 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.436 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.437 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-05-21T15:26:43.437 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.437 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.437 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.437 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.438 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.438 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.438 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.438 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.438 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.438 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.438 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.439 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.439 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.439 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.439 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.439 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.439 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.439 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.440 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.440 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.440 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.440 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.440 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.440 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.441 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.442 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.442 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.442 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.443 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.443 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.443 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.443 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.443 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.444 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.444 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.444 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.444 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.444 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.444 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.444 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.445 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.445 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.446 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.446 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.446 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.446 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.447 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.447 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.447 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-05-21T15:26:43.447 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.448 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.448 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.448 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-05-21T15:26:43.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-05-21T15:26:43.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.448 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.448 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T15:26:43.449 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.449 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.449 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.449 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.449 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.449 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.449 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.450 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:43.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:43.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.450 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.450 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-05-21T15:26:43.451 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.451 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.451 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.451 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.451 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.451 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.452 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.452 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.452 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.452 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.452 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.452 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.453 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.453 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.453 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.453 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.453 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.453 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.453 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.453 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.454 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.454 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.454 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.454 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.455 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.455 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.455 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.456 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.458 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.458 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.458 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.458 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.458 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.458 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.458 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.459 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.459 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.459 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.460 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.460 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.460 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.460 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.460 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.460 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.461 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.461 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.462 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-05-21T15:26:43.462 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.462 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.462 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.462 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-05-21T15:26:43.462 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-05-21T15:26:43.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.463 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.463 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.463 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.463 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.463 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.463 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T15:26:43.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.464 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-05-21T15:26:43.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-05-21T15:26:43.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.465 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T15:26:43.465 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.466 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-05-21T15:26:43.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-05-21T15:26:43.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.466 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.467 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.467 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T15:26:43.467 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.468 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-05-21T15:26:43.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-05-21T15:26:43.468 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.468 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.468 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.469 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T15:26:43.469 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.469 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.469 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.470 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-05-21T15:26:43.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-05-21T15:26:43.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.470 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.470 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.470 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.471 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T15:26:43.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.471 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.472 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-05-21T15:26:43.472 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-05-21T15:26:43.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.472 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.472 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.472 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.473 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-05-21T15:26:43.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-05-21T15:26:43.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.474 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.475 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.475 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.475 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.475 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-05-21T15:26:43.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-05-21T15:26:43.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.476 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T15:26:43.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.476 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.477 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.477 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.477 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-05-21T15:26:43.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-05-21T15:26:43.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.479 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.479 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:43.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:43.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.479 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.480 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-05-21T15:26:43.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-05-21T15:26:43.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.480 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.480 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.481 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.481 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-05-21T15:26:43.481 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-05-21T15:26:43.481 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.481 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.481 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.482 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-05-21T15:26:43.482 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.482 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.482 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.482 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.482 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.483 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.483 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-05-21T15:26:43.483 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-05-21T15:26:43.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.485 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.485 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.486 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.486 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.486 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.486 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.486 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.487 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-05-21T15:26:43.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-05-21T15:26:43.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.487 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.488 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.488 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.488 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.489 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.489 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.489 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.489 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.489 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.489 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.490 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.490 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.490 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.490 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.490 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.490 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.490 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.490 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.491 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.491 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.491 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.491 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.491 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.492 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.492 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.492 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.492 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.492 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.493 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.493 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.493 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.494 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.494 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.494 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.494 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.494 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.495 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.496 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.496 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.496 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.496 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.496 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.497 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.497 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.497 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.497 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.497 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.498 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-05-21T15:26:43.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-05-21T15:26:43.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.498 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.498 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.498 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.498 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-05-21T15:26:43.499 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.500 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.501 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T15:26:43.502 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.502 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.502 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.502 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.502 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.502 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.502 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.503 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.503 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-05-21T15:26:43.503 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.503 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.503 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.504 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.504 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.504 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.504 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.504 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.505 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.505 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.505 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.505 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.505 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.505 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.505 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.506 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.506 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.506 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.506 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.507 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.509 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.509 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.509 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.509 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.509 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.510 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.510 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.510 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.510 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.510 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.510 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.512 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.513 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.513 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.513 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.513 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.513 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.514 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.514 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.514 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.514 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.514 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.514 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.514 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.514 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.515 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.515 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.515 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.515 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.515 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.518 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.518 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.518 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.518 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.518 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.519 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.520 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.521 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.521 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.521 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-05-21T15:26:43.521 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-05-21T15:26:43.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.521 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.522 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-05-21T15:26:43.522 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.522 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.522 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.522 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T15:26:43.522 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.523 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-05-21T15:26:43.523 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-05-21T15:26:43.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.523 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.524 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.524 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-05-21T15:26:43.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.525 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-05-21T15:26:43.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-05-21T15:26:43.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.525 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.526 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-05-21T15:26:43.526 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.526 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T15:26:43.526 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-05-21T15:26:43.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.528 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T15:26:43.528 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.528 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.528 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.529 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.529 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-05-21T15:26:43.529 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-05-21T15:26:43.529 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-05-21T15:26:43.529 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-05-21T15:26:43.530 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-05-21T15:26:43.530 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-05-21T15:26:43.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-05-21T15:26:43.530 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.530 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-05-21T15:26:43.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.531 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.531 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.531 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.532 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-05-21T15:26:43.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-05-21T15:26:43.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.532 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.532 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.532 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.533 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.533 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.533 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-05-21T15:26:43.533 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-05-21T15:26:43.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-05-21T15:26:43.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-05-21T15:26:43.534 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.534 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.534 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.536 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.537 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-05-21T15:26:43.538 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-05-21T15:26:43.538 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.538 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.538 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.538 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.538 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-05-21T15:26:43.540 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-05-21T15:26:43.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-05-21T15:26:43.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-05-21T15:26:43.542 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-05-21T15:26:43.542 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-05-21T15:26:43.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-05-21T15:26:43.543 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.544 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.544 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-05-21T15:26:43.544 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.544 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.544 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.544 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.545 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.545 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-05-21T15:26:43.545 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-05-21T15:26:43.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-05-21T15:26:43.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-05-21T15:26:43.547 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-05-21T15:26:43.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-05-21T15:26:43.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-05-21T15:26:43.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-05-21T15:26:43.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-05-21T15:26:43.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.548 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.549 INFO:teuthology.orchestra.run.smithi078.stdout: "services": { 2024-05-21T15:26:43.549 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard": "https://172.21.15.78:8443/" 2024-05-21T15:26:43.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.549 INFO:teuthology.orchestra.run.smithi078.stdout: "always_on_modules": { 2024-05-21T15:26:43.549 INFO:teuthology.orchestra.run.smithi078.stdout: "octopus": [ 2024-05-21T15:26:43.549 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-05-21T15:26:43.549 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-05-21T15:26:43.549 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-05-21T15:26:43.550 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-05-21T15:26:43.550 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-05-21T15:26:43.550 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-05-21T15:26:43.550 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-05-21T15:26:43.550 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-05-21T15:26:43.550 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-05-21T15:26:43.551 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-05-21T15:26:43.551 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.551 INFO:teuthology.orchestra.run.smithi078.stdout: "pacific": [ 2024-05-21T15:26:43.551 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-05-21T15:26:43.551 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-05-21T15:26:43.551 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-05-21T15:26:43.551 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-05-21T15:26:43.551 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-05-21T15:26:43.552 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-05-21T15:26:43.552 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-05-21T15:26:43.552 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-05-21T15:26:43.552 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-05-21T15:26:43.552 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-05-21T15:26:43.552 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.552 INFO:teuthology.orchestra.run.smithi078.stdout: "quincy": [ 2024-05-21T15:26:43.553 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-05-21T15:26:43.553 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-05-21T15:26:43.553 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-05-21T15:26:43.553 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-05-21T15:26:43.553 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-05-21T15:26:43.553 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-05-21T15:26:43.553 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-05-21T15:26:43.553 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-05-21T15:26:43.554 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-05-21T15:26:43.554 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-05-21T15:26:43.554 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.554 INFO:teuthology.orchestra.run.smithi078.stdout: "reef": [ 2024-05-21T15:26:43.554 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-05-21T15:26:43.554 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-05-21T15:26:43.554 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-05-21T15:26:43.555 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-05-21T15:26:43.555 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-05-21T15:26:43.555 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-05-21T15:26:43.555 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-05-21T15:26:43.555 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-05-21T15:26:43.555 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-05-21T15:26:43.555 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-05-21T15:26:43.555 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-05-21T15:26:43.556 INFO:teuthology.orchestra.run.smithi078.stdout: "squid": [ 2024-05-21T15:26:43.556 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-05-21T15:26:43.556 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-05-21T15:26:43.556 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-05-21T15:26:43.556 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-05-21T15:26:43.556 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-05-21T15:26:43.556 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: "last_failure_osd_epoch": 88, 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: "active_clients": [ 2024-05-21T15:26:43.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-05-21T15:26:43.558 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-05-21T15:26:43.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-05-21T15:26:43.558 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-05-21T15:26:43.558 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 924437614 2024-05-21T15:26:43.558 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.558 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "libcephsqlite", 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 481727339 2024-05-21T15:26:43.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-05-21T15:26:43.560 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 422731806 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-05-21T15:26:43.561 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-05-21T15:26:43.562 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1463812226 2024-05-21T15:26:43.562 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.562 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:43.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-05-21T15:26:43.563 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-05-21T15:26:43.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-05-21T15:26:43.563 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-05-21T15:26:43.563 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 309518786 2024-05-21T15:26:43.563 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.563 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:43.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:43.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-05-21T15:26:43.564 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-05-21T15:26:43.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-05-21T15:26:43.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-05-21T15:26:43.564 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-05-21T15:26:43.564 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1288105135 2024-05-21T15:26:43.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.565 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:43.565 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:43.565 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-05-21T15:26:43.565 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:43.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:40 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/733452537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:40 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3015452812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:40 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3995275996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:41 smithi172 ceph-mon[36402]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:43.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:41 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3284699295' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T15:26:43.567 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:40 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/733452537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.568 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:40 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3015452812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.568 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:40 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3995275996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.568 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:41 smithi084 ceph-mon[36334]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:43.568 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:41 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3284699295' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T15:26:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f35d40a2d80 msgr2=0x7f35d40a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f35d40a2d80 0x7f35d40a5290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35d0005c40 tx=0x7f35d0005bd0 comp rx=0 tx=0).stop 2024-05-21T15:26:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81079d0 msgr2=0x7f35e8198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:26:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81079d0 0x7f35e8198b30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f35d800d740 tx=0x7f35d800dc00 comp rx=0 tx=0).stop 2024-05-21T15:26:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 shutdown_connections 2024-05-21T15:26:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f35d40a2d80 0x7f35d40a5290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e810c330 0x7f35e819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81079d0 0x7f35e8198b30 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 --2- 172.21.15.78:0/3284699295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35e8101aa0 0x7f35e81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:26:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 >> 172.21.15.78:0/3284699295 conn(0x7f35e80770c0 msgr2=0x7f35e8103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:26:43.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 shutdown_connections 2024-05-21T15:26:43.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:26:41.065+0000 7f35eebd5640 1 -- 172.21.15.78:0/3284699295 wait complete. 2024-05-21T15:26:43.577 DEBUG:tasks.check_counter:Getting stats from a 2024-05-21T15:26:43.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-21T15:26:43.578 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:40 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/733452537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.578 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:40 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3015452812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.578 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:40 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3995275996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:26:43.578 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:41 smithi078 ceph-mon[31997]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:43.578 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:41 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3284699295' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T15:26:43.833 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:26:44.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:43 smithi078 ceph-mon[31997]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:44.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:43 smithi084 ceph-mon[36334]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:44.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:43 smithi172 ceph-mon[36402]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:45.626 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:45.627 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:26:45.627 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1546, 2024-05-21T15:26:45.627 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1548, 2024-05-21T15:26:45.628 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1668317, 2024-05-21T15:26:45.628 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 491629, 2024-05-21T15:26:45.628 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 6, 2024-05-21T15:26:45.628 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-05-21T15:26:45.628 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.148686716, 2024-05-21T15:26:45.628 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.045190958, 2024-05-21T15:26:45.629 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.041650230, 2024-05-21T15:26:45.629 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.048989351, 2024-05-21T15:26:45.629 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:45.629 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1546, 2024-05-21T15:26:45.629 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.057727374, 2024-05-21T15:26:45.629 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000037339 2024-05-21T15:26:45.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.629 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:45.630 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.630 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.630 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.630 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:26:45.630 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:26:45.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.631 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:26:45.631 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 635, 2024-05-21T15:26:45.631 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 790, 2024-05-21T15:26:45.631 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 156690, 2024-05-21T15:26:45.631 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 643906, 2024-05-21T15:26:45.631 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 7, 2024-05-21T15:26:45.631 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-05-21T15:26:45.631 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.077320073, 2024-05-21T15:26:45.632 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.024485013, 2024-05-21T15:26:45.632 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.018048589, 2024-05-21T15:26:45.632 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.027121411, 2024-05-21T15:26:45.632 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:45.632 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 787, 2024-05-21T15:26:45.632 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.035540262, 2024-05-21T15:26:45.632 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000045159 2024-05-21T15:26:45.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.633 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:45.633 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.633 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.633 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.633 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T15:26:45.634 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 442544 2024-05-21T15:26:45.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.634 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:26:45.634 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 250, 2024-05-21T15:26:45.634 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 226, 2024-05-21T15:26:45.634 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1548908, 2024-05-21T15:26:45.634 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 129870, 2024-05-21T15:26:45.635 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-05-21T15:26:45.635 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-05-21T15:26:45.635 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.043145787, 2024-05-21T15:26:45.635 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.013033970, 2024-05-21T15:26:45.635 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.036901328, 2024-05-21T15:26:45.635 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.004252173, 2024-05-21T15:26:45.635 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:45.635 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 226, 2024-05-21T15:26:45.636 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.013715783, 2024-05-21T15:26:45.636 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000060689 2024-05-21T15:26:45.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.636 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:45.636 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.636 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.636 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.637 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1548640, 2024-05-21T15:26:45.637 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 129552 2024-05-21T15:26:45.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.637 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-05-21T15:26:45.637 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-05-21T15:26:45.638 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-05-21T15:26:45.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.638 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-05-21T15:26:45.638 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-05-21T15:26:45.638 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-05-21T15:26:45.638 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3573, 2024-05-21T15:26:45.639 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.108655395, 2024-05-21T15:26:45.639 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000030410 2024-05-21T15:26:45.639 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.639 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-05-21T15:26:45.639 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-05-21T15:26:45.640 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-05-21T15:26:45.640 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.640 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.640 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.640 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.640 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-05-21T15:26:45.641 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-05-21T15:26:45.641 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-05-21T15:26:45.641 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-05-21T15:26:45.641 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.641 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.641 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.642 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-05-21T15:26:45.642 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-05-21T15:26:45.642 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-05-21T15:26:45.642 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-05-21T15:26:45.642 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-05-21T15:26:45.642 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-05-21T15:26:45.643 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-05-21T15:26:45.643 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 111, 2024-05-21T15:26:45.643 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-05-21T15:26:45.643 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 111, 2024-05-21T15:26:45.643 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-05-21T15:26:45.643 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 111, 2024-05-21T15:26:45.643 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-05-21T15:26:45.644 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-05-21T15:26:45.644 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-05-21T15:26:45.644 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 4, 2024-05-21T15:26:45.644 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-05-21T15:26:45.644 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-05-21T15:26:45.644 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-05-21T15:26:45.645 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-05-21T15:26:45.645 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:26:45.645 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-05-21T15:26:45.645 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-05-21T15:26:45.645 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-05-21T15:26:45.645 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-05-21T15:26:45.645 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-05-21T15:26:45.646 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-05-21T15:26:45.646 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-05-21T15:26:45.646 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-05-21T15:26:45.646 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-05-21T15:26:45.646 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:26:45.646 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-05-21T15:26:45.646 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-05-21T15:26:45.647 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-05-21T15:26:45.647 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-05-21T15:26:45.647 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:26:45.647 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-05-21T15:26:45.647 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:26:45.647 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:26:45.648 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:26:45.648 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:26:45.648 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:26:45.648 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-05-21T15:26:45.648 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:26:45.648 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:26:45.648 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:26:45.649 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:26:45.649 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:26:45.649 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:26:45.649 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-05-21T15:26:45.649 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:26:45.649 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-05-21T15:26:45.649 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-05-21T15:26:45.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.650 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-05-21T15:26:45.650 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-05-21T15:26:45.650 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-05-21T15:26:45.650 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-05-21T15:26:45.650 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-05-21T15:26:45.651 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-05-21T15:26:45.651 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-05-21T15:26:45.651 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-05-21T15:26:45.651 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:26:45.651 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-05-21T15:26:45.652 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-05-21T15:26:45.652 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-05-21T15:26:45.652 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-05-21T15:26:45.652 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-05-21T15:26:45.652 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:26:45.652 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:26:45.653 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-05-21T15:26:45.653 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-05-21T15:26:45.653 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:26:45.653 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:26:45.653 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:26:45.654 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-05-21T15:26:45.654 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-05-21T15:26:45.654 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:26:45.654 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-05-21T15:26:45.654 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-05-21T15:26:45.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.655 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-05-21T15:26:45.655 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-05-21T15:26:45.655 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-05-21T15:26:45.656 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-05-21T15:26:45.656 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-05-21T15:26:45.656 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 962, 2024-05-21T15:26:45.656 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-05-21T15:26:45.657 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-05-21T15:26:45.657 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-05-21T15:26:45.657 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-05-21T15:26:45.657 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-05-21T15:26:45.657 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 2, 2024-05-21T15:26:45.658 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-05-21T15:26:45.658 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-05-21T15:26:45.658 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-05-21T15:26:45.658 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-05-21T15:26:45.658 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 5403724, 2024-05-21T15:26:45.659 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 5403150, 2024-05-21T15:26:45.659 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-05-21T15:26:45.659 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.659 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.659 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.660 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 962 2024-05-21T15:26:45.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.660 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-05-21T15:26:45.660 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 114, 2024-05-21T15:26:45.660 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 114, 2024-05-21T15:26:45.661 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 0, 2024-05-21T15:26:45.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 11, 2024-05-21T15:26:45.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 11, 2024-05-21T15:26:45.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 0, 2024-05-21T15:26:45.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 111, 2024-05-21T15:26:45.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 111, 2024-05-21T15:26:45.661 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 0, 2024-05-21T15:26:45.662 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-05-21T15:26:45.662 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-05-21T15:26:45.662 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-05-21T15:26:45.662 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 120624, 2024-05-21T15:26:45.662 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215444 2024-05-21T15:26:45.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.663 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-05-21T15:26:45.663 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-05-21T15:26:45.663 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-05-21T15:26:45.663 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-05-21T15:26:45.663 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-05-21T15:26:45.663 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-05-21T15:26:45.664 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-05-21T15:26:45.664 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.664 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.664 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.664 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-05-21T15:26:45.665 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.665 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.665 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.665 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-05-21T15:26:45.665 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.666 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.666 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.666 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-05-21T15:26:45.666 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.667 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.667 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.667 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-05-21T15:26:45.667 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.667 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.667 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.668 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-05-21T15:26:45.668 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.668 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.668 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.669 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-05-21T15:26:45.669 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.669 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.669 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.669 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-05-21T15:26:45.670 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.670 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.670 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.670 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-05-21T15:26:45.670 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.671 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.671 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.671 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-05-21T15:26:45.671 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.672 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.672 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.672 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-05-21T15:26:45.672 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.672 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.672 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.673 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-05-21T15:26:45.674 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.674 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.674 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.675 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-05-21T15:26:45.675 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.675 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.675 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.676 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-05-21T15:26:45.676 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.676 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.677 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-05-21T15:26:45.677 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.677 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.678 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.678 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-05-21T15:26:45.678 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.678 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.679 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.679 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-05-21T15:26:45.679 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.679 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.679 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.680 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-05-21T15:26:45.680 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.680 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.680 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.680 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-05-21T15:26:45.681 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.681 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.681 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.681 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-05-21T15:26:45.681 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.682 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.682 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.682 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-05-21T15:26:45.682 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.682 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.683 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.683 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-05-21T15:26:45.683 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.683 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.683 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.684 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-05-21T15:26:45.684 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.684 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.684 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.684 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-05-21T15:26:45.685 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.685 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.685 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.685 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-05-21T15:26:45.686 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.687 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.687 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.687 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-05-21T15:26:45.687 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.687 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.687 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.688 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.689 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-05-21T15:26:45.689 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.689 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.689 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.689 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:26:45.689 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:26:45.690 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-05-21T15:26:45.690 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.690 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.690 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.690 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-05-21T15:26:45.690 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 2, 2024-05-21T15:26:45.691 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 3, 2024-05-21T15:26:45.691 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 1, 2024-05-21T15:26:45.691 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 3, 2024-05-21T15:26:45.691 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-05-21T15:26:45.691 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-05-21T15:26:45.691 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-05-21T15:26:45.691 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 2320, 2024-05-21T15:26:45.691 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-05-21T15:26:45.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.692 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-05-21T15:26:45.692 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:26:45.692 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-05-21T15:26:45.692 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:26:45.693 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:26:45.693 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:26:45.693 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:26:45.693 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:26:45.693 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:26:45.693 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:26:45.693 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:26:45.694 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:26:45.694 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:26:45.694 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:26:45.694 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:26:45.694 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:26:45.694 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-05-21T15:26:45.694 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:26:45.694 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-05-21T15:26:45.695 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:26:45.695 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:26:45.695 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:26:45.695 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:26:45.695 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:26:45.695 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:26:45.696 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:26:45.696 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-05-21T15:26:45.696 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:26:45.696 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:26:45.697 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:26:45.697 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-05-21T15:26:45.697 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-05-21T15:26:45.697 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-05-21T15:26:45.697 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:26:45.697 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:26:45.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:26:45.698 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:26:45.698 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 193520, 2024-05-21T15:26:45.698 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 163, 2024-05-21T15:26:45.698 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:26:45.698 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-05-21T15:26:45.698 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-05-21T15:26:45.698 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8512, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 177, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:26:45.699 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-05-21T15:26:45.700 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-05-21T15:26:45.700 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 354491, 2024-05-21T15:26:45.700 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2967, 2024-05-21T15:26:45.700 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-05-21T15:26:45.700 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-05-21T15:26:45.700 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-05-21T15:26:45.700 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-05-21T15:26:45.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.701 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-05-21T15:26:45.701 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-05-21T15:26:45.701 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-05-21T15:26:45.701 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 2180, 2024-05-21T15:26:45.701 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-05-21T15:26:45.701 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-05-21T15:26:45.701 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 2180, 2024-05-21T15:26:45.701 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-05-21T15:26:45.702 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2180, 2024-05-21T15:26:45.702 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 8.108883401, 2024-05-21T15:26:45.702 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.003719671 2024-05-21T15:26:45.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.702 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-05-21T15:26:45.702 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-05-21T15:26:45.702 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2180, 2024-05-21T15:26:45.702 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2181 2024-05-21T15:26:45.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.703 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 2180, 2024-05-21T15:26:45.703 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 2180, 2024-05-21T15:26:45.703 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-05-21T15:26:45.703 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-05-21T15:26:45.703 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-05-21T15:26:45.704 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 1371, 2024-05-21T15:26:45.704 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-05-21T15:26:45.704 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 807, 2024-05-21T15:26:45.704 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-05-21T15:26:45.704 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-05-21T15:26:45.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-05-21T15:26:45.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-05-21T15:26:45.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-05-21T15:26:45.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-05-21T15:26:45.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-05-21T15:26:45.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-05-21T15:26:45.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-05-21T15:26:45.707 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-05-21T15:26:45.707 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-05-21T15:26:45.707 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-05-21T15:26:45.707 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:26:45.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-05-21T15:26:45.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:26:45.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-05-21T15:26:45.708 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-05-21T15:26:45.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-05-21T15:26:45.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:26:45.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-05-21T15:26:45.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:26:45.710 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-05-21T15:26:45.710 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-05-21T15:26:45.710 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-05-21T15:26:45.711 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-05-21T15:26:45.711 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-05-21T15:26:45.711 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-05-21T15:26:45.711 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-05-21T15:26:45.712 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-05-21T15:26:45.712 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-05-21T15:26:45.712 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-05-21T15:26:45.712 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-05-21T15:26:45.713 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-05-21T15:26:45.713 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-05-21T15:26:45.713 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-05-21T15:26:45.714 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-05-21T15:26:45.714 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-05-21T15:26:45.714 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-05-21T15:26:45.714 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 98, 2024-05-21T15:26:45.715 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-05-21T15:26:45.715 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 13, 2024-05-21T15:26:45.715 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 5, 2024-05-21T15:26:45.716 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 5, 2024-05-21T15:26:45.716 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-05-21T15:26:45.716 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-05-21T15:26:45.716 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-05-21T15:26:45.716 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-05-21T15:26:45.717 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-05-21T15:26:45.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.717 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-05-21T15:26:45.717 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-05-21T15:26:45.717 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:26:45.717 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-05-21T15:26:45.717 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-05-21T15:26:45.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.718 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-05-21T15:26:45.718 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-05-21T15:26:45.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.718 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-05-21T15:26:45.718 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-05-21T15:26:45.718 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:45.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-05-21T15:26:45.720 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:45.720 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 2427, 2024-05-21T15:26:45.720 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 3139975, 2024-05-21T15:26:45.720 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:45.721 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 2427, 2024-05-21T15:26:45.721 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-05-21T15:26:45.721 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-05-21T15:26:45.721 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2427, 2024-05-21T15:26:45.721 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 3139975, 2024-05-21T15:26:45.721 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-05-21T15:26:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:45.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-05-21T15:26:45.723 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:45.723 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-05-21T15:26:45.723 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-05-21T15:26:45.723 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:45.723 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:45.723 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 2180, 2024-05-21T15:26:45.723 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 1209425, 2024-05-21T15:26:45.723 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2180, 2024-05-21T15:26:45.724 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1209425, 2024-05-21T15:26:45.724 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:45.724 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.724 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.724 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:45.725 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:45.726 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 2180, 2024-05-21T15:26:45.726 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 2180, 2024-05-21T15:26:45.726 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2180, 2024-05-21T15:26:45.726 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 2180, 2024-05-21T15:26:45.726 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:45.726 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.726 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.727 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.727 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.727 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:26:45.727 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:45.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-05-21T15:26:45.728 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:45.728 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-05-21T15:26:45.728 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-05-21T15:26:45.728 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:45.729 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:45.729 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-05-21T15:26:45.729 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-05-21T15:26:45.729 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-05-21T15:26:45.729 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-05-21T15:26:45.729 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:45.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.730 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.730 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:45.730 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x5648c227a9c0": { 2024-05-21T15:26:45.730 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:45.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-05-21T15:26:45.731 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:45.731 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-05-21T15:26:45.731 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-05-21T15:26:45.731 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:45.731 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:45.731 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-05-21T15:26:45.731 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-05-21T15:26:45.732 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-05-21T15:26:45.732 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-05-21T15:26:45.732 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:45.732 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:45.732 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:45.733 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:45.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:45.733 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:46.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:45 smithi078 ceph-mon[31997]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:46.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:45 smithi084 ceph-mon[36334]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:46.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:45 smithi172 ceph-mon[36402]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:46.461 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-21T15:26:46.461 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-21T15:26:46.461 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-21T15:26:46.462 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-21T15:26:46.462 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-21T15:26:46.462 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:26:46.462 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:26:46.462 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-05-21T15:26:46.462 DEBUG:tasks.check_counter:Getting stats from d 2024-05-21T15:26:46.463 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-21T15:26:46.705 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:26:48.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:47 smithi078 ceph-mon[31997]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:48.384 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:48.385 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:26:48.385 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 220, 2024-05-21T15:26:48.385 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 210, 2024-05-21T15:26:48.385 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1442528, 2024-05-21T15:26:48.385 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 129184, 2024-05-21T15:26:48.385 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-05-21T15:26:48.386 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-05-21T15:26:48.386 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.045093574, 2024-05-21T15:26:48.386 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.012861594, 2024-05-21T15:26:48.386 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.036370265, 2024-05-21T15:26:48.386 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.005033619, 2024-05-21T15:26:48.386 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:48.386 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 209, 2024-05-21T15:26:48.386 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.011498858, 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000055018 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1442528, 2024-05-21T15:26:48.387 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 129184 2024-05-21T15:26:48.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.388 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:26:48.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 6, 2024-05-21T15:26:48.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 163, 2024-05-21T15:26:48.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3344, 2024-05-21T15:26:48.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 184352, 2024-05-21T15:26:48.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-05-21T15:26:48.388 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.013754609, 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.009115221, 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.002364117, 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 161, 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.016594152, 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000103069 2024-05-21T15:26:48.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:48.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:48.390 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:48.390 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:48.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-05-21T15:26:48.390 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 184352 2024-05-21T15:26:48.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.000421394, 2024-05-21T15:26:48.391 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-05-21T15:26:48.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000337862, 2024-05-21T15:26:48.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T15:26:48.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:48.392 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:48.392 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:48.392 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:48.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.392 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-05-21T15:26:48.393 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-05-21T15:26:48.394 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-05-21T15:26:48.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.394 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-05-21T15:26:48.394 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:26:48.394 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-05-21T15:26:48.394 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:26:48.394 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:26:48.394 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:26:48.395 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:26:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:26:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-05-21T15:26:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:26:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-05-21T15:26:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:26:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:26:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:26:48.396 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:26:48.397 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:26:48.397 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:26:48.397 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:26:48.397 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-05-21T15:26:48.397 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:26:48.397 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:26:48.397 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:26:48.397 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-05-21T15:26:48.398 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-05-21T15:26:48.398 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-05-21T15:26:48.398 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:26:48.398 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:26:48.398 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:26:48.398 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:26:48.398 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148977, 2024-05-21T15:26:48.398 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2024-05-21T15:26:48.399 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:26:48.399 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-05-21T15:26:48.399 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-05-21T15:26:48.399 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-05-21T15:26:48.399 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:26:48.399 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-05-21T15:26:48.399 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:26:48.399 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-05-21T15:26:48.400 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-05-21T15:26:48.400 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-05-21T15:26:48.400 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:26:48.400 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:26:48.400 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-05-21T15:26:48.400 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-05-21T15:26:48.400 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-05-21T15:26:48.401 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-05-21T15:26:48.401 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-05-21T15:26:48.401 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-05-21T15:26:48.401 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-05-21T15:26:48.401 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-05-21T15:26:48.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:48.401 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:26:48.401 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:48.402 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-05-21T15:26:48.402 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:48.402 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 222, 2024-05-21T15:26:48.402 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 1411519, 2024-05-21T15:26:48.402 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:48.402 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 222, 2024-05-21T15:26:48.402 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-05-21T15:26:48.402 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 222, 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1411519, 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:48.403 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:48.404 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:47 smithi084 ceph-mon[36334]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:48.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:47 smithi172 ceph-mon[36402]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:48.993 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-21T15:26:48.994 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-21T15:26:48.994 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-21T15:26:48.994 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-21T15:26:48.994 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-21T15:26:48.994 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-21T15:26:48.994 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-21T15:26:48.994 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-21T15:26:48.994 DEBUG:tasks.check_counter:Getting stats from g 2024-05-21T15:26:48.995 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-21T15:26:49.234 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:26:50.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:49 smithi078 ceph-mon[31997]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:50.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:49 smithi084 ceph-mon[36334]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:50.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:49 smithi172 ceph-mon[36402]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:50.934 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:50.934 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:26:50.935 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 974, 2024-05-21T15:26:50.935 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 951, 2024-05-21T15:26:50.935 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1790514, 2024-05-21T15:26:50.935 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 361252, 2024-05-21T15:26:50.935 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-05-21T15:26:50.935 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-05-21T15:26:50.935 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.123634528, 2024-05-21T15:26:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.046409154, 2024-05-21T15:26:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.053601369, 2024-05-21T15:26:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.026145289, 2024-05-21T15:26:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 950, 2024-05-21T15:26:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.059475102, 2024-05-21T15:26:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000062605 2024-05-21T15:26:50.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1546160, 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 130384 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.937 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:26:50.938 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1404, 2024-05-21T15:26:50.938 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1560, 2024-05-21T15:26:50.938 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 997415, 2024-05-21T15:26:50.938 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 888613, 2024-05-21T15:26:50.938 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 7, 2024-05-21T15:26:50.938 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 5, 2024-05-21T15:26:50.938 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.105504887, 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.031979375, 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.028071704, 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.037060387, 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1557, 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.057946000, 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000037216 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.939 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:50.940 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.940 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.940 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.940 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T15:26:50.940 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 442720 2024-05-21T15:26:50.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.941 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:26:50.941 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 8, 2024-05-21T15:26:50.941 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 8, 2024-05-21T15:26:50.941 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 4653, 2024-05-21T15:26:50.941 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1732, 2024-05-21T15:26:50.941 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-05-21T15:26:50.941 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-05-21T15:26:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001716202, 2024-05-21T15:26:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000106508, 2024-05-21T15:26:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001471598, 2024-05-21T15:26:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000092960, 2024-05-21T15:26:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 7, 2024-05-21T15:26:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.002817368, 2024-05-21T15:26:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000402481 2024-05-21T15:26:50.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.943 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:50.943 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.943 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.943 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.943 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-05-21T15:26:50.944 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-05-21T15:26:50.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.944 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-05-21T15:26:50.944 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-05-21T15:26:50.944 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-05-21T15:26:50.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.944 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-05-21T15:26:50.944 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3621, 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.088680132, 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000024490 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-05-21T15:26:50.945 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-05-21T15:26:50.946 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.946 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.946 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:50.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.946 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-05-21T15:26:50.947 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-05-21T15:26:50.947 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-05-21T15:26:50.947 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-05-21T15:26:50.947 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.947 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.947 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.948 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-05-21T15:26:50.948 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-05-21T15:26:50.948 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-05-21T15:26:50.948 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-05-21T15:26:50.948 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-05-21T15:26:50.948 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-05-21T15:26:50.948 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-05-21T15:26:50.948 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 64, 2024-05-21T15:26:50.949 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-05-21T15:26:50.949 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 64, 2024-05-21T15:26:50.949 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-05-21T15:26:50.949 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 64, 2024-05-21T15:26:50.949 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-05-21T15:26:50.949 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-05-21T15:26:50.949 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-05-21T15:26:50.950 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 4, 2024-05-21T15:26:50.950 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-05-21T15:26:50.950 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-05-21T15:26:50.950 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-05-21T15:26:50.950 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-05-21T15:26:50.950 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:26:50.951 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-05-21T15:26:50.951 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-05-21T15:26:50.951 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-05-21T15:26:50.951 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-05-21T15:26:50.951 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-05-21T15:26:50.951 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-05-21T15:26:50.951 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-05-21T15:26:50.951 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-05-21T15:26:50.952 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-05-21T15:26:50.952 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:26:50.952 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-05-21T15:26:50.952 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-05-21T15:26:50.952 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-05-21T15:26:50.952 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-05-21T15:26:50.952 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:26:50.953 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-05-21T15:26:50.953 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:26:50.953 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:26:50.953 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:26:50.953 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:26:50.953 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:26:50.953 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-05-21T15:26:50.953 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:26:50.954 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:26:50.954 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:26:50.954 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:26:50.954 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:26:50.954 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:26:50.954 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-05-21T15:26:50.954 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:26:50.955 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-05-21T15:26:50.955 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-05-21T15:26:50.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.955 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-05-21T15:26:50.955 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-05-21T15:26:50.955 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-05-21T15:26:50.955 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-05-21T15:26:50.955 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-05-21T15:26:50.956 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-05-21T15:26:50.956 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-05-21T15:26:50.956 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-05-21T15:26:50.956 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:26:50.956 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-05-21T15:26:50.956 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-05-21T15:26:50.956 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-05-21T15:26:50.956 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-05-21T15:26:50.957 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-05-21T15:26:50.957 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:26:50.957 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:26:50.957 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-05-21T15:26:50.957 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-05-21T15:26:50.957 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:26:50.957 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:26:50.957 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:26:50.958 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-05-21T15:26:50.958 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-05-21T15:26:50.958 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:26:50.958 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-05-21T15:26:50.958 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-05-21T15:26:50.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.958 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-05-21T15:26:50.959 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-05-21T15:26:50.959 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-05-21T15:26:50.959 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-05-21T15:26:50.959 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-05-21T15:26:50.959 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 522, 2024-05-21T15:26:50.959 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-05-21T15:26:50.959 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 2, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-05-21T15:26:50.960 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 4834827, 2024-05-21T15:26:50.961 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 4834253, 2024-05-21T15:26:50.961 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-05-21T15:26:50.961 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.961 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.961 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.961 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 522 2024-05-21T15:26:50.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.962 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-05-21T15:26:50.962 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 67, 2024-05-21T15:26:50.962 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 67, 2024-05-21T15:26:50.962 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 0, 2024-05-21T15:26:50.962 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 11, 2024-05-21T15:26:50.962 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 11, 2024-05-21T15:26:50.962 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 0, 2024-05-21T15:26:50.962 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 64, 2024-05-21T15:26:50.963 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 64, 2024-05-21T15:26:50.963 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 0, 2024-05-21T15:26:50.963 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-05-21T15:26:50.963 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-05-21T15:26:50.963 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-05-21T15:26:50.963 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 121748, 2024-05-21T15:26:50.963 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215444 2024-05-21T15:26:50.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.964 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-05-21T15:26:50.964 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-05-21T15:26:50.964 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-05-21T15:26:50.964 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-05-21T15:26:50.964 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-05-21T15:26:50.964 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-05-21T15:26:50.964 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-05-21T15:26:50.964 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.965 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.965 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.965 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-05-21T15:26:50.965 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.965 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.965 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.966 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-05-21T15:26:50.966 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.966 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.966 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.966 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-05-21T15:26:50.966 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.966 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-05-21T15:26:50.967 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.968 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.968 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.968 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-05-21T15:26:50.968 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.968 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.968 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.969 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-05-21T15:26:50.969 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.969 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.969 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.969 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-05-21T15:26:50.969 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.969 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.970 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.970 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-05-21T15:26:50.970 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.970 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.970 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.970 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-05-21T15:26:50.971 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.971 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.971 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.971 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-05-21T15:26:50.971 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.973 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.973 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.973 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-05-21T15:26:50.973 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.973 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.973 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.974 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-05-21T15:26:50.974 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.974 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.974 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.974 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-05-21T15:26:50.974 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.974 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.975 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.975 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-05-21T15:26:50.975 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.975 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.975 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.975 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-05-21T15:26:50.976 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.976 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.976 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.976 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-05-21T15:26:50.976 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.976 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.976 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.977 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-05-21T15:26:50.977 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.977 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.977 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.977 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-05-21T15:26:50.977 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.978 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.978 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.978 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-05-21T15:26:50.978 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.978 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.978 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.979 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.980 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.980 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-05-21T15:26:50.980 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.980 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.980 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.981 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.982 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-05-21T15:26:50.982 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.982 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.982 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.982 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-05-21T15:26:50.982 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.983 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.983 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.983 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-05-21T15:26:50.983 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.983 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.983 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.984 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:26:50.984 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:26:50.984 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-05-21T15:26:50.984 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:50.984 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:50.984 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:50.985 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:50.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.985 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-05-21T15:26:50.985 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-05-21T15:26:50.985 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 3, 2024-05-21T15:26:50.985 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 2, 2024-05-21T15:26:50.985 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 3, 2024-05-21T15:26:50.985 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-05-21T15:26:50.986 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-05-21T15:26:50.986 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-05-21T15:26:50.986 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 4008, 2024-05-21T15:26:50.986 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-05-21T15:26:50.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.986 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-05-21T15:26:50.986 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:26:50.986 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-05-21T15:26:50.987 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:26:50.987 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:26:50.987 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:26:50.987 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:26:50.987 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:26:50.987 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:26:50.987 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:26:50.987 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:26:50.988 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:26:50.988 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:26:50.988 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:26:50.988 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:26:50.988 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:26:50.988 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-05-21T15:26:50.988 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:26:50.989 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-05-21T15:26:50.989 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:26:50.989 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:26:50.989 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:26:50.989 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:26:50.989 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:26:50.989 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:26:50.989 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:26:50.990 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-05-21T15:26:50.990 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:26:50.990 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:26:50.990 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:26:50.990 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-05-21T15:26:50.990 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-05-21T15:26:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-05-21T15:26:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:26:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:26:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:26:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:26:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 199804, 2024-05-21T15:26:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 118, 2024-05-21T15:26:50.991 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:26:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-05-21T15:26:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-05-21T15:26:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-05-21T15:26:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:26:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-05-21T15:26:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:26:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-05-21T15:26:50.992 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8512, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 177, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 214228, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 1739, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-05-21T15:26:50.993 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-05-21T15:26:50.994 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-05-21T15:26:50.994 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-05-21T15:26:50.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.994 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-05-21T15:26:50.994 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-05-21T15:26:50.994 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-05-21T15:26:50.994 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 2132, 2024-05-21T15:26:50.994 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-05-21T15:26:50.995 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-05-21T15:26:50.995 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 2132, 2024-05-21T15:26:50.995 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-05-21T15:26:50.995 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2132, 2024-05-21T15:26:50.995 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 30.498561479, 2024-05-21T15:26:50.995 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.014305141 2024-05-21T15:26:50.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.995 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2132, 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 2133 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 2132, 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 2132, 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-05-21T15:26:50.996 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 1345, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 785, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-05-21T15:26:50.997 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-05-21T15:26:50.998 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-05-21T15:26:50.998 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-05-21T15:26:50.998 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-05-21T15:26:50.998 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-05-21T15:26:50.998 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-05-21T15:26:50.998 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-05-21T15:26:50.998 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:26:50.998 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-05-21T15:26:50.999 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:26:50.999 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-05-21T15:26:50.999 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-05-21T15:26:50.999 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-05-21T15:26:50.999 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:26:50.999 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-05-21T15:26:50.999 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:26:50.999 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-05-21T15:26:51.000 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-05-21T15:26:51.001 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-05-21T15:26:51.001 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-05-21T15:26:51.001 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-05-21T15:26:51.001 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-05-21T15:26:51.001 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-05-21T15:26:51.001 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-05-21T15:26:51.001 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-05-21T15:26:51.001 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 98, 2024-05-21T15:26:51.002 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-05-21T15:26:51.002 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 13, 2024-05-21T15:26:51.002 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 6, 2024-05-21T15:26:51.002 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 6, 2024-05-21T15:26:51.002 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-05-21T15:26:51.002 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-05-21T15:26:51.002 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-05-21T15:26:51.002 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-05-21T15:26:51.003 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-05-21T15:26:51.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:51.003 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-05-21T15:26:51.003 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-05-21T15:26:51.003 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:26:51.003 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-05-21T15:26:51.003 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-05-21T15:26:51.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:51.004 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-05-21T15:26:51.004 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-05-21T15:26:51.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:51.004 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-05-21T15:26:51.004 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-05-21T15:26:51.004 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:26:51.004 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-05-21T15:26:51.004 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:51.005 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 2382, 2024-05-21T15:26:51.006 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 2562588, 2024-05-21T15:26:51.006 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:51.006 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 2382, 2024-05-21T15:26:51.006 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-05-21T15:26:51.006 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-05-21T15:26:51.006 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2382, 2024-05-21T15:26:51.006 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 2562588, 2024-05-21T15:26:51.006 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:51.007 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:51.007 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:51.007 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:51.007 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:51.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:51.007 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-05-21T15:26:51.007 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:51.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-05-21T15:26:51.008 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:51.008 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-05-21T15:26:51.008 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-05-21T15:26:51.008 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:51.008 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:51.008 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 2132, 2024-05-21T15:26:51.008 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 640528, 2024-05-21T15:26:51.008 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2132, 2024-05-21T15:26:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 640528, 2024-05-21T15:26:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:51.009 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-05-21T15:26:51.010 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:51.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-05-21T15:26:51.010 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:51.010 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-05-21T15:26:51.010 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-05-21T15:26:51.010 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:51.010 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:51.010 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 2132, 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 2132, 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 2132, 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 2132, 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:51.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:51.012 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:26:51.012 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:51.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-05-21T15:26:51.012 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:51.012 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-05-21T15:26:51.012 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-05-21T15:26:51.012 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:51.012 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:51.013 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-05-21T15:26:51.013 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-05-21T15:26:51.013 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-05-21T15:26:51.013 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-05-21T15:26:51.013 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:51.013 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:51.013 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:51.013 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:51.014 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:51.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:51.014 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x559ea0b449c0": { 2024-05-21T15:26:51.014 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:51.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-05-21T15:26:51.014 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:51.014 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-05-21T15:26:51.014 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:51.015 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:51.016 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:51.016 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:51.016 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:51.016 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:51.660 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-21T15:26:51.660 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-21T15:26:51.660 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-21T15:26:51.660 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-21T15:26:51.660 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-05-21T15:26:51.660 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:26:51.661 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:26:51.661 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-05-21T15:26:51.661 DEBUG:tasks.check_counter:Getting stats from j 2024-05-21T15:26:51.661 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-21T15:26:51.901 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T15:26:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:51 smithi078 ceph-mon[31997]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:52.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:51 smithi084 ceph-mon[36334]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:52.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:51 smithi172 ceph-mon[36402]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:53.507 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-05-21T15:26:53.507 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:26:53.507 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-05-21T15:26:53.507 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4, 2024-05-21T15:26:53.507 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3024, 2024-05-21T15:26:53.507 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 448, 2024-05-21T15:26:53.508 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-05-21T15:26:53.508 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-05-21T15:26:53.508 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001629296, 2024-05-21T15:26:53.508 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000032299, 2024-05-21T15:26:53.509 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000747209, 2024-05-21T15:26:53.509 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T15:26:53.509 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:53.509 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3, 2024-05-21T15:26:53.510 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000967639, 2024-05-21T15:26:53.510 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000322546 2024-05-21T15:26:53.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.510 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:53.511 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:53.511 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:53.511 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:53.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.511 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-05-21T15:26:53.511 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-05-21T15:26:53.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.512 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:26:53.512 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2, 2024-05-21T15:26:53.512 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 162, 2024-05-21T15:26:53.512 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 320, 2024-05-21T15:26:53.512 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 184080, 2024-05-21T15:26:53.512 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-05-21T15:26:53.514 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-05-21T15:26:53.514 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.013605855, 2024-05-21T15:26:53.514 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.009346480, 2024-05-21T15:26:53.514 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001879475, 2024-05-21T15:26:53.514 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T15:26:53.514 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:53.514 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 159, 2024-05-21T15:26:53.515 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.015779362, 2024-05-21T15:26:53.515 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000099241 2024-05-21T15:26:53.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.515 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:53.515 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:53.515 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:53.515 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:53.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.517 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T15:26:53.517 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 184080 2024-05-21T15:26:53.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.517 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:26:53.517 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 223, 2024-05-21T15:26:53.517 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 213, 2024-05-21T15:26:53.518 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1443408, 2024-05-21T15:26:53.518 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 130624, 2024-05-21T15:26:53.518 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-05-21T15:26:53.518 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-05-21T15:26:53.518 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.044955936, 2024-05-21T15:26:53.518 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.013376892, 2024-05-21T15:26:53.518 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.037479272, 2024-05-21T15:26:53.518 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.004487415, 2024-05-21T15:26:53.519 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:53.519 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 212, 2024-05-21T15:26:53.519 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.010000538, 2024-05-21T15:26:53.519 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000047172 2024-05-21T15:26:53.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.519 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:53.519 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:53.519 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:53.520 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:53.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.520 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 1443408, 2024-05-21T15:26:53.520 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 130624 2024-05-21T15:26:53.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.520 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-05-21T15:26:53.520 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-05-21T15:26:53.520 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:26:53.521 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:26:53.522 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:26:53.522 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:26:53.522 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:26:53.522 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:26:53.522 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:26:53.522 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:26:53.522 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:26:53.522 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:26:53.523 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-05-21T15:26:53.523 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:26:53.523 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-05-21T15:26:53.523 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:26:53.523 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:26:53.523 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:26:53.523 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:26:53.523 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:26:53.524 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:26:53.524 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:26:53.524 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-05-21T15:26:53.524 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:26:53.524 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:26:53.524 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:26:53.524 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-05-21T15:26:53.524 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148955, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:26:53.525 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-05-21T15:26:53.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-05-21T15:26:53.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-05-21T15:26:53.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:26:53.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-05-21T15:26:53.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:26:53.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-05-21T15:26:53.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-05-21T15:26:53.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-05-21T15:26:53.527 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:26:53.527 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:26:53.527 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-05-21T15:26:53.527 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-05-21T15:26:53.527 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-05-21T15:26:53.527 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-05-21T15:26:53.527 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-05-21T15:26:53.527 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 225, 2024-05-21T15:26:53.528 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 1411973, 2024-05-21T15:26:53.529 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:53.529 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 225, 2024-05-21T15:26:53.529 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-05-21T15:26:53.529 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-05-21T15:26:53.529 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 225, 2024-05-21T15:26:53.529 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1411973, 2024-05-21T15:26:53.529 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-05-21T15:26:53.529 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-05-21T15:26:53.530 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-05-21T15:26:53.530 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-05-21T15:26:53.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:53.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-05-21T15:26:53.530 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-05-21T15:26:54.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:54 smithi078 ceph-mon[31997]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:54.179 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-21T15:26:54.179 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-21T15:26:54.179 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-21T15:26:54.179 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-21T15:26:54.179 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-21T15:26:54.180 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-21T15:26:54.180 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-21T15:26:54.180 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-21T15:26:54.180 DEBUG:tasks.check_counter:Getting stats from b 2024-05-21T15:26:54.180 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-21T15:26:54.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:54 smithi084 ceph-mon[36334]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:54.426 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:26:54.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:54 smithi172 ceph-mon[36402]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:56.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:56 smithi078 ceph-mon[31997]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:56.356 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:56 smithi084 ceph-mon[36334]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:56.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:56 smithi172 ceph-mon[36402]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:26:56.507 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-21T15:26:56.508 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:26:56.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 3765, 2024-05-21T15:26:56.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 4348, 2024-05-21T15:26:56.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 2789110, 2024-05-21T15:26:56.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 6389951, 2024-05-21T15:26:56.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 14, 2024-05-21T15:26:56.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-05-21T15:26:56.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.355342927, 2024-05-21T15:26:56.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.117934062, 2024-05-21T15:26:56.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.154775235, 2024-05-21T15:26:56.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.069221333, 2024-05-21T15:26:56.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:56.509 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4347, 2024-05-21T15:26:56.509 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.773620485, 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000177966 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3770, 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000860741, 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000228 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1595920, 2024-05-21T15:26:56.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 484256 2024-05-21T15:26:56.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.511 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:26:56.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 12519, 2024-05-21T15:26:56.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 16059, 2024-05-21T15:26:56.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 4453594, 2024-05-21T15:26:56.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 11222267, 2024-05-21T15:26:56.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 34, 2024-05-21T15:26:56.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 1.026962422, 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.396160826, 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.734823078, 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.148106672, 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 16060, 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 4.755352487, 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000296099 2024-05-21T15:26:56.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.513 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:56.513 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 22079, 2024-05-21T15:26:56.513 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.005072539, 2024-05-21T15:26:56.513 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000229 2024-05-21T15:26:56.513 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.513 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 414832, 2024-05-21T15:26:56.513 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 242816 2024-05-21T15:26:56.513 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.514 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:26:56.514 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 5116, 2024-05-21T15:26:56.514 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 5971, 2024-05-21T15:26:56.514 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1752771, 2024-05-21T15:26:56.514 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 16658234, 2024-05-21T15:26:56.514 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 178, 2024-05-21T15:26:56.514 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 7, 2024-05-21T15:26:56.514 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.495184410, 2024-05-21T15:26:56.515 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.162963447, 2024-05-21T15:26:56.515 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.234881634, 2024-05-21T15:26:56.515 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.089664482, 2024-05-21T15:26:56.515 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:56.515 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5966, 2024-05-21T15:26:56.515 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 1.237714602, 2024-05-21T15:26:56.515 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000207461 2024-05-21T15:26:56.515 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.516 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:56.516 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 7494, 2024-05-21T15:26:56.516 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.001627250, 2024-05-21T15:26:56.516 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000217 2024-05-21T15:26:56.516 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.516 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 42224, 2024-05-21T15:26:56.516 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 976 2024-05-21T15:26:56.516 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.517 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-05-21T15:26:56.517 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-05-21T15:26:56.517 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-05-21T15:26:56.517 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.517 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-05-21T15:26:56.517 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-05-21T15:26:56.517 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-05-21T15:26:56.517 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4073, 2024-05-21T15:26:56.518 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 4.215711159, 2024-05-21T15:26:56.518 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001035038 2024-05-21T15:26:56.518 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.518 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-05-21T15:26:56.518 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-05-21T15:26:56.518 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-05-21T15:26:56.518 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 877, 2024-05-21T15:26:56.519 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.082235392, 2024-05-21T15:26:56.519 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000093768 2024-05-21T15:26:56.519 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.519 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-05-21T15:26:56.519 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 2227, 2024-05-21T15:26:56.519 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 2225, 2024-05-21T15:26:56.519 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-05-21T15:26:56.520 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2225, 2024-05-21T15:26:56.520 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 530.875600804, 2024-05-21T15:26:56.520 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.238595775 2024-05-21T15:26:56.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.520 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-05-21T15:26:56.520 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 2, 2024-05-21T15:26:56.520 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 9, 2024-05-21T15:26:56.520 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 2, 2024-05-21T15:26:56.521 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 217, 2024-05-21T15:26:56.521 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 1, 2024-05-21T15:26:56.521 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-05-21T15:26:56.521 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 150, 2024-05-21T15:26:56.521 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 51, 2024-05-21T15:26:56.521 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 76, 2024-05-21T15:26:56.521 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 23, 2024-05-21T15:26:56.521 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 76, 2024-05-21T15:26:56.522 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 466, 2024-05-21T15:26:56.522 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 49, 2024-05-21T15:26:56.522 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 49, 2024-05-21T15:26:56.522 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 18, 2024-05-21T15:26:56.522 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 3045, 2024-05-21T15:26:56.522 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 2822, 2024-05-21T15:26:56.522 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 2, 2024-05-21T15:26:56.522 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 0, 2024-05-21T15:26:56.523 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:26:56.523 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-05-21T15:26:56.523 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 45, 2024-05-21T15:26:56.523 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-05-21T15:26:56.523 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-05-21T15:26:56.523 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 15, 2024-05-21T15:26:56.523 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 499, 2024-05-21T15:26:56.523 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 2, 2024-05-21T15:26:56.524 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 22, 2024-05-21T15:26:56.524 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-05-21T15:26:56.524 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 807, 2024-05-21T15:26:56.524 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-05-21T15:26:56.524 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 325, 2024-05-21T15:26:56.524 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 13424230508, 2024-05-21T15:26:56.524 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 6, 2024-05-21T15:26:56.524 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 807, 2024-05-21T15:26:56.525 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-05-21T15:26:56.525 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:26:56.525 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:26:56.525 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 291, 2024-05-21T15:26:56.525 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 19, 2024-05-21T15:26:56.525 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 272, 2024-05-21T15:26:56.525 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 516, 2024-05-21T15:26:56.526 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 1, 2024-05-21T15:26:56.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 310, 2024-05-21T15:26:56.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 3020, 2024-05-21T15:26:56.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 6, 2024-05-21T15:26:56.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:26:56.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:26:56.526 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 5598, 2024-05-21T15:26:56.527 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 4860, 2024-05-21T15:26:56.527 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 106, 2024-05-21T15:26:56.527 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 953 2024-05-21T15:26:56.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.527 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-05-21T15:26:56.527 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 338, 2024-05-21T15:26:56.527 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-05-21T15:26:56.527 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-05-21T15:26:56.528 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 0, 2024-05-21T15:26:56.528 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 649, 2024-05-21T15:26:56.528 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-05-21T15:26:56.528 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-05-21T15:26:56.528 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:26:56.528 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 530, 2024-05-21T15:26:56.528 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 469, 2024-05-21T15:26:56.528 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 61, 2024-05-21T15:26:56.529 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-05-21T15:26:56.529 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-05-21T15:26:56.529 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:26:56.529 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:26:56.529 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-05-21T15:26:56.529 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-05-21T15:26:56.529 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:26:56.529 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:26:56.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 19, 2024-05-21T15:26:56.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 15, 2024-05-21T15:26:56.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-05-21T15:26:56.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 1, 2024-05-21T15:26:56.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-05-21T15:26:56.530 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-05-21T15:26:56.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.530 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 7436, 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 7312, 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 7312, 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 124, 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 33, 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 25, 2024-05-21T15:26:56.531 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 24, 2024-05-21T15:26:56.532 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 24, 2024-05-21T15:26:56.532 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 1, 2024-05-21T15:26:56.532 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 1, 2024-05-21T15:26:56.532 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-05-21T15:26:56.532 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 1, 2024-05-21T15:26:56.532 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 23912563, 2024-05-21T15:26:56.532 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 24762133, 2024-05-21T15:26:56.532 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 0, 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1904, 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 39.005168548, 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.020485907 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 0 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-05-21T15:26:56.533 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 153, 2024-05-21T15:26:56.534 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 1014, 2024-05-21T15:26:56.534 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 861, 2024-05-21T15:26:56.534 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 37, 2024-05-21T15:26:56.534 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 135, 2024-05-21T15:26:56.534 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 98, 2024-05-21T15:26:56.534 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 150, 2024-05-21T15:26:56.534 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 1309, 2024-05-21T15:26:56.534 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 1159, 2024-05-21T15:26:56.535 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 49, 2024-05-21T15:26:56.535 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 937, 2024-05-21T15:26:56.535 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 888, 2024-05-21T15:26:56.535 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 142068, 2024-05-21T15:26:56.535 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 231828 2024-05-21T15:26:56.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.535 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-05-21T15:26:56.535 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 2784, 2024-05-21T15:26:56.536 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 4, 2024-05-21T15:26:56.536 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 2227, 2024-05-21T15:26:56.536 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 44, 2024-05-21T15:26:56.536 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 45, 2024-05-21T15:26:56.536 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-05-21T15:26:56.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 765, 2024-05-21T15:26:56.536 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.879520908, 2024-05-21T15:26:56.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001149700 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 158, 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.096132719, 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000608434 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.537 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.538 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-05-21T15:26:56.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 61, 2024-05-21T15:26:56.538 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.051022239, 2024-05-21T15:26:56.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000836430 2024-05-21T15:26:56.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.538 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-05-21T15:26:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 259, 2024-05-21T15:26:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 515.464243471, 2024-05-21T15:26:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 1.990209434 2024-05-21T15:26:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-05-21T15:26:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.540 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-05-21T15:26:56.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.540 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.540 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-05-21T15:26:56.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.541 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-05-21T15:26:56.542 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.542 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.542 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.542 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-05-21T15:26:56.543 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 10, 2024-05-21T15:26:56.543 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.001847806, 2024-05-21T15:26:56.543 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000184780 2024-05-21T15:26:56.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.543 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-05-21T15:26:56.543 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 105, 2024-05-21T15:26:56.544 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.130903900, 2024-05-21T15:26:56.544 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001246703 2024-05-21T15:26:56.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.544 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-05-21T15:26:56.544 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.544 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.544 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.545 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-05-21T15:26:56.545 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 10, 2024-05-21T15:26:56.545 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.302079099, 2024-05-21T15:26:56.545 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.030207909 2024-05-21T15:26:56.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.545 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-05-21T15:26:56.545 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 52, 2024-05-21T15:26:56.545 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.205606384, 2024-05-21T15:26:56.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003953968 2024-05-21T15:26:56.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.546 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-05-21T15:26:56.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 160, 2024-05-21T15:26:56.546 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.961979112, 2024-05-21T15:26:56.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.006012369 2024-05-21T15:26:56.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.546 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-05-21T15:26:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 113, 2024-05-21T15:26:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 11.751427778, 2024-05-21T15:26:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.103994936 2024-05-21T15:26:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-05-21T15:26:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.547 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.548 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-05-21T15:26:56.548 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.548 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.548 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.548 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-05-21T15:26:56.548 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 16, 2024-05-21T15:26:56.549 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.163078322, 2024-05-21T15:26:56.549 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.010192395 2024-05-21T15:26:56.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.549 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-05-21T15:26:56.549 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4, 2024-05-21T15:26:56.549 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.019658047, 2024-05-21T15:26:56.549 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.004914511 2024-05-21T15:26:56.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.550 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-05-21T15:26:56.550 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.550 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.550 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.550 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-05-21T15:26:56.550 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 7, 2024-05-21T15:26:56.551 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.032322965, 2024-05-21T15:26:56.551 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.004617566 2024-05-21T15:26:56.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.551 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-05-21T15:26:56.551 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.551 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.551 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.552 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-05-21T15:26:56.552 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4, 2024-05-21T15:26:56.552 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.001258479, 2024-05-21T15:26:56.552 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000314619 2024-05-21T15:26:56.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.552 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-05-21T15:26:56.552 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.552 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.553 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.553 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-05-21T15:26:56.553 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 7, 2024-05-21T15:26:56.553 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.021486468, 2024-05-21T15:26:56.553 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003069495 2024-05-21T15:26:56.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.554 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-05-21T15:26:56.554 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 13, 2024-05-21T15:26:56.554 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.009090321, 2024-05-21T15:26:56.554 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000699255 2024-05-21T15:26:56.554 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.554 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-05-21T15:26:56.554 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 481, 2024-05-21T15:26:56.554 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.783942786, 2024-05-21T15:26:56.555 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.001629818 2024-05-21T15:26:56.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.555 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:26:56.555 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:26:56.555 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-05-21T15:26:56.555 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.555 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.555 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.556 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.556 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-05-21T15:26:56.556 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 2, 2024-05-21T15:26:56.556 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 5, 2024-05-21T15:26:56.556 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 3, 2024-05-21T15:26:56.556 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 2, 2024-05-21T15:26:56.556 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-05-21T15:26:56.557 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 35, 2024-05-21T15:26:56.557 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 17, 2024-05-21T15:26:56.557 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 618, 2024-05-21T15:26:56.557 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-05-21T15:26:56.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.557 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-05-21T15:26:56.557 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:26:56.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-05-21T15:26:56.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:26:56.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:26:56.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:26:56.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:26:56.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:26:56.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:26:56.558 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:26:56.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:26:56.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:26:56.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:26:56.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:26:56.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:26:56.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:26:56.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-05-21T15:26:56.559 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:26:56.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-05-21T15:26:56.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:26:56.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:26:56.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:26:56.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:26:56.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:26:56.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:26:56.560 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:26:56.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-05-21T15:26:56.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:26:56.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:26:56.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:26:56.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-05-21T15:26:56.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-05-21T15:26:56.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-05-21T15:26:56.561 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:26:56.562 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:26:56.562 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:26:56.562 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:26:56.562 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 320496, 2024-05-21T15:26:56.562 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 240, 2024-05-21T15:26:56.562 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:26:56.562 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-05-21T15:26:56.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-05-21T15:26:56.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-05-21T15:26:56.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:26:56.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-05-21T15:26:56.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:26:56.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-05-21T15:26:56.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 8512, 2024-05-21T15:26:56.563 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 177, 2024-05-21T15:26:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:26:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:26:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-05-21T15:26:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-05-21T15:26:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 571736, 2024-05-21T15:26:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 6254, 2024-05-21T15:26:56.564 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-05-21T15:26:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-05-21T15:26:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-05-21T15:26:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-05-21T15:26:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-05-21T15:26:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-05-21T15:26:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-05-21T15:26:56.565 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 4796, 2024-05-21T15:26:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 21688650, 2024-05-21T15:26:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 37, 2024-05-21T15:26:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 4796, 2024-05-21T15:26:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-05-21T15:26:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4796, 2024-05-21T15:26:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 83.338613920, 2024-05-21T15:26:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.017376691 2024-05-21T15:26:56.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-05-21T15:26:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-05-21T15:26:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 4796, 2024-05-21T15:26:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 6473 2024-05-21T15:26:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 4796, 2024-05-21T15:26:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 873, 2024-05-21T15:26:56.567 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 3923, 2024-05-21T15:26:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-05-21T15:26:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-05-21T15:26:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 110, 2024-05-21T15:26:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 584, 2024-05-21T15:26:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 34, 2024-05-21T15:26:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 1919, 2024-05-21T15:26:56.568 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 87, 2024-05-21T15:26:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-05-21T15:26:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-05-21T15:26:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-05-21T15:26:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 1, 2024-05-21T15:26:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 989, 2024-05-21T15:26:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-05-21T15:26:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-05-21T15:26:56.569 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-05-21T15:26:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 835, 2024-05-21T15:26:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 1168, 2024-05-21T15:26:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:26:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-05-21T15:26:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:26:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-05-21T15:26:56.570 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-05-21T15:26:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-05-21T15:26:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:26:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-05-21T15:26:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:26:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 6, 2024-05-21T15:26:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-05-21T15:26:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-05-21T15:26:56.571 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-05-21T15:26:56.572 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-05-21T15:26:56.573 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-05-21T15:26:56.573 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-05-21T15:26:56.573 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-05-21T15:26:56.573 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-05-21T15:26:56.573 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 98, 2024-05-21T15:26:56.573 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-05-21T15:26:56.573 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 14, 2024-05-21T15:26:56.573 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 12, 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 12, 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 593, 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 24, 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 123 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-05-21T15:26:56.574 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 1, 2024-05-21T15:26:56.575 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:26:56.575 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-05-21T15:26:56.575 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-05-21T15:26:56.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.575 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-05-21T15:26:56.575 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 32 2024-05-21T15:26:56.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.575 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-05-21T15:26:56.576 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-05-21T15:26:56.576 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 6, 2024-05-21T15:26:56.576 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-05-21T15:26:56.576 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 6, 2024-05-21T15:26:56.576 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 469, 2024-05-21T15:26:56.576 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 469, 2024-05-21T15:26:56.576 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-05-21T15:26:56.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 21396, 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 7002807, 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 21396, 2024-05-21T15:26:56.577 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:26:56.578 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:26:56.578 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 21396, 2024-05-21T15:26:56.578 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 7002807, 2024-05-21T15:26:56.578 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:56.578 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.578 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.578 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.578 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.579 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-05-21T15:26:56.579 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:56.579 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-21T15:26:56.579 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:56.579 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:26:56.579 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:26:56.579 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:56.580 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:56.580 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 4796, 2024-05-21T15:26:56.580 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 21735808, 2024-05-21T15:26:56.580 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 4796, 2024-05-21T15:26:56.580 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 21735808, 2024-05-21T15:26:56.580 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:56.580 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.580 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.581 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.581 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.581 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.581 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-05-21T15:26:56.581 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:56.581 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-05-21T15:26:56.581 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:56.581 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:26:56.582 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:26:56.582 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:56.582 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:56.582 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 4796, 2024-05-21T15:26:56.582 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 4796, 2024-05-21T15:26:56.582 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 4796, 2024-05-21T15:26:56.582 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 4796, 2024-05-21T15:26:56.583 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:56.583 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.583 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.583 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.583 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.583 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:26:56.583 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:56.584 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-05-21T15:26:56.584 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:56.584 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 469, 2024-05-21T15:26:56.584 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 47369, 2024-05-21T15:26:56.584 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:56.584 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 469, 2024-05-21T15:26:56.584 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:26:56.584 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:26:56.585 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 15, 2024-05-21T15:26:56.585 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 47369, 2024-05-21T15:26:56.585 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:56.585 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.585 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.585 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.585 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:56.586 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x5611b84c8b60": { 2024-05-21T15:26:56.586 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:56.586 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-05-21T15:26:56.586 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:56.586 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 7436, 2024-05-21T15:26:56.586 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 20567829, 2024-05-21T15:26:56.586 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 7436, 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 1904, 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 20567829, 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:56.587 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:56.588 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.588 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:56.588 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-21T15:26:57.207 INFO:tasks.check_counter:Daemon mds.b mds.exported=15 2024-05-21T15:26:57.207 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-05-21T15:26:57.207 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=338 2024-05-21T15:26:57.207 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-21T15:26:57.207 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-21T15:26:57.208 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-05-21T15:26:57.208 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-05-21T15:26:57.208 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-05-21T15:26:57.208 DEBUG:tasks.check_counter:Getting stats from e 2024-05-21T15:26:57.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-21T15:26:57.449 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:26:58.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:26:58 smithi084 ceph-mon[36334]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:58.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:26:58 smithi078 ceph-mon[31997]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:58.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:26:58 smithi172 ceph-mon[36402]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T15:26:59.088 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-21T15:26:59.088 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:26:59.088 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 263, 2024-05-21T15:26:59.088 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 234, 2024-05-21T15:26:59.089 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1577106, 2024-05-21T15:26:59.089 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 134516, 2024-05-21T15:26:59.089 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-05-21T15:26:59.089 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-05-21T15:26:59.089 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.047161312, 2024-05-21T15:26:59.089 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.013518286, 2024-05-21T15:26:59.089 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.037552330, 2024-05-21T15:26:59.090 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.005326939, 2024-05-21T15:26:59.090 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:59.090 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 234, 2024-05-21T15:26:59.090 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.013741114, 2024-05-21T15:26:59.090 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000058722 2024-05-21T15:26:59.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.090 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:59.090 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.091 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.091 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.091 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1576272, 2024-05-21T15:26:59.091 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 134192 2024-05-21T15:26:59.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.091 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:26:59.092 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 649, 2024-05-21T15:26:59.092 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 809, 2024-05-21T15:26:59.092 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 159504, 2024-05-21T15:26:59.092 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 652407, 2024-05-21T15:26:59.092 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 7, 2024-05-21T15:26:59.092 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-05-21T15:26:59.092 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.068658112, 2024-05-21T15:26:59.092 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.022260704, 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.015513604, 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.023798597, 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 806, 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.040119878, 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000049776 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:59.093 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.094 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.094 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.094 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.094 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T15:26:59.094 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 446656 2024-05-21T15:26:59.094 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.094 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 1585, 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1587, 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1684985, 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 504088, 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 4, 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.164485541, 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.054851194, 2024-05-21T15:26:59.095 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.043640091, 2024-05-21T15:26:59.096 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.052135527, 2024-05-21T15:26:59.096 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:26:59.096 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1585, 2024-05-21T15:26:59.096 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.094513164, 2024-05-21T15:26:59.096 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000059629 2024-05-21T15:26:59.096 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.096 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:26:59.096 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.097 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.097 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.097 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:26:59.097 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:26:59.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.097 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-05-21T15:26:59.097 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-05-21T15:26:59.098 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-05-21T15:26:59.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.098 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-05-21T15:26:59.098 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-05-21T15:26:59.098 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-05-21T15:26:59.098 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3743, 2024-05-21T15:26:59.098 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.110596678, 2024-05-21T15:26:59.098 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000029547 2024-05-21T15:26:59.099 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.099 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-05-21T15:26:59.099 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-05-21T15:26:59.099 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-05-21T15:26:59.099 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.099 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.099 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.100 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.100 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-05-21T15:26:59.100 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 0, 2024-05-21T15:26:59.100 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 0, 2024-05-21T15:26:59.100 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-05-21T15:26:59.100 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.100 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-05-21T15:26:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 0, 2024-05-21T15:26:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 0, 2024-05-21T15:26:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-05-21T15:26:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 0, 2024-05-21T15:26:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 0, 2024-05-21T15:26:59.102 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-05-21T15:26:59.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 92, 2024-05-21T15:26:59.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 0, 2024-05-21T15:26:59.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 92, 2024-05-21T15:26:59.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 0, 2024-05-21T15:26:59.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 92, 2024-05-21T15:26:59.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 0, 2024-05-21T15:26:59.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 0, 2024-05-21T15:26:59.103 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 0, 2024-05-21T15:26:59.103 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 3, 2024-05-21T15:26:59.103 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 0, 2024-05-21T15:26:59.103 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 0, 2024-05-21T15:26:59.103 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 0, 2024-05-21T15:26:59.103 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 0, 2024-05-21T15:26:59.103 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:26:59.103 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-05-21T15:26:59.104 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-05-21T15:26:59.104 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-05-21T15:26:59.104 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-05-21T15:26:59.104 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 0, 2024-05-21T15:26:59.104 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 0, 2024-05-21T15:26:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 0, 2024-05-21T15:26:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 0, 2024-05-21T15:26:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-05-21T15:26:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:26:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-05-21T15:26:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 0, 2024-05-21T15:26:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 0, 2024-05-21T15:26:59.105 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 0, 2024-05-21T15:26:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:26:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-05-21T15:26:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:26:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:26:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:26:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:26:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:26:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 0, 2024-05-21T15:26:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:26:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:26:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:26:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:26:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:26:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:26:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 0, 2024-05-21T15:26:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:26:59.108 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 0, 2024-05-21T15:26:59.108 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 0 2024-05-21T15:26:59.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.108 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-05-21T15:26:59.108 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-05-21T15:26:59.108 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-05-21T15:26:59.108 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-05-21T15:26:59.108 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 0, 2024-05-21T15:26:59.109 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 0, 2024-05-21T15:26:59.109 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-05-21T15:26:59.109 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-05-21T15:26:59.109 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:26:59.109 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 0, 2024-05-21T15:26:59.109 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 0, 2024-05-21T15:26:59.109 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-05-21T15:26:59.109 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-05-21T15:26:59.110 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-05-21T15:26:59.110 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:26:59.110 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:26:59.110 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-05-21T15:26:59.110 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-05-21T15:26:59.110 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:26:59.110 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:26:59.110 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:26:59.111 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 0, 2024-05-21T15:26:59.111 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-05-21T15:26:59.111 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:26:59.111 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-05-21T15:26:59.111 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-05-21T15:26:59.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.111 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-05-21T15:26:59.111 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-05-21T15:26:59.112 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 0, 2024-05-21T15:26:59.112 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-05-21T15:26:59.112 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-05-21T15:26:59.112 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 953, 2024-05-21T15:26:59.112 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-05-21T15:26:59.112 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-05-21T15:26:59.112 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 0, 2024-05-21T15:26:59.112 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-05-21T15:26:59.113 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-05-21T15:26:59.113 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 2, 2024-05-21T15:26:59.113 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 2, 2024-05-21T15:26:59.113 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-05-21T15:26:59.113 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-05-21T15:26:59.114 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-05-21T15:26:59.114 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 5408932, 2024-05-21T15:26:59.114 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 5408358, 2024-05-21T15:26:59.114 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-05-21T15:26:59.114 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.114 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.114 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.115 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 953 2024-05-21T15:26:59.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.115 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-05-21T15:26:59.115 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 95, 2024-05-21T15:26:59.115 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 116, 2024-05-21T15:26:59.115 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 21, 2024-05-21T15:26:59.115 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 10, 2024-05-21T15:26:59.115 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 11, 2024-05-21T15:26:59.116 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 1, 2024-05-21T15:26:59.116 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 92, 2024-05-21T15:26:59.116 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 113, 2024-05-21T15:26:59.116 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 21, 2024-05-21T15:26:59.116 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 0, 2024-05-21T15:26:59.116 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 0, 2024-05-21T15:26:59.116 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 0, 2024-05-21T15:26:59.117 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 121596, 2024-05-21T15:26:59.117 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 215444 2024-05-21T15:26:59.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.117 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-05-21T15:26:59.117 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 0, 2024-05-21T15:26:59.117 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 0, 2024-05-21T15:26:59.117 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 0, 2024-05-21T15:26:59.117 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 0, 2024-05-21T15:26:59.118 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 0, 2024-05-21T15:26:59.118 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-05-21T15:26:59.118 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.118 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.118 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.118 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-05-21T15:26:59.118 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.119 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.119 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.119 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-05-21T15:26:59.119 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.119 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.119 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.120 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-05-21T15:26:59.120 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.120 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.120 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.120 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-05-21T15:26:59.120 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.120 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.121 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.121 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-05-21T15:26:59.121 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.121 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.121 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.121 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-05-21T15:26:59.122 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.122 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.122 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.122 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-05-21T15:26:59.122 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.122 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.122 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.123 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-05-21T15:26:59.123 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.123 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.123 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.123 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-05-21T15:26:59.123 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.124 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.124 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.124 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-05-21T15:26:59.124 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.124 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.124 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.125 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-05-21T15:26:59.125 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.125 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.126 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.126 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-05-21T15:26:59.126 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.126 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.126 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.127 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-05-21T15:26:59.127 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.127 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.127 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.127 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-05-21T15:26:59.127 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.127 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.128 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.128 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-05-21T15:26:59.128 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.128 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.128 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.128 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-05-21T15:26:59.129 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.129 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.129 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.129 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-05-21T15:26:59.129 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.129 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.129 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.130 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.131 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.131 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-05-21T15:26:59.131 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.131 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.131 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.131 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-05-21T15:26:59.132 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.132 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.132 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.132 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-05-21T15:26:59.132 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.132 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.132 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.133 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-05-21T15:26:59.133 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.133 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.133 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.133 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-05-21T15:26:59.133 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.134 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.134 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.134 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-05-21T15:26:59.134 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.134 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.134 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.135 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-05-21T15:26:59.135 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.135 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.135 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.135 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-05-21T15:26:59.135 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.135 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.136 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.136 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-05-21T15:26:59.136 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.136 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.136 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.136 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:26:59.137 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:26:59.137 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-05-21T15:26:59.137 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.137 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.137 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.137 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.138 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-05-21T15:26:59.138 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 1, 2024-05-21T15:26:59.138 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 2, 2024-05-21T15:26:59.138 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 1, 2024-05-21T15:26:59.138 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 2, 2024-05-21T15:26:59.138 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-05-21T15:26:59.138 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 0, 2024-05-21T15:26:59.139 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 0, 2024-05-21T15:26:59.139 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 3629, 2024-05-21T15:26:59.139 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-05-21T15:26:59.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.139 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-05-21T15:26:59.139 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:26:59.139 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-05-21T15:26:59.139 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:26:59.140 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:26:59.140 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:26:59.140 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:26:59.140 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:26:59.140 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:26:59.140 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:26:59.140 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:26:59.140 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:26:59.141 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:26:59.141 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:26:59.141 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:26:59.141 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:26:59.141 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-05-21T15:26:59.141 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:26:59.141 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-05-21T15:26:59.141 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:26:59.142 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:26:59.142 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:26:59.142 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:26:59.142 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:26:59.142 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:26:59.142 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:26:59.142 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-05-21T15:26:59.142 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:26:59.143 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:26:59.143 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:26:59.143 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-05-21T15:26:59.143 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-05-21T15:26:59.143 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-05-21T15:26:59.143 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:26:59.143 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:26:59.143 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:26:59.144 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:26:59.144 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 192800, 2024-05-21T15:26:59.144 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 143, 2024-05-21T15:26:59.144 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:26:59.144 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-05-21T15:26:59.144 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-05-21T15:26:59.144 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-05-21T15:26:59.144 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:26:59.145 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-05-21T15:26:59.145 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:26:59.145 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-05-21T15:26:59.145 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 8512, 2024-05-21T15:26:59.145 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 177, 2024-05-21T15:26:59.145 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:26:59.145 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:26:59.145 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-05-21T15:26:59.146 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-05-21T15:26:59.146 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 296488, 2024-05-21T15:26:59.146 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 2462, 2024-05-21T15:26:59.146 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-05-21T15:26:59.146 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-05-21T15:26:59.146 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-05-21T15:26:59.146 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-05-21T15:26:59.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.147 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-05-21T15:26:59.147 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-05-21T15:26:59.147 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-05-21T15:26:59.147 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 2233, 2024-05-21T15:26:59.147 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 21, 2024-05-21T15:26:59.147 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 3, 2024-05-21T15:26:59.147 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 2233, 2024-05-21T15:26:59.147 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-05-21T15:26:59.148 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2233, 2024-05-21T15:26:59.148 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 5.875905759, 2024-05-21T15:26:59.148 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.002631395 2024-05-21T15:26:59.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.148 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-05-21T15:26:59.148 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-05-21T15:26:59.148 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2233, 2024-05-21T15:26:59.148 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 2234 2024-05-21T15:26:59.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.149 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 2233, 2024-05-21T15:26:59.149 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 2233, 2024-05-21T15:26:59.149 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 0, 2024-05-21T15:26:59.149 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-05-21T15:26:59.149 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-05-21T15:26:59.149 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 1403, 2024-05-21T15:26:59.149 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 0, 2024-05-21T15:26:59.150 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 828, 2024-05-21T15:26:59.150 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 0, 2024-05-21T15:26:59.150 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 0, 2024-05-21T15:26:59.150 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-05-21T15:26:59.150 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-05-21T15:26:59.150 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-05-21T15:26:59.150 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-05-21T15:26:59.150 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 0, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 1, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 0, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:26:59.151 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-05-21T15:26:59.152 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-05-21T15:26:59.152 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-05-21T15:26:59.152 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:26:59.152 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-05-21T15:26:59.152 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:26:59.152 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-05-21T15:26:59.152 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-05-21T15:26:59.152 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-05-21T15:26:59.153 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-05-21T15:26:59.153 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-05-21T15:26:59.153 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-05-21T15:26:59.153 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-05-21T15:26:59.153 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-05-21T15:26:59.153 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-05-21T15:26:59.153 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-05-21T15:26:59.153 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-05-21T15:26:59.154 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-05-21T15:26:59.154 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-05-21T15:26:59.154 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-05-21T15:26:59.154 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-05-21T15:26:59.154 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-05-21T15:26:59.154 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-05-21T15:26:59.154 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 98, 2024-05-21T15:26:59.154 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-05-21T15:26:59.155 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 13, 2024-05-21T15:26:59.155 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 5, 2024-05-21T15:26:59.155 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 5, 2024-05-21T15:26:59.155 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-05-21T15:26:59.155 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-05-21T15:26:59.155 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 0, 2024-05-21T15:26:59.155 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 2, 2024-05-21T15:26:59.155 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 0 2024-05-21T15:26:59.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.156 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-05-21T15:26:59.156 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 0, 2024-05-21T15:26:59.156 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:26:59.156 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-05-21T15:26:59.156 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-05-21T15:26:59.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.156 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-05-21T15:26:59.157 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-05-21T15:26:59.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.157 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-05-21T15:26:59.157 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-05-21T15:26:59.157 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:26:59.157 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-05-21T15:26:59.157 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 0, 2024-05-21T15:26:59.157 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 0, 2024-05-21T15:26:59.158 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 0, 2024-05-21T15:26:59.158 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-05-21T15:26:59.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.158 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:26:59.158 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:59.158 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-21T15:26:59.158 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:59.158 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 2493, 2024-05-21T15:26:59.159 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 3181064, 2024-05-21T15:26:59.159 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:59.159 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 2493, 2024-05-21T15:26:59.159 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:26:59.159 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:26:59.159 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 2493, 2024-05-21T15:26:59.159 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 3181064, 2024-05-21T15:26:59.159 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:59.160 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.160 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.160 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.160 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.160 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-05-21T15:26:59.160 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:59.160 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-21T15:26:59.161 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:59.161 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:26:59.161 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:26:59.161 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:59.161 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:59.161 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 2233, 2024-05-21T15:26:59.161 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 1214633, 2024-05-21T15:26:59.161 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 2233, 2024-05-21T15:26:59.162 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1214633, 2024-05-21T15:26:59.162 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:59.162 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.162 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.162 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.162 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.162 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-05-21T15:26:59.163 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:59.163 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-05-21T15:26:59.163 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:59.163 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:26:59.163 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:26:59.163 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:59.163 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:59.164 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 2233, 2024-05-21T15:26:59.164 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 2233, 2024-05-21T15:26:59.164 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 2233, 2024-05-21T15:26:59.164 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 2233, 2024-05-21T15:26:59.164 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:59.164 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.164 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.164 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.165 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.165 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:26:59.165 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:59.165 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-05-21T15:26:59.165 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:59.165 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:26:59.165 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:26:59.166 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:59.166 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:59.166 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:26:59.166 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:26:59.166 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-05-21T15:26:59.166 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-05-21T15:26:59.166 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:59.166 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.167 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.167 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.167 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:26:59.167 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x563418832c30": { 2024-05-21T15:26:59.167 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:26:59.167 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-05-21T15:26:59.167 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-05-21T15:26:59.168 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:26:59.169 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:26:59.169 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:26:59.169 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:26:59.169 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.169 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:26:59.169 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-21T15:26:59.882 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-21T15:26:59.882 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-21T15:26:59.882 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-21T15:26:59.882 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-21T15:26:59.882 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-05-21T15:26:59.882 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:26:59.882 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:26:59.883 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-21T15:26:59.883 DEBUG:tasks.check_counter:Getting stats from h 2024-05-21T15:26:59.883 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-21T15:27:00.119 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:27:00.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:00 smithi084 ceph-mon[36334]: pgmap v338: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:27:00.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:00 smithi078 ceph-mon[31997]: pgmap v338: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:27:00.434 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:00 smithi172 ceph-mon[36402]: pgmap v338: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:27:01.922 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-21T15:27:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:27:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2283, 2024-05-21T15:27:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2283, 2024-05-21T15:27:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 2201592, 2024-05-21T15:27:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 725177, 2024-05-21T15:27:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 6, 2024-05-21T15:27:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 2, 2024-05-21T15:27:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.175306292, 2024-05-21T15:27:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.052319242, 2024-05-21T15:27:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.046885769, 2024-05-21T15:27:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.062869141, 2024-05-21T15:27:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2282, 2024-05-21T15:27:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.072893659, 2024-05-21T15:27:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000031942 2024-05-21T15:27:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-05-21T15:27:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 448 2024-05-21T15:27:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:27:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 4, 2024-05-21T15:27:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 164, 2024-05-21T15:27:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 876, 2024-05-21T15:27:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 448970, 2024-05-21T15:27:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 6, 2024-05-21T15:27:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-05-21T15:27:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.009633882, 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.006062471, 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.001889929, 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000045181, 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 161, 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.016865276, 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000104753 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 448272 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 258, 2024-05-21T15:27:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 231, 2024-05-21T15:27:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1573244, 2024-05-21T15:27:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 133422, 2024-05-21T15:27:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 3, 2024-05-21T15:27:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 2, 2024-05-21T15:27:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.045254680, 2024-05-21T15:27:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.013418024, 2024-05-21T15:27:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.036400909, 2024-05-21T15:27:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.005115733, 2024-05-21T15:27:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 230, 2024-05-21T15:27:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.011607389, 2024-05-21T15:27:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000050466 2024-05-21T15:27:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1572976, 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 133104 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-05-21T15:27:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3851, 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.095197459, 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000024720 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-05-21T15:27:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 0, 2024-05-21T15:27:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 0, 2024-05-21T15:27:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-05-21T15:27:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-05-21T15:27:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 0, 2024-05-21T15:27:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 0, 2024-05-21T15:27:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-05-21T15:27:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 0, 2024-05-21T15:27:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 0, 2024-05-21T15:27:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-05-21T15:27:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 526, 2024-05-21T15:27:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 87, 2024-05-21T15:27:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 439, 2024-05-21T15:27:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 0, 2024-05-21T15:27:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 463, 2024-05-21T15:27:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 0, 2024-05-21T15:27:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 0, 2024-05-21T15:27:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 0, 2024-05-21T15:27:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 13, 2024-05-21T15:27:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 0, 2024-05-21T15:27:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 0, 2024-05-21T15:27:01.937 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 0, 2024-05-21T15:27:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 0, 2024-05-21T15:27:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 0, 2024-05-21T15:27:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-05-21T15:27:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:27:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-05-21T15:27:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 0, 2024-05-21T15:27:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 0, 2024-05-21T15:27:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 0, 2024-05-21T15:27:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:27:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-05-21T15:27:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:27:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:27:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:27:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:27:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:27:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 0, 2024-05-21T15:27:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:27:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:27:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:27:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:27:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:27:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:27:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 0, 2024-05-21T15:27:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:27:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 0, 2024-05-21T15:27:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 0 2024-05-21T15:27:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-05-21T15:27:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-05-21T15:27:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-05-21T15:27:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-05-21T15:27:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 0, 2024-05-21T15:27:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 0, 2024-05-21T15:27:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-05-21T15:27:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-05-21T15:27:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:27:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 0, 2024-05-21T15:27:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 0, 2024-05-21T15:27:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-05-21T15:27:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-05-21T15:27:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-05-21T15:27:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:27:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:27:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-05-21T15:27:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-05-21T15:27:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:27:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:27:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 0, 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-05-21T15:27:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 0, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 888, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 0, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-05-21T15:27:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 2, 2024-05-21T15:27:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 2, 2024-05-21T15:27:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-05-21T15:27:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-05-21T15:27:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-05-21T15:27:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 5750949, 2024-05-21T15:27:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 5750375, 2024-05-21T15:27:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-05-21T15:27:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 888 2024-05-21T15:27:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-05-21T15:27:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 453, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 455, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 2, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 98, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 101, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 3, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 526, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 528, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 2, 2024-05-21T15:27:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 0, 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 0, 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 0, 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 123572, 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 215444 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 0, 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 0, 2024-05-21T15:27:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 0, 2024-05-21T15:27:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 0, 2024-05-21T15:27:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 0, 2024-05-21T15:27:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-05-21T15:27:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-05-21T15:27:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-05-21T15:27:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-05-21T15:27:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-05-21T15:27:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-05-21T15:27:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-05-21T15:27:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-05-21T15:27:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-05-21T15:27:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-05-21T15:27:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-05-21T15:27:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-05-21T15:27:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-05-21T15:27:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-05-21T15:27:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-05-21T15:27:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-05-21T15:27:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-05-21T15:27:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-05-21T15:27:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.968 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.968 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:27:01.968 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:27:01.968 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-05-21T15:27:01.968 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.968 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.968 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.969 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:01.969 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.969 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-05-21T15:27:01.969 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 2, 2024-05-21T15:27:01.969 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 5, 2024-05-21T15:27:01.969 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 3, 2024-05-21T15:27:01.969 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 2, 2024-05-21T15:27:01.970 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-05-21T15:27:01.970 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 0, 2024-05-21T15:27:01.970 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 0, 2024-05-21T15:27:01.970 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 5123, 2024-05-21T15:27:01.970 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-05-21T15:27:01.970 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.970 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-05-21T15:27:01.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:27:01.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-05-21T15:27:01.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:27:01.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:27:01.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:27:01.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:27:01.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:27:01.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-05-21T15:27:01.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:27:01.973 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-05-21T15:27:01.973 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:27:01.973 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:27:01.973 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:27:01.973 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:27:01.973 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:27:01.974 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:27:01.974 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:27:01.974 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-05-21T15:27:01.974 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:27:01.975 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:27:01.975 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:27:01.975 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-05-21T15:27:01.975 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-05-21T15:27:01.975 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-05-21T15:27:01.976 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:27:01.976 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:27:01.976 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:27:01.976 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:27:01.976 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 200523, 2024-05-21T15:27:01.976 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 566, 2024-05-21T15:27:01.976 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:27:01.976 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 8512, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 177, 2024-05-21T15:27:01.977 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:27:01.978 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:27:01.978 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-05-21T15:27:01.978 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-05-21T15:27:01.978 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 1617658, 2024-05-21T15:27:01.978 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 23071, 2024-05-21T15:27:01.978 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-05-21T15:27:01.978 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-05-21T15:27:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-05-21T15:27:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-05-21T15:27:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-05-21T15:27:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-05-21T15:27:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-05-21T15:27:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 2282, 2024-05-21T15:27:01.979 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 21, 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 3, 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 2282, 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2282, 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 8.901857725, 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.003900901 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-05-21T15:27:01.980 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-05-21T15:27:01.981 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2282, 2024-05-21T15:27:01.981 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 2283 2024-05-21T15:27:01.981 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.981 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 2282, 2024-05-21T15:27:01.981 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 2282, 2024-05-21T15:27:01.981 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 0, 2024-05-21T15:27:01.981 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-05-21T15:27:01.981 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-05-21T15:27:01.982 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 1433, 2024-05-21T15:27:01.982 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 0, 2024-05-21T15:27:01.982 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 847, 2024-05-21T15:27:01.982 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 0, 2024-05-21T15:27:01.982 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 0, 2024-05-21T15:27:01.982 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-05-21T15:27:01.982 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-05-21T15:27:01.982 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-05-21T15:27:01.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-05-21T15:27:01.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 0, 2024-05-21T15:27:01.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-05-21T15:27:01.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-05-21T15:27:01.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-05-21T15:27:01.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 1, 2024-05-21T15:27:01.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 0, 2024-05-21T15:27:01.983 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:27:01.984 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-05-21T15:27:01.985 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-05-21T15:27:01.985 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-05-21T15:27:01.985 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-05-21T15:27:01.985 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-05-21T15:27:01.985 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-05-21T15:27:01.985 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-05-21T15:27:01.985 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-05-21T15:27:01.986 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-05-21T15:27:01.986 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-05-21T15:27:01.986 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-05-21T15:27:01.986 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-05-21T15:27:01.986 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-05-21T15:27:01.986 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-05-21T15:27:01.986 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-05-21T15:27:01.986 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 98, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 13, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 4, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 4, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-05-21T15:27:01.987 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 0, 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 2, 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 0 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 0, 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-05-21T15:27:01.988 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.989 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-05-21T15:27:01.989 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-05-21T15:27:01.989 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.989 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-05-21T15:27:01.989 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-05-21T15:27:01.989 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:27:01.989 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-05-21T15:27:01.989 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 0, 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 0, 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 0, 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:01.990 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 2541, 2024-05-21T15:27:01.991 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 3530913, 2024-05-21T15:27:01.991 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:01.991 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 2541, 2024-05-21T15:27:01.991 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:27:01.991 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:27:01.991 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 2541, 2024-05-21T15:27:01.991 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 3530913, 2024-05-21T15:27:01.991 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:01.992 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.992 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.992 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.992 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:01.992 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.992 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-05-21T15:27:01.992 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:01.992 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-21T15:27:01.993 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:01.993 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:27:01.993 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:27:01.993 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:01.993 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:01.993 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 2282, 2024-05-21T15:27:01.994 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 1556650, 2024-05-21T15:27:01.994 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 2282, 2024-05-21T15:27:01.994 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1556650, 2024-05-21T15:27:01.994 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:01.994 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.994 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.994 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.994 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:01.995 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 2282, 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 2282, 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 2282, 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 2282, 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.996 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:01.997 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:01.998 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:27:01.998 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:27:01.998 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-05-21T15:27:01.998 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-05-21T15:27:01.998 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:01.998 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:01.998 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:01.999 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:01.999 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:01.999 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:01.999 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x560b998c4b60": { 2024-05-21T15:27:01.999 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:02.000 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-05-21T15:27:02.000 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:02.000 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:27:02.000 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:27:02.001 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:02.001 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:02.001 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:27:02.001 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:27:02.002 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-05-21T15:27:02.002 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-05-21T15:27:02.002 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:02.002 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:02.003 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:02.003 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:02.003 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:02.003 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:02.004 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-21T15:27:02.308 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:02 smithi084 ceph-mon[36334]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:27:02.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:02 smithi078 ceph-mon[31997]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:27:02.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:02 smithi172 ceph-mon[36402]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T15:27:02.634 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-21T15:27:02.634 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-21T15:27:02.634 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-21T15:27:02.634 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-21T15:27:02.635 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-05-21T15:27:02.635 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:27:02.635 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:27:02.635 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-21T15:27:02.635 DEBUG:tasks.check_counter:Getting stats from k 2024-05-21T15:27:02.635 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-21T15:27:02.871 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T15:27:04.251 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:04 smithi084 ceph-mon[36334]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:27:04.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:04 smithi078 ceph-mon[31997]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:27:04.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:04 smithi172 ceph-mon[36402]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:27:04.530 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-05-21T15:27:04.530 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:27:04.530 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2220, 2024-05-21T15:27:04.530 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1812, 2024-05-21T15:27:04.530 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 903751, 2024-05-21T15:27:04.530 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 1573284, 2024-05-21T15:27:04.531 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 11, 2024-05-21T15:27:04.531 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-05-21T15:27:04.531 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.179235935, 2024-05-21T15:27:04.531 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.080771048, 2024-05-21T15:27:04.531 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.099591518, 2024-05-21T15:27:04.531 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.009349451, 2024-05-21T15:27:04.531 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:04.532 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1811, 2024-05-21T15:27:04.532 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.094052694, 2024-05-21T15:27:04.532 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000051934 2024-05-21T15:27:04.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.532 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:04.532 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3420, 2024-05-21T15:27:04.532 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000810982, 2024-05-21T15:27:04.532 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000237 2024-05-21T15:27:04.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.533 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-05-21T15:27:04.533 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 323408 2024-05-21T15:27:04.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.533 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:27:04.533 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 1050, 2024-05-21T15:27:04.533 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1112, 2024-05-21T15:27:04.533 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1892764, 2024-05-21T15:27:04.534 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 594360, 2024-05-21T15:27:04.534 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 17, 2024-05-21T15:27:04.534 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-05-21T15:27:04.534 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.114271727, 2024-05-21T15:27:04.534 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.036172411, 2024-05-21T15:27:04.534 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.094338878, 2024-05-21T15:27:04.534 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.008446749, 2024-05-21T15:27:04.534 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1111, 2024-05-21T15:27:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.338615473, 2024-05-21T15:27:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000304784 2024-05-21T15:27:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1128, 2024-05-21T15:27:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000326972, 2024-05-21T15:27:04.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000289 2024-05-21T15:27:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1569840, 2024-05-21T15:27:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 153744 2024-05-21T15:27:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:27:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 386, 2024-05-21T15:27:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 348, 2024-05-21T15:27:04.536 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 216860, 2024-05-21T15:27:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 680890, 2024-05-21T15:27:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 46, 2024-05-21T15:27:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-05-21T15:27:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.046746830, 2024-05-21T15:27:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.010513871, 2024-05-21T15:27:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.028418321, 2024-05-21T15:27:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.005465388, 2024-05-21T15:27:04.537 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:04.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 345, 2024-05-21T15:27:04.538 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.031517590, 2024-05-21T15:27:04.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000091355 2024-05-21T15:27:04.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.538 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:04.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 257, 2024-05-21T15:27:04.538 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000067862, 2024-05-21T15:27:04.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000264 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 130544 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.539 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 446, 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.190541245, 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000427222 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-05-21T15:27:04.540 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2, 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000006680, 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000003340 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 27, 2024-05-21T15:27:04.541 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 25, 2024-05-21T15:27:04.542 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-05-21T15:27:04.542 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 25, 2024-05-21T15:27:04.542 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.663464215, 2024-05-21T15:27:04.542 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.026538568 2024-05-21T15:27:04.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.542 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-05-21T15:27:04.542 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 2, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 1, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 11, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 0, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 154, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 51, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 103, 2024-05-21T15:27:04.543 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 0, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 82, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 16, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 16, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 16, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 13, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 226, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 129, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 1, 2024-05-21T15:27:04.544 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 93, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 0, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 0, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 2, 2024-05-21T15:27:04.545 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 22, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 285, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 325, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 13424230508, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 6, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 285, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-05-21T15:27:04.546 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:27:04.547 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:27:04.547 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:27:04.547 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:27:04.547 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:27:04.547 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 285, 2024-05-21T15:27:04.547 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 4, 2024-05-21T15:27:04.547 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 41, 2024-05-21T15:27:04.547 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 91, 2024-05-21T15:27:04.548 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:27:04.548 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-05-21T15:27:04.548 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:27:04.548 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 508, 2024-05-21T15:27:04.548 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 414, 2024-05-21T15:27:04.548 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 3, 2024-05-21T15:27:04.548 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 1 2024-05-21T15:27:04.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 258, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 174, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 164, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 0, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:27:04.549 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 0, 2024-05-21T15:27:04.550 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 0, 2024-05-21T15:27:04.550 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-05-21T15:27:04.550 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-05-21T15:27:04.550 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-05-21T15:27:04.550 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:27:04.550 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:27:04.550 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-05-21T15:27:04.550 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-05-21T15:27:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:27:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:27:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:27:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 0, 2024-05-21T15:27:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-05-21T15:27:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:27:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-05-21T15:27:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-05-21T15:27:04.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.552 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-05-21T15:27:04.552 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-05-21T15:27:04.552 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 522, 2024-05-21T15:27:04.552 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-05-21T15:27:04.552 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-05-21T15:27:04.552 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 522, 2024-05-21T15:27:04.552 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-05-21T15:27:04.553 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-05-21T15:27:04.553 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 2, 2024-05-21T15:27:04.553 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-05-21T15:27:04.553 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-05-21T15:27:04.553 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 2, 2024-05-21T15:27:04.553 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 2, 2024-05-21T15:27:04.553 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-05-21T15:27:04.554 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-05-21T15:27:04.554 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-05-21T15:27:04.554 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 4834827, 2024-05-21T15:27:04.554 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 0, 2024-05-21T15:27:04.554 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-05-21T15:27:04.554 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 78, 2024-05-21T15:27:04.554 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 17.257276093, 2024-05-21T15:27:04.554 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.221247129 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 0 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 158, 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 166, 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 8, 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 103, 2024-05-21T15:27:04.555 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 120, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 17, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 154, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 170, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 16, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 16, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 23, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 7, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 126392, 2024-05-21T15:27:04.556 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 223636 2024-05-21T15:27:04.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.557 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-05-21T15:27:04.557 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 35, 2024-05-21T15:27:04.557 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 3, 2024-05-21T15:27:04.557 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 27, 2024-05-21T15:27:04.557 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 17, 2024-05-21T15:27:04.557 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 7, 2024-05-21T15:27:04.557 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-05-21T15:27:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-05-21T15:27:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 5, 2024-05-21T15:27:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.651155127, 2024-05-21T15:27:04.558 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.130231025 2024-05-21T15:27:04.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.559 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-05-21T15:27:04.559 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.559 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.559 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.559 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-05-21T15:27:04.559 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-05-21T15:27:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 19, 2024-05-21T15:27:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.011735322, 2024-05-21T15:27:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000617648 2024-05-21T15:27:04.560 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-05-21T15:27:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-05-21T15:27:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.561 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.562 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.562 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-05-21T15:27:04.562 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.562 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.562 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.562 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-05-21T15:27:04.563 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.563 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.563 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.563 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-05-21T15:27:04.563 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.563 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.564 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.564 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-05-21T15:27:04.564 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.564 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.565 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.565 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-05-21T15:27:04.565 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.565 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.565 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.566 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-05-21T15:27:04.566 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.566 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.566 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.566 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-05-21T15:27:04.566 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.567 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.567 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.567 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-05-21T15:27:04.567 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1, 2024-05-21T15:27:04.567 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000573766, 2024-05-21T15:27:04.567 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000573766 2024-05-21T15:27:04.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.568 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-05-21T15:27:04.568 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.568 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.568 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.568 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-05-21T15:27:04.568 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.568 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.569 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.569 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-05-21T15:27:04.569 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.569 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.569 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.569 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-05-21T15:27:04.570 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.570 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.570 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.570 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-05-21T15:27:04.570 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.570 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.570 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.571 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-05-21T15:27:04.571 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.571 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.571 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.571 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-05-21T15:27:04.571 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.572 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.572 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.572 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.572 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-05-21T15:27:04.572 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.572 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.572 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.572 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.573 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-05-21T15:27:04.573 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.573 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.573 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.573 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.573 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-05-21T15:27:04.573 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.573 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.574 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.574 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-05-21T15:27:04.574 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.574 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.574 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.575 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-05-21T15:27:04.575 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.575 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.575 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.575 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-05-21T15:27:04.575 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-05-21T15:27:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.577 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:27:04.577 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:27:04.577 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-05-21T15:27:04.577 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.577 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.577 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.577 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.578 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-05-21T15:27:04.578 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 1, 2024-05-21T15:27:04.578 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 3, 2024-05-21T15:27:04.578 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 2, 2024-05-21T15:27:04.578 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 1, 2024-05-21T15:27:04.578 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-05-21T15:27:04.578 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 13, 2024-05-21T15:27:04.578 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 13, 2024-05-21T15:27:04.579 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 657, 2024-05-21T15:27:04.579 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-05-21T15:27:04.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.579 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-05-21T15:27:04.579 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:27:04.579 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-05-21T15:27:04.579 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:27:04.579 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:27:04.580 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:27:04.580 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:27:04.580 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:27:04.580 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:27:04.580 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:27:04.580 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:27:04.580 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:27:04.580 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:27:04.581 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:27:04.581 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:27:04.581 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:27:04.581 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-05-21T15:27:04.581 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:27:04.581 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-05-21T15:27:04.581 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:27:04.581 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:27:04.582 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 310557, 2024-05-21T15:27:04.583 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 227, 2024-05-21T15:27:04.584 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:27:04.584 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-05-21T15:27:04.584 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-05-21T15:27:04.584 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-05-21T15:27:04.584 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:27:04.584 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-05-21T15:27:04.584 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:27:04.584 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-05-21T15:27:04.585 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 8512, 2024-05-21T15:27:04.585 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 177, 2024-05-21T15:27:04.585 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:27:04.585 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:27:04.585 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-05-21T15:27:04.585 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-05-21T15:27:04.585 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 617943, 2024-05-21T15:27:04.585 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 4218, 2024-05-21T15:27:04.586 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-05-21T15:27:04.586 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-05-21T15:27:04.586 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-05-21T15:27:04.586 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-05-21T15:27:04.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.586 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-05-21T15:27:04.586 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-05-21T15:27:04.586 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-05-21T15:27:04.587 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 453, 2024-05-21T15:27:04.587 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 658978, 2024-05-21T15:27:04.587 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 33, 2024-05-21T15:27:04.587 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 453, 2024-05-21T15:27:04.587 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-05-21T15:27:04.587 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 453, 2024-05-21T15:27:04.587 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 21.284681871, 2024-05-21T15:27:04.587 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.046986052 2024-05-21T15:27:04.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.588 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-05-21T15:27:04.588 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-05-21T15:27:04.588 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 453, 2024-05-21T15:27:04.588 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 518 2024-05-21T15:27:04.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.588 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 453, 2024-05-21T15:27:04.588 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 287, 2024-05-21T15:27:04.589 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 166, 2024-05-21T15:27:04.589 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-05-21T15:27:04.589 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-05-21T15:27:04.589 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 0, 2024-05-21T15:27:04.589 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 10, 2024-05-21T15:27:04.589 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 0, 2024-05-21T15:27:04.589 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 78, 2024-05-21T15:27:04.589 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 19, 2024-05-21T15:27:04.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-05-21T15:27:04.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-05-21T15:27:04.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-05-21T15:27:04.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-05-21T15:27:04.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 6, 2024-05-21T15:27:04.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-05-21T15:27:04.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-05-21T15:27:04.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-05-21T15:27:04.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 285, 2024-05-21T15:27:04.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 20, 2024-05-21T15:27:04.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:27:04.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-05-21T15:27:04.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:27:04.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-05-21T15:27:04.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-05-21T15:27:04.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-05-21T15:27:04.592 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:27:04.592 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-05-21T15:27:04.592 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:27:04.592 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-05-21T15:27:04.592 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-05-21T15:27:04.592 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-05-21T15:27:04.592 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-05-21T15:27:04.592 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-05-21T15:27:04.593 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-05-21T15:27:04.593 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-05-21T15:27:04.593 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-05-21T15:27:04.593 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-05-21T15:27:04.593 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-05-21T15:27:04.593 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-05-21T15:27:04.593 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-05-21T15:27:04.593 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-05-21T15:27:04.594 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-05-21T15:27:04.594 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-05-21T15:27:04.594 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-05-21T15:27:04.594 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-05-21T15:27:04.594 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 98, 2024-05-21T15:27:04.594 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-05-21T15:27:04.594 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 13, 2024-05-21T15:27:04.594 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 10, 2024-05-21T15:27:04.595 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 10, 2024-05-21T15:27:04.595 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 0, 2024-05-21T15:27:04.595 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-05-21T15:27:04.595 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 75, 2024-05-21T15:27:04.595 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 3, 2024-05-21T15:27:04.595 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 22 2024-05-21T15:27:04.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.595 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 1, 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.596 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 0, 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 0, 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 0, 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.597 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 3652, 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 2665250, 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 3652, 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:27:04.598 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:27:04.599 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 3652, 2024-05-21T15:27:04.599 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 2665250, 2024-05-21T15:27:04.599 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:04.599 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.599 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.599 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.599 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:04.600 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 453, 2024-05-21T15:27:04.601 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 658934, 2024-05-21T15:27:04.601 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 453, 2024-05-21T15:27:04.601 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 658934, 2024-05-21T15:27:04.601 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:04.601 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.601 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.601 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.601 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.602 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-05-21T15:27:04.602 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:04.602 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-05-21T15:27:04.602 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:04.602 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:27:04.602 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:27:04.602 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:04.603 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:04.603 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 453, 2024-05-21T15:27:04.603 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 453, 2024-05-21T15:27:04.603 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 453, 2024-05-21T15:27:04.603 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 453, 2024-05-21T15:27:04.603 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:04.603 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.603 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.604 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.604 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.604 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:27:04.604 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:04.604 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-05-21T15:27:04.604 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:04.604 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:04.605 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.606 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.606 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.606 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-05-21T15:27:04.606 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x562583a04b60": { 2024-05-21T15:27:04.606 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-05-21T15:27:04.606 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-05-21T15:27:04.606 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-05-21T15:27:04.607 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 522, 2024-05-21T15:27:04.607 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 640523, 2024-05-21T15:27:04.607 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:04.607 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 522, 2024-05-21T15:27:04.607 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-05-21T15:27:04.607 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-05-21T15:27:04.607 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 78, 2024-05-21T15:27:04.607 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 640523, 2024-05-21T15:27:04.608 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-05-21T15:27:04.608 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-05-21T15:27:04.608 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-05-21T15:27:04.608 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-05-21T15:27:04.608 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.608 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-05-21T15:27:04.608 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-05-21T15:27:05.299 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-21T15:27:05.299 INFO:tasks.check_counter:Daemon mds.k mds.imported=2 2024-05-21T15:27:05.300 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-21T15:27:05.300 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=258 2024-05-21T15:27:05.300 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-05-21T15:27:05.300 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:27:05.300 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:27:05.300 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-05-21T15:27:05.300 DEBUG:tasks.check_counter:Getting stats from c 2024-05-21T15:27:05.300 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-21T15:27:05.558 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:27:06.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:06 smithi172 ceph-mon[36402]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T15:27:06.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:06 smithi078 ceph-mon[31997]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T15:27:06.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:06 smithi084 ceph-mon[36334]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T15:27:07.389 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-05-21T15:27:07.389 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:27:07.389 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 326, 2024-05-21T15:27:07.389 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 465, 2024-05-21T15:27:07.389 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 180912, 2024-05-21T15:27:07.390 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1728543, 2024-05-21T15:27:07.390 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 16, 2024-05-21T15:27:07.390 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-05-21T15:27:07.390 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.054430435, 2024-05-21T15:27:07.390 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.022080949, 2024-05-21T15:27:07.390 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.013309474, 2024-05-21T15:27:07.390 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.009515125, 2024-05-21T15:27:07.390 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:07.391 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 459, 2024-05-21T15:27:07.391 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.062519195, 2024-05-21T15:27:07.391 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000136207 2024-05-21T15:27:07.391 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.391 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:07.391 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 106, 2024-05-21T15:27:07.391 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000025938, 2024-05-21T15:27:07.392 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000244 2024-05-21T15:27:07.392 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.392 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:27:07.392 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 324784 2024-05-21T15:27:07.392 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.392 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:27:07.392 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1736, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1867, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 2163469, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 786590, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 9, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 7, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.173697133, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.060554026, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.164557270, 2024-05-21T15:27:07.393 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.010704875, 2024-05-21T15:27:07.394 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:07.394 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1867, 2024-05-21T15:27:07.394 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.783848573, 2024-05-21T15:27:07.394 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000419843 2024-05-21T15:27:07.394 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.394 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:07.394 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2050, 2024-05-21T15:27:07.395 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000634319, 2024-05-21T15:27:07.395 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000309 2024-05-21T15:27:07.395 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.395 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1588896, 2024-05-21T15:27:07.395 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 152960 2024-05-21T15:27:07.395 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.395 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:27:07.395 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2632, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 2113, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1109338, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1298683, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 50, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.228633283, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.073698114, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.336082012, 2024-05-21T15:27:07.396 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.013906448, 2024-05-21T15:27:07.397 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:07.397 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2114, 2024-05-21T15:27:07.397 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.349976832, 2024-05-21T15:27:07.397 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000165551 2024-05-21T15:27:07.397 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.397 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:07.397 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4028, 2024-05-21T15:27:07.397 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001185018, 2024-05-21T15:27:07.398 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000294 2024-05-21T15:27:07.398 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.398 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-21T15:27:07.398 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 126992 2024-05-21T15:27:07.398 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.398 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-05-21T15:27:07.398 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 637, 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.501687237, 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000787578 2024-05-21T15:27:07.399 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.400 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.400 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-05-21T15:27:07.400 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-05-21T15:27:07.400 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-05-21T15:27:07.400 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2, 2024-05-21T15:27:07.400 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000010803, 2024-05-21T15:27:07.400 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000005401 2024-05-21T15:27:07.400 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.401 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.401 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-05-21T15:27:07.401 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 78, 2024-05-21T15:27:07.401 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 64, 2024-05-21T15:27:07.401 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-05-21T15:27:07.401 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 64, 2024-05-21T15:27:07.401 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 13.419412115, 2024-05-21T15:27:07.401 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.209678314 2024-05-21T15:27:07.402 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.402 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-05-21T15:27:07.402 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 14, 2024-05-21T15:27:07.402 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 2, 2024-05-21T15:27:07.402 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-05-21T15:27:07.402 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 11, 2024-05-21T15:27:07.402 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-05-21T15:27:07.402 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-05-21T15:27:07.403 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 181, 2024-05-21T15:27:07.403 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 51, 2024-05-21T15:27:07.403 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 120, 2024-05-21T15:27:07.403 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 10, 2024-05-21T15:27:07.403 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 109, 2024-05-21T15:27:07.403 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 38, 2024-05-21T15:27:07.403 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 25, 2024-05-21T15:27:07.403 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 25, 2024-05-21T15:27:07.404 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 13, 2024-05-21T15:27:07.404 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 308, 2024-05-21T15:27:07.404 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 200, 2024-05-21T15:27:07.404 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 14, 2024-05-21T15:27:07.404 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 94, 2024-05-21T15:27:07.404 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:27:07.404 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-05-21T15:27:07.404 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-05-21T15:27:07.405 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-05-21T15:27:07.405 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-05-21T15:27:07.405 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 1, 2024-05-21T15:27:07.405 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 22, 2024-05-21T15:27:07.405 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 2, 2024-05-21T15:27:07.405 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 52, 2024-05-21T15:27:07.405 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-05-21T15:27:07.405 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 438, 2024-05-21T15:27:07.406 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-05-21T15:27:07.406 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 325, 2024-05-21T15:27:07.406 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 13424230508, 2024-05-21T15:27:07.406 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 6, 2024-05-21T15:27:07.406 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 438, 2024-05-21T15:27:07.406 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-05-21T15:27:07.406 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:27:07.406 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:27:07.407 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:27:07.407 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:27:07.407 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:27:07.407 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 438, 2024-05-21T15:27:07.407 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 30, 2024-05-21T15:27:07.407 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 146, 2024-05-21T15:27:07.407 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 122, 2024-05-21T15:27:07.407 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:27:07.408 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:27:07.408 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:27:07.408 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 1055, 2024-05-21T15:27:07.408 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 849, 2024-05-21T15:27:07.408 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 2, 2024-05-21T15:27:07.408 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-05-21T15:27:07.408 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.408 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 3, 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 260, 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 175, 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 160, 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 3, 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:27:07.409 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-05-21T15:27:07.410 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-05-21T15:27:07.410 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-05-21T15:27:07.410 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-05-21T15:27:07.410 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-05-21T15:27:07.410 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:27:07.410 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:27:07.410 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-05-21T15:27:07.410 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 1, 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-05-21T15:27:07.411 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.412 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-05-21T15:27:07.412 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-05-21T15:27:07.412 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 953, 2024-05-21T15:27:07.412 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-05-21T15:27:07.412 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-05-21T15:27:07.412 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 953, 2024-05-21T15:27:07.412 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 2, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-05-21T15:27:07.413 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-05-21T15:27:07.414 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 5408932, 2024-05-21T15:27:07.414 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-05-21T15:27:07.414 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-05-21T15:27:07.415 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 144, 2024-05-21T15:27:07.415 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 4.277698556, 2024-05-21T15:27:07.415 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.029706239 2024-05-21T15:27:07.415 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.415 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-05-21T15:27:07.415 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.415 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-05-21T15:27:07.415 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 185, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 215, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 30, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 101, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 119, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 18, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 181, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 219, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 38, 2024-05-21T15:27:07.416 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 25, 2024-05-21T15:27:07.417 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 48, 2024-05-21T15:27:07.417 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 23, 2024-05-21T15:27:07.417 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 126576, 2024-05-21T15:27:07.417 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223636 2024-05-21T15:27:07.417 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.417 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-05-21T15:27:07.417 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 114, 2024-05-21T15:27:07.417 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 23, 2024-05-21T15:27:07.418 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 78, 2024-05-21T15:27:07.418 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 14, 2024-05-21T15:27:07.418 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 47, 2024-05-21T15:27:07.418 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-05-21T15:27:07.418 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.418 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.418 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.418 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.419 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-05-21T15:27:07.419 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 44, 2024-05-21T15:27:07.419 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 13.227600421, 2024-05-21T15:27:07.419 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.300627282 2024-05-21T15:27:07.419 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.419 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-05-21T15:27:07.419 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.419 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.420 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.420 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.420 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-05-21T15:27:07.420 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.420 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.420 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.420 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.420 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 19, 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.191662599, 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.010087505 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.421 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.422 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-05-21T15:27:07.422 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-05-21T15:27:07.422 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000149095, 2024-05-21T15:27:07.422 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000149095 2024-05-21T15:27:07.422 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.422 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-05-21T15:27:07.422 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.422 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.423 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.423 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.423 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-05-21T15:27:07.423 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.423 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.423 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.423 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.423 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-05-21T15:27:07.424 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.425 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.425 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.425 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.425 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-05-21T15:27:07.425 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.426 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.426 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.427 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.427 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-05-21T15:27:07.427 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.427 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.427 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.427 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.427 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-05-21T15:27:07.427 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.428 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.428 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.428 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.428 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-05-21T15:27:07.428 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.428 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.428 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.428 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-05-21T15:27:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-05-21T15:27:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.430 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.430 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.430 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-05-21T15:27:07.430 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.430 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.430 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.430 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.431 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-05-21T15:27:07.431 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.431 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.431 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.431 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.431 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-05-21T15:27:07.431 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.431 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.432 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.432 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.432 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-05-21T15:27:07.432 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.432 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.432 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.432 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.432 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-05-21T15:27:07.433 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.433 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.433 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.433 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.433 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-05-21T15:27:07.433 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.433 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.433 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.434 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.434 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-05-21T15:27:07.434 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.434 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.434 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.434 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.434 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-05-21T15:27:07.434 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.435 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.435 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.435 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.435 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-05-21T15:27:07.435 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.435 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.435 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.435 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.436 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-05-21T15:27:07.436 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.436 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.436 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.436 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.436 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-05-21T15:27:07.436 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.436 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-05-21T15:27:07.437 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.438 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.438 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.438 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.438 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:27:07.438 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:27:07.438 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-05-21T15:27:07.438 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 1, 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 2, 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 1, 2024-05-21T15:27:07.439 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 1, 2024-05-21T15:27:07.440 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-05-21T15:27:07.440 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 31, 2024-05-21T15:27:07.440 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 31, 2024-05-21T15:27:07.440 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 646, 2024-05-21T15:27:07.440 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-05-21T15:27:07.440 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.440 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-05-21T15:27:07.440 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:27:07.441 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-05-21T15:27:07.441 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:27:07.441 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:27:07.441 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:27:07.441 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:27:07.441 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:27:07.441 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:27:07.441 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:27:07.442 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:27:07.442 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:27:07.442 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:27:07.442 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:27:07.442 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:27:07.442 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:27:07.442 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-05-21T15:27:07.442 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:27:07.443 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-05-21T15:27:07.443 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:27:07.443 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:27:07.443 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:27:07.443 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:27:07.443 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:27:07.443 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:27:07.443 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:27:07.444 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-05-21T15:27:07.444 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:27:07.444 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:27:07.444 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:27:07.444 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-05-21T15:27:07.444 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-05-21T15:27:07.444 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-05-21T15:27:07.444 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:27:07.445 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:27:07.445 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:27:07.445 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:27:07.445 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 294669, 2024-05-21T15:27:07.445 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 228, 2024-05-21T15:27:07.445 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:27:07.445 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-05-21T15:27:07.445 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-05-21T15:27:07.446 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-05-21T15:27:07.446 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:27:07.446 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-05-21T15:27:07.446 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:27:07.446 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-05-21T15:27:07.446 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8512, 2024-05-21T15:27:07.446 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 177, 2024-05-21T15:27:07.446 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:27:07.447 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:27:07.447 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-05-21T15:27:07.447 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-05-21T15:27:07.447 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 697127, 2024-05-21T15:27:07.447 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 4750, 2024-05-21T15:27:07.447 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-05-21T15:27:07.447 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-05-21T15:27:07.447 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 663, 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 1234566, 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 33, 2024-05-21T15:27:07.448 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 663, 2024-05-21T15:27:07.449 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-05-21T15:27:07.449 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 663, 2024-05-21T15:27:07.449 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 10.368755121, 2024-05-21T15:27:07.449 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.015639147 2024-05-21T15:27:07.449 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.449 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-05-21T15:27:07.449 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-05-21T15:27:07.449 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 663, 2024-05-21T15:27:07.450 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 715 2024-05-21T15:27:07.450 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.450 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 663, 2024-05-21T15:27:07.450 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 444, 2024-05-21T15:27:07.450 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 219, 2024-05-21T15:27:07.450 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-05-21T15:27:07.450 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-05-21T15:27:07.450 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-05-21T15:27:07.451 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-05-21T15:27:07.451 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 0, 2024-05-21T15:27:07.451 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 144, 2024-05-21T15:27:07.451 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 19, 2024-05-21T15:27:07.451 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-05-21T15:27:07.451 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-05-21T15:27:07.451 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-05-21T15:27:07.451 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-05-21T15:27:07.452 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 6, 2024-05-21T15:27:07.452 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-05-21T15:27:07.452 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-05-21T15:27:07.452 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-05-21T15:27:07.452 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 438, 2024-05-21T15:27:07.452 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-05-21T15:27:07.452 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:27:07.452 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-05-21T15:27:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:27:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-05-21T15:27:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-05-21T15:27:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-05-21T15:27:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:27:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-05-21T15:27:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:27:07.453 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-05-21T15:27:07.454 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-05-21T15:27:07.455 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-05-21T15:27:07.455 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-05-21T15:27:07.455 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-05-21T15:27:07.455 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-05-21T15:27:07.455 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-05-21T15:27:07.455 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-05-21T15:27:07.455 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-05-21T15:27:07.455 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 98, 2024-05-21T15:27:07.456 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-05-21T15:27:07.456 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 13, 2024-05-21T15:27:07.456 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 12, 2024-05-21T15:27:07.456 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 12, 2024-05-21T15:27:07.456 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-05-21T15:27:07.456 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-05-21T15:27:07.456 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 53, 2024-05-21T15:27:07.456 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 8, 2024-05-21T15:27:07.457 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 17 2024-05-21T15:27:07.457 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.457 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-05-21T15:27:07.457 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-05-21T15:27:07.457 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:27:07.457 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-05-21T15:27:07.457 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-05-21T15:27:07.457 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.458 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-05-21T15:27:07.458 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-05-21T15:27:07.458 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.458 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-05-21T15:27:07.458 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-05-21T15:27:07.458 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:27:07.458 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-05-21T15:27:07.459 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-05-21T15:27:07.459 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-05-21T15:27:07.459 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-05-21T15:27:07.459 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-05-21T15:27:07.459 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.459 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:27:07.459 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:07.459 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-05-21T15:27:07.460 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:07.460 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 4690, 2024-05-21T15:27:07.460 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 3012508, 2024-05-21T15:27:07.460 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:07.460 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 4690, 2024-05-21T15:27:07.460 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:07.460 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:07.460 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4690, 2024-05-21T15:27:07.461 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 3012508, 2024-05-21T15:27:07.461 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:07.461 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.461 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.461 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.461 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.461 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.461 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-05-21T15:27:07.462 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:07.462 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-05-21T15:27:07.462 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:07.462 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:07.462 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:07.462 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:07.462 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:07.463 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 663, 2024-05-21T15:27:07.463 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 1234413, 2024-05-21T15:27:07.463 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 663, 2024-05-21T15:27:07.463 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1234413, 2024-05-21T15:27:07.463 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:07.463 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.463 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.463 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.464 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.464 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.464 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-05-21T15:27:07.464 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:07.464 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-05-21T15:27:07.464 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:07.464 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:07.464 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:07.465 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:07.465 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:07.465 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 663, 2024-05-21T15:27:07.465 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 663, 2024-05-21T15:27:07.465 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 663, 2024-05-21T15:27:07.465 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 663, 2024-05-21T15:27:07.465 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:07.465 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.466 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.466 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.466 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.466 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.466 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:27:07.466 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:07.466 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-05-21T15:27:07.466 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-05-21T15:27:07.467 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:07.468 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.468 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.468 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.468 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.468 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:07.468 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x558f11aa28f0": { 2024-05-21T15:27:07.468 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:07.468 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-05-21T15:27:07.469 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:07.469 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 953, 2024-05-21T15:27:07.469 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 1214628, 2024-05-21T15:27:07.469 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:07.469 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 953, 2024-05-21T15:27:07.469 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:07.469 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:07.469 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 144, 2024-05-21T15:27:07.470 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1214628, 2024-05-21T15:27:07.470 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:07.470 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:07.470 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:07.470 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:07.470 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.470 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:07.470 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-05-21T15:27:08.193 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-05-21T15:27:08.194 INFO:tasks.check_counter:Daemon mds.c mds.imported=2 2024-05-21T15:27:08.194 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=3 2024-05-21T15:27:08.194 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=260 2024-05-21T15:27:08.194 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-21T15:27:08.194 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:27:08.194 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:27:08.194 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-21T15:27:08.195 DEBUG:tasks.check_counter:Getting stats from f 2024-05-21T15:27:08.195 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-21T15:27:08.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:08 smithi078 ceph-mon[31997]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:27:08.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:08 smithi084 ceph-mon[36334]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:27:08.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:08 smithi172 ceph-mon[36402]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T15:27:08.443 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:27:09.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:09 smithi172 ceph-mon[36402]: osdmap e99: 12 total, 12 up, 12 in 2024-05-21T15:27:09.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:09 smithi078 ceph-mon[31997]: osdmap e99: 12 total, 12 up, 12 in 2024-05-21T15:27:09.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:09 smithi084 ceph-mon[36334]: osdmap e99: 12 total, 12 up, 12 in 2024-05-21T15:27:10.242 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:10 smithi172 ceph-mon[36402]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T15:27:10.316 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-05-21T15:27:10.317 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:27:10.317 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1654, 2024-05-21T15:27:10.317 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1656, 2024-05-21T15:27:10.317 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 8477770, 2024-05-21T15:27:10.317 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 525982, 2024-05-21T15:27:10.317 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 6, 2024-05-21T15:27:10.317 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 2, 2024-05-21T15:27:10.317 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.182466613, 2024-05-21T15:27:10.318 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.052718543, 2024-05-21T15:27:10.318 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.054988419, 2024-05-21T15:27:10.318 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.058894580, 2024-05-21T15:27:10.318 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:10.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1654, 2024-05-21T15:27:10.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.072380632, 2024-05-21T15:27:10.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000043760 2024-05-21T15:27:10.318 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.319 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:10.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.319 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.319 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T15:27:10.319 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T15:27:10.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.320 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:27:10.320 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 63, 2024-05-21T15:27:10.320 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 226, 2024-05-21T15:27:10.320 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 8405219, 2024-05-21T15:27:10.320 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 471496, 2024-05-21T15:27:10.320 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 7, 2024-05-21T15:27:10.320 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-05-21T15:27:10.320 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.021073374, 2024-05-21T15:27:10.321 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.007662973, 2024-05-21T15:27:10.321 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.009429138, 2024-05-21T15:27:10.321 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.001816649, 2024-05-21T15:27:10.321 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:10.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 223, 2024-05-21T15:27:10.321 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.018963384, 2024-05-21T15:27:10.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000085037 2024-05-21T15:27:10.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.322 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:10.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.322 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.322 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.322 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T15:27:10.322 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 452032 2024-05-21T15:27:10.322 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.323 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:27:10.323 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1019, 2024-05-21T15:27:10.323 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 990, 2024-05-21T15:27:10.323 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 5957863, 2024-05-21T15:27:10.323 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 375585, 2024-05-21T15:27:10.323 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-05-21T15:27:10.323 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-05-21T15:27:10.323 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.131494936, 2024-05-21T15:27:10.324 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.037292383, 2024-05-21T15:27:10.324 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.062690530, 2024-05-21T15:27:10.324 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.035791940, 2024-05-21T15:27:10.324 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:10.324 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 990, 2024-05-21T15:27:10.324 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.045674931, 2024-05-21T15:27:10.324 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000046136 2024-05-21T15:27:10.324 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.325 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:10.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.325 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.325 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.325 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1578112, 2024-05-21T15:27:10.325 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 136448 2024-05-21T15:27:10.325 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4102, 2024-05-21T15:27:10.326 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.141174535, 2024-05-21T15:27:10.327 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000034416 2024-05-21T15:27:10.327 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.327 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.327 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-05-21T15:27:10.327 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-05-21T15:27:10.327 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-05-21T15:27:10.327 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.327 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.328 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.329 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.329 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.329 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-05-21T15:27:10.329 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-05-21T15:27:10.329 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-05-21T15:27:10.329 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-05-21T15:27:10.329 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-05-21T15:27:10.329 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-05-21T15:27:10.330 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-05-21T15:27:10.330 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 599, 2024-05-21T15:27:10.330 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 394, 2024-05-21T15:27:10.330 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 205, 2024-05-21T15:27:10.330 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-05-21T15:27:10.330 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 37, 2024-05-21T15:27:10.330 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-05-21T15:27:10.330 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-05-21T15:27:10.331 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-05-21T15:27:10.331 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 18, 2024-05-21T15:27:10.331 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-05-21T15:27:10.331 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-05-21T15:27:10.331 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-05-21T15:27:10.331 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-05-21T15:27:10.331 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:27:10.331 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-05-21T15:27:10.332 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-05-21T15:27:10.333 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-05-21T15:27:10.333 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 325, 2024-05-21T15:27:10.333 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 13424230508, 2024-05-21T15:27:10.333 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 6, 2024-05-21T15:27:10.333 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T15:27:10.333 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-05-21T15:27:10.333 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:27:10.333 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:27:10.334 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:27:10.334 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:27:10.334 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:27:10.334 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-05-21T15:27:10.334 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-05-21T15:27:10.334 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T15:27:10.334 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-05-21T15:27:10.334 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-05-21T15:27:10.335 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-05-21T15:27:10.336 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-05-21T15:27:10.336 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-05-21T15:27:10.336 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-05-21T15:27:10.336 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-05-21T15:27:10.336 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-05-21T15:27:10.336 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-05-21T15:27:10.336 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:27:10.336 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-05-21T15:27:10.337 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-05-21T15:27:10.337 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-05-21T15:27:10.337 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-05-21T15:27:10.337 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-05-21T15:27:10.337 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:27:10.337 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:27:10.337 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-05-21T15:27:10.337 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-05-21T15:27:10.338 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:27:10.338 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:27:10.338 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:27:10.338 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-05-21T15:27:10.338 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-05-21T15:27:10.338 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:27:10.338 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-05-21T15:27:10.338 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-05-21T15:27:10.339 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.339 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-05-21T15:27:10.339 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-05-21T15:27:10.339 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-05-21T15:27:10.339 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-05-21T15:27:10.339 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-05-21T15:27:10.339 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 7444, 2024-05-21T15:27:10.339 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 19, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-05-21T15:27:10.340 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 23912563, 2024-05-21T15:27:10.341 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 24766345, 2024-05-21T15:27:10.341 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 24766285, 2024-05-21T15:27:10.341 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-05-21T15:27:10.341 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.341 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.341 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.341 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.341 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 7444 2024-05-21T15:27:10.342 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.342 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-05-21T15:27:10.342 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 71, 2024-05-21T15:27:10.342 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 990, 2024-05-21T15:27:10.342 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 919, 2024-05-21T15:27:10.342 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 30, 2024-05-21T15:27:10.342 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 126, 2024-05-21T15:27:10.342 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 96, 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 599, 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 1233, 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 634, 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 123540, 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215444 2024-05-21T15:27:10.343 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.344 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-05-21T15:27:10.344 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-05-21T15:27:10.344 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-05-21T15:27:10.344 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-05-21T15:27:10.344 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-05-21T15:27:10.344 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-05-21T15:27:10.344 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-05-21T15:27:10.344 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.345 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.345 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.345 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.345 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-05-21T15:27:10.345 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.345 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.345 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.345 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.346 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-05-21T15:27:10.346 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.346 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.346 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.346 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.346 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-05-21T15:27:10.346 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.346 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.347 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.347 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.347 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-05-21T15:27:10.347 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.347 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.347 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.347 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.347 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-05-21T15:27:10.348 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.348 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.348 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.348 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.348 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-05-21T15:27:10.348 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.348 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.348 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.349 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.349 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-05-21T15:27:10.349 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.349 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.349 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.349 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.349 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-05-21T15:27:10.349 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.350 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.350 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.350 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.350 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-05-21T15:27:10.350 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.350 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.350 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.350 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.351 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-05-21T15:27:10.351 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.351 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.351 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.351 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:10 smithi078 ceph-mon[31997]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T15:27:10.352 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-05-21T15:27:10.352 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.352 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-05-21T15:27:10.353 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.354 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.354 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.354 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.354 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-05-21T15:27:10.354 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.354 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.354 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.354 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.355 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-05-21T15:27:10.355 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.355 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.355 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.355 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.355 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-05-21T15:27:10.355 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.355 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.356 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.356 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.356 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-05-21T15:27:10.356 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.356 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.356 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.356 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.356 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.357 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.358 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-05-21T15:27:10.358 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.358 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.358 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.358 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.358 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-05-21T15:27:10.358 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.359 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.359 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.359 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.359 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-05-21T15:27:10.359 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.359 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.359 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.359 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.360 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-05-21T15:27:10.360 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.360 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.360 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.360 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.360 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-05-21T15:27:10.360 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.360 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-05-21T15:27:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-05-21T15:27:10.362 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.362 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.362 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.362 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.362 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-05-21T15:27:10.362 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.362 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.362 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.363 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.363 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-05-21T15:27:10.363 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.363 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.363 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.363 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.363 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:27:10.363 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-05-21T15:27:10.364 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 5, 2024-05-21T15:27:10.365 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 3, 2024-05-21T15:27:10.365 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 4, 2024-05-21T15:27:10.365 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-05-21T15:27:10.365 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-05-21T15:27:10.365 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-05-21T15:27:10.365 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 3598, 2024-05-21T15:27:10.365 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-05-21T15:27:10.365 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.366 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-05-21T15:27:10.366 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:27:10.366 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-05-21T15:27:10.366 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:27:10.366 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:27:10.366 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:27:10.366 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:27:10.367 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:27:10.368 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-05-21T15:27:10.368 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:27:10.368 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-05-21T15:27:10.368 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:27:10.368 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:27:10.368 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:27:10.368 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:27:10.368 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-05-21T15:27:10.369 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-05-21T15:27:10.370 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:27:10.370 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:27:10.370 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:27:10.370 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:27:10.370 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 201153, 2024-05-21T15:27:10.370 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 135, 2024-05-21T15:27:10.370 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:27:10.371 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-05-21T15:27:10.371 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-05-21T15:27:10.371 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-05-21T15:27:10.371 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:27:10.371 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-05-21T15:27:10.371 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:27:10.371 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-05-21T15:27:10.371 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8632, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 180, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 659146, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 4339, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-05-21T15:27:10.372 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-05-21T15:27:10.373 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-05-21T15:27:10.373 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-05-21T15:27:10.373 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.373 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-05-21T15:27:10.373 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-05-21T15:27:10.373 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-05-21T15:27:10.373 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 2467, 2024-05-21T15:27:10.373 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 26, 2024-05-21T15:27:10.374 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 4, 2024-05-21T15:27:10.374 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 2467, 2024-05-21T15:27:10.374 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-05-21T15:27:10.374 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2467, 2024-05-21T15:27:10.374 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 6.677841974, 2024-05-21T15:27:10.374 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002706867 2024-05-21T15:27:10.374 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.374 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-05-21T15:27:10.375 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-05-21T15:27:10.375 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2467, 2024-05-21T15:27:10.375 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2468 2024-05-21T15:27:10.375 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.375 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 2467, 2024-05-21T15:27:10.375 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 2467, 2024-05-21T15:27:10.375 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-05-21T15:27:10.375 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-05-21T15:27:10.376 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-05-21T15:27:10.376 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 1532, 2024-05-21T15:27:10.376 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-05-21T15:27:10.376 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 932, 2024-05-21T15:27:10.376 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-05-21T15:27:10.376 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-05-21T15:27:10.376 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-05-21T15:27:10.376 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 2, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-05-21T15:27:10.377 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:27:10.378 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-05-21T15:27:10.378 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:27:10.378 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-05-21T15:27:10.378 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-05-21T15:27:10.378 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-05-21T15:27:10.378 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:27:10.378 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-05-21T15:27:10.378 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:27:10.379 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-05-21T15:27:10.379 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-05-21T15:27:10.379 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-05-21T15:27:10.379 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-05-21T15:27:10.379 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-05-21T15:27:10.379 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-05-21T15:27:10.379 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-05-21T15:27:10.379 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-05-21T15:27:10.380 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-05-21T15:27:10.380 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-05-21T15:27:10.380 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-05-21T15:27:10.380 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-05-21T15:27:10.380 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-05-21T15:27:10.380 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-05-21T15:27:10.380 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-05-21T15:27:10.380 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 99, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 14, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 6, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 6, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-05-21T15:27:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-05-21T15:27:10.382 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-05-21T15:27:10.382 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-05-21T15:27:10.382 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.382 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-05-21T15:27:10.382 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-05-21T15:27:10.382 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:27:10.382 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-05-21T15:27:10.382 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-05-21T15:27:10.383 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.383 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-05-21T15:27:10.383 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-05-21T15:27:10.383 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.383 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-05-21T15:27:10.383 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-05-21T15:27:10.383 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:27:10.383 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-05-21T15:27:10.384 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:10.385 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 2732, 2024-05-21T15:27:10.385 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 22578558, 2024-05-21T15:27:10.385 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:10.385 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 2732, 2024-05-21T15:27:10.385 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:10.385 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:10.385 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 2732, 2024-05-21T15:27:10.385 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 22578558, 2024-05-21T15:27:10.386 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:10.386 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.386 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.386 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.386 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.386 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.386 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-05-21T15:27:10.386 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:10.387 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-05-21T15:27:10.387 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:10.387 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:10.387 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:10.387 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:10.387 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:10.387 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 2467, 2024-05-21T15:27:10.387 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 20572056, 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 2467, 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 20572056, 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.388 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-05-21T15:27:10.389 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:10.389 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-05-21T15:27:10.389 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:10.389 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:10.389 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:10.389 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:10.389 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:10.389 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 2467, 2024-05-21T15:27:10.390 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 2467, 2024-05-21T15:27:10.390 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 2467, 2024-05-21T15:27:10.390 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 2467, 2024-05-21T15:27:10.390 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:10.390 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.390 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.390 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.390 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:10.391 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:10.392 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:10.392 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:10.392 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-05-21T15:27:10.392 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-05-21T15:27:10.392 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:10.392 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.392 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.392 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55acfcdba9c0": { 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:10.393 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:10.394 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:10.394 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:10.394 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:10.394 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-05-21T15:27:10.394 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-05-21T15:27:10.394 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:10.394 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:10.395 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:10.395 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:10.395 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.395 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:10.395 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-05-21T15:27:10.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:10 smithi084 ceph-mon[36334]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T15:27:11.107 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-21T15:27:11.108 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-21T15:27:11.108 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-21T15:27:11.108 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-21T15:27:11.108 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-21T15:27:11.108 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:27:11.108 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:27:11.108 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-05-21T15:27:11.108 DEBUG:tasks.check_counter:Getting stats from i 2024-05-21T15:27:11.109 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-21T15:27:11.347 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:27:12.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:12 smithi078 ceph-mon[31997]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T15:27:12.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:12 smithi084 ceph-mon[36334]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T15:27:12.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:12 smithi172 ceph-mon[36402]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T15:27:13.105 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-05-21T15:27:13.105 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:27:13.105 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 3465, 2024-05-21T15:27:13.106 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 3195, 2024-05-21T15:27:13.106 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1865809, 2024-05-21T15:27:13.106 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 3458694, 2024-05-21T15:27:13.106 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 41, 2024-05-21T15:27:13.106 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 7, 2024-05-21T15:27:13.106 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.288476414, 2024-05-21T15:27:13.106 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.100775994, 2024-05-21T15:27:13.106 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.163066553, 2024-05-21T15:27:13.107 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.030221938, 2024-05-21T15:27:13.107 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:13.107 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3194, 2024-05-21T15:27:13.107 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.207738783, 2024-05-21T15:27:13.107 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000065040 2024-05-21T15:27:13.107 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.107 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:13.108 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4049, 2024-05-21T15:27:13.108 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001247401, 2024-05-21T15:27:13.108 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000308 2024-05-21T15:27:13.108 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.108 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-05-21T15:27:13.108 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 327088 2024-05-21T15:27:13.109 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.109 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:27:13.109 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2219, 2024-05-21T15:27:13.109 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 2279, 2024-05-21T15:27:13.109 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 2759870, 2024-05-21T15:27:13.109 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1009642, 2024-05-21T15:27:13.109 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 20, 2024-05-21T15:27:13.109 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-05-21T15:27:13.110 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.175668187, 2024-05-21T15:27:13.110 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.055695602, 2024-05-21T15:27:13.110 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.081172207, 2024-05-21T15:27:13.110 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.036198646, 2024-05-21T15:27:13.110 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:13.110 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2278, 2024-05-21T15:27:13.110 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.125089374, 2024-05-21T15:27:13.111 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000054911 2024-05-21T15:27:13.111 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.111 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:13.111 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 275, 2024-05-21T15:27:13.111 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000076920, 2024-05-21T15:27:13.111 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000279 2024-05-21T15:27:13.111 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.112 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1594768, 2024-05-21T15:27:13.112 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 165104 2024-05-21T15:27:13.112 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.112 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:27:13.112 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2634, 2024-05-21T15:27:13.112 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 3052, 2024-05-21T15:27:13.113 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1335765, 2024-05-21T15:27:13.113 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1361171, 2024-05-21T15:27:13.113 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 17, 2024-05-21T15:27:13.113 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-05-21T15:27:13.113 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.204267399, 2024-05-21T15:27:13.113 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.064563616, 2024-05-21T15:27:13.113 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.118751600, 2024-05-21T15:27:13.114 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.033179267, 2024-05-21T15:27:13.114 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:13.114 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3049, 2024-05-21T15:27:13.114 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.522600118, 2024-05-21T15:27:13.114 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000171400 2024-05-21T15:27:13.114 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.115 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:13.115 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1762, 2024-05-21T15:27:13.115 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000535618, 2024-05-21T15:27:13.115 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000303 2024-05-21T15:27:13.115 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.115 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 2000, 2024-05-21T15:27:13.115 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 128704 2024-05-21T15:27:13.116 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.116 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-05-21T15:27:13.116 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-05-21T15:27:13.116 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-05-21T15:27:13.116 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.117 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-05-21T15:27:13.117 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-05-21T15:27:13.117 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-05-21T15:27:13.117 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2855, 2024-05-21T15:27:13.117 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 1.023880006, 2024-05-21T15:27:13.117 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000358626 2024-05-21T15:27:13.117 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.118 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.118 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-05-21T15:27:13.118 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-05-21T15:27:13.118 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-05-21T15:27:13.118 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2, 2024-05-21T15:27:13.118 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000010755, 2024-05-21T15:27:13.118 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000005377 2024-05-21T15:27:13.119 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.119 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.119 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-05-21T15:27:13.119 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 55, 2024-05-21T15:27:13.119 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 42, 2024-05-21T15:27:13.119 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-05-21T15:27:13.120 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 42, 2024-05-21T15:27:13.120 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2.155521099, 2024-05-21T15:27:13.120 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.051321930 2024-05-21T15:27:13.120 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.120 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-05-21T15:27:13.120 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 13, 2024-05-21T15:27:13.120 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 1, 2024-05-21T15:27:13.121 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-05-21T15:27:13.121 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 19, 2024-05-21T15:27:13.121 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 1, 2024-05-21T15:27:13.121 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-05-21T15:27:13.121 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 544, 2024-05-21T15:27:13.121 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 165, 2024-05-21T15:27:13.121 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 379, 2024-05-21T15:27:13.121 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-05-21T15:27:13.122 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 308, 2024-05-21T15:27:13.122 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 16, 2024-05-21T15:27:13.122 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 179, 2024-05-21T15:27:13.122 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 179, 2024-05-21T15:27:13.122 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 19, 2024-05-21T15:27:13.122 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 275, 2024-05-21T15:27:13.123 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 164, 2024-05-21T15:27:13.123 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 11, 2024-05-21T15:27:13.123 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 98, 2024-05-21T15:27:13.123 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:27:13.123 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-05-21T15:27:13.123 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-05-21T15:27:13.123 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-05-21T15:27:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-05-21T15:27:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 1, 2024-05-21T15:27:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-05-21T15:27:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 9, 2024-05-21T15:27:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 381, 2024-05-21T15:27:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-05-21T15:27:13.125 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 4428, 2024-05-21T15:27:13.125 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-05-21T15:27:13.125 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 325, 2024-05-21T15:27:13.125 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 13424230508, 2024-05-21T15:27:13.125 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 6, 2024-05-21T15:27:13.125 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 4428, 2024-05-21T15:27:13.125 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-05-21T15:27:13.126 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:27:13.126 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:27:13.126 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 1169, 2024-05-21T15:27:13.126 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:27:13.127 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 1169, 2024-05-21T15:27:13.127 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 3259, 2024-05-21T15:27:13.127 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 7, 2024-05-21T15:27:13.127 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 78, 2024-05-21T15:27:13.127 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 147, 2024-05-21T15:27:13.127 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:27:13.128 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:27:13.128 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:27:13.128 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 729, 2024-05-21T15:27:13.128 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 514, 2024-05-21T15:27:13.128 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 14, 2024-05-21T15:27:13.128 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 2 2024-05-21T15:27:13.128 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.129 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-05-21T15:27:13.129 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-05-21T15:27:13.129 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 263, 2024-05-21T15:27:13.129 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 177, 2024-05-21T15:27:13.129 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 166, 2024-05-21T15:27:13.129 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-05-21T15:27:13.129 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-05-21T15:27:13.130 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-05-21T15:27:13.130 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:27:13.130 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-05-21T15:27:13.130 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-05-21T15:27:13.130 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-05-21T15:27:13.130 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-05-21T15:27:13.131 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-05-21T15:27:13.131 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:27:13.131 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:27:13.131 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-05-21T15:27:13.131 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-05-21T15:27:13.132 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:27:13.132 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:27:13.132 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:27:13.132 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 1, 2024-05-21T15:27:13.132 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-05-21T15:27:13.132 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 1, 2024-05-21T15:27:13.132 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-05-21T15:27:13.132 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-05-21T15:27:13.133 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.133 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-05-21T15:27:13.133 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-05-21T15:27:13.133 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 888, 2024-05-21T15:27:13.133 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-05-21T15:27:13.133 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-05-21T15:27:13.133 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 888, 2024-05-21T15:27:13.134 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-05-21T15:27:13.134 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-05-21T15:27:13.134 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 2, 2024-05-21T15:27:13.134 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-05-21T15:27:13.134 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-05-21T15:27:13.134 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-05-21T15:27:13.134 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-05-21T15:27:13.134 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-05-21T15:27:13.135 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-05-21T15:27:13.135 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-05-21T15:27:13.135 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 5750949, 2024-05-21T15:27:13.135 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-05-21T15:27:13.135 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-05-21T15:27:13.135 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 136, 2024-05-21T15:27:13.135 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 6.342786376, 2024-05-21T15:27:13.135 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.046638135 2024-05-21T15:27:13.136 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.136 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-05-21T15:27:13.136 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.136 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-05-21T15:27:13.136 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 472, 2024-05-21T15:27:13.136 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 480, 2024-05-21T15:27:13.136 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 8, 2024-05-21T15:27:13.136 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 119, 2024-05-21T15:27:13.137 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 129, 2024-05-21T15:27:13.137 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 10, 2024-05-21T15:27:13.137 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 544, 2024-05-21T15:27:13.137 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 560, 2024-05-21T15:27:13.137 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 16, 2024-05-21T15:27:13.137 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 179, 2024-05-21T15:27:13.137 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 292, 2024-05-21T15:27:13.137 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 113, 2024-05-21T15:27:13.138 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 129440, 2024-05-21T15:27:13.138 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223636 2024-05-21T15:27:13.138 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.138 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-05-21T15:27:13.138 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 70, 2024-05-21T15:27:13.138 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 5, 2024-05-21T15:27:13.138 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 55, 2024-05-21T15:27:13.138 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 29, 2024-05-21T15:27:13.139 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 12, 2024-05-21T15:27:13.139 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-05-21T15:27:13.139 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.139 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.139 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.139 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.139 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-05-21T15:27:13.139 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 17, 2024-05-21T15:27:13.140 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2.136883805, 2024-05-21T15:27:13.140 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.125699047 2024-05-21T15:27:13.140 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.140 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-05-21T15:27:13.140 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.140 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.140 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.140 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.141 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-05-21T15:27:13.141 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.141 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.141 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.141 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.141 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-05-21T15:27:13.141 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 23, 2024-05-21T15:27:13.141 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.017353690, 2024-05-21T15:27:13.142 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000754508 2024-05-21T15:27:13.142 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.142 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-05-21T15:27:13.142 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.142 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.142 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.142 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.142 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-05-21T15:27:13.143 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.143 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.143 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.143 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.143 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-05-21T15:27:13.143 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.143 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.143 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.144 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.144 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-05-21T15:27:13.144 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.144 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.144 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.144 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.144 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-05-21T15:27:13.144 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.145 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.145 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.145 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.145 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-05-21T15:27:13.145 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.146 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.146 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.146 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.146 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-05-21T15:27:13.146 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.146 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.146 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.147 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.147 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-05-21T15:27:13.147 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.147 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.147 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.147 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.147 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-05-21T15:27:13.147 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2, 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001283604, 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000641802 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.148 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-05-21T15:27:13.149 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.149 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.149 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.149 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.149 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-05-21T15:27:13.149 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.149 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.149 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.150 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.150 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-05-21T15:27:13.150 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.150 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.150 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.150 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.150 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-05-21T15:27:13.150 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.151 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-05-21T15:27:13.152 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.152 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.152 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.152 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.152 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-05-21T15:27:13.152 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.152 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.152 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.153 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.153 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-05-21T15:27:13.153 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.153 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.153 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.153 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.153 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-05-21T15:27:13.153 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.154 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.154 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.154 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.154 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-05-21T15:27:13.154 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.154 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.154 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.154 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.155 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.156 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.156 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-05-21T15:27:13.156 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.156 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.156 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.156 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.156 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.157 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.158 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.158 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.158 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.158 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-05-21T15:27:13.158 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-05-21T15:27:13.158 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 5, 2024-05-21T15:27:13.158 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 3, 2024-05-21T15:27:13.158 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 22, 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 11, 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 586, 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:27:13.159 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-05-21T15:27:13.160 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:27:13.160 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:27:13.160 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:27:13.160 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:27:13.160 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:27:13.160 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:27:13.160 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:27:13.160 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:27:13.161 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:27:13.161 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:27:13.161 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:27:13.161 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:27:13.161 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:27:13.161 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-05-21T15:27:13.161 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:27:13.161 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-05-21T15:27:13.162 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:27:13.163 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:27:13.163 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:27:13.163 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-05-21T15:27:13.163 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-05-21T15:27:13.163 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-05-21T15:27:13.163 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:27:13.163 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:27:13.163 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:27:13.164 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:27:13.164 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 334021, 2024-05-21T15:27:13.164 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 447, 2024-05-21T15:27:13.165 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:27:13.165 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-05-21T15:27:13.166 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-05-21T15:27:13.166 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-05-21T15:27:13.166 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:27:13.166 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-05-21T15:27:13.166 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:27:13.166 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-05-21T15:27:13.166 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8632, 2024-05-21T15:27:13.166 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 180, 2024-05-21T15:27:13.167 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:27:13.167 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:27:13.167 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-05-21T15:27:13.167 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-05-21T15:27:13.167 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 1806928, 2024-05-21T15:27:13.167 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 26099, 2024-05-21T15:27:13.167 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-05-21T15:27:13.167 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 4672, 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 1727628, 2024-05-21T15:27:13.168 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 28, 2024-05-21T15:27:13.169 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 4672, 2024-05-21T15:27:13.169 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-05-21T15:27:13.169 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4672, 2024-05-21T15:27:13.169 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 13.184688123, 2024-05-21T15:27:13.169 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002822065 2024-05-21T15:27:13.169 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.169 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-05-21T15:27:13.169 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-05-21T15:27:13.170 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4672, 2024-05-21T15:27:13.170 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 4755 2024-05-21T15:27:13.170 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.170 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 4672, 2024-05-21T15:27:13.170 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 4430, 2024-05-21T15:27:13.170 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 242, 2024-05-21T15:27:13.170 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-05-21T15:27:13.171 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-05-21T15:27:13.171 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-05-21T15:27:13.171 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-05-21T15:27:13.171 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 0, 2024-05-21T15:27:13.171 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 136, 2024-05-21T15:27:13.171 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 23, 2024-05-21T15:27:13.171 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-05-21T15:27:13.171 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-05-21T15:27:13.172 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-05-21T15:27:13.172 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 1, 2024-05-21T15:27:13.172 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 6, 2024-05-21T15:27:13.172 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-05-21T15:27:13.172 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-05-21T15:27:13.172 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-05-21T15:27:13.172 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 4428, 2024-05-21T15:27:13.172 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-05-21T15:27:13.173 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-05-21T15:27:13.174 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-05-21T15:27:13.175 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 99, 2024-05-21T15:27:13.176 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-05-21T15:27:13.176 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 14, 2024-05-21T15:27:13.176 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 12, 2024-05-21T15:27:13.176 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 12, 2024-05-21T15:27:13.176 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-05-21T15:27:13.176 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-05-21T15:27:13.176 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 99, 2024-05-21T15:27:13.176 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 3, 2024-05-21T15:27:13.177 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 29 2024-05-21T15:27:13.177 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.177 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-05-21T15:27:13.177 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-05-21T15:27:13.177 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:27:13.177 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-05-21T15:27:13.177 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-05-21T15:27:13.177 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.178 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-05-21T15:27:13.178 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-05-21T15:27:13.178 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.178 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-05-21T15:27:13.178 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-05-21T15:27:13.178 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:27:13.178 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-05-21T15:27:13.178 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:13.179 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 8314, 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 5193127, 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 8314, 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 8314, 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 5193127, 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:13.180 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:13.181 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:13.182 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:13.182 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:13.182 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:13.182 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 4672, 2024-05-21T15:27:13.182 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 1727581, 2024-05-21T15:27:13.182 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4672, 2024-05-21T15:27:13.182 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1727581, 2024-05-21T15:27:13.182 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-05-21T15:27:13.183 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 4672, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 4672, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4672, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 4672, 2024-05-21T15:27:13.184 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-05-21T15:27:13.185 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:13.186 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:13.186 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:13.186 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:13.186 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:13.186 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:13.186 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:13.186 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-05-21T15:27:13.186 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55940fe90b60": { 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:13.187 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 888, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 1556645, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 888, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 136, 2024-05-21T15:27:13.188 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1556645, 2024-05-21T15:27:13.189 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:13.189 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:13.189 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:13.189 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:13.189 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.189 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:13.189 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-05-21T15:27:14.043 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-05-21T15:27:14.043 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2024-05-21T15:27:14.043 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-21T15:27:14.043 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=263 2024-05-21T15:27:14.043 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-21T15:27:14.043 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:27:14.043 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:27:14.044 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-05-21T15:27:14.044 DEBUG:tasks.check_counter:Getting stats from l 2024-05-21T15:27:14.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d shell --fsid 852c9762-1783-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-21T15:27:14.282 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T15:27:14.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:14 smithi078 ceph-mon[31997]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:14.395 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:14 smithi084 ceph-mon[36334]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:14.404 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:14 smithi172 ceph-mon[36402]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:16.048 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-05-21T15:27:16.049 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T15:27:16.049 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 521, 2024-05-21T15:27:16.049 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 588, 2024-05-21T15:27:16.049 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 277835, 2024-05-21T15:27:16.049 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 2153842, 2024-05-21T15:27:16.049 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 48, 2024-05-21T15:27:16.050 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-05-21T15:27:16.050 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.074297220, 2024-05-21T15:27:16.050 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.023343025, 2024-05-21T15:27:16.050 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.031102251, 2024-05-21T15:27:16.050 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.010897041, 2024-05-21T15:27:16.050 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:16.050 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 582, 2024-05-21T15:27:16.051 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.068174426, 2024-05-21T15:27:16.051 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000117138 2024-05-21T15:27:16.051 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.051 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:16.051 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 232, 2024-05-21T15:27:16.051 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000052198, 2024-05-21T15:27:16.052 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000224 2024-05-21T15:27:16.052 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.052 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:27:16.052 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 328336 2024-05-21T15:27:16.052 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.052 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T15:27:16.052 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1389, 2024-05-21T15:27:16.052 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1532, 2024-05-21T15:27:16.053 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 646338, 2024-05-21T15:27:16.053 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 769336, 2024-05-21T15:27:16.053 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 11, 2024-05-21T15:27:16.053 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-05-21T15:27:16.053 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.128813222, 2024-05-21T15:27:16.053 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.050704444, 2024-05-21T15:27:16.053 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.092903312, 2024-05-21T15:27:16.054 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.005925105, 2024-05-21T15:27:16.054 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:16.054 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1531, 2024-05-21T15:27:16.054 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.705515120, 2024-05-21T15:27:16.054 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000460819 2024-05-21T15:27:16.054 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.055 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:16.055 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2047, 2024-05-21T15:27:16.055 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000583184, 2024-05-21T15:27:16.055 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000284 2024-05-21T15:27:16.055 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.055 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-21T15:27:16.055 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 126720 2024-05-21T15:27:16.055 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.056 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T15:27:16.056 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2647, 2024-05-21T15:27:16.056 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 2187, 2024-05-21T15:27:16.056 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 2512386, 2024-05-21T15:27:16.056 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 937369, 2024-05-21T15:27:16.056 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 17, 2024-05-21T15:27:16.056 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-05-21T15:27:16.056 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.240611568, 2024-05-21T15:27:16.057 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.085685079, 2024-05-21T15:27:16.057 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.164443520, 2024-05-21T15:27:16.057 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.018539189, 2024-05-21T15:27:16.057 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T15:27:16.057 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2189, 2024-05-21T15:27:16.057 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.174178098, 2024-05-21T15:27:16.057 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000079569 2024-05-21T15:27:16.058 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.058 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-05-21T15:27:16.058 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3620, 2024-05-21T15:27:16.058 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000896836, 2024-05-21T15:27:16.058 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000247 2024-05-21T15:27:16.059 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.059 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1589296, 2024-05-21T15:27:16.059 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 170240 2024-05-21T15:27:16.059 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.060 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-05-21T15:27:16.060 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-05-21T15:27:16.060 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-05-21T15:27:16.060 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.061 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-05-21T15:27:16.061 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-05-21T15:27:16.061 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-05-21T15:27:16.061 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 614, 2024-05-21T15:27:16.061 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.447928469, 2024-05-21T15:27:16.062 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000729525 2024-05-21T15:27:16.062 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.062 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.062 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-05-21T15:27:16.062 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-05-21T15:27:16.063 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-05-21T15:27:16.063 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2, 2024-05-21T15:27:16.063 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000008840, 2024-05-21T15:27:16.063 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000004420 2024-05-21T15:27:16.063 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.063 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.063 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-05-21T15:27:16.064 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 56, 2024-05-21T15:27:16.064 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 40, 2024-05-21T15:27:16.064 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-05-21T15:27:16.064 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 40, 2024-05-21T15:27:16.064 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.172901109, 2024-05-21T15:27:16.064 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004322527 2024-05-21T15:27:16.064 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.064 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-05-21T15:27:16.065 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 16, 2024-05-21T15:27:16.065 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 2, 2024-05-21T15:27:16.065 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-05-21T15:27:16.065 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 11, 2024-05-21T15:27:16.065 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-05-21T15:27:16.065 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-05-21T15:27:16.065 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 210, 2024-05-21T15:27:16.065 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 58, 2024-05-21T15:27:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 152, 2024-05-21T15:27:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-05-21T15:27:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 128, 2024-05-21T15:27:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 15, 2024-05-21T15:27:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 26, 2024-05-21T15:27:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 27, 2024-05-21T15:27:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 13, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 266, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 158, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 13, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 94, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-05-21T15:27:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-05-21T15:27:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-05-21T15:27:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-05-21T15:27:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 2, 2024-05-21T15:27:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 44, 2024-05-21T15:27:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-05-21T15:27:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 570, 2024-05-21T15:27:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-05-21T15:27:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 325, 2024-05-21T15:27:16.069 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 13424230508, 2024-05-21T15:27:16.069 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 6, 2024-05-21T15:27:16.069 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 570, 2024-05-21T15:27:16.069 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-05-21T15:27:16.069 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T15:27:16.069 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-05-21T15:27:16.069 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-05-21T15:27:16.069 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T15:27:16.070 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T15:27:16.070 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 570, 2024-05-21T15:27:16.070 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 7, 2024-05-21T15:27:16.070 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 59, 2024-05-21T15:27:16.070 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 147, 2024-05-21T15:27:16.070 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T15:27:16.070 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T15:27:16.070 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T15:27:16.071 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 978, 2024-05-21T15:27:16.071 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 819, 2024-05-21T15:27:16.071 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 4, 2024-05-21T15:27:16.071 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-05-21T15:27:16.071 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.071 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-05-21T15:27:16.071 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 263, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 177, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 162, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-05-21T15:27:16.072 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-05-21T15:27:16.073 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-05-21T15:27:16.073 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-05-21T15:27:16.073 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-05-21T15:27:16.073 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-05-21T15:27:16.073 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-05-21T15:27:16.073 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-05-21T15:27:16.073 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-05-21T15:27:16.073 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-05-21T15:27:16.074 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-05-21T15:27:16.074 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T15:27:16.074 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-05-21T15:27:16.074 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-05-21T15:27:16.074 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-05-21T15:27:16.074 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-05-21T15:27:16.074 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-05-21T15:27:16.074 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.075 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-05-21T15:27:16.075 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-05-21T15:27:16.075 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 962, 2024-05-21T15:27:16.075 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-05-21T15:27:16.075 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-05-21T15:27:16.075 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 962, 2024-05-21T15:27:16.075 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-05-21T15:27:16.075 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-05-21T15:27:16.076 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 2, 2024-05-21T15:27:16.076 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-05-21T15:27:16.076 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-05-21T15:27:16.076 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-05-21T15:27:16.076 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-05-21T15:27:16.076 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-05-21T15:27:16.076 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-05-21T15:27:16.076 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-05-21T15:27:16.077 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 5403724, 2024-05-21T15:27:16.077 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-05-21T15:27:16.077 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-05-21T15:27:16.077 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 93, 2024-05-21T15:27:16.077 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3.096910203, 2024-05-21T15:27:16.077 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.033300109 2024-05-21T15:27:16.077 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.077 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-05-21T15:27:16.078 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.078 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-05-21T15:27:16.078 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 214, 2024-05-21T15:27:16.078 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 222, 2024-05-21T15:27:16.078 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 8, 2024-05-21T15:27:16.078 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 102, 2024-05-21T15:27:16.078 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 118, 2024-05-21T15:27:16.078 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 16, 2024-05-21T15:27:16.079 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 210, 2024-05-21T15:27:16.079 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 225, 2024-05-21T15:27:16.079 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 15, 2024-05-21T15:27:16.079 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 27, 2024-05-21T15:27:16.079 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 42, 2024-05-21T15:27:16.079 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 15, 2024-05-21T15:27:16.079 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 126448, 2024-05-21T15:27:16.079 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223636 2024-05-21T15:27:16.080 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.080 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-05-21T15:27:16.080 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 57, 2024-05-21T15:27:16.080 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 6, 2024-05-21T15:27:16.080 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 56, 2024-05-21T15:27:16.080 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 22, 2024-05-21T15:27:16.080 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 12, 2024-05-21T15:27:16.080 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-05-21T15:27:16.081 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.081 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.081 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.081 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.081 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-05-21T15:27:16.081 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 5, 2024-05-21T15:27:16.081 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.002277295, 2024-05-21T15:27:16.081 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000455459 2024-05-21T15:27:16.082 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.082 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-05-21T15:27:16.082 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.082 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.082 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.082 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.082 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-05-21T15:27:16.083 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.083 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.083 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.083 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.083 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-05-21T15:27:16.083 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 31, 2024-05-21T15:27:16.083 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.160922299, 2024-05-21T15:27:16.083 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.005191041 2024-05-21T15:27:16.084 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.084 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-05-21T15:27:16.084 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.084 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.084 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.084 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.084 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-05-21T15:27:16.084 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-05-21T15:27:16.085 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000112654, 2024-05-21T15:27:16.085 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000112654 2024-05-21T15:27:16.085 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.085 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-05-21T15:27:16.085 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.085 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.085 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.085 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.086 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-05-21T15:27:16.086 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.086 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.086 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.086 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.086 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-05-21T15:27:16.086 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.087 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.087 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.087 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.087 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-05-21T15:27:16.087 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.087 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.088 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.088 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.088 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-05-21T15:27:16.088 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.088 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.088 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.089 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.089 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-05-21T15:27:16.089 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.089 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.089 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.089 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.089 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-05-21T15:27:16.089 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.090 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.090 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.090 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.090 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-05-21T15:27:16.090 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3, 2024-05-21T15:27:16.090 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.009588861, 2024-05-21T15:27:16.090 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.003196287 2024-05-21T15:27:16.091 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.091 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-05-21T15:27:16.091 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.091 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.091 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.091 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.091 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-05-21T15:27:16.091 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.092 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.092 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.092 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.092 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-05-21T15:27:16.092 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.092 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.092 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.092 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.093 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-05-21T15:27:16.093 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.093 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.093 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.093 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.093 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-05-21T15:27:16.093 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.094 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.094 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.094 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.094 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-05-21T15:27:16.094 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.094 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.094 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.095 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.095 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-05-21T15:27:16.095 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.095 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.095 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.095 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.095 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-05-21T15:27:16.095 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.096 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.096 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.096 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.096 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-05-21T15:27:16.096 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.096 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.096 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.096 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.097 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-05-21T15:27:16.097 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.097 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.097 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.097 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.097 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-05-21T15:27:16.097 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.097 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-05-21T15:27:16.098 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.099 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.099 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.099 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.099 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-05-21T15:27:16.099 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.099 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.099 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.099 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.100 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-05-21T15:27:16.100 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-05-21T15:27:16.100 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-05-21T15:27:16.100 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.100 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.100 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.100 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.100 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 3, 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 1, 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 20, 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 10, 2024-05-21T15:27:16.101 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 637, 2024-05-21T15:27:16.102 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-05-21T15:27:16.102 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.102 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-05-21T15:27:16.102 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-05-21T15:27:16.102 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-05-21T15:27:16.103 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T15:27:16.103 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-05-21T15:27:16.103 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T15:27:16.103 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-05-21T15:27:16.103 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T15:27:16.104 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T15:27:16.104 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T15:27:16.104 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T15:27:16.104 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T15:27:16.104 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-05-21T15:27:16.104 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T15:27:16.104 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T15:27:16.105 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-05-21T15:27:16.105 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-05-21T15:27:16.105 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-05-21T15:27:16.105 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-05-21T15:27:16.105 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T15:27:16.105 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T15:27:16.105 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T15:27:16.105 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T15:27:16.106 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T15:27:16.106 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-05-21T15:27:16.106 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-05-21T15:27:16.106 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-05-21T15:27:16.106 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T15:27:16.106 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T15:27:16.106 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-05-21T15:27:16.106 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-05-21T15:27:16.107 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-05-21T15:27:16.107 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-05-21T15:27:16.107 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T15:27:16.107 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-05-21T15:27:16.107 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T15:27:16.107 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-05-21T15:27:16.107 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 303056, 2024-05-21T15:27:16.107 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 254, 2024-05-21T15:27:16.108 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-05-21T15:27:16.108 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-05-21T15:27:16.108 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-05-21T15:27:16.108 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-05-21T15:27:16.108 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-05-21T15:27:16.109 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-05-21T15:27:16.109 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-05-21T15:27:16.109 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-05-21T15:27:16.109 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8632, 2024-05-21T15:27:16.109 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 180, 2024-05-21T15:27:16.109 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T15:27:16.110 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-05-21T15:27:16.110 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-05-21T15:27:16.110 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-05-21T15:27:16.110 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 786030, 2024-05-21T15:27:16.110 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 5593, 2024-05-21T15:27:16.110 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-05-21T15:27:16.110 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-05-21T15:27:16.111 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-05-21T15:27:16.111 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-05-21T15:27:16.111 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.111 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-05-21T15:27:16.111 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-05-21T15:27:16.111 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-05-21T15:27:16.112 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 760, 2024-05-21T15:27:16.112 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 1231445, 2024-05-21T15:27:16.112 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 33, 2024-05-21T15:27:16.112 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 760, 2024-05-21T15:27:16.112 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-05-21T15:27:16.112 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 760, 2024-05-21T15:27:16.112 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 6.237852529, 2024-05-21T15:27:16.113 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.008207700 2024-05-21T15:27:16.113 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.113 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-05-21T15:27:16.113 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-05-21T15:27:16.113 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 760, 2024-05-21T15:27:16.113 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 831 2024-05-21T15:27:16.113 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.113 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 760, 2024-05-21T15:27:16.114 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 576, 2024-05-21T15:27:16.114 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 184, 2024-05-21T15:27:16.114 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-05-21T15:27:16.114 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-05-21T15:27:16.114 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-05-21T15:27:16.114 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-05-21T15:27:16.114 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 0, 2024-05-21T15:27:16.114 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 93, 2024-05-21T15:27:16.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 19, 2024-05-21T15:27:16.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-05-21T15:27:16.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-05-21T15:27:16.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-05-21T15:27:16.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-05-21T15:27:16.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 6, 2024-05-21T15:27:16.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-05-21T15:27:16.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-05-21T15:27:16.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-05-21T15:27:16.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 570, 2024-05-21T15:27:16.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-05-21T15:27:16.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-05-21T15:27:16.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-05-21T15:27:16.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-05-21T15:27:16.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-05-21T15:27:16.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-05-21T15:27:16.117 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-05-21T15:27:16.118 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-05-21T15:27:16.118 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-05-21T15:27:16.118 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-05-21T15:27:16.118 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-05-21T15:27:16.118 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-05-21T15:27:16.118 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-05-21T15:27:16.118 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-05-21T15:27:16.118 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-05-21T15:27:16.119 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-05-21T15:27:16.119 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-05-21T15:27:16.119 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-05-21T15:27:16.119 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-05-21T15:27:16.119 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 99, 2024-05-21T15:27:16.119 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-05-21T15:27:16.119 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 14, 2024-05-21T15:27:16.119 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 12, 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 12, 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 0, 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 75, 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 8, 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 30 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-05-21T15:27:16.120 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-05-21T15:27:16.121 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-05-21T15:27:16.121 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-05-21T15:27:16.121 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-05-21T15:27:16.121 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.121 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-05-21T15:27:16.121 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-05-21T15:27:16.121 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.121 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-05-21T15:27:16.122 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-05-21T15:27:16.122 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T15:27:16.122 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-05-21T15:27:16.122 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-05-21T15:27:16.122 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-05-21T15:27:16.122 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-05-21T15:27:16.122 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-05-21T15:27:16.122 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.123 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T15:27:16.123 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:16.123 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-05-21T15:27:16.123 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:16.123 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 4553, 2024-05-21T15:27:16.123 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 3006694, 2024-05-21T15:27:16.123 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:16.123 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 4553, 2024-05-21T15:27:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4553, 2024-05-21T15:27:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 3006694, 2024-05-21T15:27:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.125 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.125 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.125 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-05-21T15:27:16.125 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:16.125 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-05-21T15:27:16.125 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:16.125 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:16.125 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:16.126 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:16.126 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:16.126 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 760, 2024-05-21T15:27:16.126 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 1231296, 2024-05-21T15:27:16.126 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 760, 2024-05-21T15:27:16.126 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1231296, 2024-05-21T15:27:16.126 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:16.126 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.127 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.127 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.127 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.127 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.127 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-05-21T15:27:16.127 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:16.127 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-05-21T15:27:16.127 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:16.128 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:16.128 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:16.128 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:16.128 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:16.128 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 760, 2024-05-21T15:27:16.128 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 760, 2024-05-21T15:27:16.128 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 760, 2024-05-21T15:27:16.128 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 760, 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:16.129 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-05-21T15:27:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-05-21T15:27:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-05-21T15:27:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-05-21T15:27:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-05-21T15:27:16.131 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-05-21T15:27:16.131 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:16.131 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.131 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.131 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.131 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.131 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-05-21T15:27:16.131 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x560273e2c8f0": { 2024-05-21T15:27:16.132 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-05-21T15:27:16.132 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-05-21T15:27:16.132 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-05-21T15:27:16.132 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 962, 2024-05-21T15:27:16.132 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 1209420, 2024-05-21T15:27:16.132 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-05-21T15:27:16.132 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 962, 2024-05-21T15:27:16.132 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 93, 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1209420, 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.133 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-05-21T15:27:16.134 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-05-21T15:27:16.347 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:16 smithi078 ceph-mon[31997]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:16.396 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:16 smithi084 ceph-mon[36334]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:16.433 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:16 smithi172 ceph-mon[36402]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:16.848 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-21T15:27:16.849 INFO:tasks.check_counter:Daemon mds.l mds.imported=2 2024-05-21T15:27:16.849 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-21T15:27:16.849 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=263 2024-05-21T15:27:16.849 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-05-21T15:27:16.849 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-21T15:27:16.849 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T15:27:16.849 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-05-21T15:27:16.850 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-21T15:27:16.860 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-21T15:27:16.860 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:27:16.860 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T15:27:16.882 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-05-21T15:27:16.882 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T15:27:16.942 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:27:17.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 -- 172.21.15.78:0/1424396331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc1079d0 msgr2=0x7f41cc101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 --2- 172.21.15.78:0/1424396331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc1079d0 0x7f41cc101560 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009900 tx=0x7f41bc02a410 comp rx=0 tx=0).stop 2024-05-21T15:27:17.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 -- 172.21.15.78:0/1424396331 shutdown_connections 2024-05-21T15:27:17.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 --2- 172.21.15.78:0/1424396331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41cc101aa0 0x7f41cc101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 --2- 172.21.15.78:0/1424396331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc10c330 0x7f41cc10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 --2- 172.21.15.78:0/1424396331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc1079d0 0x7f41cc101560 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 -- 172.21.15.78:0/1424396331 >> 172.21.15.78:0/1424396331 conn(0x7f41cc0770c0 msgr2=0x7f41cc077500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:17.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 -- 172.21.15.78:0/1424396331 shutdown_connections 2024-05-21T15:27:17.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.111+0000 7f41d18d7640 1 -- 172.21.15.78:0/1424396331 wait complete. 2024-05-21T15:27:17.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41d18d7640 1 Processor -- start 2024-05-21T15:27:17.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41d18d7640 1 -- start start 2024-05-21T15:27:17.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41d18d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc101aa0 0x7f41cc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41d18d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41cc1079d0 0x7f41cc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41d18d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc10c330 0x7f41cc19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41d18d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc111d60 con 0x7f41cc101aa0 2024-05-21T15:27:17.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41d18d7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc111be0 con 0x7f41cc1079d0 2024-05-21T15:27:17.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41d18d7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc111ee0 con 0x7f41cc10c330 2024-05-21T15:27:17.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc101aa0 0x7f41cc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc101aa0 0x7f41cc1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54376/0 (socket says 172.21.15.78:54376) 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41caffd640 1 -- 172.21.15.78:0/230784822 learned_addr learned my addr 172.21.15.78:0/230784822 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41cb7fe640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc10c330 0x7f41cc19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41c27fc640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41cc1079d0 0x7f41cc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41caffd640 1 -- 172.21.15.78:0/230784822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc10c330 msgr2=0x7f41cc19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41caffd640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc10c330 0x7f41cc19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41caffd640 1 -- 172.21.15.78:0/230784822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41cc1079d0 msgr2=0x7f41cc198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.112+0000 7f41caffd640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41cc1079d0 0x7f41cc198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.113+0000 7f41caffd640 1 -- 172.21.15.78:0/230784822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc19d520 con 0x7f41cc101aa0 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.113+0000 7f41c27fc640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41cc1079d0 0x7f41cc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.113+0000 7f41caffd640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc101aa0 0x7f41cc1986c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f41bc005940 tx=0x7f41bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.113+0000 7f41d08d5640 1 -- 172.21.15.78:0/230784822 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc03f030 con 0x7f41cc101aa0 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.113+0000 7f41d08d5640 1 -- 172.21.15.78:0/230784822 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41bc003ce0 con 0x7f41cc101aa0 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.113+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc19d7b0 con 0x7f41cc101aa0 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.113+0000 7f41d08d5640 1 -- 172.21.15.78:0/230784822 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc035070 con 0x7f41cc101aa0 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.113+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc19dd20 con 0x7f41cc101aa0 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.114+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41cc102c80 con 0x7f41cc101aa0 2024-05-21T15:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.115+0000 7f41d08d5640 1 -- 172.21.15.78:0/230784822 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f41bc042030 con 0x7f41cc101aa0 2024-05-21T15:27:17.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.116+0000 7f41d08d5640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41b80a2d00 0x7f41b80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.116+0000 7f41c27fc640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41b80a2d00 0x7f41b80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.116+0000 7f41d08d5640 1 -- 172.21.15.78:0/230784822 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7f41bc0efd10 con 0x7f41cc101aa0 2024-05-21T15:27:17.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.117+0000 7f41c27fc640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41b80a2d00 0x7f41b80a5210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f41cc10d6f0 tx=0x7f41b4005e70 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.120+0000 7f41d08d5640 1 -- 172.21.15.78:0/230784822 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f41bc0bc8a0 con 0x7f41cc101aa0 2024-05-21T15:27:17.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.297+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41cc105e90 con 0x7f41cc101aa0 2024-05-21T15:27:17.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.299+0000 7f41d08d5640 1 -- 172.21.15.78:0/230784822 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23225 (secure 0 0 0) 0x7f41bc0bc1a0 con 0x7f41cc101aa0 2024-05-21T15:27:17.302 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:27:17.303 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":100,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:27:16.443513+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6801/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2012910515":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2194549734":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/3526992601":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2604499013":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:27:17.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41b80a2d00 msgr2=0x7f41b80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41b80a2d00 0x7f41b80a5210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f41cc10d6f0 tx=0x7f41b4005e70 comp rx=0 tx=0).stop 2024-05-21T15:27:17.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc101aa0 msgr2=0x7f41cc1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc101aa0 0x7f41cc1986c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f41bc005940 tx=0x7f41bc0042f0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 shutdown_connections 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41cc1079d0 0x7f41cc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f41b80a2d00 0x7f41b80a5210 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc10c330 0x7f41cc19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 --2- 172.21.15.78:0/230784822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41cc101aa0 0x7f41cc1986c0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 >> 172.21.15.78:0/230784822 conn(0x7f41cc0770c0 msgr2=0x7f41cc103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 shutdown_connections 2024-05-21T15:27:17.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.301+0000 7f41d18d7640 1 -- 172.21.15.78:0/230784822 wait complete. 2024-05-21T15:27:17.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T15:09:41.478205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T15:14:09.791400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T15:14:11.122406+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T15:27:17.317 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T15:27:17.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.428+0000 7ff843875640 1 -- 172.21.15.78:0/2886444501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 msgr2=0x7ff83c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.428+0000 7ff843875640 1 --2- 172.21.15.78:0/2886444501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 0x7ff83c101560 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff838009ca0 tx=0x7ff83802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 -- 172.21.15.78:0/2886444501 shutdown_connections 2024-05-21T15:27:17.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 --2- 172.21.15.78:0/2886444501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c10c330 0x7ff83c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 --2- 172.21.15.78:0/2886444501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff83c101aa0 0x7ff83c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 --2- 172.21.15.78:0/2886444501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 0x7ff83c101560 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 -- 172.21.15.78:0/2886444501 >> 172.21.15.78:0/2886444501 conn(0x7ff83c0770b0 msgr2=0x7ff83c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 -- 172.21.15.78:0/2886444501 shutdown_connections 2024-05-21T15:27:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 -- 172.21.15.78:0/2886444501 wait complete. 2024-05-21T15:27:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 Processor -- start 2024-05-21T15:27:17.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.429+0000 7ff843875640 1 -- start start 2024-05-21T15:27:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff843875640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff83c101aa0 0x7ff83c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff843875640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 0x7ff83c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff843875640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c10c330 0x7ff83c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff843875640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c111ce0 con 0x7ff83c1079d0 2024-05-21T15:27:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff843875640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c111b60 con 0x7ff83c10c330 2024-05-21T15:27:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff843875640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c111e60 con 0x7ff83c101aa0 2024-05-21T15:27:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 0x7ff83c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 0x7ff83c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54400/0 (socket says 172.21.15.78:54400) 2024-05-21T15:27:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 -- 172.21.15.78:0/3479210600 learned_addr learned my addr 172.21.15.78:0/3479210600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff841e12640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c10c330 0x7ff83c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff841611640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff83c101aa0 0x7ff83c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 -- 172.21.15.78:0/3479210600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff83c101aa0 msgr2=0x7ff83c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff83c101aa0 0x7ff83c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 -- 172.21.15.78:0/3479210600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c10c330 msgr2=0x7ff83c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c10c330 0x7ff83c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 -- 172.21.15.78:0/3479210600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c19d500 con 0x7ff83c1079d0 2024-05-21T15:27:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff841611640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff83c101aa0 0x7ff83c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:27:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff840e10640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 0x7ff83c198c00 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00ea90 tx=0x7ff82c00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff8327fc640 1 -- 172.21.15.78:0/3479210600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c01c070 con 0x7ff83c1079d0 2024-05-21T15:27:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff8327fc640 1 -- 172.21.15.78:0/3479210600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff82c00bc90 con 0x7ff83c1079d0 2024-05-21T15:27:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c19d7f0 con 0x7ff83c1079d0 2024-05-21T15:27:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff8327fc640 1 -- 172.21.15.78:0/3479210600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0166a0 con 0x7ff83c1079d0 2024-05-21T15:27:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.430+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c068e40 con 0x7ff83c1079d0 2024-05-21T15:27:17.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.431+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c102c80 con 0x7ff83c1079d0 2024-05-21T15:27:17.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.433+0000 7ff8327fc640 1 -- 172.21.15.78:0/3479210600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff82c01f070 con 0x7ff83c1079d0 2024-05-21T15:27:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.433+0000 7ff8327fc640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8280a2d00 0x7ff8280a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.434+0000 7ff841611640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8280a2d00 0x7ff8280a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.434+0000 7ff8327fc640 1 -- 172.21.15.78:0/3479210600 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7ff82c0d0eb0 con 0x7ff83c1079d0 2024-05-21T15:27:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.434+0000 7ff841611640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8280a2d00 0x7ff8280a5210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff838000f80 tx=0x7ff8380093f0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:17.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.437+0000 7ff8327fc640 1 -- 172.21.15.78:0/3479210600 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff82c0ff850 con 0x7ff83c1079d0 2024-05-21T15:27:17.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:17 smithi078 ceph-mon[31997]: osdmap e100: 12 total, 12 up, 12 in 2024-05-21T15:27:17.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:17 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/230784822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:27:17.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.612+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff83c105a10 con 0x7ff83c1079d0 2024-05-21T15:27:17.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.613+0000 7ff8327fc640 1 -- 172.21.15.78:0/3479210600 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7ff82c09d3f0 con 0x7ff83c1079d0 2024-05-21T15:27:17.616 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-05-21T15:27:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8280a2d00 msgr2=0x7ff8280a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8280a2d00 0x7ff8280a5210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff838000f80 tx=0x7ff8380093f0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 msgr2=0x7ff83c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 0x7ff83c198c00 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00ea90 tx=0x7ff82c00ef50 comp rx=0 tx=0).stop 2024-05-21T15:27:17.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 shutdown_connections 2024-05-21T15:27:17.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff83c10c330 0x7ff83c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7ff8280a2d00 0x7ff8280a5210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff83c1079d0 0x7ff83c198c00 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 --2- 172.21.15.78:0/3479210600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff83c101aa0 0x7ff83c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 >> 172.21.15.78:0/3479210600 conn(0x7ff83c0770b0 msgr2=0x7ff83c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 shutdown_connections 2024-05-21T15:27:17.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.615+0000 7ff843875640 1 -- 172.21.15.78:0/3479210600 wait complete. 2024-05-21T15:27:17.630 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T15:27:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.740+0000 7f04f592f640 1 -- 172.21.15.78:0/1280509589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0103210 msgr2=0x7f04f010f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.740+0000 7f04f592f640 1 --2- 172.21.15.78:0/1280509589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0103210 0x7f04f010f5e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009ca0 tx=0x7f04e802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.740+0000 7f04f592f640 1 -- 172.21.15.78:0/1280509589 shutdown_connections 2024-05-21T15:27:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.740+0000 7f04f592f640 1 --2- 172.21.15.78:0/1280509589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0103210 0x7f04f010f5e0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.740+0000 7f04f592f640 1 --2- 172.21.15.78:0/1280509589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04f0108f60 0x7f04f0102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.740+0000 7f04f592f640 1 --2- 172.21.15.78:0/1280509589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04f0108ac0 0x7f04f0102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.740+0000 7f04f592f640 1 -- 172.21.15.78:0/1280509589 >> 172.21.15.78:0/1280509589 conn(0x7f04f00fe1c0 msgr2=0x7f04f0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.741+0000 7f04f592f640 1 -- 172.21.15.78:0/1280509589 shutdown_connections 2024-05-21T15:27:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.741+0000 7f04f592f640 1 -- 172.21.15.78:0/1280509589 wait complete. 2024-05-21T15:27:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.741+0000 7f04f592f640 1 Processor -- start 2024-05-21T15:27:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.741+0000 7f04f592f640 1 -- start start 2024-05-21T15:27:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f592f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04f0103210 0x7f04f019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f592f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04f0108ac0 0x7f04f019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f592f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0108f60 0x7f04f0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f592f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0112ea0 con 0x7f04f0108f60 2024-05-21T15:27:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f592f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0112d20 con 0x7f04f0108ac0 2024-05-21T15:27:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f592f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0113020 con 0x7f04f0103210 2024-05-21T15:27:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0108f60 0x7f04f0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0108f60 0x7f04f0199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54420/0 (socket says 172.21.15.78:54420) 2024-05-21T15:27:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 -- 172.21.15.78:0/2798941310 learned_addr learned my addr 172.21.15.78:0/2798941310 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04eeffd640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04f0103210 0x7f04f019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04e67fc640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04f0108ac0 0x7f04f019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 -- 172.21.15.78:0/2798941310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04f0103210 msgr2=0x7f04f019f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04f0103210 0x7f04f019f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 -- 172.21.15.78:0/2798941310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04f0108ac0 msgr2=0x7f04f019fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04f0108ac0 0x7f04f019fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 -- 172.21.15.78:0/2798941310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f0199e30 con 0x7f04f0108f60 2024-05-21T15:27:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04eeffd640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04f0103210 0x7f04f019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:27:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04ef7fe640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0108f60 0x7f04f0199700 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f04e8031a20 tx=0x7f04e8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f492d640 1 -- 172.21.15.78:0/2798941310 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8003c30 con 0x7f04f0108f60 2024-05-21T15:27:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f492d640 1 -- 172.21.15.78:0/2798941310 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e8003dd0 con 0x7f04f0108f60 2024-05-21T15:27:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f019a060 con 0x7f04f0108f60 2024-05-21T15:27:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f492d640 1 -- 172.21.15.78:0/2798941310 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e802fc40 con 0x7f04f0108f60 2024-05-21T15:27:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.742+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01a6320 con 0x7f04f0108f60 2024-05-21T15:27:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.744+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f0103da0 con 0x7f04f0108f60 2024-05-21T15:27:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.745+0000 7f04f492d640 1 -- 172.21.15.78:0/2798941310 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f04e8004050 con 0x7f04f0108f60 2024-05-21T15:27:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.745+0000 7f04f492d640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f04dc0a2d00 0x7f04dc0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.746+0000 7f04eeffd640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f04dc0a2d00 0x7f04dc0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.746+0000 7f04f492d640 1 -- 172.21.15.78:0/2798941310 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7f04e80f0700 con 0x7f04f0108f60 2024-05-21T15:27:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.746+0000 7f04eeffd640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f04dc0a2d00 0x7f04dc0a5210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f04f01037a0 tx=0x7f04e0008040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.750+0000 7f04f492d640 1 -- 172.21.15.78:0/2798941310 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04e80bd430 con 0x7f04f0108f60 2024-05-21T15:27:17.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:17 smithi084 ceph-mon[36334]: osdmap e100: 12 total, 12 up, 12 in 2024-05-21T15:27:17.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:17 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/230784822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:27:17.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.923+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f04f010abd0 con 0x7f04f0108f60 2024-05-21T15:27:17.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.923+0000 7f04f492d640 1 -- 172.21.15.78:0/2798941310 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f04e80bcd30 con 0x7f04f0108f60 2024-05-21T15:27:17.926 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-05-21T15:27:17.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f04dc0a2d00 msgr2=0x7f04dc0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f04dc0a2d00 0x7f04dc0a5210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f04f01037a0 tx=0x7f04e0008040 comp rx=0 tx=0).stop 2024-05-21T15:27:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0108f60 msgr2=0x7f04f0199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0108f60 0x7f04f0199700 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f04e8031a20 tx=0x7f04e8031a50 comp rx=0 tx=0).stop 2024-05-21T15:27:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 shutdown_connections 2024-05-21T15:27:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04f0108ac0 0x7f04f019fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f04dc0a2d00 0x7f04dc0a5210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04f0108f60 0x7f04f0199700 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 --2- 172.21.15.78:0/2798941310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04f0103210 0x7f04f019f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 >> 172.21.15.78:0/2798941310 conn(0x7f04f00fe1c0 msgr2=0x7f04f0105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:17.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 shutdown_connections 2024-05-21T15:27:17.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:17.925+0000 7f04f592f640 1 -- 172.21.15.78:0/2798941310 wait complete. 2024-05-21T15:27:17.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:17 smithi172 ceph-mon[36402]: osdmap e100: 12 total, 12 up, 12 in 2024-05-21T15:27:17.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:17 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/230784822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:27:17.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T15:27:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.051+0000 7f7f41bf5640 1 -- 172.21.15.78:0/4095228008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c1016d0 msgr2=0x7f7f3c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.051+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/4095228008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c1016d0 0x7f7f3c10f520 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009ca0 tx=0x7f7f3402a8c0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.051+0000 7f7f41bf5640 1 -- 172.21.15.78:0/4095228008 shutdown_connections 2024-05-21T15:27:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.051+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/4095228008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c1016d0 0x7f7f3c10f520 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.051+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/4095228008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c108f40 0x7f7f3c101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.051+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/4095228008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3c108aa0 0x7f7f3c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.051+0000 7f7f41bf5640 1 -- 172.21.15.78:0/4095228008 >> 172.21.15.78:0/4095228008 conn(0x7f7f3c076830 msgr2=0x7f7f3c076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:18.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.052+0000 7f7f41bf5640 1 -- 172.21.15.78:0/4095228008 shutdown_connections 2024-05-21T15:27:18.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.052+0000 7f7f41bf5640 1 -- 172.21.15.78:0/4095228008 wait complete. 2024-05-21T15:27:18.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.052+0000 7f7f41bf5640 1 Processor -- start 2024-05-21T15:27:18.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.052+0000 7f7f41bf5640 1 -- start start 2024-05-21T15:27:18.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f41bf5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3c1016d0 0x7f7f3c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:18.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f41bf5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c108aa0 0x7f7f3c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:18.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f41bf5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c108f40 0x7f7f3c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f41bf5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c112de0 con 0x7f7f3c108f40 2024-05-21T15:27:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f41bf5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c112c60 con 0x7f7f3c1016d0 2024-05-21T15:27:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f41bf5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c112f60 con 0x7f7f3c108aa0 2024-05-21T15:27:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c108f40 0x7f7f3c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c108f40 0x7f7f3c19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54442/0 (socket says 172.21.15.78:54442) 2024-05-21T15:27:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 -- 172.21.15.78:0/815400070 learned_addr learned my addr 172.21.15.78:0/815400070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3b7fe640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3c1016d0 0x7f7f3c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:18.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3affd640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c108aa0 0x7f7f3c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:18.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 -- 172.21.15.78:0/815400070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c108aa0 msgr2=0x7f7f3c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c108aa0 0x7f7f3c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 -- 172.21.15.78:0/815400070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3c1016d0 msgr2=0x7f7f3c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3c1016d0 0x7f7f3c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 -- 172.21.15.78:0/815400070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c19e4c0 con 0x7f7f3c108f40 2024-05-21T15:27:18.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3b7fe640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3c1016d0 0x7f7f3c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T15:27:18.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f3bfff640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c108f40 0x7f7f3c19df00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7f34031a60 tx=0x7f7f34031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:18.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f38ff9640 1 -- 172.21.15.78:0/815400070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34003c30 con 0x7f7f3c108f40 2024-05-21T15:27:18.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f38ff9640 1 -- 172.21.15.78:0/815400070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f34003dd0 con 0x7f7f3c108f40 2024-05-21T15:27:18.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f38ff9640 1 -- 172.21.15.78:0/815400070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34004850 con 0x7f7f3c108f40 2024-05-21T15:27:18.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.053+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c19e750 con 0x7f7f3c108f40 2024-05-21T15:27:18.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.054+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c19ec50 con 0x7f7f3c108f40 2024-05-21T15:27:18.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.054+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c103e40 con 0x7f7f3c108f40 2024-05-21T15:27:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.057+0000 7f7f38ff9640 1 -- 172.21.15.78:0/815400070 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7f34004050 con 0x7f7f3c108f40 2024-05-21T15:27:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.057+0000 7f7f38ff9640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7f280a2f20 0x7f7f280a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.058+0000 7f7f3b7fe640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7f280a2f20 0x7f7f280a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.058+0000 7f7f38ff9640 1 -- 172.21.15.78:0/815400070 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7f7f340f0b20 con 0x7f7f3c108f40 2024-05-21T15:27:18.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.058+0000 7f7f3b7fe640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7f280a2f20 0x7f7f280a5430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0058c0 tx=0x7f7f2c005ec0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.061+0000 7f7f38ff9640 1 -- 172.21.15.78:0/815400070 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f340bd7c0 con 0x7f7f3c108f40 2024-05-21T15:27:18.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.236+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f3c10aaf0 con 0x7f7f3c108f40 2024-05-21T15:27:18.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.236+0000 7f7f38ff9640 1 -- 172.21.15.78:0/815400070 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f7f3c10aaf0 con 0x7f7f3c108f40 2024-05-21T15:27:18.239 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-05-21T15:27:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7f280a2f20 msgr2=0x7f7f280a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7f280a2f20 0x7f7f280a5430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0058c0 tx=0x7f7f2c005ec0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c108f40 msgr2=0x7f7f3c19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c108f40 0x7f7f3c19df00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7f34031a60 tx=0x7f7f34031a90 comp rx=0 tx=0).stop 2024-05-21T15:27:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 shutdown_connections 2024-05-21T15:27:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f7f280a2f20 0x7f7f280a5430 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f3c108f40 0x7f7f3c19df00 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f3c108aa0 0x7f7f3c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.238+0000 7f7f41bf5640 1 --2- 172.21.15.78:0/815400070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3c1016d0 0x7f7f3c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.239+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 >> 172.21.15.78:0/815400070 conn(0x7f7f3c076830 msgr2=0x7f7f3c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.239+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 shutdown_connections 2024-05-21T15:27:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.239+0000 7f7f41bf5640 1 -- 172.21.15.78:0/815400070 wait complete. 2024-05-21T15:27:18.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-21T15:27:18.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 -- 172.21.15.78:0/3147374319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46441079e0 msgr2=0x7f4644101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 --2- 172.21.15.78:0/3147374319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46441079e0 0x7f4644101570 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f4638009900 tx=0x7f463802a410 comp rx=0 tx=0).stop 2024-05-21T15:27:18.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 -- 172.21.15.78:0/3147374319 shutdown_connections 2024-05-21T15:27:18.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 --2- 172.21.15.78:0/3147374319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f464410c340 0x7f464410e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 --2- 172.21.15.78:0/3147374319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4644101ab0 0x7f4644101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 --2- 172.21.15.78:0/3147374319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46441079e0 0x7f4644101570 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 -- 172.21.15.78:0/3147374319 >> 172.21.15.78:0/3147374319 conn(0x7f46440fd210 msgr2=0x7f46440ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:18.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 -- 172.21.15.78:0/3147374319 shutdown_connections 2024-05-21T15:27:18.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.362+0000 7f464ab38640 1 -- 172.21.15.78:0/3147374319 wait complete. 2024-05-21T15:27:18.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f464ab38640 1 Processor -- start 2024-05-21T15:27:18.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f464ab38640 1 -- start start 2024-05-21T15:27:18.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f464ab38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4644101ab0 0x7f4644199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:18.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f464ab38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46441079e0 0x7f4644199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:18.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f464ab38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464410c340 0x7f464419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:18.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f464ab38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4644111cf0 con 0x7f4644101ab0 2024-05-21T15:27:18.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f464ab38640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4644111b70 con 0x7f46441079e0 2024-05-21T15:27:18.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f464ab38640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4644111e70 con 0x7f464410c340 2024-05-21T15:27:18.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f46488d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4644101ab0 0x7f4644199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:18.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f46488d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4644101ab0 0x7f4644199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54464/0 (socket says 172.21.15.78:54464) 2024-05-21T15:27:18.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.363+0000 7f46488d4640 1 -- 172.21.15.78:0/2286446779 learned_addr learned my addr 172.21.15.78:0/2286446779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:18.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f46488d4640 1 -- 172.21.15.78:0/2286446779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464410c340 msgr2=0x7f464419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f46490d5640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464410c340 0x7f464419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f46488d4640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464410c340 0x7f464419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f4643fff640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46441079e0 0x7f4644199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f46488d4640 1 -- 172.21.15.78:0/2286446779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46441079e0 msgr2=0x7f4644199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f46488d4640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46441079e0 0x7f4644199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f46488d4640 1 -- 172.21.15.78:0/2286446779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464419e500 con 0x7f4644101ab0 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f46488d4640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4644101ab0 0x7f4644199720 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f46380042f0 tx=0x7f4638004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f4641ffb640 1 -- 172.21.15.78:0/2286446779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463803f030 con 0x7f4644101ab0 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f4641ffb640 1 -- 172.21.15.78:0/2286446779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4638003ce0 con 0x7f4644101ab0 2024-05-21T15:27:18.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464419e790 con 0x7f4644101ab0 2024-05-21T15:27:18.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f4641ffb640 1 -- 172.21.15.78:0/2286446779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638035070 con 0x7f4644101ab0 2024-05-21T15:27:18.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.364+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464419ec80 con 0x7f4644101ab0 2024-05-21T15:27:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.365+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4644102c90 con 0x7f4644101ab0 2024-05-21T15:27:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.367+0000 7f4641ffb640 1 -- 172.21.15.78:0/2286446779 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4638042030 con 0x7f4644101ab0 2024-05-21T15:27:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.367+0000 7f4641ffb640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46340a2d00 0x7f46340a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.368+0000 7f4643fff640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46340a2d00 0x7f46340a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.368+0000 7f4641ffb640 1 -- 172.21.15.78:0/2286446779 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7f46380efd90 con 0x7f4644101ab0 2024-05-21T15:27:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.368+0000 7f4643fff640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46340a2d00 0x7f46340a5210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4630005fd0 tx=0x7f4630005ce0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.371+0000 7f4641ffb640 1 -- 172.21.15.78:0/2286446779 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f46380bc920 con 0x7f4644101ab0 2024-05-21T15:27:18.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.560+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4644069110 con 0x7f4644101ab0 2024-05-21T15:27:18.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.560+0000 7f4641ffb640 1 -- 172.21.15.78:0/2286446779 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+944 (secure 0 0 0) 0x7f46380bc220 con 0x7f4644101ab0 2024-05-21T15:27:18.564 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/3694112952 2024-05-22T15:07:15.158228+0000 2024-05-21T15:27:18.564 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/2228258510 2024-05-22T15:16:08.698984+0000 2024-05-21T15:27:18.564 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/2228258510 2024-05-22T15:16:08.698984+0000 2024-05-21T15:27:18.564 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1173534629 2024-05-22T15:07:15.158228+0000 2024-05-21T15:27:18.564 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3341206163 2024-05-22T15:07:15.158228+0000 2024-05-21T15:27:18.564 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/888544333 2024-05-22T15:06:47.981082+0000 2024-05-21T15:27:18.565 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2012910515 2024-05-22T15:16:08.698984+0000 2024-05-21T15:27:18.565 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/3694112952 2024-05-22T15:07:15.158228+0000 2024-05-21T15:27:18.565 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1077949258 2024-05-22T15:06:47.981082+0000 2024-05-21T15:27:18.565 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/888544333 2024-05-22T15:06:47.981082+0000 2024-05-21T15:27:18.565 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2800502330 2024-05-22T15:07:15.158228+0000 2024-05-21T15:27:18.565 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2303085949 2024-05-22T15:06:47.981082+0000 2024-05-21T15:27:18.565 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2194549734 2024-05-22T15:16:08.698984+0000 2024-05-21T15:27:18.566 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3526992601 2024-05-22T15:16:08.698984+0000 2024-05-21T15:27:18.566 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2604499013 2024-05-22T15:16:08.698984+0000 2024-05-21T15:27:18.566 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2144862462 2024-05-22T15:06:47.981082+0000 2024-05-21T15:27:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:listed 16 entries 2024-05-21T15:27:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46340a2d00 msgr2=0x7f46340a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46340a2d00 0x7f46340a5210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4630005fd0 tx=0x7f4630005ce0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4644101ab0 msgr2=0x7f4644199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4644101ab0 0x7f4644199720 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f46380042f0 tx=0x7f4638004320 comp rx=0 tx=0).stop 2024-05-21T15:27:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 shutdown_connections 2024-05-21T15:27:18.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46441079e0 0x7f4644199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f46340a2d00 0x7f46340a5210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464410c340 0x7f464419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.562+0000 7f464ab38640 1 --2- 172.21.15.78:0/2286446779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4644101ab0 0x7f4644199720 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.563+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 >> 172.21.15.78:0/2286446779 conn(0x7f46440fd210 msgr2=0x7f46440ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.563+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 shutdown_connections 2024-05-21T15:27:18.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:18.563+0000 7f464ab38640 1 -- 172.21.15.78:0/2286446779 wait complete. 2024-05-21T15:27:18.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:18 smithi078 ceph-mon[31997]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T15:27:18.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:18 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3479210600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:27:18.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:18 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2798941310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:27:18.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:18 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/815400070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:27:18.577 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-21T15:27:18.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:27:18.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-21T15:27:18.722 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:18 smithi172 ceph-mon[36402]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T15:27:18.723 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:18 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3479210600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:27:18.723 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:18 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2798941310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:27:18.723 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:18 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/815400070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:27:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:18 smithi084 ceph-mon[36334]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T15:27:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:18 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3479210600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T15:27:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:18 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2798941310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T15:27:18.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:18 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/815400070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T15:27:19.010 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:27:19.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:27:19.010 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-21T15:27:19.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:27:19.051 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-05-21T15:27:19.124 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-21T15:27:19.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:27:19.125 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-21T15:27:19.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T15:27:19.192 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-05-21T15:27:19.291 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-21T15:27:19.302 INFO:tasks.cephadm:Teardown begin 2024-05-21T15:27:19.303 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-21T15:27:19.303 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:27:19.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 -- 172.21.15.78:0/2978902388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 msgr2=0x7f0838101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 --2- 172.21.15.78:0/2978902388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 0x7f0838101f40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f082c009900 tx=0x7f082c02a410 comp rx=0 tx=0).stop 2024-05-21T15:27:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 -- 172.21.15.78:0/2978902388 shutdown_connections 2024-05-21T15:27:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 --2- 172.21.15.78:0/2978902388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f083810c370 0x7f083810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 --2- 172.21.15.78:0/2978902388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 0x7f0838101f40 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 --2- 172.21.15.78:0/2978902388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0838107a10 0x7f08381015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 -- 172.21.15.78:0/2978902388 >> 172.21.15.78:0/2978902388 conn(0x7f0838077130 msgr2=0x7f0838077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 -- 172.21.15.78:0/2978902388 shutdown_connections 2024-05-21T15:27:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 -- 172.21.15.78:0/2978902388 wait complete. 2024-05-21T15:27:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 Processor -- start 2024-05-21T15:27:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.413+0000 7f083e806640 1 -- start start 2024-05-21T15:27:19.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f083e806640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 0x7f0838199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f083e806640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0838107a10 0x7f0838199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f083e806640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f083810c370 0x7f083819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f083e806640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0838112e30 con 0x7f0838101b30 2024-05-21T15:27:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f083e806640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0838112cb0 con 0x7f0838107a10 2024-05-21T15:27:19.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f083e806640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0838112fb0 con 0x7f083810c370 2024-05-21T15:27:19.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 0x7f0838199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:19.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 0x7f0838199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40992/0 (socket says 172.21.15.78:40992) 2024-05-21T15:27:19.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 -- 172.21.15.78:0/2921942152 learned_addr learned my addr 172.21.15.78:0/2921942152 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:19.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f08377fe640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0838107a10 0x7f0838199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:19.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f083cda3640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f083810c370 0x7f083819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:19.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 -- 172.21.15.78:0/2921942152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f083810c370 msgr2=0x7f083819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f083810c370 0x7f083819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 -- 172.21.15.78:0/2921942152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0838107a10 msgr2=0x7f0838199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0838107a10 0x7f0838199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 -- 172.21.15.78:0/2921942152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083819e5f0 con 0x7f0838101b30 2024-05-21T15:27:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.414+0000 7f0837fff640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 0x7f0838199790 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f082800d740 tx=0x7f082800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.415+0000 7f08357fa640 1 -- 172.21.15.78:0/2921942152 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828017070 con 0x7f0838101b30 2024-05-21T15:27:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.415+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083819e8e0 con 0x7f0838101b30 2024-05-21T15:27:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.415+0000 7f08357fa640 1 -- 172.21.15.78:0/2921942152 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082800de50 con 0x7f0838101b30 2024-05-21T15:27:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.415+0000 7f08357fa640 1 -- 172.21.15.78:0/2921942152 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828004e00 con 0x7f0838101b30 2024-05-21T15:27:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.415+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08381a6260 con 0x7f0838101b30 2024-05-21T15:27:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.416+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0838102cc0 con 0x7f0838101b30 2024-05-21T15:27:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.417+0000 7f08357fa640 1 -- 172.21.15.78:0/2921942152 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0828004fa0 con 0x7f0838101b30 2024-05-21T15:27:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.417+0000 7f08357fa640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f08200a2d00 0x7f08200a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.418+0000 7f08377fe640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f08200a2d00 0x7f08200a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.418+0000 7f08357fa640 1 -- 172.21.15.78:0/2921942152 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7f08280ceee0 con 0x7f0838101b30 2024-05-21T15:27:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.418+0000 7f08377fe640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f08200a2d00 0x7f08200a5210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f082c005940 tx=0x7f082c0058d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.422+0000 7f08357fa640 1 -- 172.21.15.78:0/2921942152 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f082809ba70 con 0x7f0838101b30 2024-05-21T15:27:19.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:19 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2286446779' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T15:27:19.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.639+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0838105ed0 con 0x7f0838101b30 2024-05-21T15:27:19.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.639+0000 7f08357fa640 1 -- 172.21.15.78:0/2921942152 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7f082809b370 con 0x7f0838101b30 2024-05-21T15:27:19.642 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:27:19.643 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:27:19.643 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:27:19.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f08200a2d00 msgr2=0x7f08200a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f08200a2d00 0x7f08200a5210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f082c005940 tx=0x7f082c0058d0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 msgr2=0x7f0838199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 0x7f0838199790 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f082800d740 tx=0x7f082800dc00 comp rx=0 tx=0).stop 2024-05-21T15:27:19.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 shutdown_connections 2024-05-21T15:27:19.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0838107a10 0x7f0838199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f08200a2d00 0x7f08200a5210 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f083810c370 0x7f083819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 --2- 172.21.15.78:0/2921942152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0838101b30 0x7f0838199790 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 >> 172.21.15.78:0/2921942152 conn(0x7f0838077130 msgr2=0x7f08381030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:19.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 shutdown_connections 2024-05-21T15:27:19.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.642+0000 7f083e806640 1 -- 172.21.15.78:0/2921942152 wait complete. 2024-05-21T15:27:19.657 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T15:27:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/915421559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d8810d500 msgr2=0x7f5d8810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/915421559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d8810d500 0x7f5d8810f920 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009900 tx=0x7f5d7802a410 comp rx=0 tx=0).stop 2024-05-21T15:27:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/915421559 shutdown_connections 2024-05-21T15:27:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/915421559 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d8810d500 0x7f5d8810f920 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/915421559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d88101000 0x7f5d88101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/915421559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88108b50 0x7f5d88100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/915421559 >> 172.21.15.78:0/915421559 conn(0x7f5d880fc810 msgr2=0x7f5d880fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/915421559 shutdown_connections 2024-05-21T15:27:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.766+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/915421559 wait complete. 2024-05-21T15:27:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8f8ab640 1 Processor -- start 2024-05-21T15:27:19.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8f8ab640 1 -- start start 2024-05-21T15:27:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8f8ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d88101000 0x7f5d88199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8f8ab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d88108b50 0x7f5d88199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8f8ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d8810d500 0x7f5d8819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8f8ab640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d88112e10 con 0x7f5d88108b50 2024-05-21T15:27:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8f8ab640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5d88112c90 con 0x7f5d8810d500 2024-05-21T15:27:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8f8ab640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5d88112f90 con 0x7f5d88101000 2024-05-21T15:27:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.767+0000 7f5d8ce46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d88108b50 0x7f5d88199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8ce46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d88108b50 0x7f5d88199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41012/0 (socket says 172.21.15.78:41012) 2024-05-21T15:27:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8ce46640 1 -- 172.21.15.78:0/498273595 learned_addr learned my addr 172.21.15.78:0/498273595 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8d647640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d88101000 0x7f5d88199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8de48640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d8810d500 0x7f5d8819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8ce46640 1 -- 172.21.15.78:0/498273595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d88101000 msgr2=0x7f5d88199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8ce46640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d88101000 0x7f5d88199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8ce46640 1 -- 172.21.15.78:0/498273595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d8810d500 msgr2=0x7f5d8819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8ce46640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d8810d500 0x7f5d8819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8ce46640 1 -- 172.21.15.78:0/498273595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8819e5d0 con 0x7f5d88108b50 2024-05-21T15:27:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8ce46640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d88108b50 0x7f5d88199cd0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00eb90 tx=0x7f5d7c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d867fc640 1 -- 172.21.15.78:0/498273595 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c010630 con 0x7f5d88108b50 2024-05-21T15:27:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8819e8c0 con 0x7f5d88108b50 2024-05-21T15:27:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d867fc640 1 -- 172.21.15.78:0/498273595 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c00ee90 con 0x7f5d88108b50 2024-05-21T15:27:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d867fc640 1 -- 172.21.15.78:0/498273595 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c018b90 con 0x7f5d88108b50 2024-05-21T15:27:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.768+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d881a6240 con 0x7f5d88108b50 2024-05-21T15:27:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.769+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d88069140 con 0x7f5d88108b50 2024-05-21T15:27:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.771+0000 7f5d867fc640 1 -- 172.21.15.78:0/498273595 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5d7c0107d0 con 0x7f5d88108b50 2024-05-21T15:27:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.771+0000 7f5d867fc640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5d700a2d00 0x7f5d700a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.771+0000 7f5d8d647640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5d700a2d00 0x7f5d700a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.771+0000 7f5d867fc640 1 -- 172.21.15.78:0/498273595 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7f5d7c0cff40 con 0x7f5d88108b50 2024-05-21T15:27:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.772+0000 7f5d8d647640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5d700a2d00 0x7f5d700a5210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d8806ace0 tx=0x7f5d80005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.775+0000 7f5d867fc640 1 -- 172.21.15.78:0/498273595 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5d7c09cad0 con 0x7f5d88108b50 2024-05-21T15:27:19.895 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:19 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2286446779' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T15:27:19.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:19 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2286446779' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T15:27:19.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.987+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d8819aac0 con 0x7f5d88108b50 2024-05-21T15:27:19.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.987+0000 7f5d867fc640 1 -- 172.21.15.78:0/498273595 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11925 (secure 0 0 0) 0x7f5d7c09c3d0 con 0x7f5d88108b50 2024-05-21T15:27:19.991 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:27:19.991 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4003323899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4003323899},{"type":"v1","addr":"172.21.15.78:6835","nonce":4003323899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3299007119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3299007119},{"type":"v1","addr":"172.21.15.78:6837","nonce":3299007119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T15:14:11.918954+0000","modified":"2024-05-21T15:21:07.021188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24526,"mds_2":24502,"mds_3":24448,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14667":{"gid":14667,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/3881579182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":3881579182},{"type":"v1","addr":"172.21.15.78:6839","nonce":3881579182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14679":{"gid":14679,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/2631303889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2631303889},{"type":"v1","addr":"172.21.15.78:6841","nonce":2631303889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6833/502215426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":502215426},{"type":"v1","addr":"172.21.15.84:6833","nonce":502215426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.172:6833/1961121180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1961121180},{"type":"v1","addr":"172.21.15.172:6833","nonce":1961121180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6835/4071213416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":4071213416},{"type":"v1","addr":"172.21.15.84:6835","nonce":4071213416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6837/3570369413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3570369413},{"type":"v1","addr":"172.21.15.84:6837","nonce":3570369413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/204442136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":204442136},{"type":"v1","addr":"172.21.15.172:6835","nonce":204442136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.172:6837/3093965276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":3093965276},{"type":"v1","addr":"172.21.15.172:6837","nonce":3093965276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6839/3459432480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":3459432480},{"type":"v1","addr":"172.21.15.172:6839","nonce":3459432480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6839/652490614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":652490614},{"type":"v1","addr":"172.21.15.84:6839","nonce":652490614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24526,24527,24502,24448,24419]},"id":1}]} 2024-05-21T15:27:19.991 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-05-21T15:27:19.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5d700a2d00 msgr2=0x7f5d700a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5d700a2d00 0x7f5d700a5210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d8806ace0 tx=0x7f5d80005f90 comp rx=0 tx=0).stop 2024-05-21T15:27:19.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d88108b50 msgr2=0x7f5d88199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:19.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d88108b50 0x7f5d88199cd0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00eb90 tx=0x7f5d7c00c6a0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 shutdown_connections 2024-05-21T15:27:19.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d8810d500 0x7f5d8819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f5d700a2d00 0x7f5d700a5210 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d88108b50 0x7f5d88199cd0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 --2- 172.21.15.78:0/498273595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d88101000 0x7f5d88199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:19.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.989+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 >> 172.21.15.78:0/498273595 conn(0x7f5d880fc810 msgr2=0x7f5d880fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:19.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.990+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 shutdown_connections 2024-05-21T15:27:19.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:19.990+0000 7f5d8f8ab640 1 -- 172.21.15.78:0/498273595 wait complete. 2024-05-21T15:27:20.004 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T15:27:20.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 -- 172.21.15.78:0/1632706472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 msgr2=0x7f123810f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 --2- 172.21.15.78:0/1632706472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 0x7f123810f5c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f1230009900 tx=0x7f123002a410 comp rx=0 tx=0).stop 2024-05-21T15:27:20.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 -- 172.21.15.78:0/1632706472 shutdown_connections 2024-05-21T15:27:20.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 --2- 172.21.15.78:0/1632706472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 0x7f123810f5c0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 --2- 172.21.15.78:0/1632706472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1238108f40 0x7f12381010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 --2- 172.21.15.78:0/1632706472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1238108aa0 0x7f1238100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 -- 172.21.15.78:0/1632706472 >> 172.21.15.78:0/1632706472 conn(0x7f12380fc7c0 msgr2=0x7f12380fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:20.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 -- 172.21.15.78:0/1632706472 shutdown_connections 2024-05-21T15:27:20.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.112+0000 7f123cbce640 1 -- 172.21.15.78:0/1632706472 wait complete. 2024-05-21T15:27:20.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123cbce640 1 Processor -- start 2024-05-21T15:27:20.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123cbce640 1 -- start start 2024-05-21T15:27:20.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123cbce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 0x7f1238199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123cbce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1238108aa0 0x7f1238199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123cbce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1238108f40 0x7f123819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123cbce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1238112de0 con 0x7f1238101740 2024-05-21T15:27:20.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123cbce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1238112c60 con 0x7f1238108f40 2024-05-21T15:27:20.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123cbce640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1238112f60 con 0x7f1238108aa0 2024-05-21T15:27:20.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 0x7f1238199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 0x7f1238199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41044/0 (socket says 172.21.15.78:41044) 2024-05-21T15:27:20.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123659c640 1 -- 172.21.15.78:0/1131454901 learned_addr learned my addr 172.21.15.78:0/1131454901 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:20.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f1235d9b640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1238108aa0 0x7f1238199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f1236d9d640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1238108f40 0x7f123819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123659c640 1 -- 172.21.15.78:0/1131454901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1238108aa0 msgr2=0x7f1238199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123659c640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1238108aa0 0x7f1238199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123659c640 1 -- 172.21.15.78:0/1131454901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1238108f40 msgr2=0x7f123819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.113+0000 7f123659c640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1238108f40 0x7f123819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.114+0000 7f123659c640 1 -- 172.21.15.78:0/1131454901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123819e5c0 con 0x7f1238101740 2024-05-21T15:27:20.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.114+0000 7f1236d9d640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1238108f40 0x7f123819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:27:20.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.114+0000 7f123659c640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 0x7f1238199760 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f122800ebc0 tx=0x7f122800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:20.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.114+0000 7f12277fe640 1 -- 172.21.15.78:0/1131454901 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228010630 con 0x7f1238101740 2024-05-21T15:27:20.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.114+0000 7f12277fe640 1 -- 172.21.15.78:0/1131454901 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1228010cb0 con 0x7f1238101740 2024-05-21T15:27:20.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.114+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123819e8b0 con 0x7f1238101740 2024-05-21T15:27:20.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.114+0000 7f12277fe640 1 -- 172.21.15.78:0/1131454901 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228018bd0 con 0x7f1238101740 2024-05-21T15:27:20.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.114+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381a6230 con 0x7f1238101740 2024-05-21T15:27:20.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.115+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12380682c0 con 0x7f1238101740 2024-05-21T15:27:20.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.116+0000 7f12277fe640 1 -- 172.21.15.78:0/1131454901 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f12280107d0 con 0x7f1238101740 2024-05-21T15:27:20.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.117+0000 7f12277fe640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12200a2d80 0x7f12200a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.117+0000 7f1235d9b640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12200a2d80 0x7f12200a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.117+0000 7f12277fe640 1 -- 172.21.15.78:0/1131454901 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7f12280cfce0 con 0x7f1238101740 2024-05-21T15:27:20.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.117+0000 7f1235d9b640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12200a2d80 0x7f12200a5290 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1238107b50 tx=0x7f122c005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:20.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.120+0000 7f12277fe640 1 -- 172.21.15.78:0/1131454901 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f122809c870 con 0x7f1238101740 2024-05-21T15:27:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.296+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f123810abd0 con 0x7f1238101740 2024-05-21T15:27:20.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.297+0000 7f12277fe640 1 -- 172.21.15.78:0/1131454901 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23225 (secure 0 0 0) 0x7f122809c170 con 0x7f1238101740 2024-05-21T15:27:20.301 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-05-21T15:27:20.301 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":100,"fsid":"852c9762-1783-11ef-bc9a-c7b262605968","created":"2024-05-21T15:06:20.984577+0000","modified":"2024-05-21T15:27:16.443513+0000","last_up_change":"2024-05-21T15:13:23.304965+0000","last_in_change":"2024-05-21T15:13:04.100293+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T15:09:41.478205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T15:14:09.791400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T15:14:11.122406+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3fb7e10-ca25-4585-becc-d4ff4784cb9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6803","nonce":780655399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6805","nonce":780655399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6809","nonce":780655399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":780655399},{"type":"v1","addr":"172.21.15.78:6807","nonce":780655399}]},"public_addr":"172.21.15.78:6803/780655399","cluster_addr":"172.21.15.78:6805/780655399","heartbeat_back_addr":"172.21.15.78:6809/780655399","heartbeat_front_addr":"172.21.15.78:6807/780655399","state":["exists","up"]},{"osd":1,"uuid":"549cfd32-c874-4e14-9489-c7fb33b2ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6801","nonce":3339409975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6803","nonce":3339409975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6807","nonce":3339409975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3339409975},{"type":"v1","addr":"172.21.15.84:6805","nonce":3339409975}]},"public_addr":"172.21.15.84:6801/3339409975","cluster_addr":"172.21.15.84:6803/3339409975","heartbeat_back_addr":"172.21.15.84:6807/3339409975","heartbeat_front_addr":"172.21.15.84:6805/3339409975","state":["exists","up"]},{"osd":2,"uuid":"233933e5-4dde-47f6-b007-015683591374","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6801","nonce":55763200}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6803","nonce":55763200}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6807","nonce":55763200}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":55763200},{"type":"v1","addr":"172.21.15.172:6805","nonce":55763200}]},"public_addr":"172.21.15.172:6801/55763200","cluster_addr":"172.21.15.172:6803/55763200","heartbeat_back_addr":"172.21.15.172:6807/55763200","heartbeat_front_addr":"172.21.15.172:6805/55763200","state":["exists","up"]},{"osd":3,"uuid":"8bb0ba8c-de4c-4334-a4cf-dda60352271a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6811","nonce":586567339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6813","nonce":586567339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6817","nonce":586567339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":586567339},{"type":"v1","addr":"172.21.15.78:6815","nonce":586567339}]},"public_addr":"172.21.15.78:6811/586567339","cluster_addr":"172.21.15.78:6813/586567339","heartbeat_back_addr":"172.21.15.78:6817/586567339","heartbeat_front_addr":"172.21.15.78:6815/586567339","state":["exists","up"]},{"osd":4,"uuid":"727cecb4-86c9-438c-9c09-7468e39aad27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6809","nonce":2716060044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6811","nonce":2716060044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6815","nonce":2716060044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2716060044},{"type":"v1","addr":"172.21.15.84:6813","nonce":2716060044}]},"public_addr":"172.21.15.84:6809/2716060044","cluster_addr":"172.21.15.84:6811/2716060044","heartbeat_back_addr":"172.21.15.84:6815/2716060044","heartbeat_front_addr":"172.21.15.84:6813/2716060044","state":["exists","up"]},{"osd":5,"uuid":"2cc5c54e-742b-4406-80d2-e84e0ba50b8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6809","nonce":3395704434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6811","nonce":3395704434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6815","nonce":3395704434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":3395704434},{"type":"v1","addr":"172.21.15.172:6813","nonce":3395704434}]},"public_addr":"172.21.15.172:6809/3395704434","cluster_addr":"172.21.15.172:6811/3395704434","heartbeat_back_addr":"172.21.15.172:6815/3395704434","heartbeat_front_addr":"172.21.15.172:6813/3395704434","state":["exists","up"]},{"osd":6,"uuid":"2a3eb06b-7754-4293-bb69-301f02709e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6819","nonce":4029644708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6821","nonce":4029644708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6825","nonce":4029644708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4029644708},{"type":"v1","addr":"172.21.15.78:6823","nonce":4029644708}]},"public_addr":"172.21.15.78:6819/4029644708","cluster_addr":"172.21.15.78:6821/4029644708","heartbeat_back_addr":"172.21.15.78:6825/4029644708","heartbeat_front_addr":"172.21.15.78:6823/4029644708","state":["exists","up"]},{"osd":7,"uuid":"9155447f-a10d-4e00-8f4f-a891a095b823","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6817","nonce":429797253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6819","nonce":429797253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6823","nonce":429797253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":429797253},{"type":"v1","addr":"172.21.15.84:6821","nonce":429797253}]},"public_addr":"172.21.15.84:6817/429797253","cluster_addr":"172.21.15.84:6819/429797253","heartbeat_back_addr":"172.21.15.84:6823/429797253","heartbeat_front_addr":"172.21.15.84:6821/429797253","state":["exists","up"]},{"osd":8,"uuid":"26bbfd7f-c916-46e3-8908-19f65e5350a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6817","nonce":3263777193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6819","nonce":3263777193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6823","nonce":3263777193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3263777193},{"type":"v1","addr":"172.21.15.172:6821","nonce":3263777193}]},"public_addr":"172.21.15.172:6817/3263777193","cluster_addr":"172.21.15.172:6819/3263777193","heartbeat_back_addr":"172.21.15.172:6823/3263777193","heartbeat_front_addr":"172.21.15.172:6821/3263777193","state":["exists","up"]},{"osd":9,"uuid":"92534fe2-a86c-45e6-b3f1-b436d49fb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6827","nonce":2889072877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6829","nonce":2889072877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6833","nonce":2889072877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2889072877},{"type":"v1","addr":"172.21.15.78:6831","nonce":2889072877}]},"public_addr":"172.21.15.78:6827/2889072877","cluster_addr":"172.21.15.78:6829/2889072877","heartbeat_back_addr":"172.21.15.78:6833/2889072877","heartbeat_front_addr":"172.21.15.78:6831/2889072877","state":["exists","up"]},{"osd":10,"uuid":"e378331e-ef86-4d80-a184-6acc0d9283b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6825","nonce":1416740807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6827","nonce":1416740807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6831","nonce":1416740807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1416740807},{"type":"v1","addr":"172.21.15.84:6829","nonce":1416740807}]},"public_addr":"172.21.15.84:6825/1416740807","cluster_addr":"172.21.15.84:6827/1416740807","heartbeat_back_addr":"172.21.15.84:6831/1416740807","heartbeat_front_addr":"172.21.15.84:6829/1416740807","state":["exists","up"]},{"osd":11,"uuid":"07bc1da2-963e-42e0-9b37-16a9a1be16c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6825","nonce":1663785983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6827","nonce":1663785983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6831","nonce":1663785983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1663785983},{"type":"v1","addr":"172.21.15.172:6829","nonce":1663785983}]},"public_addr":"172.21.15.172:6825/1663785983","cluster_addr":"172.21.15.172:6827/1663785983","heartbeat_back_addr":"172.21.15.172:6831/1663785983","heartbeat_front_addr":"172.21.15.172:6829/1663785983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:08:49.030021+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:13.517125+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:09:37.653806+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:02.067942+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:26.815395+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:10:50.941555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:15.933013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:11:40.500677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:04.975227+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:31.475610+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:12:56.780991+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T15:13:21.989868+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.a","mappings":[{"from":1,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":3,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6801/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/2228258510":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/1173534629":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/3341206163":"2024-05-22T15:07:15.158228+0000","172.21.15.78:6800/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2012910515":"2024-05-22T15:16:08.698984+0000","172.21.15.78:6800/3694112952":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/1077949258":"2024-05-22T15:06:47.981082+0000","172.21.15.78:6801/888544333":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2800502330":"2024-05-22T15:07:15.158228+0000","172.21.15.78:0/2303085949":"2024-05-22T15:06:47.981082+0000","172.21.15.78:0/2194549734":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/3526992601":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2604499013":"2024-05-22T15:16:08.698984+0000","172.21.15.78:0/2144862462":"2024-05-22T15:06:47.981082+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T15:27:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.299+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12200a2d80 msgr2=0x7f12200a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.299+0000 7f123cbce640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12200a2d80 0x7f12200a5290 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1238107b50 tx=0x7f122c005f90 comp rx=0 tx=0).stop 2024-05-21T15:27:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.299+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 msgr2=0x7f1238199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.299+0000 7f123cbce640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 0x7f1238199760 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f122800ebc0 tx=0x7f122800c6a0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.299+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 shutdown_connections 2024-05-21T15:27:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.300+0000 7f123cbce640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1238108f40 0x7f123819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.300+0000 7f123cbce640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f12200a2d80 0x7f12200a5290 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.300+0000 7f123cbce640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1238108aa0 0x7f1238199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.300+0000 7f123cbce640 1 --2- 172.21.15.78:0/1131454901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1238101740 0x7f1238199760 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.300+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 >> 172.21.15.78:0/1131454901 conn(0x7f12380fc7c0 msgr2=0x7f12380fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:20.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.300+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 shutdown_connections 2024-05-21T15:27:20.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.300+0000 7f123cbce640 1 -- 172.21.15.78:0/1131454901 wait complete. 2024-05-21T15:27:20.315 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-21T15:27:20.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 -- 172.21.15.78:0/3462930169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 msgr2=0x7fad4010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 --2- 172.21.15.78:0/3462930169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 0x7fad4010e750 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fad38009ca0 tx=0x7fad3802a8c0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 -- 172.21.15.78:0/3462930169 shutdown_connections 2024-05-21T15:27:20.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 --2- 172.21.15.78:0/3462930169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad40101aa0 0x7fad40101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 --2- 172.21.15.78:0/3462930169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 0x7fad4010e750 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 --2- 172.21.15.78:0/3462930169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad401079d0 0x7fad40101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 -- 172.21.15.78:0/3462930169 >> 172.21.15.78:0/3462930169 conn(0x7fad400770b0 msgr2=0x7fad400774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 -- 172.21.15.78:0/3462930169 shutdown_connections 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 -- 172.21.15.78:0/3462930169 wait complete. 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 Processor -- start 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.424+0000 7fad4558f640 1 -- start start 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad4558f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40101aa0 0x7fad4019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad4558f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad401079d0 0x7fad4019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad4558f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 0x7fad4019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad4558f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad40111ce0 con 0x7fad4010c330 2024-05-21T15:27:20.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad4558f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fad40111b60 con 0x7fad401079d0 2024-05-21T15:27:20.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad4558f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fad40111e60 con 0x7fad40101aa0 2024-05-21T15:27:20.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 0x7fad4019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 0x7fad4019b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41056/0 (socket says 172.21.15.78:41056) 2024-05-21T15:27:20.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 -- 172.21.15.78:0/88698138 learned_addr learned my addr 172.21.15.78:0/88698138 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:20.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3effd640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40101aa0 0x7fad4019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad37fff640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad401079d0 0x7fad4019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 -- 172.21.15.78:0/88698138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40101aa0 msgr2=0x7fad4019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40101aa0 0x7fad4019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 -- 172.21.15.78:0/88698138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad401079d0 msgr2=0x7fad4019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad401079d0 0x7fad4019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 -- 172.21.15.78:0/88698138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad401a58f0 con 0x7fad4010c330 2024-05-21T15:27:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad37fff640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad401079d0 0x7fad4019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:27:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3f7fe640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 0x7fad4019b060 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fad38031a20 tx=0x7fad38031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3cff9640 1 -- 172.21.15.78:0/88698138 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38003c30 con 0x7fad4010c330 2024-05-21T15:27:20.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3cff9640 1 -- 172.21.15.78:0/88698138 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad38003dd0 con 0x7fad4010c330 2024-05-21T15:27:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad401a5b80 con 0x7fad4010c330 2024-05-21T15:27:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad3cff9640 1 -- 172.21.15.78:0/88698138 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3802fcf0 con 0x7fad4010c330 2024-05-21T15:27:20.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.425+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad401a60d0 con 0x7fad4010c330 2024-05-21T15:27:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.426+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad40069f50 con 0x7fad4010c330 2024-05-21T15:27:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.428+0000 7fad3cff9640 1 -- 172.21.15.78:0/88698138 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fad38004050 con 0x7fad4010c330 2024-05-21T15:27:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.429+0000 7fad3cff9640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fad2c0a2d00 0x7fad2c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.429+0000 7fad3effd640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fad2c0a2d00 0x7fad2c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.429+0000 7fad3cff9640 1 -- 172.21.15.78:0/88698138 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8658+0+0 (secure 0 0 0) 0x7fad380f0780 con 0x7fad4010c330 2024-05-21T15:27:20.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.429+0000 7fad3effd640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fad2c0a2d00 0x7fad2c0a5210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fad401029d0 tx=0x7fad3000b390 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:20.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.434+0000 7fad3cff9640 1 -- 172.21.15.78:0/88698138 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fad380bd4b0 con 0x7fad4010c330 2024-05-21T15:27:20.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-mon[31997]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:20.597 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/2921942152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/498273595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.598 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/1131454901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.648+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fad40105900 con 0x7fad4010c330 2024-05-21T15:27:20.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.651+0000 7fad3cff9640 1 -- 172.21.15.78:0/88698138 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7fad3802fe90 con 0x7fad4010c330 2024-05-21T15:27:20.653 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-21T15:27:20.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fad2c0a2d00 msgr2=0x7fad2c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fad2c0a2d00 0x7fad2c0a5210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fad401029d0 tx=0x7fad3000b390 comp rx=0 tx=0).stop 2024-05-21T15:27:20.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 msgr2=0x7fad4019b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 0x7fad4019b060 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fad38031a20 tx=0x7fad38031a50 comp rx=0 tx=0).stop 2024-05-21T15:27:20.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 shutdown_connections 2024-05-21T15:27:20.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad401079d0 0x7fad4019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7fad2c0a2d00 0x7fad2c0a5210 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4010c330 0x7fad4019b060 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 --2- 172.21.15.78:0/88698138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40101aa0 0x7fad4019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 >> 172.21.15.78:0/88698138 conn(0x7fad400770b0 msgr2=0x7fad40103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:20.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 shutdown_connections 2024-05-21T15:27:20.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.654+0000 7fad4558f640 1 -- 172.21.15.78:0/88698138 wait complete. 2024-05-21T15:27:20.673 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-21T15:27:20.725 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:20 smithi084 ceph-mon[36334]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:20.725 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:20 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/2921942152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.725 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:20 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/498273595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.725 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:20 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/1131454901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.728 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:20 smithi172 ceph-mon[36402]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T15:27:20.728 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:20 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/2921942152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.728 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:20 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/498273595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.728 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:20 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/1131454901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T15:27:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.791+0000 7f89eec92640 1 -- 172.21.15.78:0/2120423760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e810c340 msgr2=0x7f89e810e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.791+0000 7f89eec92640 1 --2- 172.21.15.78:0/2120423760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e810c340 0x7f89e810e760 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009ca0 tx=0x7f89e002a8c0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.791+0000 7f89eec92640 1 -- 172.21.15.78:0/2120423760 shutdown_connections 2024-05-21T15:27:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.791+0000 7f89eec92640 1 --2- 172.21.15.78:0/2120423760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e8101ab0 0x7f89e8101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.791+0000 7f89eec92640 1 --2- 172.21.15.78:0/2120423760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e810c340 0x7f89e810e760 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.791+0000 7f89eec92640 1 --2- 172.21.15.78:0/2120423760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e81079e0 0x7f89e8101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.791+0000 7f89eec92640 1 -- 172.21.15.78:0/2120423760 >> 172.21.15.78:0/2120423760 conn(0x7f89e80fd210 msgr2=0x7f89e80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:20.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.792+0000 7f89eec92640 1 -- 172.21.15.78:0/2120423760 shutdown_connections 2024-05-21T15:27:20.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.792+0000 7f89eec92640 1 -- 172.21.15.78:0/2120423760 wait complete. 2024-05-21T15:27:20.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.792+0000 7f89eec92640 1 Processor -- start 2024-05-21T15:27:20.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.792+0000 7f89eec92640 1 -- start start 2024-05-21T15:27:20.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.792+0000 7f89eec92640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e8101ab0 0x7f89e8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.792+0000 7f89eec92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e81079e0 0x7f89e8199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89eec92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e810c340 0x7f89e819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89eec92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8111d10 con 0x7f89e81079e0 2024-05-21T15:27:20.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89eec92640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8111b90 con 0x7f89e810c340 2024-05-21T15:27:20.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89eec92640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8111e90 con 0x7f89e8101ab0 2024-05-21T15:27:20.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e81079e0 0x7f89e8199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e81079e0 0x7f89e8199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41110/0 (socket says 172.21.15.78:41110) 2024-05-21T15:27:20.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 -- 172.21.15.78:0/3980561012 learned_addr learned my addr 172.21.15.78:0/3980561012 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:20.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89ed22f640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e810c340 0x7f89e819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89eca2e640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e8101ab0 0x7f89e8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 -- 172.21.15.78:0/3980561012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e8101ab0 msgr2=0x7f89e8199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e8101ab0 0x7f89e8199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 -- 172.21.15.78:0/3980561012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e810c340 msgr2=0x7f89e819df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:20.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e810c340 0x7f89e819df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:20.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 -- 172.21.15.78:0/3980561012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e819e5a0 con 0x7f89e81079e0 2024-05-21T15:27:20.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89eca2e640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e8101ab0 0x7f89e8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T15:27:20.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e7fff640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e81079e0 0x7f89e8199ca0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f89d400e9f0 tx=0x7f89d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:20.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e5ffb640 1 -- 172.21.15.78:0/3980561012 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d400ccc0 con 0x7f89e81079e0 2024-05-21T15:27:20.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e5ffb640 1 -- 172.21.15.78:0/3980561012 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89d40035f0 con 0x7f89e81079e0 2024-05-21T15:27:20.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e819e890 con 0x7f89e81079e0 2024-05-21T15:27:20.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.793+0000 7f89e5ffb640 1 -- 172.21.15.78:0/3980561012 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d400ce60 con 0x7f89e81079e0 2024-05-21T15:27:20.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.794+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e81a6210 con 0x7f89e81079e0 2024-05-21T15:27:20.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.794+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e8102c90 con 0x7f89e81079e0 2024-05-21T15:27:20.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.796+0000 7f89e5ffb640 1 -- 172.21.15.78:0/3980561012 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f89d40037a0 con 0x7f89e81079e0 2024-05-21T15:27:20.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.796+0000 7f89e5ffb640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f89d80a2d00 0x7f89d80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:20.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.796+0000 7f89eca2e640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f89d80a2d00 0x7f89d80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:20.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.796+0000 7f89e5ffb640 1 -- 172.21.15.78:0/3980561012 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9044+0+0 (secure 0 0 0) 0x7f89d40cfa10 con 0x7f89e81079e0 2024-05-21T15:27:20.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.797+0000 7f89eca2e640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f89d80a2d00 0x7f89d80a5210 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f89e81029e0 tx=0x7f89dc005e50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:20.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:20.801+0000 7f89e5ffb640 1 -- 172.21.15.78:0/3980561012 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f89d409c420 con 0x7f89e81079e0 2024-05-21T15:27:21.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.017+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f89e81a6440 con 0x7f89e81079e0 2024-05-21T15:27:21.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a[31993]: 2024-05-21T15:27:20.649+0000 7f66183e9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T15:27:21.098 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a[73770]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.098 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a[73770]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.098 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-mds[73774]: -- 172.21.15.78:0/3374707760 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55659655ba40 con 0x5565965d3a80 2024-05-21T15:27:21.098 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:27:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a[73770]: starting mds.a at 2024-05-21T15:27:21.099 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:27:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-g[73414]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.099 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:27:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-g[73414]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.099 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:27:20 smithi078 ceph-mds[73420]: -- 172.21.15.78:0/3778958597 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e010fba40 con 0x558e01173180 2024-05-21T15:27:21.099 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:27:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-g[73414]: starting mds.g at 2024-05-21T15:27:21.146 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k[66229]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.146 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k[66229]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.146 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:27:20 smithi084 ceph-mds[66233]: -- 172.21.15.84:0/162654824 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3a8a43a40 con 0x55f3a8abb600 2024-05-21T15:27:21.146 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k[66229]: starting mds.k at 2024-05-21T15:27:21.147 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.147 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.147 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:27:20 smithi084 ceph-mds[67165]: -- 172.21.15.84:0/3680659043 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564243a03a40 con 0x564243a7b600 2024-05-21T15:27:21.147 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: starting mds.b at 2024-05-21T15:27:21.148 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e[66544]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.148 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e[66544]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.148 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:27:20 smithi084 ceph-mds[66548]: -- 172.21.15.84:0/4193222179 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9ba6d9a40 con 0x55a9ba751180 2024-05-21T15:27:21.148 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e[66544]: starting mds.e at 2024-05-21T15:27:21.148 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h[66847]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.148 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h[66847]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.149 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:27:20 smithi084 ceph-mds[66851]: -- 172.21.15.84:0/1450618260 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c257587a40 con 0x55c2575ffa80 2024-05-21T15:27:21.149 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:27:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h[66847]: starting mds.h at 2024-05-21T15:27:21.183 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c[66713]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.184 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c[66713]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.184 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:27:20 smithi172 ceph-mds[66717]: -- 172.21.15.172:0/1221077801 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf75967a40 con 0x55cf759df600 2024-05-21T15:27:21.184 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c[66713]: starting mds.c at 2024-05-21T15:27:21.184 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f[66412]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.185 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f[66412]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.185 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:27:20 smithi172 ceph-mds[66416]: -- 172.21.15.172:0/3578766430 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56345ecc3a40 con 0x56345ed3b180 2024-05-21T15:27:21.185 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f[66412]: starting mds.f at 2024-05-21T15:27:21.185 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i[67028]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.185 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i[67028]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.186 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:27:20 smithi172 ceph-mds[67032]: -- 172.21.15.172:0/2142677578 <== mon.2 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614c4cf1a40 con 0x5614c4d69a80 2024-05-21T15:27:21.186 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i[67028]: starting mds.i at 2024-05-21T15:27:21.186 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l[66108]: ignoring --setuser ceph since I am not root 2024-05-21T15:27:21.186 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l[66108]: ignoring --setgroup ceph since I am not root 2024-05-21T15:27:21.186 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:27:20 smithi172 ceph-mds[66112]: -- 172.21.15.172:0/3112558419 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9692c9a40 con 0x55f969341600 2024-05-21T15:27:21.186 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:27:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l[66108]: starting mds.l at 2024-05-21T15:27:21.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.656+0000 7f89e5ffb640 1 -- 172.21.15.78:0/3980561012 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f89d409bd20 con 0x7f89e81079e0 2024-05-21T15:27:21.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f89d80a2d00 msgr2=0x7f89d80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f89d80a2d00 0x7f89d80a5210 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f89e81029e0 tx=0x7f89dc005e50 comp rx=0 tx=0).stop 2024-05-21T15:27:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e81079e0 msgr2=0x7f89e8199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e81079e0 0x7f89e8199ca0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f89d400e9f0 tx=0x7f89d400eeb0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 shutdown_connections 2024-05-21T15:27:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e810c340 0x7f89e819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f89d80a2d00 0x7f89d80a5210 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89e81079e0 0x7f89e8199ca0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 --2- 172.21.15.78:0/3980561012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e8101ab0 0x7f89e8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 >> 172.21.15.78:0/3980561012 conn(0x7f89e80fd210 msgr2=0x7f89e8073d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:21.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 shutdown_connections 2024-05-21T15:27:21.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.660+0000 7f89eec92640 1 -- 172.21.15.78:0/3980561012 wait complete. 2024-05-21T15:27:21.681 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-21T15:27:21.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 -- 172.21.15.78:0/537783819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c1079f0 msgr2=0x7f880c101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:21.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 --2- 172.21.15.78:0/537783819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c1079f0 0x7f880c101cd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009900 tx=0x7f87fc02a410 comp rx=0 tx=0).stop 2024-05-21T15:27:21.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 -- 172.21.15.78:0/537783819 shutdown_connections 2024-05-21T15:27:21.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 --2- 172.21.15.78:0/537783819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f880c10c350 0x7f880c10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 --2- 172.21.15.78:0/537783819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f880c102210 0x7f880c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 --2- 172.21.15.78:0/537783819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c1079f0 0x7f880c101cd0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 -- 172.21.15.78:0/537783819 >> 172.21.15.78:0/537783819 conn(0x7f880c076830 msgr2=0x7f880c076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:21.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 -- 172.21.15.78:0/537783819 shutdown_connections 2024-05-21T15:27:21.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 -- 172.21.15.78:0/537783819 wait complete. 2024-05-21T15:27:21.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 Processor -- start 2024-05-21T15:27:21.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.802+0000 7f881148d640 1 -- start start 2024-05-21T15:27:21.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f881148d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f880c102210 0x7f880c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:21.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f881148d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f880c1079f0 0x7f880c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f881148d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c10c350 0x7f880c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f881148d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f880c111d00 con 0x7f880c10c350 2024-05-21T15:27:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f881148d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f880c111b80 con 0x7f880c1079f0 2024-05-21T15:27:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f881148d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f880c111e80 con 0x7f880c102210 2024-05-21T15:27:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c10c350 0x7f880c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f8803fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f880c1079f0 0x7f880c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:21.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c10c350 0x7f880c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41132/0 (socket says 172.21.15.78:41132) 2024-05-21T15:27:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 -- 172.21.15.78:0/152825918 learned_addr learned my addr 172.21.15.78:0/152825918 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880affd640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f880c102210 0x7f880c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 -- 172.21.15.78:0/152825918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f880c102210 msgr2=0x7f880c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f880c102210 0x7f880c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 -- 172.21.15.78:0/152825918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f880c1079f0 msgr2=0x7f880c199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f880c1079f0 0x7f880c199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 -- 172.21.15.78:0/152825918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c19e610 con 0x7f880c10c350 2024-05-21T15:27:21.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.803+0000 7f880b7fe640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c10c350 0x7f880c19dfd0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f880400eb10 tx=0x7f880400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.804+0000 7f8808ff9640 1 -- 172.21.15.78:0/152825918 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004b20 con 0x7f880c10c350 2024-05-21T15:27:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.804+0000 7f8808ff9640 1 -- 172.21.15.78:0/152825918 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804004cc0 con 0x7f880c10c350 2024-05-21T15:27:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.804+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c19e900 con 0x7f880c10c350 2024-05-21T15:27:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.804+0000 7f8808ff9640 1 -- 172.21.15.78:0/152825918 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804010760 con 0x7f880c10c350 2024-05-21T15:27:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.804+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c19ee00 con 0x7f880c10c350 2024-05-21T15:27:21.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.805+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c102ce0 con 0x7f880c10c350 2024-05-21T15:27:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.806+0000 7f8808ff9640 1 -- 172.21.15.78:0/152825918 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8804005000 con 0x7f880c10c350 2024-05-21T15:27:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.807+0000 7f8808ff9640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f87f80a2d00 0x7f87f80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.807+0000 7f880affd640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f87f80a2d00 0x7f87f80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:21.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.807+0000 7f8808ff9640 1 -- 172.21.15.78:0/152825918 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9044+0+0 (secure 0 0 0) 0x7f88040cfe00 con 0x7f880c10c350 2024-05-21T15:27:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.807+0000 7f880affd640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f87f80a2d00 0x7f87f80a5210 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87fc004770 tx=0x7f87fc0058d0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:21.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.811+0000 7f8808ff9640 1 -- 172.21.15.78:0/152825918 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8804053850 con 0x7f880c10c350 2024-05-21T15:27:21.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:21 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/88698138' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T15:27:21.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:21 smithi078 ceph-mon[31997]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T15:27:21.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:21 smithi078 ceph-mon[31997]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T15:27:21.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:21 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/88698138' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T15:27:21.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:21 smithi078 ceph-mon[31997]: osdmap e101: 12 total, 12 up, 12 in 2024-05-21T15:27:21.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:21 smithi078 ceph-mon[31997]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T15:27:21.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:21 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3980561012' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:21.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:21 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/88698138' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T15:27:21.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:21 smithi084 ceph-mon[36334]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T15:27:21.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:21 smithi084 ceph-mon[36334]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T15:27:21.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:21 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/88698138' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T15:27:21.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:21 smithi084 ceph-mon[36334]: osdmap e101: 12 total, 12 up, 12 in 2024-05-21T15:27:21.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:21 smithi084 ceph-mon[36334]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T15:27:21.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:21 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3980561012' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:21.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:21 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/88698138' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T15:27:21.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:21 smithi172 ceph-mon[36402]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T15:27:21.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:21 smithi172 ceph-mon[36402]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T15:27:21.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:21 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/88698138' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T15:27:21.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:21 smithi172 ceph-mon[36402]: osdmap e101: 12 total, 12 up, 12 in 2024-05-21T15:27:21.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:21 smithi172 ceph-mon[36402]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T15:27:21.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:21 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3980561012' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T15:27:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:21.997+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f880c19ab00 con 0x7f880c10c350 2024-05-21T15:27:22.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.657+0000 7f8808ff9640 1 -- 172.21.15.78:0/152825918 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f880c19ab00 con 0x7f880c10c350 2024-05-21T15:27:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-05-21T15:27:22.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f87f80a2d00 msgr2=0x7f87f80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:22.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f87f80a2d00 0x7f87f80a5210 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87fc004770 tx=0x7f87fc0058d0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c10c350 msgr2=0x7f880c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c10c350 0x7f880c19dfd0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f880400eb10 tx=0x7f880400c2d0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 shutdown_connections 2024-05-21T15:27:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f880c1079f0 0x7f880c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7f87f80a2d00 0x7f87f80a5210 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f880c10c350 0x7f880c19dfd0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 --2- 172.21.15.78:0/152825918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f880c102210 0x7f880c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.660+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 >> 172.21.15.78:0/152825918 conn(0x7f880c076830 msgr2=0x7f880c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.661+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 shutdown_connections 2024-05-21T15:27:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.661+0000 7f881148d640 1 -- 172.21.15.78:0/152825918 wait complete. 2024-05-21T15:27:22.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-21T15:27:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 -- 172.21.15.78:0/1524426853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98101000 msgr2=0x7faf98101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/1524426853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98101000 0x7faf98101410 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7faf84009900 tx=0x7faf8402a410 comp rx=0 tx=0).stop 2024-05-21T15:27:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 -- 172.21.15.78:0/1524426853 shutdown_connections 2024-05-21T15:27:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/1524426853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf9810d500 0x7faf9810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/1524426853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98101000 0x7faf98101410 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/1524426853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf98108b50 0x7faf98100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 -- 172.21.15.78:0/1524426853 >> 172.21.15.78:0/1524426853 conn(0x7faf98076830 msgr2=0x7faf98076c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 -- 172.21.15.78:0/1524426853 shutdown_connections 2024-05-21T15:27:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 -- 172.21.15.78:0/1524426853 wait complete. 2024-05-21T15:27:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 Processor -- start 2024-05-21T15:27:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.793+0000 7faf9ec2b640 1 -- start start 2024-05-21T15:27:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9ec2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf98101000 0x7faf98101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9ec2b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98108b50 0x7faf981021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9ec2b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9810d500 0x7faf98102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9ec2b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf98112e10 con 0x7faf98108b50 2024-05-21T15:27:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9ec2b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf98112c90 con 0x7faf98101000 2024-05-21T15:27:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9ec2b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faf98112f90 con 0x7faf9810d500 2024-05-21T15:27:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98108b50 0x7faf981021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98108b50 0x7faf981021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41154/0 (socket says 172.21.15.78:41154) 2024-05-21T15:27:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 -- 172.21.15.78:0/4107309246 learned_addr learned my addr 172.21.15.78:0/4107309246 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-05-21T15:27:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9c9c7640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf98101000 0x7faf98101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:22.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9d1c8640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9810d500 0x7faf98102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 -- 172.21.15.78:0/4107309246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9810d500 msgr2=0x7faf98102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9810d500 0x7faf98102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 -- 172.21.15.78:0/4107309246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf98101000 msgr2=0x7faf98101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf98101000 0x7faf98101ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 -- 172.21.15.78:0/4107309246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf981a7b70 con 0x7faf98108b50 2024-05-21T15:27:22.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf97fff640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98108b50 0x7faf981021e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7faf84005940 tx=0x7faf840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf95ffb640 1 -- 172.21.15.78:0/4107309246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8403f030 con 0x7faf98108b50 2024-05-21T15:27:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf95ffb640 1 -- 172.21.15.78:0/4107309246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf840044c0 con 0x7faf98108b50 2024-05-21T15:27:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.794+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf981a7e00 con 0x7faf98108b50 2024-05-21T15:27:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.795+0000 7faf95ffb640 1 -- 172.21.15.78:0/4107309246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84035070 con 0x7faf98108b50 2024-05-21T15:27:22.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.795+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981a8260 con 0x7faf98108b50 2024-05-21T15:27:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.796+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf98068290 con 0x7faf98108b50 2024-05-21T15:27:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.797+0000 7faf95ffb640 1 -- 172.21.15.78:0/4107309246 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faf84042030 con 0x7faf98108b50 2024-05-21T15:27:22.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.798+0000 7faf95ffb640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faf880a2d00 0x7faf880a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T15:27:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.798+0000 7faf9c9c7640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faf880a2d00 0x7faf880a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T15:27:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.798+0000 7faf95ffb640 1 -- 172.21.15.78:0/4107309246 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8490+0+0 (secure 0 0 0) 0x7faf840ef940 con 0x7faf98108b50 2024-05-21T15:27:22.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.799+0000 7faf9c9c7640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faf880a2d00 0x7faf880a5210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faf980744d0 tx=0x7faf8c005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T15:27:22.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.802+0000 7faf95ffb640 1 -- 172.21.15.78:0/4107309246 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faf840bc520 con 0x7faf98108b50 2024-05-21T15:27:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-21T15:27:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T15:27:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T15:27:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: Cluster is now healthy 2024-05-21T15:27:22.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/3980561012' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.78:6838/470714917,v1:172.21.15.78:6839/470714917] up:boot 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.78:6840/4109845384,v1:172.21.15.78:6841/4109845384] up:boot 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6832/2446076663,v1:172.21.15.84:6834/2446076663] up:boot 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6833/3429374306,v1:172.21.15.84:6835/3429374306] up:boot 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6832/1337292664,v1:172.21.15.172:6833/1337292664] up:boot 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6836/372041369,v1:172.21.15.84:6837/372041369] up:boot 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.84:6838/2811951689,v1:172.21.15.84:6839/2811951689] up:boot 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6834/1489826338,v1:172.21.15.172:6835/1489826338] up:boot 2024-05-21T15:27:22.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6836/1914104403,v1:172.21.15.172:6837/1914104403] up:boot 2024-05-21T15:27:22.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: mds.? [v2:172.21.15.172:6838/3758356183,v1:172.21.15.172:6839/3758356183] up:boot 2024-05-21T15:27:22.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: fsmap 12 up:standby 2024-05-21T15:27:22.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:27:22.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:27:22.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:27:22.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:27:22.850 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:27:22.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:27:22.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:27:22.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:27:22.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:27:22.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:27:22.851 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:22 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/152825918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:27:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-21T15:27:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T15:27:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T15:27:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: Cluster is now healthy 2024-05-21T15:27:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/3980561012' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:22.896 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.78:6838/470714917,v1:172.21.15.78:6839/470714917] up:boot 2024-05-21T15:27:22.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.78:6840/4109845384,v1:172.21.15.78:6841/4109845384] up:boot 2024-05-21T15:27:22.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6832/2446076663,v1:172.21.15.84:6834/2446076663] up:boot 2024-05-21T15:27:22.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6833/3429374306,v1:172.21.15.84:6835/3429374306] up:boot 2024-05-21T15:27:22.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6832/1337292664,v1:172.21.15.172:6833/1337292664] up:boot 2024-05-21T15:27:22.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6836/372041369,v1:172.21.15.84:6837/372041369] up:boot 2024-05-21T15:27:22.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.84:6838/2811951689,v1:172.21.15.84:6839/2811951689] up:boot 2024-05-21T15:27:22.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6834/1489826338,v1:172.21.15.172:6835/1489826338] up:boot 2024-05-21T15:27:22.897 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6836/1914104403,v1:172.21.15.172:6837/1914104403] up:boot 2024-05-21T15:27:22.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: mds.? [v2:172.21.15.172:6838/3758356183,v1:172.21.15.172:6839/3758356183] up:boot 2024-05-21T15:27:22.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: fsmap 12 up:standby 2024-05-21T15:27:22.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:27:22.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:27:22.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:27:22.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:27:22.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:27:22.898 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:27:22.899 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:27:22.899 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:27:22.899 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:27:22.899 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:27:22.899 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:22 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/152825918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:27:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-21T15:27:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T15:27:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T15:27:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: Cluster is now healthy 2024-05-21T15:27:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/3980561012' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T15:27:22.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.78:6838/470714917,v1:172.21.15.78:6839/470714917] up:boot 2024-05-21T15:27:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.78:6840/4109845384,v1:172.21.15.78:6841/4109845384] up:boot 2024-05-21T15:27:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6832/2446076663,v1:172.21.15.84:6834/2446076663] up:boot 2024-05-21T15:27:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6833/3429374306,v1:172.21.15.84:6835/3429374306] up:boot 2024-05-21T15:27:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6832/1337292664,v1:172.21.15.172:6833/1337292664] up:boot 2024-05-21T15:27:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6836/372041369,v1:172.21.15.84:6837/372041369] up:boot 2024-05-21T15:27:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.84:6838/2811951689,v1:172.21.15.84:6839/2811951689] up:boot 2024-05-21T15:27:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6834/1489826338,v1:172.21.15.172:6835/1489826338] up:boot 2024-05-21T15:27:22.935 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6836/1914104403,v1:172.21.15.172:6837/1914104403] up:boot 2024-05-21T15:27:22.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: mds.? [v2:172.21.15.172:6838/3758356183,v1:172.21.15.172:6839/3758356183] up:boot 2024-05-21T15:27:22.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: fsmap 12 up:standby 2024-05-21T15:27:22.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T15:27:22.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T15:27:22.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T15:27:22.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T15:27:22.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T15:27:22.936 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T15:27:22.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T15:27:22.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T15:27:22.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T15:27:22.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='mgr.15399 172.21.15.78:0/3968074240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T15:27:22.937 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:22 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/152825918' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:27:22.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:22.984+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faf9810bd60 con 0x7faf98108b50 2024-05-21T15:27:23.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.664+0000 7faf95ffb640 1 -- 172.21.15.78:0/4107309246 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7faf840bbe20 con 0x7faf98108b50 2024-05-21T15:27:23.667 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-05-21T15:27:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.667+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faf880a2d00 msgr2=0x7faf880a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faf880a2d00 0x7faf880a5210 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faf980744d0 tx=0x7faf8c005f90 comp rx=0 tx=0).stop 2024-05-21T15:27:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98108b50 msgr2=0x7faf981021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T15:27:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98108b50 0x7faf981021e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7faf84005940 tx=0x7faf840040c0 comp rx=0 tx=0).stop 2024-05-21T15:27:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 shutdown_connections 2024-05-21T15:27:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:6800/1100997097,v1:172.21.15.78:6801/1100997097] conn(0x7faf880a2d00 0x7faf880a5210 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9810d500 0x7faf98102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:23.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf98108b50 0x7faf981021e0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 --2- 172.21.15.78:0/4107309246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf98101000 0x7faf98101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T15:27:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 >> 172.21.15.78:0/4107309246 conn(0x7faf98076830 msgr2=0x7faf981094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T15:27:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 shutdown_connections 2024-05-21T15:27:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-05-21T15:27:23.668+0000 7faf9ec2b640 1 -- 172.21.15.78:0/4107309246 wait complete. 2024-05-21T15:27:23.691 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:27:23.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:27:23.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:27:23.804 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-21T15:27:23.804 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T15:27:23.825 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T15:27:23.845 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T15:27:23.865 INFO:tasks.cephadm:Stopping all daemons... 2024-05-21T15:27:23.865 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-21T15:27:23.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.a 2024-05-21T15:27:23.933 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:23 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/152825918' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T15:27:23.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:23 smithi172 ceph-mon[36402]: osdmap e102: 12 total, 12 up, 12 in 2024-05-21T15:27:23.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:23 smithi172 ceph-mon[36402]: pgmap v354: 65 pgs: 65 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:27:23.934 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:23 smithi172 ceph-mon[36402]: from='client.? 172.21.15.78:0/4107309246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:27:24.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:23 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/152825918' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T15:27:24.097 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:23 smithi078 ceph-mon[31997]: osdmap e102: 12 total, 12 up, 12 in 2024-05-21T15:27:24.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:23 smithi078 ceph-mon[31997]: pgmap v354: 65 pgs: 65 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:27:24.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:23 smithi078 ceph-mon[31997]: from='client.? 172.21.15.78:0/4107309246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:27:24.098 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:23 smithi078 systemd[1]: Stopping Ceph mon.a for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:27:24.145 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:23 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/152825918' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T15:27:24.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:23 smithi084 ceph-mon[36334]: osdmap e102: 12 total, 12 up, 12 in 2024-05-21T15:27:24.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:23 smithi084 ceph-mon[36334]: pgmap v354: 65 pgs: 65 active+clean; 13 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T15:27:24.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:23 smithi084 ceph-mon[36334]: from='client.? 172.21.15.78:0/4107309246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T15:27:24.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:24 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a[31993]: 2024-05-21T15:27:24.432+0000 7f661dbf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:27:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:24 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a[31993]: 2024-05-21T15:27:24.432+0000 7f661dbf4700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-21T15:27:24.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:24 smithi078 podman[105159]: 2024-05-21 15:27:24.494128165 +0000 UTC m=+0.234002131 container died 1cd58a23b21134cfd745909164202cfacf7cf75c5970da8614312a181c4c509b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0) 2024-05-21T15:27:25.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:24 smithi078 podman[105159]: 2024-05-21 15:27:24.899525148 +0000 UTC m=+0.639399114 container cleanup 1cd58a23b21134cfd745909164202cfacf7cf75c5970da8614312a181c4c509b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T15:27:25.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:24 smithi078 bash[105159]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a 2024-05-21T15:27:25.348 INFO:journalctl@ceph.mon.a.smithi078.stdout:May 21 15:27:25 smithi078 podman[105178]: 2024-05-21 15:27:25.134673272 +0000 UTC m=+0.630859319 container remove 1cd58a23b21134cfd745909164202cfacf7cf75c5970da8614312a181c4c509b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-21T15:27:25.616 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-21T15:27:25.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:27:25.685 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-21T15:27:25.685 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-21T15:27:25.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.b 2024-05-21T15:27:26.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:25 smithi084 systemd[1]: Stopping Ceph mon.b for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:27:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:26 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-b[36330]: 2024-05-21T15:27:26.311+0000 7fcffbb09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:27:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:26 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-b[36330]: 2024-05-21T15:27:26.311+0000 7fcffbb09700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-21T15:27:26.646 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:26 smithi084 podman[72693]: 2024-05-21 15:27:26.546491415 +0000 UTC m=+0.391393970 container died 85123a736b782adda8df75cfdbcc90ae68b04a8ac362501d2781819fb07a4f59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-b, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-21T15:27:27.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:26 smithi084 podman[72693]: 2024-05-21 15:27:26.904154272 +0000 UTC m=+0.749056816 container cleanup 85123a736b782adda8df75cfdbcc90ae68b04a8ac362501d2781819fb07a4f59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:27:27.146 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:26 smithi084 bash[72693]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-b 2024-05-21T15:27:27.437 INFO:journalctl@ceph.mon.b.smithi084.stdout:May 21 15:27:27 smithi084 podman[72705]: 2024-05-21 15:27:27.147214181 +0000 UTC m=+0.588902156 container remove 85123a736b782adda8df75cfdbcc90ae68b04a8ac362501d2781819fb07a4f59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-21T15:27:27.453 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-21T15:27:27.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:27:27.496 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-21T15:27:27.496 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-21T15:27:27.497 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.c 2024-05-21T15:27:27.810 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:27 smithi172 systemd[1]: Stopping Ceph mon.c for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:27:28.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c[36385]: 2024-05-21T15:27:27.806+0000 7fc19f9c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:27:28.183 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c[36385]: 2024-05-21T15:27:27.806+0000 7fc19f9c1700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-21T15:27:28.184 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:28 smithi172 podman[72704]: 2024-05-21 15:27:27.999887402 +0000 UTC m=+0.313067062 container died 14e48ae402610e5ce2c57bc349f15268681567c72498128f6af9cf3b3aaea333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T15:27:28.492 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:28 smithi172 podman[72704]: 2024-05-21 15:27:28.389688616 +0000 UTC m=+0.702868259 container cleanup 14e48ae402610e5ce2c57bc349f15268681567c72498128f6af9cf3b3aaea333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-21T15:27:28.492 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:28 smithi172 bash[72704]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c 2024-05-21T15:27:28.746 INFO:journalctl@ceph.mon.c.smithi172.stdout:May 21 15:27:28 smithi172 podman[72717]: 2024-05-21 15:27:28.616322603 +0000 UTC m=+0.616202901 container remove 14e48ae402610e5ce2c57bc349f15268681567c72498128f6af9cf3b3aaea333 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-21T15:27:28.763 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-21T15:27:28.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:27:28.813 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-21T15:27:28.813 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-21T15:27:28.813 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T15:27:29.098 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:27:28 smithi078 systemd[1]: Stopping Ceph mgr.x for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:27:29.524 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:27:29 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x[32226]: 2024-05-21T15:27:29.163+0000 7f7d7dd3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:27:29.525 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:27:29 smithi078 podman[105353]: 2024-05-21 15:27:29.204012487 +0000 UTC m=+0.189325910 container died a09b5065f38abe2728df05b6d950b802123e100b4f31334efcf947d8fbb6ff2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-21T15:27:29.842 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:27:29 smithi078 podman[105353]: 2024-05-21 15:27:29.604973999 +0000 UTC m=+0.590287421 container cleanup a09b5065f38abe2728df05b6d950b802123e100b4f31334efcf947d8fbb6ff2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2b507c724ef6c8295ee0a35d185cdab167fd61d, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-05-21T15:27:29.842 INFO:journalctl@ceph.mgr.x.smithi078.stdout:May 21 15:27:29 smithi078 bash[105353]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-x 2024-05-21T15:27:29.972 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-21T15:27:30.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:27:30.011 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-21T15:27:30.012 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-21T15:27:30.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.y 2024-05-21T15:27:30.341 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:27:30 smithi084 systemd[1]: Stopping Ceph mgr.y for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:27:30.620 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:27:30 smithi084 podman[72809]: 2024-05-21 15:27:30.339168271 +0000 UTC m=+0.148107873 container died 472ee36adf996ddabcb20e79f69dd35086a3d5d0128e5c834c94862d177830b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-21T15:27:30.896 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:27:30 smithi084 podman[72809]: 2024-05-21 15:27:30.725117661 +0000 UTC m=+0.534057260 container cleanup 472ee36adf996ddabcb20e79f69dd35086a3d5d0128e5c834c94862d177830b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:27:30.896 INFO:journalctl@ceph.mgr.y.smithi084.stdout:May 21 15:27:30 smithi084 bash[72809]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-y 2024-05-21T15:27:31.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-21T15:27:31.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:27:31.120 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-21T15:27:31.120 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-21T15:27:31.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.z 2024-05-21T15:27:31.433 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:27:31 smithi172 systemd[1]: Stopping Ceph mgr.z for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:27:31.747 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:27:31 smithi172 podman[72836]: 2024-05-21 15:27:31.450334146 +0000 UTC m=+0.144210396 container died 3e525d029db1c8e458786783b28cc7fb1e53304ecd736d620c5b575d4e9ce51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-21T15:27:32.031 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:27:31 smithi172 podman[72836]: 2024-05-21 15:27:31.827536499 +0000 UTC m=+0.521412751 container cleanup 3e525d029db1c8e458786783b28cc7fb1e53304ecd736d620c5b575d4e9ce51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-21T15:27:32.031 INFO:journalctl@ceph.mgr.z.smithi172.stdout:May 21 15:27:31 smithi172 bash[72836]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mgr-z 2024-05-21T15:27:32.171 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-21T15:27:32.206 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:27:32.206 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-21T15:27:32.207 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-21T15:27:32.207 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.0 2024-05-21T15:27:32.587 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:27:32 smithi078 systemd[1]: Stopping Ceph osd.0 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:27:32.847 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:27:32 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0[47099]: 2024-05-21T15:27:32.584+0000 7fdd7353a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:27:32.847 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:27:32 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0[47099]: 2024-05-21T15:27:32.584+0000 7fdd7353a700 -1 osd.0 103 *** Got signal Terminated *** 2024-05-21T15:27:32.847 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:27:32 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0[47099]: 2024-05-21T15:27:32.584+0000 7fdd7353a700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:28:02.848 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:02 smithi078 bash[105472]: time="2024-05-21T15:28:02Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0 in 30 seconds, resorting to SIGKILL" 2024-05-21T15:28:02.848 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:02 smithi078 podman[105472]: 2024-05-21 15:28:02.718156231 +0000 UTC m=+30.326779080 container died 5830d917b4cfbb8892ec8b9a5a5736610cd7b27911a1581ba6fcc2c194449cb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-21T15:28:03.345 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:03 smithi078 podman[105472]: 2024-05-21 15:28:03.125136664 +0000 UTC m=+30.733759512 container cleanup 5830d917b4cfbb8892ec8b9a5a5736610cd7b27911a1581ba6fcc2c194449cb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-21T15:28:03.345 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:03 smithi078 bash[105472]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0 2024-05-21T15:28:03.597 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:03 smithi078 podman[105504]: 2024-05-21 15:28:03.343513305 +0000 UTC m=+0.623516349 container remove 5830d917b4cfbb8892ec8b9a5a5736610cd7b27911a1581ba6fcc2c194449cb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:28:03.597 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:03 smithi078 systemd[1]: ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.0.service: Main process exited, code=exited, status=137/n/a 2024-05-21T15:28:03.936 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:03 smithi078 podman[105590]: 2024-05-21 15:28:03.564240804 +0000 UTC m=+0.017205213 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:28:03.937 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:03 smithi078 podman[105590]: 2024-05-21 15:28:03.686252512 +0000 UTC m=+0.139216901 container create 3b98a96acbc46093a325d9f5e7f0b82acd9f2b2554da084606ad3b480efddeb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:28:03.937 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:03 smithi078 podman[105590]: 2024-05-21 15:28:03.935093556 +0000 UTC m=+0.388057955 container init 3b98a96acbc46093a325d9f5e7f0b82acd9f2b2554da084606ad3b480efddeb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-21T15:28:04.243 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:03 smithi078 podman[105590]: 2024-05-21 15:28:03.938681654 +0000 UTC m=+0.391646051 container start 3b98a96acbc46093a325d9f5e7f0b82acd9f2b2554da084606ad3b480efddeb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-21T15:28:04.243 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:04 smithi078 podman[105590]: 2024-05-21 15:28:04.022130673 +0000 UTC m=+0.475095085 container attach 3b98a96acbc46093a325d9f5e7f0b82acd9f2b2554da084606ad3b480efddeb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:28:04.594 INFO:journalctl@ceph.osd.0.smithi078.stdout:May 21 15:28:04 smithi078 podman[105590]: 2024-05-21 15:28:04.241977 +0000 UTC m=+0.694941396 container died 3b98a96acbc46093a325d9f5e7f0b82acd9f2b2554da084606ad3b480efddeb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:28:04.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-21T15:28:04.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:28:04.958 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-21T15:28:04.959 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-21T15:28:04.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.1 2024-05-21T15:28:05.245 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:04 smithi084 systemd[1]: Stopping Ceph osd.1 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:28:05.645 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1[40938]: 2024-05-21T15:28:05.242+0000 7f309d002700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:28:05.645 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1[40938]: 2024-05-21T15:28:05.242+0000 7f309d002700 -1 osd.1 103 *** Got signal Terminated *** 2024-05-21T15:28:05.645 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1[40938]: 2024-05-21T15:28:05.242+0000 7f309d002700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:28:35.645 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:35 smithi084 bash[72950]: time="2024-05-21T15:28:35Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1 in 30 seconds, resorting to SIGKILL" 2024-05-21T15:28:35.646 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:35 smithi084 podman[72950]: 2024-05-21 15:28:35.361187438 +0000 UTC m=+30.219778941 container died 24cc1940280b1135af68db3616cb48d4fa558031c4fc2805d33ebf0c4ce8e47e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:28:35.924 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:35 smithi084 podman[72950]: 2024-05-21 15:28:35.721622752 +0000 UTC m=+30.580214246 container cleanup 24cc1940280b1135af68db3616cb48d4fa558031c4fc2805d33ebf0c4ce8e47e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:28:35.924 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:35 smithi084 bash[72950]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1 2024-05-21T15:28:36.379 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:35 smithi084 podman[72967]: 2024-05-21 15:28:35.9227387 +0000 UTC m=+0.559642317 container remove 24cc1940280b1135af68db3616cb48d4fa558031c4fc2805d33ebf0c4ce8e47e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:28:36.379 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:35 smithi084 systemd[1]: ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.1.service: Main process exited, code=exited, status=137/n/a 2024-05-21T15:28:36.645 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:36 smithi084 podman[73067]: 2024-05-21 15:28:36.280553808 +0000 UTC m=+0.017892639 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:28:36.645 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:36 smithi084 podman[73067]: 2024-05-21 15:28:36.390233578 +0000 UTC m=+0.127572380 container create b7f0adb5892f5b4e0eb85bc7ec65fae20b3b416a3917290b70fce78cc7e22522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True) 2024-05-21T15:28:36.984 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:36 smithi084 podman[73067]: 2024-05-21 15:28:36.673858461 +0000 UTC m=+0.411197285 container init b7f0adb5892f5b4e0eb85bc7ec65fae20b3b416a3917290b70fce78cc7e22522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:28:36.985 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:36 smithi084 podman[73067]: 2024-05-21 15:28:36.67728887 +0000 UTC m=+0.414627683 container start b7f0adb5892f5b4e0eb85bc7ec65fae20b3b416a3917290b70fce78cc7e22522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-21T15:28:36.985 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:36 smithi084 podman[73067]: 2024-05-21 15:28:36.751092634 +0000 UTC m=+0.488431464 container attach b7f0adb5892f5b4e0eb85bc7ec65fae20b3b416a3917290b70fce78cc7e22522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-21T15:28:37.289 INFO:journalctl@ceph.osd.1.smithi084.stdout:May 21 15:28:36 smithi084 podman[73067]: 2024-05-21 15:28:36.982581759 +0000 UTC m=+0.719920573 container died b7f0adb5892f5b4e0eb85bc7ec65fae20b3b416a3917290b70fce78cc7e22522 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:28:37.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-21T15:28:37.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:28:37.637 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-21T15:28:37.637 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-21T15:28:37.638 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.2 2024-05-21T15:28:37.933 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:28:37 smithi172 systemd[1]: Stopping Ceph osd.2 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:28:38.432 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:28:37 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2[41522]: 2024-05-21T15:28:37.940+0000 7fc24760f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:28:38.433 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:28:37 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2[41522]: 2024-05-21T15:28:37.940+0000 7fc24760f700 -1 osd.2 103 *** Got signal Terminated *** 2024-05-21T15:28:38.433 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:28:37 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2[41522]: 2024-05-21T15:28:37.940+0000 7fc24760f700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:29:03.433 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:03 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:03.011+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:04.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:04 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:04.030+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:05.060 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:04 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:04.733+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:05.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:05.058+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:05.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:05 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:05.259+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:06.106 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:05.715+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:06.350 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:05.916+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:06.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:06 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:06.104+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:06.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:06 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:06.295+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:07.130 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:06 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:06.684+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:07.223 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:07.004+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:07.326 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:06.949+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:07.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:07 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:07.128+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:07.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:07.338+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:07.597 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:07.220+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:08.086 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:07 smithi172 bash[72961]: time="2024-05-21T15:29:07Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2 in 30 seconds, resorting to SIGKILL" 2024-05-21T15:29:08.087 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:07 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:07.722+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:08.096 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:07 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:07.968+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:08.374 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:08 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:08.107+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:08.375 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:08 smithi172 podman[72961]: 2024-05-21 15:29:08.085438606 +0000 UTC m=+30.249832903 container died 12ea02df0e538726ddecb4b22ed127a2711ef11d3f1be1e844ffcc12d284c493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:29:08.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:07 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:07.938+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:08.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:08.327+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:08.597 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:08.182+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:08.675 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:08 smithi172 podman[72961]: 2024-05-21 15:29:08.555156434 +0000 UTC m=+30.719550730 container cleanup 12ea02df0e538726ddecb4b22ed127a2711ef11d3f1be1e844ffcc12d284c493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-21T15:29:08.675 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:08 smithi172 bash[72961]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2 2024-05-21T15:29:08.932 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:08 smithi172 podman[72991]: 2024-05-21 15:29:08.78145175 +0000 UTC m=+0.698029305 container remove 12ea02df0e538726ddecb4b22ed127a2711ef11d3f1be1e844ffcc12d284c493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-21T15:29:08.932 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:08 smithi172 systemd[1]: ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.2.service: Main process exited, code=exited, status=137/n/a 2024-05-21T15:29:08.933 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:08 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:08.673+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:09.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:08 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:08.890+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:09.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:09 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:09.169+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:09.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:08 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:08.993+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:09.369 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:09 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:09.088+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:09.370 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:09 smithi172 podman[73084]: 2024-05-21 15:29:09.009958051 +0000 UTC m=+0.017451201 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:29:09.370 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:09 smithi172 podman[73084]: 2024-05-21 15:29:09.115600603 +0000 UTC m=+0.123093787 container create 2db411eeedb410d560f4982cd82dbc53775fb3721f35fbfe7cbdebc5b1d3dfdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:09.672 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:09 smithi172 podman[73084]: 2024-05-21 15:29:09.368445233 +0000 UTC m=+0.375938380 container init 2db411eeedb410d560f4982cd82dbc53775fb3721f35fbfe7cbdebc5b1d3dfdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-21T15:29:09.672 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:09 smithi172 podman[73084]: 2024-05-21 15:29:09.372161058 +0000 UTC m=+0.379654214 container start 2db411eeedb410d560f4982cd82dbc53775fb3721f35fbfe7cbdebc5b1d3dfdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-21T15:29:09.672 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:09 smithi172 podman[73084]: 2024-05-21 15:29:09.459747401 +0000 UTC m=+0.467240542 container attach 2db411eeedb410d560f4982cd82dbc53775fb3721f35fbfe7cbdebc5b1d3dfdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-21T15:29:09.846 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:09 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:09.364+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:09.932 INFO:journalctl@ceph.osd.2.smithi172.stdout:May 21 15:29:09 smithi172 podman[73084]: 2024-05-21 15:29:09.670924872 +0000 UTC m=+0.678418024 container died 2db411eeedb410d560f4982cd82dbc53775fb3721f35fbfe7cbdebc5b1d3dfdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-21T15:29:09.932 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:09 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:09.712+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:10.231 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-21T15:29:10.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:29:10.270 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-21T15:29:10.270 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-21T15:29:10.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.3 2024-05-21T15:29:10.299 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:10.040+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:10.299 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:10.161+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:10.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:09 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:09.899+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:10.433 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:10 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:10.056+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:10.584 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:10 smithi078 systemd[1]: Stopping Ceph osd.3 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:29:10.584 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:10.346+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:10.846 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:10.581+0000 7fee0e902700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:29:10.846 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:10.581+0000 7fee0e902700 -1 osd.3 103 *** Got signal Terminated *** 2024-05-21T15:29:10.846 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:10 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:10.581+0000 7fee0e902700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:29:11.106 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:10 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:10.750+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:11.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:10 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:10.853+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:11.329 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:11 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:11.019+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:11.330 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:11 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:11.151+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:11.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:11 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:11.103+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:11.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:11 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:11.327+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:12.142 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:11 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:11.760+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:12.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:11 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:11.815+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:12.288 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:12 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:12.150+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:12.289 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:11 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:11.976+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:12.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:12 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:12.140+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:12.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:12 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:12.286+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:13.134 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:12 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:12.749+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:13.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:12 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:12.767+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:13.195 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:12 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:12.936+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:13.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:13 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:13.131+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:13.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:13 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:13.258+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:13.596 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:13 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:13.193+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:14.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:13 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:13.790+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:14.180 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:13 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:13.746+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:14.232 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:13 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:13.929+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:14.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:14 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:14.177+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:14.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:14 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:14.298+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:14.596 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:14 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:14.231+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:15.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:14 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:14.764+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:15.150 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:14 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:14.746+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:15.239 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:14 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:14.912+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:15.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:15 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:15.147+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:15.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:15 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3[53557]: 2024-05-21T15:29:15.269+0000 7fee0b6b9700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.875880+0000 front 2024-05-21T15:27:36.875950+0000 (oldest deadline 2024-05-21T15:29:04.779305+0000) 2024-05-21T15:29:15.597 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:15 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:15.237+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:16.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:15 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:15.745+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:16.181 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:15 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:15.775+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:16.219 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:15 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:15.894+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:16.220 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:16 smithi078 podman[105707]: 2024-05-21 15:29:16.049286627 +0000 UTC m=+5.583166794 container died 6ed3e33ad927006ba85f19fdc5c59815594e3a7f7677dcbc9b6d271de5d51ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:16.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:16 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:16.179+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:16.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:16 smithi078 podman[105707]: 2024-05-21 15:29:16.390872128 +0000 UTC m=+5.924752284 container cleanup 6ed3e33ad927006ba85f19fdc5c59815594e3a7f7677dcbc9b6d271de5d51ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-21T15:29:16.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:16 smithi078 bash[105707]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3 2024-05-21T15:29:16.597 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:16.217+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:17.056 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:16 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:16.932+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:17.056 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:16 smithi078 podman[105726]: 2024-05-21 15:29:16.68459393 +0000 UTC m=+0.636265043 container remove 6ed3e33ad927006ba85f19fdc5c59815594e3a7f7677dcbc9b6d271de5d51ef6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T15:29:17.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:16 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:16.745+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:17.154 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:16 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:16.766+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:17.314 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:17 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:17.182+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:17.314 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:17 smithi078 podman[105815]: 2024-05-21 15:29:16.958131902 +0000 UTC m=+0.017157255 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:29:17.314 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:17 smithi078 podman[105815]: 2024-05-21 15:29:17.085179984 +0000 UTC m=+0.144205323 container create d908903510580125f65527483517e01a4370f94da926b23fe129d62fb89001d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-21T15:29:17.314 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:17 smithi078 podman[105815]: 2024-05-21 15:29:17.295942819 +0000 UTC m=+0.354968160 container init d908903510580125f65527483517e01a4370f94da926b23fe129d62fb89001d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-21T15:29:17.314 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:17 smithi078 podman[105815]: 2024-05-21 15:29:17.299422151 +0000 UTC m=+0.358447491 container start d908903510580125f65527483517e01a4370f94da926b23fe129d62fb89001d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:29:17.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:17 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:17.152+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:17.596 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:17 smithi078 podman[105815]: 2024-05-21 15:29:17.429660695 +0000 UTC m=+0.488686044 container attach d908903510580125f65527483517e01a4370f94da926b23fe129d62fb89001d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5) 2024-05-21T15:29:17.878 INFO:journalctl@ceph.osd.3.smithi078.stdout:May 21 15:29:17 smithi078 podman[105815]: 2024-05-21 15:29:17.597066007 +0000 UTC m=+0.656091358 container died d908903510580125f65527483517e01a4370f94da926b23fe129d62fb89001d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-21T15:29:18.138 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:17 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:17.754+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:18.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:17 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:17.754+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:18.150 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:17 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:17.976+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:18.170 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-21T15:29:18.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:29:18.250 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-21T15:29:18.251 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-21T15:29:18.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.4 2024-05-21T15:29:18.425 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:18 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:18.149+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:18.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:18 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:18.136+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:18.579 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:18 smithi084 systemd[1]: Stopping Ceph osd.4 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:29:18.894 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:18 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:18.730+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:18.895 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:18 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4[47864]: 2024-05-21T15:29:18.576+0000 7f522837d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:29:18.895 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:18 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4[47864]: 2024-05-21T15:29:18.576+0000 7f522837d700 -1 osd.4 103 *** Got signal Terminated *** 2024-05-21T15:29:18.895 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:18 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4[47864]: 2024-05-21T15:29:18.576+0000 7f522837d700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:29:19.124 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:18 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:18.725+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:19.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:18 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:18.978+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:19.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:19 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:19.112+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:19.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:19 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:19.122+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:20.134 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:19 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:19.769+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:20.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:19 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:19.684+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:20.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:19 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:19.979+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:20.347 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:20.133+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:20.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:20.131+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:21.109 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:20 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:20.815+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:21.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:20 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:20.675+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:21.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:20 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:20.944+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:21.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:21 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:21.092+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:21.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:21 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:21.107+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:21.894 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:21 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:21.640+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:21.932 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:21 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:21.777+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:22.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:21 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:21.928+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:22.347 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:22 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:22.045+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:22.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:22 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:22.060+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:23.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:22 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:22.687+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:23.182 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:23 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:23.030+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:23.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:22 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:22.822+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:23.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:22 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:22.932+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:23.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:23 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:23.080+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:24.121 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:23 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:23.735+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:24.182 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:23 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:23.997+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:24.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:23 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:23.819+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:24.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:23 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:23.919+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:24.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:24 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:24.093+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:24.394 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:24 smithi084 podman[73186]: 2024-05-21 15:29:24.120198501 +0000 UTC m=+5.680043151 container died a714699a6b6e63a41b4aacfceb5edcd273f08dadfde172b35f30b46cb95d912c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-21T15:29:24.746 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:24 smithi084 podman[73186]: 2024-05-21 15:29:24.507211709 +0000 UTC m=+6.067056427 container cleanup a714699a6b6e63a41b4aacfceb5edcd273f08dadfde172b35f30b46cb95d912c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-21T15:29:24.746 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:24 smithi084 bash[73186]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4 2024-05-21T15:29:25.075 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:24 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:24.767+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:25.075 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:24 smithi084 podman[73204]: 2024-05-21 15:29:24.745112517 +0000 UTC m=+0.626056753 container remove a714699a6b6e63a41b4aacfceb5edcd273f08dadfde172b35f30b46cb95d912c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-21T15:29:25.182 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:25 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:25.044+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:25.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:24 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:24.864+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:25.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:24 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:24.914+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:25.347 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:25 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:25.130+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:25.367 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:25 smithi084 podman[73291]: 2024-05-21 15:29:24.976562981 +0000 UTC m=+0.016991288 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:29:25.367 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:25 smithi084 podman[73291]: 2024-05-21 15:29:25.129258618 +0000 UTC m=+0.169686908 container create 04d03637d4dbf3d29e98d2876149699e47767c9e016895893c7194f7c946e0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:25.367 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:25 smithi084 podman[73291]: 2024-05-21 15:29:25.365675171 +0000 UTC m=+0.406103472 container init 04d03637d4dbf3d29e98d2876149699e47767c9e016895893c7194f7c946e0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-21T15:29:25.644 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:25 smithi084 podman[73291]: 2024-05-21 15:29:25.369083178 +0000 UTC m=+0.409511467 container start 04d03637d4dbf3d29e98d2876149699e47767c9e016895893c7194f7c946e0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-21T15:29:25.645 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:25 smithi084 podman[73291]: 2024-05-21 15:29:25.448413578 +0000 UTC m=+0.488841869 container attach 04d03637d4dbf3d29e98d2876149699e47767c9e016895893c7194f7c946e0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T15:29:25.986 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:25 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:25.802+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:25.986 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:25 smithi084 podman[73291]: 2024-05-21 15:29:25.656878557 +0000 UTC m=+0.697306848 container died 04d03637d4dbf3d29e98d2876149699e47767c9e016895893c7194f7c946e0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-21T15:29:26.182 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:26 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:26.055+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:26.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:25 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:25.821+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:26.242 INFO:journalctl@ceph.osd.4.smithi084.stdout:May 21 15:29:26 smithi084 podman[73318]: 2024-05-21 15:29:26.227753406 +0000 UTC m=+0.562350715 container remove 04d03637d4dbf3d29e98d2876149699e47767c9e016895893c7194f7c946e0e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:29:26.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-21T15:29:26.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:29:26.287 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-21T15:29:26.288 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-21T15:29:26.288 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.5 2024-05-21T15:29:26.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:25 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:25.914+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:26.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:26 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:26.121+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:26.591 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:26 smithi172 systemd[1]: Stopping Ceph osd.5 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:29:26.932 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:26 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:26.826+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:26.932 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:26 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:26.589+0000 7fa9e7f6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:29:26.933 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:26 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:26.589+0000 7fa9e7f6d700 -1 osd.5 103 *** Got signal Terminated *** 2024-05-21T15:29:26.933 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:26 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:26.589+0000 7fa9e7f6d700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:29:27.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:26 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:26.828+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:27.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:26 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:26.883+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:27.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:27 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:27.106+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:27.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:27.067+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:28.068 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:27 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:27.789+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:28.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:27 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:27.806+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:28.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:27 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:27.907+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:28.347 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:28 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:28.124+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:28.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:28 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:28.066+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:29.031 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:28 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:28.749+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:29.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:28 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:28.775+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:29.145 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:28 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:28.916+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:29.428 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:29 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:29.143+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:29.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:29 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:29.029+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:30.079 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:29 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:29.719+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:30.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:29 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:29.820+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:30.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:29 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:29.942+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:30.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:30 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:30.146+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:30.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:30 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:30.077+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:31.075 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:30 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:30.713+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:31.147 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:30 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:30.870+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:31.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:30 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:30.913+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:31.347 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:31 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:31.097+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:31.432 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:31 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5[48062]: 2024-05-21T15:29:31.072+0000 7fa9e0d1c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:36.456440+0000 front 2024-05-21T15:27:36.456474+0000 (oldest deadline 2024-05-21T15:29:02.659872+0000) 2024-05-21T15:29:31.932 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:31 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:31.675+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:32.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:31 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:31.858+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:32.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:32 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:32.049+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:32.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:31 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:31.879+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:32.532 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:32 smithi172 podman[73203]: 2024-05-21 15:29:32.190075674 +0000 UTC m=+5.707520897 container died e4fd9406f40784ab8d9141e638ab8c0f03eba7bf1b8ed0e843e5f0c957b7f50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T15:29:32.791 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:32 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:32.675+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:32.792 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:32 smithi172 podman[73203]: 2024-05-21 15:29:32.531307484 +0000 UTC m=+6.048752698 container cleanup e4fd9406f40784ab8d9141e638ab8c0f03eba7bf1b8ed0e843e5f0c957b7f50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:32.792 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:32 smithi172 bash[73203]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5 2024-05-21T15:29:33.107 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:32 smithi172 podman[73221]: 2024-05-21 15:29:32.790709867 +0000 UTC m=+0.604109549 container remove e4fd9406f40784ab8d9141e638ab8c0f03eba7bf1b8ed0e843e5f0c957b7f50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-21T15:29:33.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:32 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:32.864+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:33.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:33.065+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:33.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:32 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:32.864+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:33.394 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:33 smithi172 podman[73312]: 2024-05-21 15:29:33.009544706 +0000 UTC m=+0.017239453 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:29:33.395 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:33 smithi172 podman[73312]: 2024-05-21 15:29:33.150018346 +0000 UTC m=+0.157713143 container create f9e2038225a1bd2e037a6d561daddbd1bfec6e661e795710588d7e742499df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-05-21T15:29:33.677 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:33 smithi172 podman[73312]: 2024-05-21 15:29:33.393746697 +0000 UTC m=+0.401441445 container init f9e2038225a1bd2e037a6d561daddbd1bfec6e661e795710588d7e742499df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514) 2024-05-21T15:29:33.678 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:33 smithi172 podman[73312]: 2024-05-21 15:29:33.397039921 +0000 UTC m=+0.404734670 container start f9e2038225a1bd2e037a6d561daddbd1bfec6e661e795710588d7e742499df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:33.678 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:33 smithi172 podman[73312]: 2024-05-21 15:29:33.485515669 +0000 UTC m=+0.493210418 container attach f9e2038225a1bd2e037a6d561daddbd1bfec6e661e795710588d7e742499df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-21T15:29:33.932 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:33 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:33.675+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:33.932 INFO:journalctl@ceph.osd.5.smithi172.stdout:May 21 15:29:33 smithi172 podman[73312]: 2024-05-21 15:29:33.691643565 +0000 UTC m=+0.699338322 container died f9e2038225a1bd2e037a6d561daddbd1bfec6e661e795710588d7e742499df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T15:29:34.096 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:33.839+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:34.096 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:34 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:34.081+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:34.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:33 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:33.828+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:34.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-21T15:29:34.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:29:34.300 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-21T15:29:34.300 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-21T15:29:34.300 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.6 2024-05-21T15:29:34.596 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:34 smithi078 systemd[1]: Stopping Ceph osd.6 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:29:35.057 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:34 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:34.611+0000 7f92d5d64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:29:35.057 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:34 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:34.611+0000 7f92d5d64700 -1 osd.6 103 *** Got signal Terminated *** 2024-05-21T15:29:35.057 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:34 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:34.611+0000 7f92d5d64700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:29:35.057 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:34 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:34.857+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:35.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:34 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:34.856+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:35.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:34 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:34.683+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:35.319 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:35 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:35.055+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:36.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:35 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:35.885+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:36.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:35 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:35.713+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:36.249 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:35 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:35.895+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:36.249 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:36 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:36.093+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:37.083 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:36 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:36.707+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:37.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:36 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:36.836+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:37.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:36 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:36.900+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:37.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:37 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:37.066+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:37.347 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:37 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:37.066+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:37.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:37 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:37.354+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:37.682 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:37 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:37.457+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:38.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:37 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:37.805+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:38.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:37 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:37.805+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:38.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:37 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:37.705+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:38.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:37 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:37.705+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:38.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:37 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:37.873+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:38.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:37 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:37.873+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:06.244259+0000 front 2024-05-21T15:28:06.244124+0000 (oldest deadline 2024-05-21T15:29:37.748391+0000) 2024-05-21T15:29:38.347 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:38 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:38.104+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:38.347 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:38 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:38.104+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:38.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:38 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:38.322+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:38.682 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:38 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:38.494+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:39.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:38 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:38.825+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:39.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:38 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:38.825+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:39.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:38 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:38.731+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:39.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:38 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:38.731+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:39.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:38 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:38.887+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:35.339630+0000 front 2024-05-21T15:27:35.339666+0000 (oldest deadline 2024-05-21T15:29:06.243843+0000) 2024-05-21T15:29:39.346 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:38 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6[60277]: 2024-05-21T15:29:38.887+0000 7f92d2b1b700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:06.244259+0000 front 2024-05-21T15:28:06.244124+0000 (oldest deadline 2024-05-21T15:29:37.748391+0000) 2024-05-21T15:29:39.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:39 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:39.100+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:39.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:39 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:39.100+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:39.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:39 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:39.328+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:39.724 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:39 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:39.462+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:40.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:39 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:39.804+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:40.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:39 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:39.804+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:40.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:39 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:39.722+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:40.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:39 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:39.722+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:40.513 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:40 smithi078 podman[105933]: 2024-05-21 15:29:40.213787488 +0000 UTC m=+5.709338633 container died dccfa821094f0c9a14883da4a0ed2d4646e30a47d84d9ebb090b28102f264c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-21T15:29:40.513 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:40 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:40.097+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:40.513 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:40 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:40.097+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:40.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:40 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:40.370+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:40.764 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:40 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:40.464+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:40.838 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:40 smithi078 podman[105933]: 2024-05-21 15:29:40.569245845 +0000 UTC m=+6.064796982 container cleanup dccfa821094f0c9a14883da4a0ed2d4646e30a47d84d9ebb090b28102f264c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:40.838 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:40 smithi078 bash[105933]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6 2024-05-21T15:29:41.096 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:40 smithi078 podman[105962]: 2024-05-21 15:29:40.837423831 +0000 UTC m=+0.618558452 container remove dccfa821094f0c9a14883da4a0ed2d4646e30a47d84d9ebb090b28102f264c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-21T15:29:41.096 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:41.071+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:41.096 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:41 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:41.071+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:41.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:40 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:40.827+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:41.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:40 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:40.827+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:41.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:40 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:40.762+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:41.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:40 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:40.762+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:41.483 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:41 smithi078 podman[106047]: 2024-05-21 15:29:41.179891518 +0000 UTC m=+0.103623436 container create f944d7e29c93f4e35072845df8a73850314b75bcd2fad83bffe3d432372d70e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-21T15:29:41.483 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:41 smithi078 podman[106047]: 2024-05-21 15:29:41.09362497 +0000 UTC m=+0.017356908 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:29:41.483 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:41 smithi078 podman[106047]: 2024-05-21 15:29:41.399247243 +0000 UTC m=+0.322979163 container init f944d7e29c93f4e35072845df8a73850314b75bcd2fad83bffe3d432372d70e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-21T15:29:41.484 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:41 smithi078 podman[106047]: 2024-05-21 15:29:41.402680888 +0000 UTC m=+0.326412810 container start f944d7e29c93f4e35072845df8a73850314b75bcd2fad83bffe3d432372d70e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-21T15:29:41.484 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:41 smithi078 podman[106047]: 2024-05-21 15:29:41.482604028 +0000 UTC m=+0.406335969 container attach f944d7e29c93f4e35072845df8a73850314b75bcd2fad83bffe3d432372d70e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:41.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:41 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:41.321+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:41.797 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:41 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:41.497+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:41.846 INFO:journalctl@ceph.osd.6.smithi078.stdout:May 21 15:29:41 smithi078 podman[106047]: 2024-05-21 15:29:41.690848097 +0000 UTC m=+0.614580025 container died f944d7e29c93f4e35072845df8a73850314b75bcd2fad83bffe3d432372d70e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS) 2024-05-21T15:29:42.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:41 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:41.791+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:42.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:41 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:41.791+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:42.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:41 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:41.796+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:42.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:41 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:41.796+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:42.261 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:42 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:42.077+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:42.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:42 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:42.077+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:42.294 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-21T15:29:42.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:29:42.335 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-21T15:29:42.335 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-21T15:29:42.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.7 2024-05-21T15:29:42.630 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:42 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:42.350+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:42.631 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:42 smithi084 systemd[1]: Stopping Ceph osd.7 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:29:42.631 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:42 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:42.629+0000 7f8df6aa7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:29:42.631 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:42 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:42.629+0000 7f8df6aa7700 -1 osd.7 103 *** Got signal Terminated *** 2024-05-21T15:29:42.631 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:42 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:42.629+0000 7f8df6aa7700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:29:42.791 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:42 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:42.464+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:42.894 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:42 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:42.827+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:42.894 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:42 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:42.827+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:43.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:42 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:42.790+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:43.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:42 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:42.790+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:43.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:43 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:43.030+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:43.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:43 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:43.030+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:43.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:43 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:43.327+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:43.835 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:43 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:43.461+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:44.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:43 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:43.783+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:44.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:43 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:43.783+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:44.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:43 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:43.834+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:44.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:43 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:43.834+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:44.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:44 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:44.004+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:44.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:44 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:44.004+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:44.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:44 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:44.329+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:44.793 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:44 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:44.480+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:45.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:44 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:44.800+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:45.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:44 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:44.800+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:45.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:44 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:44.791+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:45.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:44 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:44.791+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:45.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:45 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:45.005+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:45.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:45 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:45.005+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:45.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:45 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:45.353+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:45.830 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:45 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:45.521+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:46.096 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:45 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:45.980+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:46.096 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:45 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:45.980+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:46.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:45 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:45.792+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:46.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:45 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:45.792+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:46.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:45 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:45.829+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:46.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:45 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:45.829+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:46.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:46 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:46.346+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:46.867 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:46 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:46.492+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:47.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:46 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:46.792+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:47.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:46 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:46.792+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:47.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:46 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:46.865+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:47.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:46 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:46.865+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:47.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:47 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:47.011+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:47.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:47 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:47.011+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:47.785 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:47 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:47.500+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:47.799 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:47 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7[54573]: 2024-05-21T15:29:47.395+0000 7f8def856700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.931625+0000 front 2024-05-21T15:28:05.931663+0000 (oldest deadline 2024-05-21T15:29:36.835648+0000) 2024-05-21T15:29:48.094 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:47 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:47.797+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:48.095 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:47 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:47.797+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:48.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:47 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:47.871+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:48.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:47 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:47.871+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:48.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:48 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:48.012+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:48.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:48 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:48.012+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:48.382 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:48 smithi084 podman[73410]: 2024-05-21 15:29:48.093229739 +0000 UTC m=+5.567393726 container died bed83afc2773301a4897b3608ebc3a9676c652c8590f475ad04607cdc47f3934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-21T15:29:48.644 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:48 smithi084 podman[73410]: 2024-05-21 15:29:48.470957954 +0000 UTC m=+5.945121934 container cleanup bed83afc2773301a4897b3608ebc3a9676c652c8590f475ad04607cdc47f3934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:29:48.645 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:48 smithi084 bash[73410]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7 2024-05-21T15:29:48.682 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:48 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:48.469+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:49.077 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:48 smithi084 podman[73427]: 2024-05-21 15:29:48.739656291 +0000 UTC m=+0.640191568 container remove bed83afc2773301a4897b3608ebc3a9676c652c8590f475ad04607cdc47f3934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:29:49.077 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:48 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:48.847+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:49.078 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:48 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:48.847+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:49.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:48 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:48.865+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:49.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:48 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:48.865+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:49.335 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:49 smithi084 podman[73519]: 2024-05-21 15:29:48.979485839 +0000 UTC m=+0.017680317 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:29:49.336 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:49 smithi084 podman[73519]: 2024-05-21 15:29:49.115478565 +0000 UTC m=+0.153673026 container create 07d1b11dc8c1b753912b53be9e16f05b426647ac61674ef47582b48310af8dc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:49.345 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:48 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:48.994+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:49.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:48 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:48.994+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:49.625 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:49 smithi084 podman[73519]: 2024-05-21 15:29:49.335133227 +0000 UTC m=+0.373327695 container init 07d1b11dc8c1b753912b53be9e16f05b426647ac61674ef47582b48310af8dc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-05-21T15:29:49.625 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:49 smithi084 podman[73519]: 2024-05-21 15:29:49.338159581 +0000 UTC m=+0.376354056 container start 07d1b11dc8c1b753912b53be9e16f05b426647ac61674ef47582b48310af8dc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0) 2024-05-21T15:29:49.626 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:49 smithi084 podman[73519]: 2024-05-21 15:29:49.434997264 +0000 UTC m=+0.473191745 container attach 07d1b11dc8c1b753912b53be9e16f05b426647ac61674ef47582b48310af8dc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:29:49.860 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:49 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:49.478+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:49.894 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:49 smithi084 podman[73519]: 2024-05-21 15:29:49.624750101 +0000 UTC m=+0.662944579 container died 07d1b11dc8c1b753912b53be9e16f05b426647ac61674ef47582b48310af8dc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-21T15:29:49.894 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:49 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:49.847+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:49.895 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:49 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:49.847+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:50.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:49 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:49.859+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:50.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:49 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:49.859+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:50.193 INFO:journalctl@ceph.osd.7.smithi084.stdout:May 21 15:29:50 smithi084 podman[73544]: 2024-05-21 15:29:50.171535003 +0000 UTC m=+0.541329824 container remove 07d1b11dc8c1b753912b53be9e16f05b426647ac61674ef47582b48310af8dc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-21T15:29:50.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-21T15:29:50.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:29:50.236 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-21T15:29:50.236 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-21T15:29:50.236 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.8 2024-05-21T15:29:50.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:50 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:50.031+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:50.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:50 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:50.031+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:50.550 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:50 smithi172 systemd[1]: Stopping Ceph osd.8 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:29:50.551 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:50 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:50.476+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:50.899 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:50 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:50.548+0000 7f5d030f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:29:50.899 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:50 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:50.548+0000 7f5d030f4700 -1 osd.8 103 *** Got signal Terminated *** 2024-05-21T15:29:50.899 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:50 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:50.548+0000 7f5d030f4700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:29:51.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:50 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:50.815+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:51.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:50 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:50.815+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:51.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:50 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:50.898+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:51.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:50 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:50.898+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:51.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:51 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:51.014+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:51.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:51 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:51.014+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:51.933 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:51 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:51.511+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:52.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:51 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:51.828+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:52.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:51 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:51.828+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:52.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:51 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:51.986+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:52.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:51 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:51.986+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:52.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:51 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:51.930+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:52.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:51 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:51.930+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:52.932 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:52 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:52.514+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:53.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:52 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:52.811+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:53.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:52 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:52.811+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:53.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:52.987+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:53.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:52 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:52.987+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:53.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:52 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:52.962+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:53.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:52 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:52.962+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:53.932 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:53 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:53.522+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:54.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:53 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:53.856+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:54.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:53 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:53.856+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:54.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:53 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:53.961+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:54.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:53 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:53.961+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:54.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:53 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:53.972+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:54.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:53 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:53.972+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:54.932 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:54 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:54.566+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:55.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:54 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:54.859+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:55.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:54 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:54.859+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:55.345 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:54 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:54.942+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:55.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:54 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:54.942+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:55.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:54 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:54.959+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:55.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:54 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:54.959+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:55.932 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:55 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8[55064]: 2024-05-21T15:29:55.530+0000 7f5cffeab700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:04.660777+0000 front 2024-05-21T15:28:04.660747+0000 (oldest deadline 2024-05-21T15:29:36.565830+0000) 2024-05-21T15:29:56.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:55 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:55.813+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:56.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:55 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:55.813+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:56.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:55 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:55.962+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:56.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:55 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:55.962+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:56.367 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:56 smithi172 podman[73431]: 2024-05-21 15:29:56.11504974 +0000 UTC m=+5.680463934 container died d7e486abab1771fcd82a97ac009722f410e6ea5368c518c2f6b5e9b8ab3293a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-21T15:29:56.367 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:55 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:55.951+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:56.367 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:55 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:55.951+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:56.658 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:56 smithi172 podman[73431]: 2024-05-21 15:29:56.456365694 +0000 UTC m=+6.021779883 container cleanup d7e486abab1771fcd82a97ac009722f410e6ea5368c518c2f6b5e9b8ab3293a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:29:56.658 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:56 smithi172 bash[73431]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8 2024-05-21T15:29:56.932 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:56 smithi172 podman[73446]: 2024-05-21 15:29:56.657511009 +0000 UTC m=+0.536985892 container remove d7e486abab1771fcd82a97ac009722f410e6ea5368c518c2f6b5e9b8ab3293a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-21T15:29:57.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:56 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:56.849+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:57.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:56 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:56.849+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:57.221 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:57 smithi172 podman[73538]: 2024-05-21 15:29:56.984047683 +0000 UTC m=+0.017965370 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:29:57.221 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:57 smithi172 podman[73538]: 2024-05-21 15:29:57.083225109 +0000 UTC m=+0.117142780 container create 386c6aaa0bde91db2251b87af2d025c3f2b7c60bfa931c031dc1df199bb81f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:29:57.221 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:56 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:56.951+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:57.221 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:56 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:56.951+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:57.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:57 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:57.001+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:57.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:57 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:57.001+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:57.613 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:57 smithi172 podman[73538]: 2024-05-21 15:29:57.319779501 +0000 UTC m=+0.353697192 container init 386c6aaa0bde91db2251b87af2d025c3f2b7c60bfa931c031dc1df199bb81f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_BRANCH=HEAD) 2024-05-21T15:29:57.613 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:57 smithi172 podman[73538]: 2024-05-21 15:29:57.32315442 +0000 UTC m=+0.357072093 container start 386c6aaa0bde91db2251b87af2d025c3f2b7c60bfa931c031dc1df199bb81f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-21T15:29:57.613 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:57 smithi172 podman[73538]: 2024-05-21 15:29:57.394155493 +0000 UTC m=+0.428073165 container attach 386c6aaa0bde91db2251b87af2d025c3f2b7c60bfa931c031dc1df199bb81f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD) 2024-05-21T15:29:57.897 INFO:journalctl@ceph.osd.8.smithi172.stdout:May 21 15:29:57 smithi172 podman[73538]: 2024-05-21 15:29:57.612780162 +0000 UTC m=+0.646697835 container died 386c6aaa0bde91db2251b87af2d025c3f2b7c60bfa931c031dc1df199bb81f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-21T15:29:58.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:57 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:57.889+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:58.144 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:57 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:57.889+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:29:58.164 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:57 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:57.947+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:58.164 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:57 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:57.947+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:58.182 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-21T15:29:58.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:29:58.220 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-21T15:29:58.220 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-21T15:29:58.220 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.9 2024-05-21T15:29:58.245 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:57 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:57.965+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:58.245 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:57 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:57.965+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:58.506 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:58 smithi078 systemd[1]: Stopping Ceph osd.9 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:29:58.845 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:58 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:58.505+0000 7ffa346a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:29:58.846 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:58 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:58.505+0000 7ffa346a7700 -1 osd.9 103 *** Got signal Terminated *** 2024-05-21T15:29:58.846 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:58 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:58.505+0000 7ffa346a7700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:29:59.345 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:58 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:58.984+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:29:59.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:58 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:58.984+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:29:59.355 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:58 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:58.957+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:29:59.355 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:58 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:58.957+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:29:59.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:58 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:58.933+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:29:59.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:58 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:58.933+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:00.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:59 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:59.925+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:00.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:29:59 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:29:59.925+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:00.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:59 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:59.970+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:30:00.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:29:59 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:29:59.970+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:30:00.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:59 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:59.966+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:00.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:29:59 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:29:59.966+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:01.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:00 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:30:00.977+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:30:01.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:00 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:30:00.977+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:30:01.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:01 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:01.013+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:01.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:01 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:01.013+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:01.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:00 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:00.969+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:01.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:00 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:00.969+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:02.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:01 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:30:01.980+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:30:02.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:01 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:30:01.980+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:30:02.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:02 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:02.038+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:02.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:02 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:02.038+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:02.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:01 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:01.972+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:02.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:01 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:01.972+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:03.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:03 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:30:03.022+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:32.309115+0000 front 2024-05-21T15:27:32.309325+0000 (oldest deadline 2024-05-21T15:29:07.114286+0000) 2024-05-21T15:30:03.346 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:03 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9[67001]: 2024-05-21T15:30:03.022+0000 7ffa2d456700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.114685+0000 front 2024-05-21T15:28:07.114632+0000 (oldest deadline 2024-05-21T15:29:36.818670+0000) 2024-05-21T15:30:03.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:03 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:03.027+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:03.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:03 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:03.027+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:03.431 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:03 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:03.014+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:03.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:03 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:03.014+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:04.339 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:04 smithi078 podman[106153]: 2024-05-21 15:30:04.019952662 +0000 UTC m=+5.613076247 container died d3f4652349fa5ab796b31b0c80c4df9a0f5647510a05c0d39e19714ee4eb6283 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-21T15:30:04.339 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:04 smithi078 podman[106153]: 2024-05-21 15:30:04.338966501 +0000 UTC m=+5.932090080 container cleanup d3f4652349fa5ab796b31b0c80c4df9a0f5647510a05c0d39e19714ee4eb6283 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-21T15:30:04.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:04 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:04.055+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:04.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:04 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:04.055+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:04.431 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:03 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:03.966+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:04.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:03 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:03.966+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:04.596 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:04 smithi078 bash[106153]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9 2024-05-21T15:30:04.596 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:04 smithi078 podman[106170]: 2024-05-21 15:30:04.533454381 +0000 UTC m=+0.509621751 container remove d3f4652349fa5ab796b31b0c80c4df9a0f5647510a05c0d39e19714ee4eb6283 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-21T15:30:05.237 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:04 smithi078 podman[106257]: 2024-05-21 15:30:04.858217804 +0000 UTC m=+0.016844140 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:30:05.237 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:04 smithi078 podman[106257]: 2024-05-21 15:30:04.967784292 +0000 UTC m=+0.126410632 container create 7bb9de869f142415060526d437734007335041203fb112ed0b574ed0f80191c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-05-21T15:30:05.238 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:05 smithi078 podman[106257]: 2024-05-21 15:30:05.170216578 +0000 UTC m=+0.328842901 container init 7bb9de869f142415060526d437734007335041203fb112ed0b574ed0f80191c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-21T15:30:05.238 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:05 smithi078 podman[106257]: 2024-05-21 15:30:05.173540154 +0000 UTC m=+0.332166473 container start 7bb9de869f142415060526d437734007335041203fb112ed0b574ed0f80191c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-21T15:30:05.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:05.066+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:05.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:05 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:05.066+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:05.431 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:04 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:04.932+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:05.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:04 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:04.932+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:05.596 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:05 smithi078 podman[106257]: 2024-05-21 15:30:05.236615065 +0000 UTC m=+0.395241386 container attach 7bb9de869f142415060526d437734007335041203fb112ed0b574ed0f80191c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:30:05.596 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:05 smithi078 podman[106257]: 2024-05-21 15:30:05.461393321 +0000 UTC m=+0.620019640 container died 7bb9de869f142415060526d437734007335041203fb112ed0b574ed0f80191c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:30:06.018 INFO:journalctl@ceph.osd.9.smithi078.stdout:May 21 15:30:05 smithi078 podman[106284]: 2024-05-21 15:30:05.999418764 +0000 UTC m=+0.531789189 container remove 7bb9de869f142415060526d437734007335041203fb112ed0b574ed0f80191c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:30:06.023 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-21T15:30:06.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:06.103 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-21T15:30:06.103 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-21T15:30:06.103 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.10 2024-05-21T15:30:06.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:05.915+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:06.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:05.915+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:06.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:06.108+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:06.395 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:06.108+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:06.395 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:06 smithi084 systemd[1]: Stopping Ceph osd.10 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:06.719 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:06.397+0000 7f76d66cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:06.719 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:06.397+0000 7f76d66cb700 -1 osd.10 103 *** Got signal Terminated *** 2024-05-21T15:30:06.719 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:06 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:06.397+0000 7f76d66cb700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:30:07.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:06 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:06.920+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:07.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:06 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:06.920+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:07.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:07 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:07.140+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:07.394 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:07 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:07.140+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:08.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:07 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:07.967+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:08.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:07 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:07.967+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:08.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:08 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:08.172+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:08.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:08 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:08.172+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:09.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:08 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:08.937+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:09.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:08 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:08.937+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:09.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:09 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:09.184+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:09.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:09 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:09.184+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:10.181 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:09 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:09.904+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:10.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:09 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:09.904+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:10.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:10 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:10.196+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:10.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:10 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:10.196+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:11.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:10 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:10.926+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:11.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:10 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:10.926+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:11.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:10 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:10.926+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:11.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:11 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:11.150+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:33.892464+0000 front 2024-05-21T15:27:33.892486+0000 (oldest deadline 2024-05-21T15:29:05.897103+0000) 2024-05-21T15:30:11.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:11 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:11.150+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:05.897366+0000 front 2024-05-21T15:28:05.897388+0000 (oldest deadline 2024-05-21T15:29:37.202367+0000) 2024-05-21T15:30:11.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:11 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10[61617]: 2024-05-21T15:30:11.150+0000 7f76d3482700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:41.803484+0000 front 2024-05-21T15:28:41.803440+0000 (oldest deadline 2024-05-21T15:30:10.410353+0000) 2024-05-21T15:30:12.306 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:12 smithi084 podman[73624]: 2024-05-21 15:30:12.01154172 +0000 UTC m=+5.722992011 container died 90ae1a8952b518595316b2cce1a79caf644ff148c1993c4229be9bdcd5f166b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:30:12.431 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:11 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:11.941+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:12.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:11 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:11.941+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:12.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:11 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:11.941+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:12.632 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:12 smithi084 podman[73624]: 2024-05-21 15:30:12.413633954 +0000 UTC m=+6.125084249 container cleanup 90ae1a8952b518595316b2cce1a79caf644ff148c1993c4229be9bdcd5f166b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:30:12.632 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:12 smithi084 bash[73624]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10 2024-05-21T15:30:12.894 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:12 smithi084 podman[73652]: 2024-05-21 15:30:12.631545557 +0000 UTC m=+0.611674659 container remove 90ae1a8952b518595316b2cce1a79caf644ff148c1993c4229be9bdcd5f166b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-21T15:30:13.218 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:12 smithi084 podman[73743]: 2024-05-21 15:30:12.940618994 +0000 UTC m=+0.110202365 container create 32762f33bf3ecf878c2215e42de286503f06bace6d05c6468a2159ef36a00f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-21T15:30:13.219 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:12 smithi084 podman[73743]: 2024-05-21 15:30:12.847272806 +0000 UTC m=+0.016856159 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:30:13.219 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:13 smithi084 podman[73743]: 2024-05-21 15:30:13.134454771 +0000 UTC m=+0.304038100 container init 32762f33bf3ecf878c2215e42de286503f06bace6d05c6468a2159ef36a00f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-21T15:30:13.219 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:13 smithi084 podman[73743]: 2024-05-21 15:30:13.137565244 +0000 UTC m=+0.307148579 container start 32762f33bf3ecf878c2215e42de286503f06bace6d05c6468a2159ef36a00f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-05-21T15:30:13.219 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:13 smithi084 podman[73743]: 2024-05-21 15:30:13.217841457 +0000 UTC m=+0.387424812 container attach 32762f33bf3ecf878c2215e42de286503f06bace6d05c6468a2159ef36a00f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-21T15:30:13.431 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:12 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:12.962+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:13.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:12 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:12.962+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:13.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:12 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:12.962+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:13.644 INFO:journalctl@ceph.osd.10.smithi084.stdout:May 21 15:30:13 smithi084 podman[73743]: 2024-05-21 15:30:13.42791665 +0000 UTC m=+0.597499982 container died 32762f33bf3ecf878c2215e42de286503f06bace6d05c6468a2159ef36a00f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:30:14.023 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-21T15:30:14.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:14.061 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-21T15:30:14.062 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-21T15:30:14.062 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.11 2024-05-21T15:30:14.355 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:13 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:13.930+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:14.356 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:13 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:13.930+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:14.356 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:13 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:13.930+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:14.356 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:14 smithi172 systemd[1]: Stopping Ceph osd.11 for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:14.681 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:14 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:14.353+0000 7fb195bb9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:14.682 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:14 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:14.353+0000 7fb195bb9700 -1 osd.11 103 *** Got signal Terminated *** 2024-05-21T15:30:14.682 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:14 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:14.353+0000 7fb195bb9700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T15:30:15.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:14 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:14.951+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:15.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:14 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:14.951+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:15.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:14 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:14.951+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:16.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:15 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:15.984+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:16.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:15 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:15.984+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:16.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:15 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:15.984+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:17.431 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:17 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:17.011+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:17.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:17 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:17.011+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:17.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:17 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:17.011+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:18.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:18 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:18.009+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:18.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:18 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:18.009+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:18.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:18 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:18.009+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:19.431 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:19 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:19.015+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-05-21T15:27:37.285115+0000 front 2024-05-21T15:27:37.285085+0000 (oldest deadline 2024-05-21T15:29:04.188658+0000) 2024-05-21T15:30:19.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:19 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:19.015+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.84:6804 osd.1 since back 2024-05-21T15:28:07.490876+0000 front 2024-05-21T15:28:07.490742+0000 (oldest deadline 2024-05-21T15:29:37.394230+0000) 2024-05-21T15:30:19.432 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:19 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11[61829]: 2024-05-21T15:30:19.015+0000 7fb192970700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-05-21T15:28:42.095161+0000 front 2024-05-21T15:28:42.095245+0000 (oldest deadline 2024-05-21T15:30:10.798217+0000) 2024-05-21T15:30:20.218 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:19 smithi172 podman[73656]: 2024-05-21 15:30:19.951942795 +0000 UTC m=+5.698612001 container died cb9c465bff356ac7c0c55aab39cfa468e46e3603bcce1e243b9879478362dda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:30:20.558 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:20 smithi172 podman[73656]: 2024-05-21 15:30:20.307278808 +0000 UTC m=+6.053948013 container cleanup cb9c465bff356ac7c0c55aab39cfa468e46e3603bcce1e243b9879478362dda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:30:20.559 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:20 smithi172 bash[73656]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11 2024-05-21T15:30:20.918 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:20 smithi172 podman[73673]: 2024-05-21 15:30:20.558188184 +0000 UTC m=+0.598918273 container remove cb9c465bff356ac7c0c55aab39cfa468e46e3603bcce1e243b9879478362dda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-21T15:30:21.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:20 smithi172 podman[73764]: 2024-05-21 15:30:20.820421222 +0000 UTC m=+0.017035889 image pull cfcc61b2d476f611479310eb042aa05f11dd5f60dbb65e3b0d7e0e11d024cdb3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708 2024-05-21T15:30:21.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:20 smithi172 podman[73764]: 2024-05-21 15:30:20.925580265 +0000 UTC m=+0.122194908 container create 9841a534770d09ece57e87f04f64fb324612d5e090261b0c2b56912f2755fdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T15:30:21.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:21 smithi172 podman[73764]: 2024-05-21 15:30:21.161198595 +0000 UTC m=+0.357813238 container init 9841a534770d09ece57e87f04f64fb324612d5e090261b0c2b56912f2755fdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T15:30:21.182 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:21 smithi172 podman[73764]: 2024-05-21 15:30:21.164177401 +0000 UTC m=+0.360792044 container start 9841a534770d09ece57e87f04f64fb324612d5e090261b0c2b56912f2755fdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-21T15:30:21.682 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:21 smithi172 podman[73764]: 2024-05-21 15:30:21.26141807 +0000 UTC m=+0.458032717 container attach 9841a534770d09ece57e87f04f64fb324612d5e090261b0c2b56912f2755fdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:30:21.682 INFO:journalctl@ceph.osd.11.smithi172.stdout:May 21 15:30:21 smithi172 podman[73764]: 2024-05-21 15:30:21.461464447 +0000 UTC m=+0.658079092 container died 9841a534770d09ece57e87f04f64fb324612d5e090261b0c2b56912f2755fdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:30:22.030 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-21T15:30:22.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:22.069 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-21T15:30:22.069 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-21T15:30:22.069 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.a 2024-05-21T15:30:22.377 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:30:22 smithi078 systemd[1]: Stopping Ceph mds.a for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:22.845 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:30:22 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a[73770]: 2024-05-21T15:30:22.375+0000 7f177b85c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:22.846 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:30:22 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a[73770]: 2024-05-21T15:30:22.375+0000 7f177b85c700 -1 mds.a *** got signal Terminated *** 2024-05-21T15:30:25.010 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:30:24 smithi078 podman[106376]: 2024-05-21 15:30:24.746966352 +0000 UTC m=+2.476104069 container died 4be056043a8e73470c1127e2698787486516f2422801b6d9ab80d4d30007af29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2) 2024-05-21T15:30:25.345 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:30:25 smithi078 podman[106376]: 2024-05-21 15:30:25.109457356 +0000 UTC m=+2.838595065 container cleanup 4be056043a8e73470c1127e2698787486516f2422801b6d9ab80d4d30007af29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-21T15:30:25.346 INFO:journalctl@ceph.mds.a.smithi078.stdout:May 21 15:30:25 smithi078 bash[106376]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-a 2024-05-21T15:30:25.605 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-21T15:30:25.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:25.640 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-21T15:30:25.640 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-21T15:30:25.641 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.d 2024-05-21T15:30:25.972 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:30:25 smithi078 systemd[1]: Stopping Ceph mds.d for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:26.345 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:30:25 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d[72822]: 2024-05-21T15:30:25.971+0000 7f3b717ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:26.346 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:30:25 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d[72822]: 2024-05-21T15:30:25.971+0000 7f3b717ce700 -1 mds.d *** got signal Terminated *** 2024-05-21T15:30:28.915 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:30:28 smithi078 podman[106498]: 2024-05-21 15:30:28.654408802 +0000 UTC m=+2.787419032 container died 49275e6098244e035b3c0cbed3b244cadeb22d0f1d785729007f85c35c9972a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-21T15:30:29.298 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:30:29 smithi078 podman[106498]: 2024-05-21 15:30:29.022491853 +0000 UTC m=+3.155502091 container cleanup 49275e6098244e035b3c0cbed3b244cadeb22d0f1d785729007f85c35c9972a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-21T15:30:29.299 INFO:journalctl@ceph.mds.d.smithi078.stdout:May 21 15:30:29 smithi078 bash[106498]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-d 2024-05-21T15:30:29.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-21T15:30:29.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:29.464 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-21T15:30:29.464 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-21T15:30:29.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.g 2024-05-21T15:30:29.595 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:30:29 smithi078 systemd[1]: Stopping Ceph mds.g for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:30.095 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:30:29 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-g[73414]: 2024-05-21T15:30:29.784+0000 7fb172bca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:30.096 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:30:29 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-g[73414]: 2024-05-21T15:30:29.784+0000 7fb172bca700 -1 mds.g *** got signal Terminated *** 2024-05-21T15:30:33.044 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:30:32 smithi078 podman[106619]: 2024-05-21 15:30:32.75412707 +0000 UTC m=+3.067370541 container died 4c0e61d3c68f34f0558ad8a9c5ae8ea19eea8e7f60a9f8d3f50dfda1f1bf6fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-g, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-21T15:30:33.345 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:30:33 smithi078 podman[106619]: 2024-05-21 15:30:33.109934369 +0000 UTC m=+3.423177882 container cleanup 4c0e61d3c68f34f0558ad8a9c5ae8ea19eea8e7f60a9f8d3f50dfda1f1bf6fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-21T15:30:33.346 INFO:journalctl@ceph.mds.g.smithi078.stdout:May 21 15:30:33 smithi078 bash[106619]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-g 2024-05-21T15:30:33.542 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-21T15:30:33.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:33.573 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-21T15:30:33.573 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-21T15:30:33.573 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.j 2024-05-21T15:30:33.906 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:30:33 smithi078 systemd[1]: Stopping Ceph mds.j for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:34.345 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:30:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j[72443]: 2024-05-21T15:30:33.905+0000 7f152d1cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:34.345 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:30:33 smithi078 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j[72443]: 2024-05-21T15:30:33.905+0000 7f152d1cc700 -1 mds.j *** got signal Terminated *** 2024-05-21T15:30:38.095 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:30:37 smithi078 podman[106740]: 2024-05-21 15:30:37.749467613 +0000 UTC m=+3.954844007 container died 403f2890c9064f9cfa928fb33f3704d4b0d5fc02b34eb4842ed1cee0fef13e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:30:38.591 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:30:38 smithi078 podman[106740]: 2024-05-21 15:30:38.18362963 +0000 UTC m=+4.389005986 container cleanup 403f2890c9064f9cfa928fb33f3704d4b0d5fc02b34eb4842ed1cee0fef13e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-21T15:30:38.592 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:30:38 smithi078 bash[106740]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j 2024-05-21T15:30:38.592 INFO:journalctl@ceph.mds.j.smithi078.stdout:May 21 15:30:38 smithi078 podman[106753]: 2024-05-21 15:30:38.365349286 +0000 UTC m=+0.607374077 container remove 403f2890c9064f9cfa928fb33f3704d4b0d5fc02b34eb4842ed1cee0fef13e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514) 2024-05-21T15:30:38.615 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-21T15:30:38.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:38.647 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-21T15:30:38.648 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-21T15:30:38.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.b 2024-05-21T15:30:38.953 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:30:38 smithi084 systemd[1]: Stopping Ceph mds.b for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:39.211 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:30:38 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: 2024-05-21T15:30:38.952+0000 7f2191c92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:39.212 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:30:38 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b[67161]: 2024-05-21T15:30:38.952+0000 7f2191c92700 -1 mds.b *** got signal Terminated *** 2024-05-21T15:30:41.029 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:30:40 smithi084 podman[73849]: 2024-05-21 15:30:40.753441441 +0000 UTC m=+1.907818329 container died 57cd46193d8aca42a840a813fcf80923405ce9d13da82e207ad3ea3d053b4d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-21T15:30:41.394 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:30:41 smithi084 podman[73849]: 2024-05-21 15:30:41.136018515 +0000 UTC m=+2.290395397 container cleanup 57cd46193d8aca42a840a813fcf80923405ce9d13da82e207ad3ea3d053b4d34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:30:41.394 INFO:journalctl@ceph.mds.b.smithi084.stdout:May 21 15:30:41 smithi084 bash[73849]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-b 2024-05-21T15:30:41.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-21T15:30:41.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:41.605 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-21T15:30:41.605 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-21T15:30:41.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.e 2024-05-21T15:30:41.932 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:30:41 smithi084 systemd[1]: Stopping Ceph mds.e for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:42.393 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:30:41 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e[66544]: 2024-05-21T15:30:41.931+0000 7f2495942700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:42.394 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:30:41 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e[66544]: 2024-05-21T15:30:41.931+0000 7f2495942700 -1 mds.e *** got signal Terminated *** 2024-05-21T15:30:45.024 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:30:44 smithi084 podman[73981]: 2024-05-21 15:30:44.75830191 +0000 UTC m=+2.934665193 container died a93cd8b3a9103dac29da406609b078118724a604513ccb2b0dcf9241bb6c250b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-21T15:30:45.394 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:30:45 smithi084 podman[73981]: 2024-05-21 15:30:45.122739154 +0000 UTC m=+3.299102435 container cleanup a93cd8b3a9103dac29da406609b078118724a604513ccb2b0dcf9241bb6c250b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-21T15:30:45.394 INFO:journalctl@ceph.mds.e.smithi084.stdout:May 21 15:30:45 smithi084 bash[73981]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-e 2024-05-21T15:30:45.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-21T15:30:45.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:45.586 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-21T15:30:45.586 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-21T15:30:45.586 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.h 2024-05-21T15:30:46.144 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:45 smithi084 systemd[1]: Stopping Ceph mds.h for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:46.144 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:45 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h[66847]: 2024-05-21T15:30:45.892+0000 7f1e5e72a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:46.144 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:45 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h[66847]: 2024-05-21T15:30:45.892+0000 7f1e5e72a700 -1 mds.h *** got signal Terminated *** 2024-05-21T15:30:49.010 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:48 smithi084 podman[74101]: 2024-05-21 15:30:48.753577947 +0000 UTC m=+2.952430079 container died 096aa407a45fb4e975f13e734d7d293d62b00aab5acbfab59a20a26f602728e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c) 2024-05-21T15:30:49.277 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:49 smithi084 podman[74101]: 2024-05-21 15:30:49.092293921 +0000 UTC m=+3.291146043 container cleanup 096aa407a45fb4e975f13e734d7d293d62b00aab5acbfab59a20a26f602728e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-21T15:30:49.277 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:49 smithi084 bash[74101]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h 2024-05-21T15:30:49.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-21T15:30:49.539 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:49 smithi084 podman[74113]: 2024-05-21 15:30:49.277086167 +0000 UTC m=+0.518436144 container remove 096aa407a45fb4e975f13e734d7d293d62b00aab5acbfab59a20a26f602728e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-h, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-05-21T15:30:49.540 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:49 smithi084 systemd[1]: ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-21T15:30:49.540 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:49 smithi084 systemd[1]: Stopped Ceph mds.h for 852c9762-1783-11ef-bc9a-c7b262605968. 2024-05-21T15:30:49.540 INFO:journalctl@ceph.mds.h.smithi084.stdout:May 21 15:30:49 smithi084 systemd[1]: ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.h.service: Consumed 2.314s CPU time. 2024-05-21T15:30:49.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:49.550 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-21T15:30:49.551 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-21T15:30:49.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.k 2024-05-21T15:30:49.893 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:30:49 smithi084 systemd[1]: Stopping Ceph mds.k for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:50.201 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:30:49 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k[66229]: 2024-05-21T15:30:49.900+0000 7f3f13b68700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:50.201 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:30:49 smithi084 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k[66229]: 2024-05-21T15:30:49.900+0000 7f3f13b68700 -1 mds.k *** got signal Terminated *** 2024-05-21T15:30:53.047 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:30:52 smithi084 podman[74222]: 2024-05-21 15:30:52.747499847 +0000 UTC m=+2.981095137 container died bffdc70a4ddcd904f949ea67140fb0bccfbe39d89dfb12d575ff4090955110f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-21T15:30:53.364 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:30:53 smithi084 podman[74222]: 2024-05-21 15:30:53.145988701 +0000 UTC m=+3.379583992 container cleanup bffdc70a4ddcd904f949ea67140fb0bccfbe39d89dfb12d575ff4090955110f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T15:30:53.364 INFO:journalctl@ceph.mds.k.smithi084.stdout:May 21 15:30:53 smithi084 bash[74222]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-k 2024-05-21T15:30:53.490 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-21T15:30:53.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:53.520 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-21T15:30:53.521 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-21T15:30:53.521 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.c 2024-05-21T15:30:53.805 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:30:53 smithi172 systemd[1]: Stopping Ceph mds.c for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:54.181 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:30:53 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c[66713]: 2024-05-21T15:30:53.804+0000 7f59445b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:54.181 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:30:53 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c[66713]: 2024-05-21T15:30:53.804+0000 7f59445b6700 -1 mds.c *** got signal Terminated *** 2024-05-21T15:30:57.063 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:30:56 smithi172 podman[73868]: 2024-05-21 15:30:56.763122698 +0000 UTC m=+3.054382221 container died 92ef669fa1973d297791cc5642349843b32e9af729b6f5aa1d5eadbae90e9471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:30:57.363 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:30:57 smithi172 podman[73868]: 2024-05-21 15:30:57.154109985 +0000 UTC m=+3.445369516 container cleanup 92ef669fa1973d297791cc5642349843b32e9af729b6f5aa1d5eadbae90e9471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-21T15:30:57.364 INFO:journalctl@ceph.mds.c.smithi172.stdout:May 21 15:30:57 smithi172 bash[73868]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-c 2024-05-21T15:30:57.499 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-21T15:30:57.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:30:57.534 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-21T15:30:57.534 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-21T15:30:57.534 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.f 2024-05-21T15:30:57.681 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:30:57 smithi172 systemd[1]: Stopping Ceph mds.f for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:30:58.181 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:30:57 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f[66412]: 2024-05-21T15:30:57.857+0000 7efd2c885700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:30:58.182 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:30:57 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f[66412]: 2024-05-21T15:30:57.857+0000 7efd2c885700 -1 mds.f *** got signal Terminated *** 2024-05-21T15:31:01.159 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:31:00 smithi172 podman[74002]: 2024-05-21 15:31:00.765236337 +0000 UTC m=+3.012677142 container died 354413a5be18d65fbe74ae4ab7923abd64e6137e781498dae6415556f3079e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-21T15:31:01.431 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:31:01 smithi172 podman[74002]: 2024-05-21 15:31:01.224294342 +0000 UTC m=+3.471735139 container cleanup 354413a5be18d65fbe74ae4ab7923abd64e6137e781498dae6415556f3079e33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-21T15:31:01.432 INFO:journalctl@ceph.mds.f.smithi172.stdout:May 21 15:31:01 smithi172 bash[74002]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-f 2024-05-21T15:31:01.655 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-21T15:31:01.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:31:01.688 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-21T15:31:01.688 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-21T15:31:01.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.i 2024-05-21T15:31:02.012 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:31:01 smithi172 systemd[1]: Stopping Ceph mds.i for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:31:02.431 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:31:02 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i[67028]: 2024-05-21T15:31:02.011+0000 7f7110c4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:31:02.431 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:31:02 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i[67028]: 2024-05-21T15:31:02.011+0000 7f7110c4f700 -1 mds.i *** got signal Terminated *** 2024-05-21T15:31:05.095 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:31:04 smithi172 podman[74124]: 2024-05-21 15:31:04.749264025 +0000 UTC m=+2.842295132 container died e57ba4470e22ed45cc685e3828e26f563bbae2c28fa20d7e09202b53af840ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T15:31:05.362 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:31:05 smithi172 podman[74124]: 2024-05-21 15:31:05.186153733 +0000 UTC m=+3.279184838 container cleanup e57ba4470e22ed45cc685e3828e26f563bbae2c28fa20d7e09202b53af840ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-21T15:31:05.362 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:31:05 smithi172 bash[74124]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i 2024-05-21T15:31:05.616 INFO:journalctl@ceph.mds.i.smithi172.stdout:May 21 15:31:05 smithi172 podman[74136]: 2024-05-21 15:31:05.362174636 +0000 UTC m=+0.606308391 container remove e57ba4470e22ed45cc685e3828e26f563bbae2c28fa20d7e09202b53af840ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T15:31:05.620 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-21T15:31:05.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:31:05.651 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-21T15:31:05.651 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-21T15:31:05.651 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.l 2024-05-21T15:31:05.931 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:31:05 smithi172 systemd[1]: Stopping Ceph mds.l for 852c9762-1783-11ef-bc9a-c7b262605968... 2024-05-21T15:31:06.431 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:31:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l[66108]: 2024-05-21T15:31:05.972+0000 7f034a8f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T15:31:06.431 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:31:05 smithi172 ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l[66108]: 2024-05-21T15:31:05.972+0000 7f034a8f6700 -1 mds.l *** got signal Terminated *** 2024-05-21T15:31:09.082 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:31:08 smithi172 podman[74244]: 2024-05-21 15:31:08.779271744 +0000 UTC m=+2.914424106 container died e7fb22fe21c2624183265aa930c70d8208c086b8d325d637317badacfcac08f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T15:31:09.390 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:31:09 smithi172 podman[74244]: 2024-05-21 15:31:09.181187983 +0000 UTC m=+3.316340345 container cleanup e7fb22fe21c2624183265aa930c70d8208c086b8d325d637317badacfcac08f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5395c4020ac9059aec12c85bcc41cf8b1b75bc8c18339e31fbf77c838aee708, name=ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240517.203521-squid-debug-d2b507c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-21T15:31:09.391 INFO:journalctl@ceph.mds.l.smithi172.stdout:May 21 15:31:09 smithi172 bash[74244]: ceph-852c9762-1783-11ef-bc9a-c7b262605968-mds-l 2024-05-21T15:31:09.545 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-852c9762-1783-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-21T15:31:09.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T15:31:09.577 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-21T15:31:09.578 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 852c9762-1783-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T15:31:09.775 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:31:12.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 852c9762-1783-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T15:31:12.406 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:31:14.755 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 852c9762-1783-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T15:31:14.944 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:31:17.313 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:31:17.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:31:17.378 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T15:31:17.405 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-21T15:31:17.407 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi078/crash 2024-05-21T15:31:17.408 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T15:31:17.447 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T15:31:17.447 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-05-21T15:31:17.449 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi084/crash 2024-05-21T15:31:17.450 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T15:31:17.492 INFO:teuthology.orchestra.run.smithi084.stderr:tar: /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T15:31:17.492 INFO:teuthology.orchestra.run.smithi084.stderr:tar: Error is not recoverable: exiting now 2024-05-21T15:31:17.494 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi172/crash 2024-05-21T15:31:17.495 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T15:31:17.529 INFO:teuthology.orchestra.run.smithi172.stderr:tar: /var/lib/ceph/852c9762-1783-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T15:31:17.529 INFO:teuthology.orchestra.run.smithi172.stderr:tar: Error is not recoverable: exiting now 2024-05-21T15:31:17.530 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-21T15:31:17.530 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-21T15:31:17.566 INFO:tasks.cephadm:Compressing logs... 2024-05-21T15:31:17.566 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T15:31:17.609 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T15:31:17.612 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T15:31:17.640 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T15:31:17.640 INFO:teuthology.orchestra.run.smithi078.stderr:: No such file or directory 2024-05-21T15:31:17.641 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T15:31:17.641 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-21T15:31:17.641 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-21T15:31:17.642 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-21T15:31:17.642 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-21T15:31:17.642 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-21T15:31:17.642 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-21T15:31:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-21T15:31:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.a.log: 87.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T15:31:17.643 INFO:teuthology.orchestra.run.smithi078.stderr: 90.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T15:31:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T15:31:17.643 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-21T15:31:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T15:31:17.644 INFO:teuthology.orchestra.run.smithi078.stderr: 94.8% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-21T15:31:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T15:31:17.644 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-21T15:31:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T15:31:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:: No such file or directory 2024-05-21T15:31:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T15:31:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-21T15:31:17.660 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-21T15:31:17.660 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-21T15:31:17.660 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.b.log: --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-21T15:31:17.660 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-21T15:31:17.660 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-21T15:31:17.660 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-21T15:31:17.660 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.h.log: 87.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T15:31:17.660 INFO:teuthology.orchestra.run.smithi084.stderr: 90.2% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T15:31:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T15:31:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-21T15:31:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T15:31:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T15:31:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-21T15:31:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-21T15:31:17.662 INFO:teuthology.orchestra.run.smithi172.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T15:31:17.662 INFO:teuthology.orchestra.run.smithi172.stderr:: No such file or directory 2024-05-21T15:31:17.662 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-21T15:31:17.662 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-21T15:31:17.663 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T15:31:17.663 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-21T15:31:17.663 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log: --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-21T15:31:17.663 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-21T15:31:17.663 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-21T15:31:17.663 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-21T15:31:17.663 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.z.log: 87.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T15:31:17.663 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T15:31:17.664 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T15:31:17.664 INFO:teuthology.orchestra.run.smithi172.stderr: 91.9% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-21T15:31:17.664 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-21T15:31:17.664 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-21T15:31:17.664 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log: /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.11.log: 83.9% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T15:31:17.664 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T15:31:17.664 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-21T15:31:17.665 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-21T15:31:17.665 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T15:31:17.665 INFO:teuthology.orchestra.run.smithi078.stderr: 94.9% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-21T15:31:17.666 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T15:31:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-21T15:31:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T15:31:17.677 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T15:31:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T15:31:17.689 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T15:31:17.691 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T15:31:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T15:31:17.694 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T15:31:17.721 INFO:teuthology.orchestra.run.smithi078.stderr: 94.5% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-21T15:31:17.745 INFO:teuthology.orchestra.run.smithi078.stderr: 94.8% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-21T15:31:17.756 INFO:teuthology.orchestra.run.smithi084.stderr: 94.8% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-21T15:31:17.760 INFO:teuthology.orchestra.run.smithi084.stderr: 94.4% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-21T15:31:17.999 INFO:teuthology.orchestra.run.smithi078.stderr: 90.4% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-21T15:31:18.078 INFO:teuthology.orchestra.run.smithi084.stderr: 94.3% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-21T15:31:18.104 INFO:teuthology.orchestra.run.smithi172.stderr: 91.9% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-21T15:31:18.127 INFO:teuthology.orchestra.run.smithi084.stderr: 91.9% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-21T15:31:18.187 INFO:teuthology.orchestra.run.smithi172.stderr: 95.4% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-21T15:31:18.213 INFO:teuthology.orchestra.run.smithi172.stderr: 94.7% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-21T15:31:18.395 INFO:teuthology.orchestra.run.smithi172.stderr: 94.8% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-21T15:31:18.692 INFO:teuthology.orchestra.run.smithi172.stderr: 94.4% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-21T15:31:18.891 INFO:teuthology.orchestra.run.smithi078.stderr: 90.5% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-21T15:31:23.842 INFO:teuthology.orchestra.run.smithi084.stderr: 93.3% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-21T15:31:24.357 INFO:teuthology.orchestra.run.smithi084.stderr: 93.9% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-21T15:31:24.611 INFO:teuthology.orchestra.run.smithi078.stderr: 93.2% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-21T15:31:24.884 INFO:teuthology.orchestra.run.smithi084.stderr: 93.2% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-21T15:31:25.388 INFO:teuthology.orchestra.run.smithi172.stderr: 93.2% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-21T15:31:25.602 INFO:teuthology.orchestra.run.smithi172.stderr: 93.2% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-21T15:31:26.144 INFO:teuthology.orchestra.run.smithi172.stderr: 93.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-21T15:31:26.466 INFO:teuthology.orchestra.run.smithi078.stderr: 93.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-21T15:31:27.468 INFO:teuthology.orchestra.run.smithi078.stderr: 93.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-21T15:31:27.581 INFO:teuthology.orchestra.run.smithi084.stderr: 93.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-21T15:31:28.379 INFO:teuthology.orchestra.run.smithi172.stderr: 93.2% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-21T15:31:28.381 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-05-21T15:31:28.381 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m10.754s 2024-05-21T15:31:28.381 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m36.480s 2024-05-21T15:31:28.381 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m1.362s 2024-05-21T15:31:29.224 INFO:teuthology.orchestra.run.smithi078.stderr: 93.1% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-21T15:31:29.226 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-05-21T15:31:29.226 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m11.604s 2024-05-21T15:31:29.226 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m37.621s 2024-05-21T15:31:29.226 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m1.389s 2024-05-21T15:31:30.134 INFO:teuthology.orchestra.run.smithi084.stderr: 93.2% -- replaced with /var/log/ceph/852c9762-1783-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-21T15:31:30.136 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-05-21T15:31:30.136 INFO:teuthology.orchestra.run.smithi084.stderr:real 0m12.510s 2024-05-21T15:31:30.137 INFO:teuthology.orchestra.run.smithi084.stderr:user 0m41.919s 2024-05-21T15:31:30.137 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m1.428s 2024-05-21T15:31:30.137 INFO:tasks.cephadm:Archiving logs... 2024-05-21T15:31:30.137 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi078/log 2024-05-21T15:31:30.138 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T15:31:35.979 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi084/log 2024-05-21T15:31:35.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T15:31:41.916 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi172/log 2024-05-21T15:31:41.917 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T15:31:47.478 INFO:tasks.cephadm:Removing cluster... 2024-05-21T15:31:47.478 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 852c9762-1783-11ef-bc9a-c7b262605968 --force 2024-05-21T15:31:47.668 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:31:47.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 852c9762-1783-11ef-bc9a-c7b262605968 --force 2024-05-21T15:31:48.167 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:31:48.467 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 852c9762-1783-11ef-bc9a-c7b262605968 --force 2024-05-21T15:31:48.659 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: 852c9762-1783-11ef-bc9a-c7b262605968 2024-05-21T15:31:48.963 INFO:tasks.cephadm:Removing cephadm ... 2024-05-21T15:31:48.963 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T15:31:48.982 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T15:31:48.997 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T15:31:49.012 INFO:tasks.cephadm:Teardown complete 2024-05-21T15:31:49.013 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-21T15:31:49.025 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-21T15:31:49.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T15:31:49.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T15:31:49.041 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T15:31:49.164 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-21T15:31:49.187 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-21T15:31:49.188 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:31:49.193 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:31:49.196 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T15:31:49.205 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:31:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:31:49.210 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:31:49.211 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-05-21T15:31:49.211 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv01.front.sepia.ceph.com 3 8 377 45 +1046us[+1089us] +/- 51ms 2024-05-21T15:31:49.211 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 43 -2316us[-2316us] +/- 68ms 2024-05-21T15:31:49.211 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +1191us[+1191us] +/- 69ms 2024-05-21T15:31:49.211 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:31:49.212 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-05-21T15:31:49.213 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:31:49.213 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-05-21T15:31:49.213 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +1015us[ +994us] +/- 51ms 2024-05-21T15:31:49.213 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -2362us[-2362us] +/- 68ms 2024-05-21T15:31:49.214 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +1209us[+1209us] +/- 69ms 2024-05-21T15:31:49.214 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:31:49.214 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T15:31:49.214 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-05-21T15:31:49.215 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +1007us[ +990us] +/- 51ms 2024-05-21T15:31:49.215 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -2374us[-2391us] +/- 68ms 2024-05-21T15:31:49.215 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +1186us[+1169us] +/- 69ms 2024-05-21T15:31:49.215 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T15:31:49.216 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-21T15:31:49.227 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-21T15:31:49.227 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-21T15:31:49.267 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T15:31:49.348 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T15:31:49.413 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T15:31:49.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T15:31:49.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:31:49.509 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T15:31:49.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:31:49.536 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T15:31:49.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:31:49.562 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-21T15:31:49.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-21T15:31:49.597 INFO:teuthology.task.internal:Duration was 2039.365643 seconds 2024-05-21T15:31:49.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-21T15:31:49.636 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-21T15:31:49.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T15:31:49.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T15:31:49.641 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T15:31:49.675 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T15:31:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T15:31:49.678 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T15:31:50.182 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-21T15:31:50.183 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-05-21T15:31:50.183 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T15:31:50.204 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-05-21T15:31:50.204 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T15:31:50.249 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-05-21T15:31:50.249 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T15:31:50.284 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-21T15:31:50.284 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T15:31:50.287 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T15:31:50.293 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T15:31:50.349 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-21T15:31:50.349 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T15:31:50.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T15:31:50.357 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T15:31:50.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-21T15:31:50.770 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-21T15:31:50.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T15:31:50.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T15:31:50.819 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T15:31:50.843 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-21T15:31:50.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T15:31:50.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T15:31:50.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T15:31:50.884 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-05-21T15:31:50.885 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-05-21T15:31:50.906 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-05-21T15:31:50.924 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T15:31:50.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:31:50.953 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T15:31:50.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:31:50.970 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T15:31:50.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T15:31:50.984 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-21T15:31:50.998 INFO:teuthology.task.internal:Transferring archived files... 2024-05-21T15:31:50.999 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi078 2024-05-21T15:31:51.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T15:31:51.052 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi084 2024-05-21T15:31:51.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T15:31:51.087 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-21_13:18:14-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7718560/remote/smithi172 2024-05-21T15:31:51.087 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T15:31:51.127 INFO:teuthology.task.internal:Removing archive directory... 2024-05-21T15:31:51.128 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T15:31:51.130 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T15:31:51.132 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T15:31:51.172 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-21T15:31:51.184 INFO:teuthology.task.internal:Not uploading archives. 2024-05-21T15:31:51.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-21T15:31:51.195 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-21T15:31:51.195 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T15:31:51.199 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T15:31:51.202 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T15:31:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: 265549 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 15:31 /home/ubuntu/cephtest 2024-05-21T15:31:51.214 INFO:teuthology.orchestra.run.smithi084.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 15:31 /home/ubuntu/cephtest 2024-05-21T15:31:51.226 INFO:teuthology.orchestra.run.smithi172.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 15:31 /home/ubuntu/cephtest 2024-05-21T15:31:51.228 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-21T15:31:51.250 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-21T15:31:51.335 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} duration: 2039.365642786026 failure_reason: 'Command failed (workunit test suites/ffsb.sh) on smithi078 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2b507c724ef6c8295ee0a35d185cdab167fd61d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9ed1b325c7e74cbba4e902807f8afc89 status: fail success: false 2024-05-21T15:31:51.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T15:31:51.425 INFO:teuthology.run:FAIL